This course provides an advanced introduction to kernel methods and artificial neural networks with their properties and weaknesses. Knowledge and understanding: you will gain a comprehension of the theory and applications of kernel methods and its connections to neural networks. You will learn how the choice of the model architecture, optimization strategy and initialization deeply impact its performance both from a theoretical and practical point of view. Applying knowledge and understanding: you will be able to develop end-to-end kernel or deep learning models to solve various types of machine learning problems. Communication skills: you will be able to present the results of your machine learning pipeline together with an explanation of the data analysis and strategies steps. Learning skills: you will be able to navigate the machine learning existing kernel and deep learning literature to complement and improve your data analysis methods.

Knowledge from the “Introduction to Machine Learning” course. Knowledge of Python and scientific Python. Basic knowledge of matrix operations, differential equations and Calculus.

1. Kernel Methods 2.Artificial Neural Networks 3.Biologically inspired neural networks for visual cortex 4. Geometric deep learning 5. Adversarial attacks and implicit bias of neural networks.

Recommended 1. C. M. Bishop, Pattern recognition and machine learning. New York, NY: Springer, 2009. 2. J. Friedman, T. Hastie, and R. Tibshirani, The elements of statistical learning, vol. 1. Springer series in statistics Springer, Berlin, 2001. Other good textbooks 3. Machine Learning: an algorithmic perspective, S. Marsland, Chapman and Hall/CRC, 2014 4. Understanding Deep Learning by Simon J.D. Prince, Online

1. Kernels methods: from linear regression to kernels, kernel trick; Aronszajn theorem and positive definite maps; Reproducing kernel hilbert spaces, reproducing property; Pointwise continuity; Representer theorem; Kernel PCA, Kernel Ridge Regression, Kernel SVM 2. Neural Tangent kernel; Neural Networks and tier properties (generalization robustness and interpretability): early stopping, explicit regularization, SGD implicit regularization, Data augmentation and symmetries, Learning rate, Weights initialization; Architectures: convolutional networks and weight sharing; depth and compositionality; adversarial training; Interpretability 3. Biologically inspired anns for Visual Cortex: Invariant and selective representations, Sample complexity; Neural motivation and neurally plausible algorithms. Invariant and selective representation using kernel methods; Learning invariant and selective representations; Applications to visual cortex. 4.Geometric deep learning: Curse of dimensionality; Groups, groups representations, invariance and equivariance ; Group convolutions; Graph neural networks and permutation invariance; Applications. 5. Adversarial attacks and anns implicit bias: Implicit Bias in neural networks; Implicit bias for linear fully connected and convolutional; Robustness and implicit bias; Adversarsarial attacks; Fourier bias and robustness.

Frontal lectures and hands on sessions, both individual and in groups. The balance will be roughly 60% of frontal lectures and 40% of hands-on sessions. Ideally, each lecture will have a part of frontal teaching and a part of hands-on training. This may range from getting used to new libraries and tools to analyze complex datasets in groups.

Bring your laptop

The exam will consist of two parts: 1. Each student will propose one final project for the exam typically analyzing a complex dataset or investigating a methodology in detail, and will have to give a brief presentation explaining the work done and provide commented code upon request. We will evaluate the exposition clarity, the ability of synthesis and the understanding of the underlying theory. 2. an interview where few questions will be asked to assess the preparation on the topics of the course. The final mark is obtained by averaging the score for the project (max 12) and the oral part, max (18) to reach a maximum of 30. A bonus up to 3 points will be given depending on the averaged normalised marking of homeworks. In the portal part we will ask 3 questions: one simple (8 points), one average (6 points) and one complex (4 points). If a question will be not answered correctly we will remain on the same level of difficulty decreasing the points accordingly (8-6-4). Laude can be given for an exceptional exam, typically in presence of successfully submitted homework.

This course explores topics closely related to one or more goals of the United Nations 2030 Agenda for Sustainable Development (SDGs).