Neural Networks for Machine Learning

Geoffrey Hinton

Learn about artificial neural networks and how they're being used for machine learning, as applied to speech and object recognition, image segmentation, modeling language and human motion, etc. We'll emphasize both the basic algorithms and the practical tricks needed to get them to work well.

Announcements

Welcome to the course on Neural Networks for Machine Learning

The videos

Each week, the course will have two lectures. Each lecture will be divided into about five short videos. Both lectures will become available on Monday (at 12.01 am Eastern Standard Time). There are quiz questions embedded in the videos to help you check that you are understanding the material. These quizzes do not count towards your final grade.

The two weekly quizzes

Each week, there will be two quizzes (one per lecture) that do count towards your final grade.
The two weekly quizzes will become available at the same time as the two lectures. To help keep you on track with the course, we will have due dates associated with the weekly quizzes. After the due dates, we will continue to accept late submissions with a penalty on your score.

The programming assignments

Programming assignments will become available at the beginning of weeks 2, 3, 5 & 7. You will need to answer questions about the results produced by the programs and your answers will count towards your final grade. As with the weekly quizzes, there will be due dates for the programming assignments.

The first programming assignment will be very simple and will count less towards your grade. It is mainly intended to get you to download Octave and get used to using it (see the Octave installation link). We regret that we do not have the resources to support other languages, but if you have Matlab it should be simple to adapt the Octave code we provide. You will not need to submit any code.

The final test

On the monday of the ninth week the final test will become available and should be completed by the following monday.
The final test will be 25% of the final grade, the programming assignments will be 35% and the weekly quizzes 40%.

Sun 30 Sep 2012 9:01:00 PM PDT

Syllabus

The course syllabus will be finalized sometime in the next few days. Meanwhile, the topics of the first 9 lectures are already decided and I am also including a list of topics that will be covered in the last 7 lectures.

Lecture 1: Introduction

Lecture 2: The Perceptron learning procedure

Lecture 3: The backpropagation learning proccedure

Lecture 4: Learning feature vectors for words

Lecture 5: Object recognition with neural nets

Lecture 6: Optimization: How to make the learning go faster

Lecture 7: Recurrent neural networks and advanced optimization

Lecture 8: How to make neural networks generalize better

Lecture 9: Combining multiple neural networks to improve generalization

TOPICS TO BE COVERED IN LECTURES 10-16

Deep Autoencoders (including semantic hashing and image search with binary codes)

Hopfield Nets and Simulated Annealing

Boltzmann machines and the general learning algorithm

Restricted Boltzmann machines and contrastive divergence learning

Applications of Restricted Boltzmann machines to collaborative filtering and document modeling.

Stacking restricted Boltzmann machines or shallow autoencoders to make deep nets.

The wake-sleep algorithm and its contrastive version

Recent applications of generatively pre-trained deep nets

Deep Boltzmann machines and how to pre-train them

Modeling hierarchical structure with neural nets

Sabados Literarios

Monday, October 1, 2012