Psy
5038W, Fall 2009, 3 credits
Psychology Department , University
of Minnesota
Place: S150 Elliott
Hall
Time: 9:4511:00 MW
Course home pages: courses.kersten.org
Instructor: Daniel Kersten, Office:
S212 Elliott Hall, Phone: 6252589, email: kersten@umn.edu
Office hours: Mondays 11:00 to 12:00 and by appointment.
TA: Michael Blank, Office:
email: blan0138@umn.edu
Office hours: 11:0012:00 on Tuesdays, in Elliott Hall S501, and by appointment.
Course description. Introduction to large scale parallel distributed processing models in neural and cognitive science. Topics include: linear models, statistical pattern theory, Hebbian rules, selforganization, nonlinear models, information optimization, and representation of neural information. Applications to sensory processing, perception, learning, and memory.
General Readings and Software
Grade Requirements
There will be a midterm, final examination, programming assignments, as well as a final project. The grade weights are:
(NOTE:
Updated links to lecture material below will be revised and posted on the day of the lecture
if
you want a preview, check
out lectures from 2007)
All lecture notes are in Mathematica Notebook and pdf format. You can download
the Mathematica notebook files below to view with Mathematica or MathPlayer
(which is free).
Date 
Lecture 
Additional Readings & supplementary material 
Assignments 

I.

1 
Sep 9 
Introduction (pdf file)Mathematica notebook 
Mathematica screencast 

2 
Sep 14 
The
neuron (pdf
file) Mathematica
notebook 
HodgkinHuxley.nb 

3 
Sep 16 
Neural Models, McCullochPitt (pdf file) Mathematica notebook  Koch, C., & Segev, I. (Eds.). (1998) (pdf) 

4 
Sep 21 
Generic neuron model (pdf file) Mathematica notebook  
II. 
5 
Sep 23 
Lateral inhibition (pdf file) Mathematica notebook  Hartline (1972) (pdf)


6 
Sep 28 
Matrices (pdf file) Mathematica notebook  PS 1. Introduction to Mathematica , vectors, crosscorrelation  
7 
Sep 30  Linear systems, learning & Memory (pdf file) Mathematica notebook  
III. 
8 
Oct 
Linear Associator (pdf file) Mathematica notebook  
9 
Oct 7 
Sampling, Summed vector memory (pdf file) Mathematica notebook  ProbabilityOverviewNN.nb  
10 
Oct 12 
Nonlinear networks, Perceptron (pdf file) Mathematica notebook 

PS 2. Lateral inhibition  
11 
Oct 14 
Regression, WidrowHoff (pdf file) Mathematica notebook  
12 
Oct 19 
Multilayer feedforward nets, Backpropagation (pdf file) Mathematica notebook 
Poirazi,Brannon & Mel (2003) (pdf) Williams (1992) (pdf) 

IV.

13 
Oct 21 
Science
writing (pdf) (Mathematica notebook) 
Gopen & Swan,
1990 (pdf)


14 
Oct 26 
MIDTERM  MIDTERM (16%)  
15 
Oct 28 
Networks and Visual Representation (pdf file) Mathematica notebook  Carrandini, Heeger, Movshon (1996)(pdf)  
16 
Nov 2 
Neural Representation and coding (pdf file) Mathematica notebook  Sanger (2003) (pdf) Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005).(pdf) 

17 
Nov 4 
Selforganization, Principal Components Analysis (pdf file) Mathematica notebook  Supplement: ContingentAdaptation.nb  
18 
Nov 
Discrete Hopfield network (pdf file) Mathematica notebook  
19 
Nov 11 
Graded response Hopfield network (pdf file) Mathematica notebook  PROJECT IDEAS For demonstration style projects, see the Wolfram Demonstration site. 

20 
Nov 16 
Boltzmann machine (pdf file) Mathematica notebook 
Sculpting the energy function, interpolation (pdf file) Mathematica notebook)  
21 
Nov 18 
Adaptive maps (pdf file) Mathematica notebook  smallRetinaCortexMap.nb GraylefteyeDan.jpg 
Final project title & paragraph outline (2%) 

22  Nov 23  Probability (pdf file) Mathematica notebook 
Griffiths and Yuille (2006) (pdf) Jordan, M. I. and Bishop. C. MIT Artificial Intelligence Lab Memo 1562, March 1996. Neural networks. 
PS 4 Hopfield network  
V.

23  Nov 25  More on neural coding, Generative
models,Bayes nets and inference (pdf file) Mathematica notebook 
Knill
& Pouget (2004) (pdf) Pouget et al. (2006) (pdf) 

24  Nov 30  Belief
Propagation (pdf) Mathematica notebook 
Pattern Recognition and Machine Learning, Chapter 8. Weiss Y. (pdf) 

25  Dec 2  EM (pdf) Mathematica notebook 

26  Dec 7  Bayes decision theory (pdf) Mathematica notebook 
Fisher's linear discriminant notes (pdf) Mathematica notebook  REVISED DATES in RED Complete Draft of Final Project (5%) Due December 7 

27  Dec 9  Kalman
filter, fisher discriminant (pdf) 
Kalman notes (pdf) 
Peer comments on Final Project (5%:) Due December 14  
28  Dec 14  SVMs, Bias/Variance, Wrapup & Review 
Bias/Variance
notes (pdf) Mathematica SVMs: Nilsson, Björkegren & Tegnér (2006) 
Drafts returned with Instructor comments December 16  
Dec 16  EXAM (offical last instruction day)  FINAL STUDY GUIDE  FINAL EXAM (16%)  
Dec 23  (last day of fall semester)  Submit Final Revised Draft of Project (28%)  
This course teaches you how to understand cognitive and perceptual aspects of brain processing in terms of computation. Writing a computer program encourages you to think clearly about the assumptions underlying a given theory. Getting a program to work, however, tests just one level of clear thinking. By writing about your work, you will learn to think through the broader implications of your final project, and to effectively communicate the rationale and results in words.
Your final project will involve: 1) a computer simulation and; 2) a 20003000 word final paper describing your simulation. For your computer project, you will do one of the following: 1) Devise a novel application for a neural network model studied in the course; 2) Write a program to simulate a model from the neural network literature ; 3) Design and program a method for solving some problem in perception, cognition or motor control. The results of your final project should be written up in the form of a short scientific paper, describing the motivation, methods, results, and interpretation. Your paper will be critiqued and returned for you to revise and resubmit in final form. You should write for an audience consisting of your class peers. You may elect to have your final paper published in the course's webbased electronic journal.
Completing the final paper involves 3 steps:
If you choose to write your program in Mathematica, your paper and program can be combined can be formated as a Mathematica notebook. See: Books and Tutorials on Notebooks.
Your paper will be critiqued and returned for you to revise and resubmit in final form. You should write for an audience consisting of your class peers.
Some Resources:
Student Writing Support: Center for Writing, 306b Lind Hall and satellite locations
(612.625.1893) http://writing.umn.edu.
Online Writing Center:http://www.owc.umn.edu
NOTE: Plagiarism, a form of scholastic dishonesty and a disciplinaryoffense, is described by the Regents as follows: Scholasticdishonesty means plagiarizing; cheating on assignments or examinations;engaging in unauthorized collaboration on academic work; taking,acquiring, or using test materials without faculty permission; submittingfalse or incomplete records of academic achievement; acting alone or incooperation with another to falsify records or to obtain dishonestlygrades, honors, awards, or professional endorsement; or altering,forging, or misusing a University academic record; or fabricating orfalsifying of data, research procedures, or data analysis.http://www1.umn.edu/regents/policies/academic/StudentConductCode.html
© 2003, 2005, 2007 Computational Vision Lab, University of Minnesota, Department of Psychology.