PSY 5038W - Introduction to Neural Networks

Fall 2014
Class #: 34641  
9:45AM-11:00AM MW
Elliott Hall N227, TCEASTBANK

Course home pages: courses.kersten.org

Instructor: Daniel Kersten, kersten@umn.edu, Office: S212 Elliott Hall, Phone: 625-2589
Office hours: Mondays 11:00 to 12:00 and by appointment.

Teaching assistant: Cheng Qiu, qiuxx077@umn.edu
Office hours: Tuesdays 11:00 to 12:00

Course description. Introduction to large scale parallel distributed processing models in neural and cognitive science. Topics include: linear models, statistical pattern theory, Hebbian rules, self-organization, non-linear models, information optimization, and representation of neural information. Applications to sensory processing, perception, learning, and memory.

Prerequisites: Linear algebra, multivariate calculus.

Readings

Lecture notes (see below)

Software

Mathematica

Mathematica is the primary programming environment for this course. If you wish to purchase Mathematica for Students see http://www.wolfram.com/products/student/mathforstudents/index.html).

Accessing Mathematica on the CLA servers:

mac (Note: you may have to change the forward slash to a back slash)
windows

For user help on using Mathematica, see: http://mathematica.stackexchange.com

Learning center: http://www.wolfram.com/learningcenter/

Python/IPython

http://ipython.org
http://www.scipy.org

Supplementary

http://www.pybrain.org
https://code.google.com/p/neurolab/
http://briansimulator.org

Writing

Gopen, G. D., & Swan, J. A., 1990. The Science of Scientific Writing. American Scientist, 78, 550-558. (pdf)

Supplementary

• Penrose, A. M., & Katz, S. B. (1998). Writing in the Sciences: Exploring Conventions of Scientific Discourse. New York: St. Martin's Press, Inc.
• American Psychological Association. (2009). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association
  
  

Writing assistance

THE CENTER FOR WRITING offers free one-to-one writing assistance to undergraduate and graduate students, with appointments up to 45 minutes.
Nonnative speaker specialists are available. For more information, see http://writing.umn.edu

Supplementary readings

*Anderson, James. (1995) Introduction to Neural Networks, MIT Press.
***Bishop, C. M. (2006). Pattern recognition and machine learning. New York: Springer.
*Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience : computational and mathematical modeling of neural systems. Cambridge, Mass.: MIT Press.
Freeman, J. A. (1994). Simulating Neural Networks with Mathematica . Reading, MA: Addison-Wesley Publishing Company. http://library.wolfram.com/infocenter/Books/3485/
**Gershenfeld, N. A. (1999). The nature of mathematical modeling. Cambridge ; New York: Cambridge University Press.
**Hertz, J., Krogh, A., &;Palmer, R. G. (1991). Introduction to the theory of neural computation (Santa Fe Institute Studies in the Sciences of Complexity ed.). Reading, MA: Addison-Wesley Publishing Company.
Koch, C., & Segev, I. (Eds.). (1998). Methods in Neuronal Modeling : From Ions to Networks (2nd ed.). Cambridge, MA: MIT Press.
***MacKay, D. J. C. (2003). Information theory, inference, and learning algorithms. Cambridge, UK ; New York: Cambridge University Press. http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
***Murphy, K. P. (2012). Machine Learning: a Probabilistic Perspective. MIT Press.

*Neural/Cognitive Science
**Physics/Applied Math
***Statistical/machine learning

Grade Requirements

There will be a mid-term, final examination, programming assignments, as well as a final project. The grade weights are:

• Homework/programming: 28%
• Mid-term examination (covering first half of semester): 16%
• Final examination (covering 2nd half of semester): 16%
• Final project : 40% (four parts: 2%+5%+5%+28%)
  
  The programming assignments will use the Mathematica programming environment. No prior experience with Mathematica is necessary.
  
  Assignment due BEFORE class start time (9:45 am) on the day due. You can use the downloaded Mathematica notebook for the assignment as your template, add your answers, and email your finished assignment to the TA. You can copy and paste any code bits you need from the Lecture notebooks. But of course, you cannot copy and paste code or any other answer materials from someone else.

Outline & Lecture Notes
(under construction)

http://onestop.umn.edu/calendars/#fall2014

(NOTE: Links to revised lecture material below will be posted on the day of the lecture.
Links to the pdfs for additional readings may require a password.
If you want to preview similar material, check out lectures from 2009)

Lecture notes are in Mathematica Notebook and pdf format. You can download the Mathematica notebook files below to view with Mathematica or Wolfram CDF Player (which is free).

This course teaches you how to understand cognitive and perceptual aspects of brain processing in terms of computation. Writing a computer program encourages you to think clearly about the assumptions underlying a given theory. Getting a program to work, however, tests just one level of clear thinking. By writing about your work, you will learn to think through the broader implications of your final project, and to effectively communicate the rationale and results in words.

Your final project will involve: 1) a computer simulation and; 2) a 2000-3000 word final paper describing your simulation. For your computer project, you will do one of the following: 1) Devise a novel application for a neural network model studied in the course; 2) Write a program to simulate a model from the neural network literature ; 3) Design and program a method for solving some problem in perception, cognition or motor control. The results of your final project should be written up in the form of a short scientific paper, describing the motivation, methods, results, and interpretation. Your paper will be critiqued and returned for you to revise and resubmit in final form. You should write for an audience consisting of your class peers. You may elect to have your final paper published in the course's web-based electronic journal.

Completing the final paper involves 3 steps:

1. Outline. You will submit a working title and paragraph outline by the deadline noted in the syllabus. These outlines will be critiqued in order to help you find an appropriate focus for your papers. (2% of grade). (Consult with the instructor or TA for ideas well ahead of time).
2. Complete draft. You will then submit a complete draft of your paper (2000-3000 words). Papers must include the following sections: Abstract, Introduction, Methods, Results, Discussion, and Bibliography. Use citations to motivate your problem and to justify your claims. Figures should be numbered and have figure captions. Cite authors by name and date, e.g. (Marr & Poggio, 1979). Use a standard citation format, such as APA. Papers must be typed, with a page number on each page.Each paper will be reviewed with specific recommendations for improvement. (5% of grade)
3. Peer commentary. Each student will submit a paragraph on an anonymous paired project draft (5% of grade)
4. Final draft. You will submit a final revision for grading. (28% of grade). The final draft must be turned in by the date noted on the syllabus. Students who wish to submit their final papers to be published in the class electronic journal should turn in both paper and electronic copies of their reports.

If you choose to write your program in Mathematica, your paper and program can be combined can be formated as a Mathematica notebook. See: Books and Tutorials on Notebooks.

Your paper will be critiqued and returned for you to revise and resubmit in final form. You should write for an audience consisting of your class peers.

Some Resources:

Student Writing Support: Center for Writing, 306b Lind Hall and satellite locations (612.625.1893) http://writing.umn.edu.
Online Writing Center: http://writing.umn.edu/sws/visit/online/index.html

NOTE: Plagiarism, a form of scholastic dishonesty and a disciplinaryoffense, is described by the Regents as follows: Scholasticdishonesty means plagiarizing; cheating on assignments or examinations;engaging in unauthorized collaboration on academic work; taking,acquiring, or using test materials without faculty permission; submittingfalse or incomplete records of academic achievement; acting alone or incooperation with another to falsify records or to obtain dishonestlygrades, honors, awards, or professional endorsement; or altering,forging, or misusing a University academic record; or fabricating orfalsifying of data, research procedures, or data analysis.http://regents.umn.edu/sites/regents.umn.edu/files/policies/Student_Conduct_Code.pdf

© 2014 Daniel Kersten, University of Minnesota