Introduction to Neural Networks
U. of Minnesota
Mid-term Study Guide, Fall, 2014

The mid-term will cover material through October 22, 2014

Sample short answer questions
Define and describe the relation of the following key words or phrases to neural networks. Provide examples where appropriate.
(8 items drawn from set below; 3 points each).

eigenvector linear associator autoassociator action potential synaptic modification
Hebbian rule heteroassociation superposition cable equation bias/variance trade-off
leaky (or forgetful) integrate and fire model EPSP/IPSP Hodgkin-Huxley model Mach bands Bayes theorem
dendrite classical conditioning operant conditioning reinforcement learning lateral inhibition
spike linear independence grandmother cell generic neural network neuron delta-rule
McCulloch-Pitts TLU cross-correlation perceptron learning rule supervised learning
recurrent inhibition gradient descent compartmental model "Energy" symmetric matrix
Winner-take-all least sum of squares linear system outer product learning orthogonality
joint distribution marginal distribution conditional independence covariance matrix entropy
constraint satisfaction correspondence problem Hopfield's continuous response model

Kullback-Leibler divergence

simulated annealing
Widrow-Hoff error correction diameter-limited linear separability directed graphical model generative model

Sample essay questions
(choice of 2 essays drawn from those listed below; 12 points each).

Describe the slow-potential model and discuss it in the context of current views of the computational power of single neurons (see articles by Koch & Segev, 2000; Poirazi, Brannon & Mel, 2003).

Describe Hopfield's 1984 graded response neural network model. How can it be used for memory? How does it relate to the discrete stochastic model of 1982?

Describe the Boltzmann machine algorithm for both recall using annealing, and for learning (you need not derive the learning rule). What are the pros and cons of this learning algorithm?

Describe the experiment and results of Berkes, P., Orban, G., Lengyel, M., & Fiser, J. (2011). Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science, 331(6013), 83. What does their experiment say about learning in neural networks?

Discuss a linear model of either auto- or hetero-associative learning. Give one example of its application. What are the problems with linear models and how can they be addressed?

Describe a neural network model for the lateral eye of the limulus. Discuss the relationship between the performance of feedforward and feedback models of lateral inhibition. (See chapter by Hartline & Ratliff, 1972.)

Explain why XOR can't be computed with a simple TLU. Explain how it can be solved by using an augmented input to a TLU.