Sample short answer questions
Define and describe the relation of the following key words or phrases to
neural networks. Provide examples where appropriate.
(8 items drawn from set below; 3 points each).
| eigenvector | linear associator | autoassociator | action potential | synaptic modification |
| Hebbian rule | heteroassociation | superposition | cable equation | bias/variance trade-off |
| leaky (or forgetful) integrate and fire model | EPSP/IPSP | Hodgkin-Huxley model | Mach bands | Bayes theorem |
| dendrite | classical conditioning | operant conditioning | reinforcement learning | lateral inhibition |
| spike | linear independence | grandmother cell | generic neural network neuron | delta-rule |
| McCulloch-Pitts | TLU | cross-correlation | perceptron learning rule | supervised learning |
| recurrent inhibition | gradient descent | compartmental model | "Energy" | symmetric matrix |
| Winner-take-all | least sum of squares | linear system | outer product learning | orthogonality |
| joint distribution | marginal distribution | conditional independence | covariance matrix | entropy |
| constraint satisfaction | correspondence problem | Hopfield's continuous response model | Kullback-Leibler divergence |
simulated annealing |
| Widrow-Hoff error correction | diameter-limited | linear separability | directed graphical model | generative model |
Sample essay questions
(choice of 2 essays drawn from those listed below; 12 points
each).
Describe the slow-potential model and discuss it in the context of current
views of the computational power of single neurons (see articles by Koch &
Segev, 2000; Poirazi, Brannon & Mel, 2003).
Describe Hopfield's 1984 graded response neural network model. How can it be used for memory? How does it relate to the discrete stochastic model of 1982?
Describe the Boltzmann machine algorithm for both recall using annealing, and for learning (you need not derive the learning rule). What are the pros and cons of this learning algorithm?
Describe the experiment and results of Berkes, P., Orban, G., Lengyel, M., & Fiser, J. (2011). Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science, 331(6013), 83. What does their experiment say about learning in neural networks?
Discuss a linear model of either auto- or hetero-associative learning. Give one example of its application. What are the problems with linear models and how can they be addressed?
Describe a neural network model for the lateral eye of the limulus. Discuss the relationship between the performance of feedforward and feedback models of lateral inhibition. (See chapter by Hartline & Ratliff, 1972.)
Explain why XOR can't be computed with a simple TLU. Explain how it can be solved by using an augmented input to a TLU.