Exercises 1: Mathematica

Start with simple arithmetic. For example, type 5+7 in an input cell, and then hit the "enter" key (or “shift” AND “return” together). Note that if you try division, e.g. 2/3, you get the exact answer back. To get a decimal approximation, type N[2/3].

You can go back and select an expression by clicking on the brackets on the far right. These brackets serve to organize text and calculations into a Notebook with outlining features. You can group or ungroup cells for text, graphs, and expressions in various ways to present your calculations. Explore these options under Cell in the menu.  You can see the possible cell types under the Style menu.

Seems pretty dumb so far...but you can see that Mathematica's default handling of arithmetic can make a difference in the next examples:

a)  Calculate the square roots of: 12345678987654321 and 12345678987654321.0. 

b) Compare (2^.000000000001)^1000000000000 with (2^(1/1000000000000))^1000000000000

If you don't explicitly tell Mathematica that your expressions involve floating point numbers, it will treat the numbers as true integers. And in general, Mathematica will do symbolic processing as a default, and assume you want as general an answer as possible. Because symbolic processing demands considerably more computer resources than numerical processing, this results in annoying consequences, like slow or no responsivity. So remember to include a decimal point somewhere, or use N[ ] to force a floating point representation, and consequently numerical computations.

Front-end stuff

There's a lot to explore stylistically in Notebooks, but one of the most common things you will do is to select and manipulate the brackets on the far right. You can go back and select an expression by clicking on the brackets on the right. These brackets are features of the user interface and serve to organize input data, programs, text and the outcomes of calculations into a Notebook with outlining features. You can group or ungroup cells for text, graphs, and expressions in various ways to present your calculations. Explore these options under Cell in the menu.  You can see the possible cell types under the Style menu.

By ending an expression with ; you can suppress the output:

(3/4)/6;
(3 4)/6

Before Mathematica 6, you had to be particularly careful ... without a semi-colon before evaluating, you could end up with a Notebook full of 50,000 "hi"s. It behaves nicely with later versions. Execute the next cell:

a=Table["hi",{50000}]

The most recent result of a calculation can be accessed by %, the one before by %%, and so forth.  Try it.

Built-in functions

Mathematica has a very large library of built-in functions. They all begin with an uppercase letter and the arguments are enclosed by square brackets. Knowing that, you can often guess the form of a function. Or under the Help menu, go to Find Selected Function.

If you know the function, you can enter: ?Log to get more info. And then click on >> to take you to the documentation page.  Try the next cell:

?Plot

If you type two question marks before a function, ??Plot, you'll get more information. Try it.

?NormalDistribution

??Random

Defining your own functions

Let's illustrate function definition by building a simple model of a neuron’s input/output function. We'll see the justification later.  Suppose a neuron takes a set of inputs, say x1, x2, x3,  (e.g. from the outputs of three other neurons in the network) and produces an output signal, call it y.

For a so-called linear model, the output is the weighted sum of the inputs. We'll assume the weights are fixed and given by w1, w2, w3.

w1=-1; w2 = 2; w3=-1;

y[x1_,x2_,x3_]:= w1*x1 + w2*x2 +w3*x3 + 30;

(This code is pretty primitive and not very general--don't worry, we'll get more sophisticated later). 
The underscore in x1_ is important because it tells Mathematica that x1 represents a slot for a variable, not an expression.

:= vs.  =

Note that when defining a function, we used a colon followed by equals ( := ) instead of just an equals sign (=). When you use an equals sign, the value is calculated and assigned immediately. When there is a colon in front of the equals, the value is calculated only when called on later. So we use := for function definition because we need to define the function for later use and evaluation,  when we may have new values for its arguments.

(A double equals (==) has yet another meaning and is used to represent a symbolic equation which evaluates to True if left and right hand sides are identical.)

xv:=RandomInteger[10];
xf=RandomInteger[10];

a) Now evaluate xv and xf three times each. What is the difference between the two definitions?

{xv, xf}
{xv, xf}
{xv, xf}

You can do the same thing by making a list of lists: {{xv,xf},{xv,xf},{xv,xf}}

b) Compare repeated evaluations of the above with repeated evaluations of the two expressions below.

y[xv,xv,xv]
y[xf,xf,xf]

Why does our neuron model always output a 30 when the inputs  all have the same value? In general, what family of inputs will all produce a constant output?

Hint: The weights we defined can be thought of as a discrete approximation to a 2nd derivative operator in differential calculus.

Graphics & more function definitions

The generic non-linear model neuron requires defining a function that suppresses small outputs and "squashes" or clamps large outputs to a maximum level. Here is an example:

squash[x_] := N[1/(1 + Exp[-x+4])];

Note that our squashing function was defined with N[]. Again, remember that Mathematica tries to keep everything exact as long as possible and thus will do symbol manipulation if we don't explicitly tell it that we want numerical representations and calculations.

Let's plot a graph of the squash function for -5<x<10, using the syntax we learned about above:

Plot[squash[x],{x,-5,10}]

We'll use this and similar functions to model the small-signal compression and large signal saturation characteristics of neural output.

(Since version 10, this function is built in: MouseOver)

Even though Exp grows exponentially fast with increasing x (by definition!), squash “keeps a lid” on it.

It can be important to check your definitions like this. One reason is that, as we will see later, Mathematica definitions can be built up line-by-line, incorporating multiple criteria/constraints. And sometimes you might add to a function unwittingly and it appears to misbehave. You can check your defintion by asking Mathematica for it.

Mathematica has useful and easy-to-program GUI tools. One of the most common is   Manipulate[ ]

First check the definition of the squash function already defined above:

?squash

(*Define squash2[x,λ] here*)
Manipulate[Plot[squash2[x,λ], {x, -5, 10}],{{λ,1},0.1,2}]

Check the definition:

?squash2

We’ll often want to visualize the effective synaptic weights of a model neuron. Some neurons in the visual cortex have 2D spatial pattern of weights that are modeled by a gaussian windowed sinusoid:

Turn the above into a Mathematica expression and visualize the function by replacing ZZZ with it in the routine below:

Manipulate[Plot3D[ZZZ,{x,-2,2},{y,-2,2},PlotRange->{-2,2}, ImageSize->Tiny],{a,1,6},{b,1,6}]

Try using your mouse and mouse button to rotate the above plot. 

What does the Option or Alt key do?

a) Suppose each instance of a category of patterns, that we’ll call “A”, has 2 features, such as brightness and size. Think of a feature as being represented by a dimension. How “much” of each of the features of A is represented by a pair of numbers {x,y}, x for how bright, and y for how big. And similarly for a second category, “B”. 

A particular “instance”, say the first, is then represented by a pair of feature values {(Subscript[x, 1]),(Subscript[y, 1])}.

Suppose we have 3 instances for each category.

Hare are two lists in Mathematica format:

Ainstancelist = {{0,1},{1,1.5},{1.7,1.6}};
Binstancelist = {{.5,1.1},{1,3.},{2,3.5}};

Find and use the Mathematica function to return the number of rows and columns of list Ainstancelist. Hint: the function name begins with D.

Use ListPlot[] to plot Ainstancelist and Binstancelist on the same graph. If you do it right, the A and B points will be automatically color coded.

Bring up Mathematica’s Drawing Tools from the Graphics menu. Use the straight line tool to try to draw a line between the instances of A and B. 

Are the classes linearly separable?

Instead of doing calculations to answer this question, this exercise illustrates how you can use interactive graphics to get the answer. 

The time constant for the RC circuit model in the lecture is proportional to RR x CC, the resistance (RR) times capacitance (CC). The resistance is inversely proportional to the cross-section area. 

Assume a cross-section area of AA = 0.5 units. We need to specify a response criterion--assume the response has to reach 75% of its maximum value to be sufficiently significant.

Clear[V,τ,g1,RR,CC];CC=1.0;
V[t_,AA_]=1-(E^(-t*AA/(CC)));

g1[AA_]:=Plot[V[t-1,AA],{t,1,6},PlotRange->{-1,1.5},AxesOrigin->{-1,-.5},PlotStyle->RGBColor[1,0,0]]; (*This is the output*)g2=Plot[UnitStep[t-1],{t,-.5,6},PlotRange->{-1,1.5},AxesOrigin->{-1,-.5},PlotStyle->{RGBColor[1,0,0],RGBColor[0,0,1]}]; (*This is the input*)
g3=Plot[.75,{t,-.5,6},PlotRange->{-1,1.5},AxesOrigin->{-1,-.5},PlotStyle->{RGBColor[1,0,0],RGBColor[0,1,1]}] ;(*This is the criterion response level*)

The next line plots the input (time), output (voltage), and the criterion together with a Dynamic function Locator. The output depends on the time constant which in turn depends on the cross-section area, controlled by the AA slider.

Manipulate[Show[{g1[AA],g2,g3,Graphics[Point[p]],Graphics[Text[p,{3,0}]]}],{{p,{1,1}},Locator},{{AA,0.5},10,.1}]

OK, now for the exercise. The original cross section is 0.5 units, which is the default in the above plot. Use the locator to visually locate the point on the graph where the neuron voltage response is 0.75 (the cyan line). 

For what value of the cross-section (AA) will the time taken to reach criterion be halved?

Using McCullochPitts[a, b, t], find a threshold value that will enable a McCulloch-Pitts neuron to realize the And[] function whose truth table is shown below.

truthtable=Flatten[Table[{a,b,And[a==1,b==1]},{a,0,1},{b,0,1}],1];
TableForm[truthtable,TableHeadings->{{},{"a", "b","c"}}]