Computational Vision
U. Minn. Psy 5036
Daniel Kersten

Initialize

inormalize[x_]:= 0.5*x + 0.5; (*Vision scientists usually define images in terms of contrast from -1 to 1. Image[] expects pixel intensity values between 0 and 1*)

Limits to contrast detection: comparing humans with the ideal

Background

This notebook allows you to calculate your own statistical efficiency for contrast detection, relative to the signal-known-exactly (SKE) ideal observer.

There are two ways to calculate efficiency. The first is: (((((Subscript[d, human]^('))/(Subscript[d, ideal]^(')))))^2), where the d’’s are calculated from the performance levels (e.g. the proportions correct (Subscript[P, ideal]), (Subscript[P, human])) given that both the ideal and humans are trying to detect a signal with the same contrast. In general the ideal is getting more correct, so its d’ is bigger than the human’s, so the ratio is less than one. No observer can do better than the ideal under the constraints of the generative process and task.

The second is:  (((((Subscript[s, ideal])/(Subscript[s, human]))))^2), where (Subscript[s, ideal]) is the contrast threshold of the ideal, and (Subscript[s, human]) is the contrast threshold of the human when they are both at the same performance level--e.g. getting the same proportion correct in a 2AFC experiment. In general, the ideal is achieving a lower contrast threshold than the human, so again the ratio is less than one.

There are two main equations to keep in mind. The first computes d’ from the proportion correct, P:

dprime1 = -Sqrt[2] x z(P)

which using Mathematica’s built-in functions can be calculated as:

z[p_]:=-InverseCDF[NormalDistribution[0,1],p];
dprime1[x_] := N[-Sqrt[2] z[x],2]

And the second computes d’ from the signal and noise contrast levels, in terms of a signal-to-noise ratio:

dprime2 = (((((sv.sv)/σ)))^2) = ((((s/σ)))^2) ,

where sv is a vector of signal contrast values, s is a scalar equal to the vector length of sv, and σ is the standard deviation of the gaussian distribution that generates the noise image which is added at each trial. The vector length of sv can be calculated as:

Clear[dprime2];
dprime2[sv_,σ_] := Flatten[sv].Flatten[sv]/σ;

d’ can also be written as:

d'=((r.m.s. contrast x (Sqrt[N]))/σ)

In the code below scon = r.m.s. contrast , and ncon = σ.

Stimulus set up

Define the signal image

Clear[cgabor,signalcgabor,swidth]

cgabor[x_,y_, fx_, fy_,s_] := 
Exp[-(x^2 +  y^2)/s^2] Cos[2 Pi(fx x + fy y)];
signalcgabor[x_,y_,vf_,vtheta_,swidth_]:=N[cgabor[x,y,vf Cos[vtheta],vf Sin[vtheta],swidth]];

size=256;
signal0=Table[signalcgabor[x,y,1,Pi/2, 1],{x,-2,2,(4/(size-1))},{y,-2,2,(4/(size-1))}];
signal0 =signal0/StandardDeviation[Flatten[signal0]];
(*Remove ; below to check values*)
StandardDeviation[Flatten[signal0]];
scon;
StandardDeviation[scon*Flatten[signal0]];
RootMeanSquare[Flatten[signal0]];
ListPlot[signal0[[128]],ImageSize->Small];

Now get a preliminary estimate of your  threshold for scon0, using the “method of adjustment” for a fixed noise level of ncon = 0.35.  You can use this as a starting value in the 2AFC task below.

scon0=0.01;
ncon0=.35;
Manipulate[GraphicsRow[{Image[inormalize[scon0*signal0]],Image[inormalize[ncon0*RandomVariate[NormalDistribution[0,1],{size,size}]]],Image[inormalize[scon0*signal0+ncon0*RandomVariate[NormalDistribution[0,1],{size,size}]]]}],{{scon0,.001},.001,.025},{{ncon0,.35},.1,.8}]

Define a mid-gray blank image

blank = Table[0.5,{size},{size}];
flash=blank;

Define a function to create random noise images that get added to the signal

noise := RandomVariate[NormalDistribution[0,1],{size,size}];

Define functions: stimulus presentation, response logging, and results analysis

signal = signal0;
twoflashes:=Module[{},
If[ii>numtrials,Throw[ii]];
whichflash=RandomInteger[{0,1}];If[whichflash==1,leftnumsample=leftx=scon*signal+ncon*noise;rightnumsample=rightx=ncon*noise,leftnumsample=leftx=ncon*noise;rightnumsample=rightx=scon*signal+ncon*noise];
Pause[.5];
flash=inormalize[leftnumsample];Pause[.25];flash=blank;Pause[.25];
flash=inormalize[rightnumsample];Pause[.25];flash=blank];

loganswer[myanswer_]:=Module[{}, If[myanswer==whichflash,WasICorrect=1,WasICorrect=0];
idealanswer=If[Flatten[leftx].Flatten[signal]>Flatten[rightx].Flatten[signal],1,0];
If[idealanswer==whichflash,WasIdealCorrect=1,WasIdealCorrect=0];data=Append[data,{WasICorrect,WasIdealCorrect}]];

results := Module[{},data2=Transpose[Drop[data,1]];myproportioncorrect=N[Map[Count[data2[[1]],#]&,{1}]/Dimensions[data2][[2]]][[1]];
idealproportioncorrect=N[Map[Count[data2[[2]],#]&,{1}]/Dimensions[data2][[2]]][[1]];
mydprime=dprime[myproportioncorrect];
idealdprime=dprime[idealproportioncorrect];
If[(mydprime == Infinity)|| (idealdprime == Infinity) || (idealdprime == 0),Infinity,mystatisticalefficiency=Round[100.*(mydprime/idealdprime)^2]];
mystatisticalefficiency
]

Run a block of trials

scon=0.004;
ncon=.35;
data={{"Was I Correct?","Was Ideal Correct?"}};
numtrials=15;
ii=0;

Dynamic[
Panel[Column[{Panel[Image[flash]],Button["Start",twoflashes,Method->"Queued"],
Row[{Button["First",loganswer[1];results ;twoflashes;ii++,Method->"Queued"],
Button["Second",loganswer[0];
results ;twoflashes;ii++,Method->"Queued"]}]},Alignment->Center],FrameMargins->20]]

Take a look at the raw data

data //MatrixForm

Summarize the results

Dynamic[If[Dynamic[ii]>numtrials,Print["done"]]]

Print[Style[Grid[{{"my prop correct","ideal's prop correct","my d'","ideal's d'","my efficiency (%)"},{myproportioncorrect,idealproportioncorrect,mydprime,idealdprime,mystatisticalefficiency}},Frame->All],9]];

To get a reliable estimate, you need at least 100 or more trials, i.e. numtrials=100.

How can you measure statistical efficiency if the signal-to-noise ratio is so high the ideal observer doesn't make enough mistakes to get a reliable estimate of it's d'

How can you calculate efficiency if the ideal hasn’t made any mistakes? If the ideal makes no mistakes, it’s d’ is infinity.

First, find out what the signal contrast for the ideal would be if it was getting the same proportion correct as the human, i.e. you. For example, if you were getting 75% correct, the ideal’s signal-to-noise ratio at that level would be:

dprime1[myproportioncorrect]

But your signal-to-noise ratio is determined by the square root of the signal contrast energy divided by the standard deviation of the noise:

dprime2[scon*signal,ncon]

So in percent, your efficiency would be:

100*(dprime1[myproportioncorrect]/dprime2[scon*signal,ncon])^2

Repeat your experiment with a different signal

Let’s try a signal that we might expect to be handled inefficiently by the human visual system--namely, a sample of “frozen noise”. We’ll draw just one sample and use it as our stored template.

SeedRandom[1111]; (*We fix the random number seed so that we can get an identical sample of noise if we want to repeat the experiment under the same conditions, e.g. with different observers*)
signal1 = RandomVariate[NormalDistribution[0,.2],{size,size}];
signal1 =signal1/StandardDeviation[Flatten[signal0]];
Image[inormalize[signal1]]
{Max[signal1],Min[signal1]}

In the above section (Define functions: stimulus presentation, response logging, and results analysis), set:
	 signal = signal1 above, i.e. replace signal0 with signal1. Then measure your threshold.