Demo of a state-machine-driven Opticka Behavioural Experiment.

Opticka is an object-oriented framework for the Psychophysics toolbox (PTB), allowing randomised interleaved presentation of parameter varying stimuli specified in experimenter-relevant values. It operates under Linux (PTB's preferred OS), macOS & Windows, and can interface via strobed words (using Display++, VPixx or a cheap and very reliable LabJack), or TTLs via low-cost Arduino. It uses ethernet to connect with Tobii or Eyelink eyetrackers and with with external harware for recording neurophysiological data.

In this demo, two stimulus objects (wrapped in a myStims manager object), task sequence variables (myTask object), and a screenManager (myScreen object) are passed to the runExperiment class. runExperiment uses the stateMachine class object which loads the experiment specification (for this demo it uses DefaultStateInfo.m)

DefaultStateInfo.m specifies the following experiment with six states (prefix, fixate, stimulus, correct, incorrect, breakfix), and uses the eyetracker (here using the mouse functions to transition from fixate.

   ┌───────────────────────────────────────┐
   │                                       ▼
   │                                     ┌───────────────────────────┐
┌──┼───────────────────────────────────▶ │          (1) prefix       │ ◀┐
│  │                                     └───────────────────────────┘  │
│  │                                       │                            │
│  │                                       ▼                            │
│┌───────────┐ Transition                ┌───────────────────────────┐  │
││ incorrect │ inFixFcn=>incorrect       │          fixate           │  │
││           │◀───────────────────────── │ show(stims, 2)            │  │
│└───────────┘                           └───────────────────────────┘  │
│                                          │                            │
│                                          │ Transition                 │
└──┐                                       │ inFixFcn=>stimulus         │
   │                                       ▼                            │
 ┌───────────┐ Transition                ┌───────────────────────────┐  │
 │  CORRECT  │ maintainFixFcn=>correct   │         stimulus          │  │
 │           │◀───────────────────────── │ show(stims, [1 2])        │  │
 └───────────┘                           └───────────────────────────┘  │
                                           │                            │
                                           │ Transition                 │
                                           │ maintainFixFcn=>breakfix   │
                                           ▼                            │
                                         ┌───────────────────────────┐  │
                                         │         BREAKFIX          │ ─┘
                                         └───────────────────────────┘

For this demo a dummy eyetracker object (where the mouse input transparently replaces the eye movements), is used to demonstrate behavioural control of the paradigm.

Opticka also offers an optional GUI (type `opticka` in the command window), which is a visual manager of the objects introduced here. The UI also controls other functions such as screen calibration, protocol loading/saving and managing communication with neurophysiological equipment via LabJack, Arduino and ethernet.

The source of this file can be found at : https://github.com/iandol/opticka/blob/master/optickaBehaviourTest.m

Initial clear up of any previous objects
Setup our visual stimuli
Task Initialisation
Setup screenManager Object
Setup runExperiment Object
Run the full behavioural task
Plot a timing log of every frame against the stimulus on/off times
Plot a timing log of all states and their function evaluation transition times

Initial clear up of any previous objects

Make sure we start in a clean environment, not essential

clear myStims myTask myExp myScreen
sca %PTB screen clear all

Setup our visual stimuli

First we use the metaStimulus class to create a stimulus manager object that collects and handles groups of stimuli as if they were a single 'thing', so for example when you use the draw method myStims.draw(), it tells each of its child stimuli to draw in order. You can show and hide each stimulus in the group and only thoses set to draw will do so. Note you can control each stimulus yourself (each stimulus has its own set of control functions), but using metaStimulus makes the code simpler...

myStims			= metaStimulus();

first stimulus is a smoothed 5° red disc

myStims{1}		= discStimulus('colour', [0.7 0.2 0], 'size', 5, 'sigma', 20);

second stimulus is an optimised 0.8° fixation cross from Thaler et al., 2013

myStims{2}		= fixationCrossStimulus('size', 0.8);

Task Initialisation

The taskSequence class defines a stimulus sequence (task) which is composed of randomised stimulus parameter changes (called variables) repeated over a set of blocks. A trial is an individual stimulus presentation. This

For behavioural tasks, several of the parameters like myTask.trialTime are not used as the state machine takes over the job of task timing, but the stimulus randomisation is still useful for such tasks. In this case the state info file can use taskSequence to deterimine the next stimulus value. There are functions to handle what happens if the subject responds incorrectly, where we can re-randomise the next value within the block.

myTask					= taskSequence(); %new taskSequence object instance

Our variable is xyPosition, applied to stimulus 1 only. 3 different position values so 3 trials per block...

myTask.nVar(1).name		= 'xyPosition';
myTask.nVar(1).stimulus	= 1;
myTask.nVar(1).values	= {[-10 -10], [0 0], [10 10]};

We call the method to randomise the trials into a block (3 blocks, 3 trials) structure.

myTask.nBlocks			= 3;
randomiseTask(myTask);

---> taskSequence<taskSequence#1594B8760>: Took 61.1 ms | randomiseTask

Setup screenManager Object

screenManager controls the PTB Screen(). We initialise the object with parameters to open the PTB screen with. Note distance and pixels per cm define the resultant geometry > pixel mappings. You can set several screen parameters, windowing, blending etc.

myScreen = screenManager('distance', 57.3,... % display distance from observer
	'pixelsPerCm', 27,... % calibration value for pixel density, measure using calibrateSize()
	'windowed', [],... % use fullscreen [] or window [X Y]?
	'backgroundColour', [0.5 0.5 0.5],... % initial background colour
	'blend', true,... % enable OpenGL blending, you can also set blend modes when needed
	'bitDepth', '8bit'); % FloatingPoint32bit, 8bit, FloatingPoint16bit etc.
% use retina mode for macOS
if ismac; myScreen.useRetina = true; end

PsychVulkanCore-INFO: Vulkan instance (version 1.3.290) created.

Setup runExperiment Object

We now pass our stimulus, screen and task objects to the runExperiment class. runExperiment contains the runTask() method that actually runs the behavioural task.

Importantly, the stateInfoFile 'FixationTrainingStateInfo.m' determines the behavioural protocol that the state machine will run. The state machine is available as myExp.stateMachine. Read through that StateInfo file to better understand what is being done. stateInfoFiles contain some general configuration, then a set of cell-arrays of functions, and a table of states which run these function arrays as the states are entered and exited. For this simple task, it starts in a paused state, then transitions to a blank period, then the stimulus presentation, where initiating and maintaining fixation on the cross leads to a correct state or breaking fixation leads to breakFix state, then the state machine loops back to blank etc. using the myTask object to set variable values on each trial.

myExp = runExperiment('stimuli', myStims,... %stimulus objects
	'screen', myScreen,... %screen manager object
	'task', myTask,... % task randomised stimulus sequence
	'stateInfoFile', 'DefaultStateInfo.m', ... % use the default state info file
	'debug', false,... % debug mode for testing?
	'logStateTimers',true,...
	'subjectName', 'Simulcra', ...
	'researcherName', 'Automated Test');
myExp.eyetracker.device = 'eyelink';
myExp.eyetracker.dummy = true;

Run the full behavioural task

runTask(myExp);

--->arduinoManager: Ports available: /dev/ttyS15
--->arduinoManager: Ports available: /dev/ttyS6
--->arduinoManager: Ports available: /dev/ttyS23
--->arduinoManager: Ports available: /dev/ttyS13
--->arduinoManager: Ports available: /dev/ttyS31
--->arduinoManager: Ports available: /dev/ttyS4
--->arduinoManager: Ports available: /dev/ttyS21
--->arduinoManager: Ports available: /dev/ttyS11
--->arduinoManager: Ports available: /dev/ttyS2
--->arduinoManager: Ports available: /dev/ttyS28
--->arduinoManager: Ports available: /dev/ttyS0
--->arduinoManager: Ports available: /dev/ttyS18
--->arduinoManager: Ports available: /dev/ttyS9
--->arduinoManager: Ports available: /dev/ttyS26
--->arduinoManager: Ports available: /dev/ttyS16
--->arduinoManager: Ports available: /dev/ttyACM0
--->arduinoManager: Ports available: /dev/ttyS7
--->arduinoManager: Ports available: /dev/ttyS24
--->arduinoManager: Ports available: /dev/ttyS14
--->arduinoManager: Ports available: /dev/ttyS5
--->arduinoManager: Ports available: /dev/ttyS22
--->arduinoManager: Ports available: /dev/ttyS12
--->arduinoManager: Ports available: /dev/ttyS30
--->arduinoManager: Ports available: /dev/ttyS3
--->arduinoManager: Ports available: /dev/ttyS20
--->arduinoManager: Ports available: /dev/ttyS10
--->arduinoManager: Ports available: /dev/ttyS29
--->arduinoManager: Ports available: /dev/ttyS1
--->arduinoManager: Ports available: /dev/ttyS19
--->arduinoManager: Ports available: /dev/ttyS27
--->arduinoManager: Ports available: /dev/ttyS17
--->arduinoManager: Ports available: /dev/ttyS8
--->arduinoManager: Ports available: /dev/ttyS25
PTB-INFO: Using PortAudio V19.7.0-devel, revision 147dd722548358763a8b649b3e4b41dfffbcfbb6
PTB-INFO: Using PortAudio V19.7.0-devel, revision 147dd722548358763a8b649b3e4b41dfffbcfbb6
---> audio-manager<audioManager#1594CDB34>: Audio Manager initialisation complete | constructor
PTB-INFO: Using PortAudio V19.7.0-devel, revision 147dd722548358763a8b649b3e4b41dfffbcfbb6
PTB-INFO: Using PortAudio V19.7.0-devel, revision 147dd722548358763a8b649b3e4b41dfffbcfbb6
PTB-INFO: Using PortAudio V19.7.0-devel, revision 147dd722548358763a8b649b3e4b41dfffbcfbb6
PTB-INFO: Choosing deviceIndex 0 [HDA Intel PCH: ALC3234 Analog (hw:0,0)] as default output audio device.
PTB-INFO: New audio device -1 with handle 0 opened as PortAudio stream:
PTB-INFO: For 2 channels Playback: Audio subsystem is ALSA, Audio device name is HDA Intel PCH: ALC3234 Analog (hw:0,0)
PTB-INFO: Real samplerate 44100.000000 Hz. Input latency 0.000000 msecs, Output latency 9.977324 msecs.
---> audio-manager<audioManager#1594CDB34>: Beep | beep
---> taskSequence<taskSequence#1594B8760>: Took 23.6 ms | randomiseTask
---> taskSequence.initialise: Initialised!

---> screenManager: Normal Screen Preferences used.
BitsPlusPlus: Could not find a Bits# config file under [/home/cog5/.Psychtoolbox/BitsSharpConfig.txt]. Assuming a Bits+ device instead of a Bits# is connected.
BitsPlusPlus: Please create a config file under this name if you have a Bits# and want to use it as Bits# instead of as a Bits+.
BitsPlusPlus: The most simple way is to create an empty file. A more robust way is to store the name of the Bits# serial port
BitsPlusPlus: in the first line of the text file, e.g., COM5 [Windows], or /dev/ttyACM0 [Linux] or similar.
---> screenManager: Probing for a Display++...	NO Display++
---> screenManager: Internal processing set to: 8 bits


PTB-INFO: This is Psychtoolbox-3 for GNU/Linux X11, under Matlab 64-Bit (Version 3.0.19 - Build date: Jun 22 2024).
PTB-INFO: OS support status: Linux 6.8.0-44-lowlatency Supported.
PTB-INFO: Type 'PsychtoolboxVersion' for more detailed version information.
PTB-INFO: Most parts of the Psychtoolbox distribution are licensed to you under terms of the MIT License, with
PTB-INFO: some restrictions. See file 'License.txt' in the Psychtoolbox root folder for the exact licensing conditions.

PTB-INFO: For information about paid support, support memberships and other commercial services, please type
PTB-INFO: 'PsychPaidSupportAndServices'.



PTB-INFO: OpenGL-Renderer is AMD :: AMD Radeon RX 6800 (radeonsi, navi21, LLVM 17.0.6, DRM 3.57, 6.8.0-44-lowlatency) :: 4.6 (Compatibility Profile) Mesa 24.0.9-0ubuntu0.1
PTB-INFO: VBL startline = 1080 , VBL Endline = -1
PTB-INFO: Measured monitor refresh interval from VBLsync = 16.666679 ms [59.999954 Hz]. (50 valid samples taken, stddev=0.000591 ms.)
PTB-INFO: Reported monitor refresh interval from operating system = 16.666667 ms [60.000000 Hz].
PTB-INFO: Small deviations between reported values are normal and no reason to worry.
PTB-INFO: Psychtoolbox imaging pipeline starting up for window with requested imagingmode 1025 ...
PTB-INFO: Will use 8 bits per color component framebuffer for stimulus drawing.
PTB-INFO: Will use 8 bits per color component framebuffer for stimulus post-processing (if any).

---> screenManager: Previous OpenGL blending: GL_ONE | GL_ZERO
---> screenManager: OpenGL blending now: GL_SRC_ALPHA | GL_ONE_MINUS_SRC_ALPHA
Compiling all shaders matching SmoothedDiscShader * into a GLSL program.
Building a fragment shader:Reading shader from file /home/cog5/Code/Psychtoolbox-3/Psychtoolbox/PsychOpenGL/PsychGLSLShaders/SmoothedDiscShader.frag.txt ...
Building a vertex shader:Reading shader from file /home/cog5/Code/Psychtoolbox-3/Psychtoolbox/PsychOpenGL/PsychGLSLShaders/SmoothedDiscShader.vert.txt ...

===>>> No strobe output I/O...
===> No reward TTLs will be sent...
Eyelink: Opening Eyelink in DUMMY mode
---> Eyelink<eyelinkManager#15951D0EF>: Running on a Dummy Eyelink @  0 (time offset:  0) | Initialise Method


===>>> User Functions instantiated…

======>>> Loading State File: DefaultStateInfo.m
=================>> Built state info file <<==================
  Columns 1 through 4
    {'name'     }    {'next'     }    {'time'             }    {'entryFcn'}
    {'pause'    }    {'prefix'   }    {[              Inf]}    {11x1 cell }
    {'prefix'   }    {'fixate'   }    {[0.750000000000000]}    { 8x1 cell }
    {'fixate'   }    {'breakfix' }    {[               10]}    { 1x1 cell }
    {'stimulus' }    {'incorrect'}    {[               10]}    { 2x1 cell }
    {'correct'  }    {'prefix'   }    {[0.100000000000000]}    { 3x1 cell }
    {'incorrect'}    {'timeout'  }    {[0.100000000000000]}    { 3x1 cell }
    {'breakfix' }    {'timeout'  }    {[0.100000000000000]}    { 3x1 cell }
    {'timeout'  }    {'prefix'   }    {[                2]}    { 0x0 cell }
    {'calibrate'}    {'pause'    }    {[0.500000000000000]}    { 4x1 cell }
    {'drift'    }    {'pause'    }    {[0.500000000000000]}    { 4x1 cell }
    {'offset'   }    {'pause'    }    {[0.500000000000000]}    { 4x1 cell }
    {'override' }    {'pause'    }    {[0.500000000000000]}    { 1x1 cell }
    {'flash'    }    {'pause'    }    {[0.500000000000000]}    { 1x1 cell }
    {'showgrid' }    {'pause'    }    {[               10]}    { 0x0 cell }
  Columns 5 through 7
    {'withinFcn'}    {'transitionFcn'}    {'exitFcn'}
    {0x0 cell   }    {0x0 cell       }    { 1x1 cell}
    {1x1 cell   }    {0x0 cell       }    { 0x0 cell}
    {3x1 cell   }    {1x1 cell       }    { 3x1 cell}
    {3x1 cell   }    {1x1 cell       }    { 2x1 cell}
    {1x1 cell   }    {0x0 cell       }    {11x1 cell}
    {1x1 cell   }    {0x0 cell       }    {11x1 cell}
    {1x1 cell   }    {0x0 cell       }    {11x1 cell}
    {1x1 cell   }    {0x0 cell       }    { 0x0 cell}
    {0x0 cell   }    {0x0 cell       }    { 0x0 cell}
    {0x0 cell   }    {0x0 cell       }    { 0x0 cell}
    {0x0 cell   }    {0x0 cell       }    { 0x0 cell}
    {0x0 cell   }    {0x0 cell       }    { 0x0 cell}
    {0x0 cell   }    {0x0 cell       }    { 0x0 cell}
    {1x1 cell   }    {0x0 cell       }    { 0x0 cell}
=================>> Built state info file <<=================
---> Path: /home/cog5/OptickaFiles/SavedData/Simulcra/2024-09-13/session001/ created...



===>>>>>> START BEHAVIOURAL TASK: Simulcra-session001-2024-9-13-13-54-16 <<<<<<===	Initial Path: /home/cog5/OptickaFiles/SavedData/Simulcra/2024-09-13/session001/
	Initial Comments: 


===>>> Creating Behavioural Record Plot Window...
===>>> Increasing Priority...
PTB-INFO: Gamemode optimizations enable requested. Current/Old status: Disabled
PTB-INFO: New status: Active

===>>> Warming up the GPU, Eyetracker and I/O systems... <<<===
begin state: stateMachine warmup... ...exit
middle state: stateMachine warmup... ...exit
surprise state: stateMachine warmup... ...exit
end state: stateMachine warmup... ...exit

--->>> Total time to do state traversal: 0.311916 secs 
--->>> Loops: 3001 thus ~0.103937 ms per loop
---> state machine<stateMachine#159580928>: stateMachine not running... | finish method

===>>> Save initial state in case of crash: /tmp//TEMPSimulcra-session001-2024-9-13-13-54-16.mat ...
	 ... Saved!


===>>> Igniting the State Machine... <<<===
PAUSED, press [p] to resume...
---> audio-manager<audioManager#1594CDB34>: Beep | beep
===>>> CORRECT : correct:15966479A | B:1 R:1 [1/9] | V: 3 | Time: 6.433 (267) isFix:1 isExcl:0 isFixInit:0
isBlink:0 fixLength: 1.01 > xyPosition: 10.00 10.00 | updateVariables: B:1 R:1 T:1
V:3>S1:xyPositionOut=10 10 
---> audio-manager<audioManager#1594CDB34>: Beep | beep
===>>> CORRECT : correct:1596E4F72 | B:1 R:2 [2/9] | V: 2 | Time: 10.983 (420) isFix:1 isExcl:0 isFixInit:0
isBlink:0 fixLength: 1.02 > xyPosition: 0.00 0.00 | updateVariables: B:1 R:2 T:2
V:2>S1:xyPositionOut=0 0 
---> audio-manager<audioManager#1594CDB34>: Beep | beep
===>>> CORRECT : correct:15975C98A | B:1 R:3 [3/9] | V: 1 | Time: 15.217 (633) isFix:1 isExcl:0 isFixInit:0
isBlink:0 fixLength: 1.02 > xyPosition: -10.00 -10.00 | updateVariables: B:1 R:3 T:3
V:1>S1:xyPositionOut=-10 -10 
---> audio-manager<audioManager#1594CDB34>: Beep | beep
===>>> CORRECT : correct:1597CB453 | B:2 R:1 [4/9] | V: 3 | Time: 19.133 (836) isFix:1 isExcl:0 isFixInit:0
isBlink:0 fixLength: 1.02 > xyPosition: 10.00 10.00 | updateVariables: B:2 R:1 T:4
V:3>S1:xyPositionOut=10 10 

--->>> Total time to do state traversal: 20.2832 secs 
--->>> Loops: 992 thus ~20.4468 ms per loop


---> screenManager 1594BED38: Closing screen = 1, Win = 10, Kind = 1


INFO: PTB's Screen('Flip', 10) command seems to have missed the requested stimulus presentation deadline
INFO: a total of 5 times out of a total of 949 flips during this session.

INFO: This number is fairly accurate (and indicative of real timing problems in your own code or your system)
INFO: if you provided requested stimulus onset times with the 'when' argument of Screen('Flip', window [, when]);
INFO: If you called Screen('Flip', window); without the 'when' argument, this count is more of a ''mild'' indicator
INFO: of timing behaviour than a hard reliable measurement. Large numbers may indicate problems and should at least
INFO: deserve your closer attention. Cfe. 'help SyncTrouble', the FAQ section at www.psychtoolbox.org and the
INFO: examples in the PDF presentation in PsychDocumentation/Psychtoolbox3-Slides.pdf for more info and timing tips.

PsychEyelinkDispatchCallback: Unknown eyelink command (-1)


======>>> Total ticks: 880 | stateMachine ticks: 992
======>>> Tracker Time: 0 | PTB time: 20.2834 | Drift Offset: 0


#####################
===>>> <strong>SAVED DATA to: /home/cog5/OptickaFiles/SavedData/Simulcra/2024-09-13/session001//opticka.raw.Simulcra-session001-2024-9-13-13-54-16.mat</strong>
#####################

Lets print out a table of the stimulus values for every trial run

showTable(myTask);

Plot a timing log of every frame against the stimulus on/off times

PTB has the most reliable and precise timing control of any experimental control system, and we therefore log every flip time alongside the stimulus transitions. The timing log shows every recorded frame in relation to the stimulus transitions.

showTimingLog(myExp);

Plot a timing log of all states and their function evaluation transition times

The state machine also records the timestamps when states are entered and exited. In addition, it times how long each cell array of functions take to run on enter/within/exit, to check for any potential timing problems (you do not want enter/within states to take too long in case it causes frame drops, this can be seen via these plots).

showLog(myExp.stateMachine);