Demo of a state-machine-driven Opticka Behavioural Experiment.
Opticka is an object-oriented framework for the Psychophysics toolbox (PTB), allowing randomised interleaved presentation of parameter varying stimuli specified in experimenter-relevant values. It operates under Linux (PTB's preferred OS), macOS & Windows, and can interface via strobed words (using Display++, VPixx or a cheap and very reliable LabJack), or TTLs via low-cost Arduino. It uses ethernet to connect with Tobii or Eyelink eyetrackers and with with external harware for recording neurophysiological data.
In this demo, two stimulus objects (wrapped in a myStims manager object), task sequence variables (myTask object), and a screenManager (myScreen object) are passed to the runExperiment class. runExperiment uses the stateMachine class object which loads the experiment specification (for this demo it uses DefaultStateInfo.m)
DefaultStateInfo.m specifies the following experiment with six states (prefix, fixate, stimulus, correct, incorrect, breakfix), and uses the eyetracker (here using the mouse functions to transition from fixate.
┌───────────────────────────────────────┐
│ ▼
│ ┌───────────────────────────┐
┌──┼───────────────────────────────────▶ │ (1) prefix │ ◀┐
│ │ └───────────────────────────┘ │
│ │ │ │
│ │ ▼ │
│┌───────────┐ Transition ┌───────────────────────────┐ │
││ incorrect │ inFixFcn=>incorrect │ fixate │ │
││ │◀───────────────────────── │ show(stims, 2) │ │
│└───────────┘ └───────────────────────────┘ │
│ │ │
│ │ Transition │
└──┐ │ inFixFcn=>stimulus │
│ ▼ │
┌───────────┐ Transition ┌───────────────────────────┐ │
│ CORRECT │ maintainFixFcn=>correct │ stimulus │ │
│ │◀───────────────────────── │ show(stims, [1 2]) │ │
└───────────┘ └───────────────────────────┘ │
│ │
│ Transition │
│ maintainFixFcn=>breakfix │
▼ │
┌───────────────────────────┐ │
│ BREAKFIX │ ─┘
└───────────────────────────┘For this demo a dummy eyetracker object (where the mouse input transparently replaces the eye movements), is used to demonstrate behavioural control of the paradigm.
Opticka also offers an optional GUI (type `opticka` in the command window), which is a visual manager of the objects introduced here. The UI also controls other functions such as screen calibration, protocol loading/saving and managing communication with neurophysiological equipment via LabJack, Arduino and ethernet.
The source of this file can be found at : https://github.com/iandol/opticka/blob/master/optickaBehaviourTest.m
Contents
- Initial clear up of any previous objects
- Setup our visual stimuli
- Task Initialisation
- Setup screenManager Object
- Setup runExperiment Object
- Run the full behavioural task
- Plot a timing log of every frame against the stimulus on/off times
- Plot a timing log of all states and their function evaluation transition times
Initial clear up of any previous objects
Make sure we start in a clean environment, not essential
clear myStims myTask myExp myScreen sca %PTB screen clear all
Setup our visual stimuli
First we use the metaStimulus class to create a stimulus manager object that collects and handles groups of stimuli as if they were a single 'thing', so for example when you use the draw method myStims.draw(), it tells each of its child stimuli to draw in order. You can show and hide each stimulus in the group and only thoses set to draw will do so. Note you can control each stimulus yourself (each stimulus has its own set of control functions), but using metaStimulus makes the code simpler...
myStims = metaStimulus();
first stimulus is a smoothed 5° red disc
myStims{1} = discStimulus('colour', [0.7 0.2 0], 'size', 5, 'sigma', 20);
second stimulus is an optimised 0.8° fixation cross from Thaler et al., 2013
myStims{2} = fixationCrossStimulus('size', 0.8);
Task Initialisation
The taskSequence class defines a stimulus sequence (task) which is composed of randomised stimulus parameter changes (called variables) repeated over a set of blocks. A trial is an individual stimulus presentation. This
For behavioural tasks, several of the parameters like myTask.trialTime are not used as the state machine takes over the job of task timing, but the stimulus randomisation is still useful for such tasks. In this case the state info file can use taskSequence to deterimine the next stimulus value. There are functions to handle what happens if the subject responds incorrectly, where we can re-randomise the next value within the block.
myTask = taskSequence(); %new taskSequence object instance
Our variable is xPosition, applied to stimulus 1 only. 3 different position values so 3 trials per block...
myTask.nVar(1).name = 'xPosition';
myTask.nVar(1).stimulus = 1;
myTask.nVar(1).values = [-10 0 10];
We call the method to randomise the trials into a block (3 blocks, 3 trials) structure.
myTask.nBlocks = 3; randomiseTask(myTask);
---> taskSequence <taskSequence#165864C6E>: Took 38.324 ms | randomiseTask
Setup screenManager Object
screenManager controls the PTB Screen(). We initialise the object with parameters to open the PTB screen with. Note distance and pixels per cm define the resultant geometry > pixel mappings. You can set several screen parameters, windowing, blending etc.
myScreen = screenManager('distance', 57.3,... % display distance from observer 'pixelsPerCm', 27,... % calibration value for pixel density, measure using calibrateSize() 'windowed', [],... % use fullscreen [] or window [X Y]? 'backgroundColour', [0.5 0.5 0.5],... % initial background colour 'blend', true,... % enable OpenGL blending, you can also set blend modes when needed 'bitDepth', '8bit'); % FloatingPoint32bit, 8bit, FloatingPoint16bit etc. % use retina mode for macOS if ismac; myScreen.useRetina = true; end
Setup runExperiment Object
We now pass our stimulus, screen and task objects to the runExperiment class. runExperiment contains the runTask() method that actually runs the behavioural task.
Importantly, the stateInfoFile 'FixationTrainingStateInfo.m' determines the behavioural protocol that the state machine will run. The state machine is available as myExp.stateMachine. Read through that StateInfo file to better understand what is being done. stateInfoFiles contain some general configuration, then a set of cell-arrays of functions, and a table of states which run these function arrays as the states are entered and exited. For this simple task, it starts in a paused state, then transitions to a blank period, then the stimulus presentation, where initiating and maintaining fixation on the cross leads to a correct state or breaking fixation leads to breakFix state, then the state machine loops back to blank etc. using the myTask object to set variable values on each trial.
myExp = runExperiment('stimuli', myStims,... %stimulus objects 'screen', myScreen,... %screen manager object 'task', myTask,... % task randomised stimulus sequence 'stateInfoFile', 'DefaultStateInfo.m', ... % use the default state info file 'debug', false,... % debug mode for testing? 'useEyeLink', true, ... % use the eyelink manager 'dummyMode', true, ... % use dummy mode so the mouse replaces eye movements for testing 'logStateTimers',true,... 'subjectName', 'Simulcra', ... 'researcherName', 'Joanna Doe');
---> Run Experiment <runExperiment#16586CE66>: Propery invalid! | dummyMode
Run the full behavioural task
runTask(myExp);
--->arduinoManager: Ports available: /dev/ttyS0
PTB-INFO: Using PortAudio V19.7.0-devel, revision 147dd722548358763a8b649b3e4b41dfffbcfbb6
PTB-INFO: Using PortAudio V19.7.0-devel, revision 147dd722548358763a8b649b3e4b41dfffbcfbb6
---> audio-manager <audioManager#16586FC4F>: Audio Manager initialisation complete | constructor
PTB-INFO: Using PortAudio V19.7.0-devel, revision 147dd722548358763a8b649b3e4b41dfffbcfbb6
PTB-INFO: Choosing deviceIndex 0 [HDA Intel PCH: ALC3234 Analog (hw:0,0)] as default output audio device.
PTB-INFO: New audio device -1 with handle 0 opened as PortAudio stream:
PTB-INFO: For 2 channels Playback: Audio subsystem is ALSA, Audio device name is HDA Intel PCH: ALC3234 Analog (hw:0,0)
PTB-INFO: Real samplerate 44100.000000 Hz. Input latency 0.000000 msecs, Output latency 9.977324 msecs.
===>>>>>> Start task: Simulcra-2023-3-31-14-23-45 <<<<<<===
---> taskSequence <taskSequence#165864C6E>: Took 37.022 ms | randomiseTask
---> taskSequence.initialise: Initialised!
---> screenManager: Normal Screen Preferences used.
PsychVulkanCore-ERROR: Vulkan instance creation failed: -1
BitsPlusPlus: Could not find a Bits# config file under [/home/cog5/.Psychtoolbox/BitsSharpConfig.txt]. Assuming a Bits+ device instead of a Bits# is connected.
BitsPlusPlus: Please create a config file under this name if you have a Bits# and want to use it as Bits# instead of as a Bits+.
BitsPlusPlus: The most simple way is to create an empty file. A more robust way is to store the name of the Bits# serial port
BitsPlusPlus: in the first line of the text file, e.g., COM5 [Windows], or /dev/ttyACM0 [Linux] or similar.
---> screenManager: Probing for a Display++... NO Display++
---> screenManager: Internal processing set to: 8 bits
PTB-INFO: This is Psychtoolbox-3 for GNU/Linux X11, under Matlab 64-Bit (Version 3.0.19 - Build date: Feb 19 2023).
PTB-INFO: OS support status: Linux 6.2.7-060207-generic Supported.
PTB-INFO: Type 'PsychtoolboxVersion' for more detailed version information.
PTB-INFO: Most parts of the Psychtoolbox distribution are licensed to you under terms of the MIT License, with
PTB-INFO: some restrictions. See file 'License.txt' in the Psychtoolbox root folder for the exact licensing conditions.
PTB-INFO: For information about paid support, support memberships and other commercial services, please type
PTB-INFO: 'PsychPaidSupportAndServices'.
PTB-INFO: Connected to NVidia TU104 [GeForce RTX 2070 SUPER] GPU of NV-160 family with 4 display heads.
PTB-INFO: OpenGL-Renderer is NVIDIA Corporation :: NVIDIA GeForce RTX 2070 SUPER/PCIe/SSE2 :: 4.6.0 NVIDIA 525.89.02
PTB-INFO: VBL startline = 1080 , VBL Endline = 1143
PTB-INFO: Measured monitor refresh interval from beamposition = 8.334588 ms [119.981935 Hz].
PTB-INFO: Measured monitor refresh interval from VBLsync = 8.334203 ms [119.987481 Hz]. (50 valid samples taken, stddev=0.032203 ms.)
PTB-INFO: Reported monitor refresh interval from operating system = 8.334653 ms [119.981003 Hz].
PTB-INFO: Small deviations between reported values are normal and no reason to worry.
PTB-INFO: Psychtoolbox imaging pipeline starting up for window with requested imagingmode 1025 ...
PTB-INFO: Will use 8 bits per color component framebuffer for stimulus drawing.
PTB-INFO: Will use 8 bits per color component framebuffer for stimulus post-processing (if any).
---> screenManager: Previous OpenGL blending: GL_ONE | GL_ZERO
---> screenManager: OpenGL blending now: GL_SRC_ALPHA | GL_ONE_MINUS_SRC_ALPHA
Compiling all shaders matching SmoothedDiscShader * into a GLSL program.
Building a fragment shader:Reading shader from file /home/cog5/Code/Psychtoolbox-3/Psychtoolbox/PsychOpenGL/PsychGLSLShaders/SmoothedDiscShader.frag.txt ...
Building a vertex shader:Reading shader from file /home/cog5/Code/Psychtoolbox-3/Psychtoolbox/PsychOpenGL/PsychGLSLShaders/SmoothedDiscShader.vert.txt ...
===>>> No strobe output I/O...
===> No reward TTLs will be sent...
--> dataConnection: Trying to close local connection... | close Method
--> dataConnection: Cleaning up now... | dataConnection delete Method
--> dataConnection: Trying to close local connection... | close Method
===>>> User Functions instantiated…
======>>> Loading State File: /home/cog5/Code/opticka/DefaultStateInfo.m
=================>> Built state info file <<==================
Columns 1 through 4
{'name' } {'next' } {'time' } {'entryFcn'}
{'pause' } {'prefix' } {[ Inf]} {12x1 cell }
{'prefix' } {'fixate' } {[0.5000]} { 8x1 cell }
{'fixate' } {'incorrect'} {[ 10]} { 2x1 cell }
{'stimulus' } {'incorrect'} {[ 10]} { 2x1 cell }
{'incorrect'} {'timeout' } {[0.5000]} { 9x1 cell }
{'breakfix' } {'timeout' } {[0.5000]} {10x1 cell }
{'correct' } {'prefix' } {[0.5000]} {11x1 cell }
{'timeout' } {'prefix' } {[ 2]} { 0x0 cell }
{'calibrate'} {'pause' } {[0.5000]} { 4x1 cell }
{'drift' } {'pause' } {[0.5000]} { 4x1 cell }
{'offset' } {'pause' } {[0.5000]} { 4x1 cell }
{'override' } {'pause' } {[0.5000]} { 1x1 cell }
{'flash' } {'pause' } {[0.5000]} { 1x1 cell }
{'showgrid' } {'pause' } {[ 10]} { 0x0 cell }
Columns 5 through 7
{'withinFcn'} {'transitionFcn'} {'exitFcn'}
{0x0 cell } {0x0 cell } { 1x1 cell}
{1x1 cell } {0x0 cell } { 0x0 cell}
{2x1 cell } {1x1 cell } { 4x1 cell}
{3x1 cell } {1x1 cell } { 1x1 cell}
{1x1 cell } {0x0 cell } {10x1 cell}
{1x1 cell } {0x0 cell } {10x1 cell}
{1x1 cell } {0x0 cell } {12x1 cell}
{1x1 cell } {0x0 cell } { 0x0 cell}
{0x0 cell } {0x0 cell } { 0x0 cell}
{0x0 cell } {0x0 cell } { 0x0 cell}
{0x0 cell } {0x0 cell } { 0x0 cell}
{0x0 cell } {0x0 cell } { 0x0 cell}
{0x0 cell } {0x0 cell } { 0x0 cell}
{1x1 cell } {0x0 cell } { 0x0 cell}
=================>> Built state info file <<=================
===>>> Dummy eyelink being initialised...
Eyelink: Opening Eyelink in DUMMY mode
---> Eyelink <eyelinkManager#16588F575>: Running on a Dummy Eyelink @ 0 (time offset: 0) | Initialise Method
===>>> Warming up the GPU, Eyetracker and I/O systems... <<<===
begin state: stateMachine warmup... ...exit
middle state: stateMachine warmup... ...exit
surprise state: stateMachine warmup... ...exit
end state: stateMachine warmup... ...exit
--->>> Total time to do state traversal: 0.303071 secs
--->>> Loops: 298 thus 1.01702 ms per loop
---> state machine <stateMachine#1658C7860>: stateMachine not running... | finish method
===>>> Save initial state: /tmp//Simulcra-2023-3-31-14-23-45.mat ...
... Saved!
===>>> Increasing Priority...
===>>> Igniting the State Machine... <<<===
PAUSED, press [p] to resume...
PsychHID-WARNING: Failed to setup international keyboard handling due to failed input method creation.
PsychHID-WARNING: Only US keyboard layouts will be mapped properly due to this failure for GetChar() et al.
===>INITFIX:fixate:165AF05E3 | B:1 R:1 [1/9] | V: 2 | Time: 16.544 (57) > xPosition: 0.000 | updateVariables:
B:1 R:1 T:1 V:2>S1:xPositionOut=0
===>CORRECT:correct:165B39A93 | B:1 R:1 [1/9] | V: 2 | Time: 19.145 (366) > xPosition: 0.000 |
updateVariables: B:1 R:1 T:1 V:2>S1:xPositionOut=0
===>INITFIX:fixate:165B56DB3 | B:1 R:2 [2/9] | V: 1 | Time: 20.178 (487) > xPosition: -10.000 |
updateVariables: B:1 R:2 T:2 V:1>S1:xPositionOut=-10
===>CORRECT:correct:165B92ABD | B:1 R:2 [2/9] | V: 1 | Time: 22.295 (741) > xPosition: -10.000 |
updateVariables: B:1 R:2 T:2 V:1>S1:xPositionOut=-10
===>INITFIX:fixate:165BAFDC6 | B:1 R:3 [3/9] | V: 3 | Time: 23.329 (863) > xPosition: 10.000 |
updateVariables: B:1 R:3 T:3 V:3>S1:xPositionOut=10
===>CORRECT:correct:165BEE84E | B:1 R:3 [3/9] | V: 3 | Time: 25.546 (1128) > xPosition: 10.000 |
updateVariables: B:1 R:3 T:3 V:3>S1:xPositionOut=10
===>INITFIX:fixate:165C0B794 | B:2 R:1 [4/9] | V: 2 | Time: 26.571 (1249) > xPosition: 0.000 |
updateVariables: B:2 R:1 T:4 V:2>S1:xPositionOut=0
===>CORRECT:correct:165C474D1 | B:2 R:1 [4/9] | V: 2 | Time: 28.688 (1503) > xPosition: 0.000 |
updateVariables: B:2 R:1 T:4 V:2>S1:xPositionOut=0
===>INITFIX:fixate:165C64441 | B:2 R:2 [5/9] | V: 1 | Time: 29.713 (1624) > xPosition: -10.000 |
updateVariables: B:2 R:2 T:5 V:1>S1:xPositionOut=-10
===>CORRECT:correct:165CA7673 | B:2 R:2 [5/9] | V: 1 | Time: 32.088 (1909) > xPosition: -10.000 |
updateVariables: B:2 R:2 T:5 V:1>S1:xPositionOut=-10
===>INITFIX:fixate:165CC495A | B:2 R:3 [6/9] | V: 3 | Time: 33.122 (2031) > xPosition: 10.000 |
updateVariables: B:2 R:3 T:6 V:3>S1:xPositionOut=10
===>CORRECT:correct:165D042CE | B:2 R:3 [6/9] | V: 3 | Time: 35.372 (2301) > xPosition: 10.000 |
updateVariables: B:2 R:3 T:6 V:3>S1:xPositionOut=10
===>INITFIX:fixate:165D20E7C | B:3 R:1 [7/9] | V: 2 | Time: 36.389 (2423) > xPosition: 0.000 |
updateVariables: B:3 R:1 T:7 V:2>S1:xPositionOut=0
===>CORRECT:correct:165D607E3 | B:3 R:1 [7/9] | V: 2 | Time: 38.639 (2693) > xPosition: 0.000 |
updateVariables: B:3 R:1 T:7 V:2>S1:xPositionOut=0
===>INITFIX:fixate:165D7D3A2 | B:3 R:2 [8/9] | V: 1 | Time: 39.656 (2815) > xPosition: -10.000 |
updateVariables: B:3 R:2 T:8 V:1>S1:xPositionOut=-10
===>CORRECT:correct:165DB8570 | B:3 R:2 [8/9] | V: 1 | Time: 41.748 (3066) > xPosition: -10.000 |
updateVariables: B:3 R:2 T:8 V:1>S1:xPositionOut=-10
===>INITFIX:fixate:165DD5127 | B:3 R:3 [9/9] | V: 3 | Time: 42.765 (3188) > xPosition: 10.000 |
updateVariables: B:3 R:3 T:9 V:3>S1:xPositionOut=10
===>CORRECT:correct:165E18319 | B:3 R:3 [9/9] | V: 3 | Time: 45.140 (3473) > xPosition: 10.000 |
updateVariables: B:3 R:3 T:9 V:3>S1:xPositionOut=10
---> taskSequence.updateTask: Task FINISHED, no more updates allowed
---> screenManager 16586A67A: Closing screen = 1, Win = 10, Kind = 1
INFO: PTB's Screen('Flip', 10) command seems to have missed the requested stimulus presentation deadline
INFO: a total of 10 times out of a total of 3665 flips during this session.
INFO: This number is fairly accurate (and indicative of real timing problems in your own code or your system)
INFO: if you provided requested stimulus onset times with the 'when' argument of Screen('Flip', window [, when]);
INFO: If you called Screen('Flip', window); without the 'when' argument, this count is more of a ''mild'' indicator
INFO: of timing behaviour than a hard reliable measurement. Large numbers may indicate problems and should at least
INFO: deserve your closer attention. Cfe. 'help SyncTrouble', the FAQ section at www.psychtoolbox.org and the
INFO: examples in the PDF presentation in PsychDocumentation/Psychtoolbox3-Slides.pdf for more info and timing tips.
--> dataConnection: Trying to close local connection... | close Method
======>>> Total ticks: 3536 | stateMachine ticks: 5391
======>>> Tracker Time: 0 | PTB time: 45.657 | Drift Offset: 0
Lets print out a table of the stimulus values for every trial run
showTable(myTask);
Plot a timing log of every frame against the stimulus on/off times
PTB has the most reliable and precise timing control of any experimental control system, and we therefore log every flip time alongside the stimulus transitions. The timing log shows every recorded frame in relation to the stimulus transitions.
showTimingLog(myExp);
Plot a timing log of all states and their function evaluation transition times
The state machine also records the timestamps when states are entered and exited. In addition, it times how long each cell array of functions take to run on enter/within/exit, to check for any potential timing problems (you do not want enter/within states to take too long in case it causes frame drops, this can be seen via these plots).
showLog(myExp.stateMachine);
