Tutorial 1: ERP Experiment

Corresponding file for this tutorial: TMSiPlugins/PsychoPy/experiment_psychopy_erp_experiment_apex.py

An ERP experiment consists of two phases which are data collection (stimulus presentation occurs at this phase) and data analysis. This tutorial covers the first phase only. The second phase is explained in Tutorial ERP Analysis.

This tutorial is a step-by-step guide on running the example_psychopy_erp_experiment_apex.py located inside the TMSiPlugins/PsychoPy folder. This example contains all you need to run an auditory oddball experiment in Psychopy using TMSi APEX amplifier. Before reading this tutorial it’s highly recommended to have a basic knowledge about electroencephalography (EEG), TMSi Python interface, and TMSi APEX amplifier. More information about Event Related Potentials (ERPs) can be found on TMSi’s website. Before getting started, please ensure that you have Python 3.8 installed on your PC as this is a requirement of PsychoPy. Furthermore, please ensure you have a USB-TTL module (REF 95-8650-0001-0) available.

Adding the correct directory of files

To use the TMSi Python interface folder, which is necessary for running the examples, your Python interpreter needs to know the corresponding path. Therefore, the first lines of code add this information. If you wish to store the recordings in a different folder than the default one, you can change the directory specified in the highlighted code line 38 below.

import sys
from os.path import join, dirname, realpath
Plugin_dir = dirname(realpath(__file__)) # directory of this file
modules_dir = join(Plugin_dir, '..', '..') # directory with all modules
measurements_dir = join(Plugin_dir, '../../measurements') # directory with all measurements
configs_dir = join(Plugin_dir, '../../TMSiSDK\\tmsi_resources') # directory with configurations
sys.path.append(modules_dir)

Importing the required classes and functions

The next lines import all the libraries which are required for the example script to work. Line 43 is needed so that PsychoPy uses the correct graphics library. They are used for purposes such as handling errors, writing recording files on PC, interacting with the USB-TTL module, and using the PsychoPy package.

import os
os.environ["PYQTGRAPH_QT_LIB"] = 'PySide2'

from TMSiSDK.tmsi_sdk import TMSiSDK, DeviceType, DeviceInterfaceType, DeviceState
from TMSiSDK.tmsi_errors.error import TMSiError

from PySide2.QtWidgets import *
from TMSiGui.gui import Gui
from TMSiPlotterHelpers.impedance_plotter_helper import ImpedancePlotterHelper
from TMSiPlotterHelpers.signal_plotter_helper import SignalPlotterHelper
from TMSiFileFormats.file_writer import FileWriter, FileFormat
from TMSiPlugins.PsychoPy.experiment_psychopy import PsychopyExperimentSetup
from TMSiPlugins.PsychoPy.erp_training_routine import PsychopyTrainingSetup
from TMSiPlugins.external_devices.usb_ttl_device import TTLError
from TMSiSDK.device.tmsi_device_enums import MeasurementType
import time
from threading import Thread
import easygui 
from PIL import Image

Defining a function to set the background image

Using an auditory oddball paradigm, it’s common to display an image on the screen so that the participant remains focused. This section of the example script defines a function which gets the directory of an image as an input (line 62) and then shows that image on the screen. This function is defined at the beginning as it will be called in several steps of the experiment.

def display_background_image(image_path):
    # Open and display the image using PIL
    image = Image.open(image_path)
    image.show()
    easygui.msgbox("Click 'OK' to continue once you see the image.")

Checking the sound level

As sound intensity is important for auditory oddball paradigms to be reproducible, it should be set before starting the experiment. Therefore, this section of the script is implemented to check if the sound level of user’s PC is set to the desired volume (in this case 40 as it corresponds to 60 dB, please check the paradigm settings in the P300 blog).

This is done by displaying a dialogue box. Please note that the displayed message and the text shown on both buttons can be changed (line 71). The same block of code can be used if more user confirmations need to be defined during the experiment.

# Check whether the sound level on laptop is set to 40 ~ corresponding to 60 dB
while True:
    sound_choice = easygui.buttonbox("Is the laptop sound turned on at volume 40?", choices=["Yes","No"])
    
    if sound_choice == "Yes":
        easygui.msgbox("Well done, you can continue")
        break
    elif sound_choice == "No":
        easygui.msgbox("Adjust the volume please")
        continue
    else:
        break        

Finding the paired device for Bluetooth connection

As it’s possible to have more than one device connected, it’s necessary to specify the type of connected device as well as the interface via which the device is connected to the PC (line 84). The interface type for APEX can be set to either usb or bluetooth (line 84). When choosing bluetooth, it’s required to assign a variable (dongle) to the Bluetooth dongle that is connected to the PC (line 85). Next, a list of connected devices is stored inside the variable discoveryList.

The next line of code (line 89) is a check to ensure that at least one device has been discovered. When one or multiple devices are found, the device which is paired to the Bluetooth dongle is selected. The handle to this discovered device is stored in the variable dev and a connection to the device is opened (line 97). The dongle ID is used to ensure that the connection is made to the device which is paired to that specific dongle.

    # Execute a device discovery. This returns a list of device-objects for every discovered device.
    TMSiSDK().discover(DeviceType.apex, DeviceInterfaceType.bluetooth)
    dongle = TMSiSDK().get_dongle_list(DeviceType.apex)[0]
    discoveryList = TMSiSDK().get_device_list(DeviceType.apex)

    # Set up device
    if (len(discoveryList) > 0):
        # Get the handle to the paired device.
        for i, dev_i in enumerate(discoveryList):
            if dev_i.get_dongle_serial_number() == dongle.get_serial_number():
                dev_id = i   
        dev = discoveryList[dev_id]
        
        # Open a connection to the APEX-system
        dev.open(dongle_id = dongle.get_id())

Loading desired device configuration

The pre-defined configuration files are located inside the configs folder of the TMSi Python interface (TMSiSDK/tmsi_resources). It is also possible to create custom configurations and save these as .xml files. In this example, a 32-channel configuration with EEG labeling is loaded to the device (line 101). Obviously, other configuration files could be imported given the corresponding directory.

        # Load the EEG channel set and configuration (in this case for 32 channels)
        print("load EEG config")
        dev.import_configuration(join(configs_dir, 'APEX_config_EEG32.xml'))

Initializing PsychoPy experiment

The first two lines of this section are meant to give feedback to the end-user. Next, settings used for the experiment are defined. These settings include number of trials, interval between stimulus presentation, tone duration, and probability of target stimulus occurrence. Users can set these values based on the desired auditory paradigm. This part of the code is where the experiment can be tailored to the user’s liking (assuming an auditory odd-ball experiment is desired).

        # Initialize PsychoPy Experiment, for arguments description see class
        print('\n Initializing PsychoPy experiment and TTL module \n')
        print('\n  Please check if red and green LEDs are turned on ... \n')

        # Experiment settings
        n_trials = 60
        interval = 1.5 
        duration = 0.05
        probability = 0.2

Once the settings are defined, they will need to be passed on to the PsychopyExperimentSetup class that is created by TMSi to set up the experiment properties needed to use the PsychoPy library and to initialize the trigger setup for an oddball experiment.

Please note, when APEX is used available input options for both target and non-target values are only even numbers between 2 and 30. These numbers are assigned to ensure that the user can distinguish target and non-target values when looking at the STATUS channel of APEX.

It’s important to specify the correct COM_port on which the USB-TTL module is to be found. This should be checked prior to the experiment, which can be done via Windows’ Device manager - Ports (COM & LPT).

        experiment = PsychopyExperimentSetup(TMSiDevice="APEX", COM_port = 'COM5', n_trials = n_trials, target_value = 16, nontarget_value= 2,
                                             interval = interval, probability = probability, duration = duration)

Initializing impedance plotter

In most EEG measurements, it is common to monitor impedances prior to signal acquisition. Therefore, once the device configuration is set, a real-time plotter is initialized. As shown, the required graphical user interface (GUI) and plotter helper were initially imported from TMSiGui and TMSiPlotterHelpers. The plotter helper defines which plotter is shown by the GUI (for more information about the plotters, please read the Tutorial Plotters). In this case, the ImpedancePlotterHelper is initialized in line 126. Three parameters can be specified for the impedance plotter. The first parameter is the device that is passed to the Helper. The second variable, the layout variable, defines if the electrode impedances should be visualized as a head (for EEG) or in a Grid form (for HD-EMG). Finally, the impedances can be stored in a file: the third parameter defines the location and name of the file.

        # Check if there is already a plotter application in existence
        app = QApplication.instance()
        
        # Initialise the plotter application if there is no other plotter application
        if not app:
            app = QApplication(sys.argv)
        
        # Initialise the helper
        plotter_helper = ImpedancePlotterHelper(device=dev,
                                                is_head_layout=True, 
                                                file_storage = join(measurements_dir,"Example_PsychoPy_ERP_experiment"))
        # Define the GUI object and show it 
        gui = Gui(plotter_helper = plotter_helper)
        # Enter the event loop
        app.exec_()

Once the impedance plotter is closed by the user, a training round of the ERP experiment will start.

Start the training round

To familiarize the participant with the oddball paradigm, a training round is designed which can be run more than one time if required. In the beginning, line 138 defines the directory that contains the background image (in this case a black cross). Next, a while loop is defined so that the participant can practice the experiment as long as he/she clicks on yes on the dialogue boxes shown. During this practice session, no data acquisition takes place.

Please note that the display_background_image function which was defined earlier is used in this section (line 147).

        # Set up background image 
        background_image_path = join(Plugin_dir, 'psychopy_resources', 'cross.png')

        while True:
            # are you ready for the training?
            ready_choice = easygui.buttonbox("Are you ready for the training?", choices=["Yes", "No"])
            
            if ready_choice == "Yes":
                
                # Display the background image
                display_background_image(background_image_path)
        
                # Create an instance of the PsychopyTrainingSetup class with the background image
                training_setup = PsychopyTrainingSetup(background_image_path)
                
                # Run the training
                training_setup.runTraining()
            
                # Ask the participant if they want to do it again or continue
                choice = easygui.buttonbox("Do you want to do the training again?", choices=["Yes", "No"])

                if choice == "No":
                    break  # Exit the loop if the participant chooses "No"
            elif ready_choice == "No":
                continue
            else:
                break

Running the auditory oddball experiment

Before running the experiment, a file writer ensures that the acquired data will be saved to disk. Here, it is also possible to specify the directory in which the recordings are stored. The default is set to the measurement_dir (defined earlier) located inside the TMSi Python interface folder (line 166). Next a while loop is defined to check whether the participant is ready to start the experiment. The dialogue boxes which appear on the screen are self-explanatory.

        # Initialise a file-writer class (Xdf-format) and state its file path
        file_writer = FileWriter(FileFormat.xdf, join(measurements_dir,"Example_PsychoPy_ERP_experiment.xdf"))
        
        # Check to see if the participant is ready for the experiment
        while True:
            exp_choice = easygui.buttonbox("Are you ready for the experiment?", choices=["Yes","No"])
            
            if exp_choice == "Yes":
                easygui.msgbox("Alright, the experiment will start in a bit. \n\nPlease make the image fullscreen after clicking 'OK' ")
                break
            elif exp_choice == "No":
                easygui.msgbox("Alright take your time")
                continue
            else:
                break   

It is required for the file_writer to get the handle to the device specified earlier in the discoveryList (line 182). Further, a thread (a separate flow of execution) is defined to run the oddball experiment. As soon as the measurement starts on APEX (line 188), the experiment will be started (line 191). The experiment runs until a certain time that is given according to the defined settings (line 194). Please note, some additional time (in this example 10 seconds) is added at the end of the experiment.

No pause is included in this experiment. Therefore, it is advised to keep the number of trials limited or to edit the code to include such a pause.

Finally, the measurement will be stopped, the file_writer will be closed, and the connection to APEX will be closed too. The data will be saved to the measurements directory which is specified in the very beginning.

        # Define the handle to the device
        file_writer.open(dev)
        
        # Define thread to run the experiment
        thread = Thread(target=experiment.runExperiment)
        
        # Start a measurement on APEX
        dev.start_measurement(MeasurementType.APEX_SIGNAL)       
        
        # Start the PsychoPy thread
        thread.start()
        
        # Acquisition time, based on experiment timing (with some additional time at the end)
        time.sleep(n_trials * (interval + duration) + 10)
        
        # Stop the measurement on APEX
        dev.stop_measurement()

        # Close the file writer after GUI termination
        file_writer.close()
        
        # Close the connection to APEX
        dev.close()

To learn how to analyze your ERP data, please refer to the tutorial for example_erp_analysis.py