workshop / neurotecheu / realtime /
Realtime EEG data processing
Unicorn Hybrid Black
The Unicorn Hybrid Black is a good-quality and affordable 8-channel EEG device that is very suited for rapid application development during teaching and hackathons. It comes with its own software (only for Windows) that helps you get started quickly visualizing the EEG signals, and streaming them over LSL or UDP. The Unicorn Recorder application also includes the OSCAR algorithm for real-time data cleaning, which allows both the raw and cleaned data to be streamed.
On the Unicorn Suite Hybrid Black GitHub repository you can find more documentation and instruction videos.
The low-level serial-over-bluetooth Unicorn interface protocol is public, allowing for alternative implementations that are independent from the gtec software. This page for example describes a native Python interface that also allows streaming data to LSL. This page contains a C/C++ applications for interfacing the Unicorn with a (virtual) audio device, allowing it to be used in Audacity, Ableton Live or TouchDesigner.
If you don’t have a windows computer, please pair up during the hackathon with someone that does. The whole experience with the Unicorn Recorder application is just way better, you will get up and running much faster, and the realtime OSCAR cleaning algorithm is very useful.
If you read this at a later moment and if you do have your own Unicorn but no Windows laptop, consider buying a refurbished one.
Install the Unicorn software
Please download the installer from the gtec website and install it.
After installation, you have to pair it with the Unicorn device over Bluetooth. There is a recommended USB Bluetooth dongle, but in my experience it works well enough with the builtin bluetooth of various laptops.
Pair device over bluetooth
To pair your laptop with the Unicorn EEG system, you switch the Unicorn on by pressing the button a few seconds until it blinks, and then wait for it to appear in the right column. Subsequently, you click on it in the right column to pair it. A generic Windows pop-up will appear and you have to confirm. The Unicorn will then show up in the middle column.
Install applications
Besides managing the hardware and licenses (more on that below), the Unicorn Suite Hybrid Black software includes a number of apps that you have to install separately. The relevant ones to install are the recorder and the bandpower app.
The recorder app allows visualization of the data without or with filters, implements OSCAR for realtime denoising, and allows data to be sent over LSL.
The bandpower app computes spectral power in specific bands, like alpha, beta, etc.
There are also a number of complete BCI applications that you can try, like the puzzle game which implements some sort of a visual steady-state evoked potential (SSVEP) interface.
Explore APIs
The Unicorn Suite Hybrid Black also includes a number of development tools and application programming interfaces (APIs). If you go through them and download and install them on your computer, you will see that a few of them are free to use, whereas others are licensed. The C/C++ and Net API are free, but the Python and Simulink interfaces require you to get a (paid) license.
Luckily these APIs are not needed for this hackathon, since we will be using the Lab Streaming Layer (LSL) interface.
There is also a separate “UnicornLSL.exe” application that you can use (see below), but even that is not needed. The Unicorn Recorder application mentioned above can be used to stream the data over LSL and UDP.
Download and install FieldTrip
This requires that you already have MATLAB installed. Follow the instructions in the offline section to download and install FieldTrip, and to add it properly to your path.
Download and install Python
Follow online instructions elsewhere on the internet to install Python on your computer. Please ensure that you are able to create virtual environments for installing specific software (and their dependencies), either using Anaconda or Miniconda, or using virtualenv.
Using LSL with MATLAB
Follow the installation instructions on the LSL website or documentation.
Transmitting markers
% instantiate the library
disp('Loading library...');
lib = lsl_loadlib();
disp(lsl_library_version(lib));
nchan = 1;
fsample = 0; % required for irregular streams
type = 'cf_string'; % can also be cf_int32 or cf_float32
% make a new stream outlet
disp('Creating a new streaminfo...');
info = lsl_streaminfo(lib, 'My marker stream', 'Marker', nchan, fsample, type, 'yourname');
disp('Opening an outlet...');
outlet = lsl_outlet(info);
% send markers into the outlet
disp('Now transmitting markers...');
% markers = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
markers = {'1', '2', '3', '4', '5', '6', '7', '8', '9', '10'};
% markers = {'left_button', 'right_button', 'visual_stimulus', 'auditory_stimulus'};
while true
% pick a random marker
thismarker = markers{randi(length(markers))};
fprintf('Now pushing %s\n', thismarker);
outlet.push_sample({thismarker}); % note that the string is wrapped into a cell-array
pause(rand()*2);
end
Transmitting EEG data
% instantiate the library
disp('Loading library...');
lib = lsl_loadlib();
disp(lsl_library_version(lib));
nchan = 8;
fsample = 250;
type = 'cf_float32'; % can also be cf_int32 or cf_string
blocksize = 25; % samples per block
% make a new stream outlet
disp('Creating a new streaminfo...');
info = lsl_streaminfo(lib, 'My EEG stream', 'EEG', nchan, fsample, type, 'yourname');
disp('Opening an outlet...');
outlet = lsl_outlet(info);
% send data into the outlet
disp('Now transmitting data...');
while true
block = round(randn(nchan, blocksize)*100)/100;;
outlet.push_sample(block);
str = sprintf('%f, ', block(:,1)); % show only the first column
fprintf('Pushed data [[%s], ...]\n', str(1:end-2)); % minus the last space and comma
pause(blocksize/fsample);
end
Receiving markers
% Instantiate the library
disp('Loading library...');
lib = lsl_loadlib();
disp(lsl_library_version(lib));
% Look for marker streams on the lab network
fprintf('Looking for marker streams...\n');
streams = lsl_resolve_byprop(lib, 'type', 'Markers');
if isempty(streams)
error('There are no marker streams\n');
else
% Create a new inlet to read from the first stream
fprintf('Found stream: %s\n', streams{1}.name());
inlet = lsl_inlet(streams{1});
end
% Continuously read incoming markers
while true
% Pull a sample (non-blocking)
[sample, timestamp] = inlet.pull_sample(0);
if ~isempty(sample)
for i=1:length(sample)
fprintf('Received marker %s\n', sample{i});
end
end
% Avoid excessive CPU usage while polling
pause(0.001);
end % while
Receiving EEG data
% Instantiate the library
disp('Loading library...');
lib = lsl_loadlib();
disp(lsl_library_version(lib));
% Look for marker streams on the lab network
fprintf('Looking for EEG data streams...\n');
streams = lsl_resolve_byprop(lib, 'type', 'EEG');
if isempty(streams)
error('There are no EEG data streams\n');
else
% Create a new inlet to read from the first stream
fprintf('Found stream: %s\n', streams{1}.name());
inlet = lsl_inlet(streams{1});
end
% Continuously read incoming samples
while true
% Pull a sample (non-blocking)
[sample, timestamp] = inlet.pull_sample();
if ~isempty(sample)
str = sprintf('%f, ', sample);
fprintf('Received data [%s]\n', str(1:end-2)); % minus the last space and comma
end
% Avoid excessive CPU usage while polling
pause(0.001);
end % while
Using LSL with Python
Follow the installation instructions on the LSL website or documentation.
Transmitting markers
import pylsl
import time
import random
# on my M3 MacBook I had to recompile the LSL library and then this is needed
# export DYLD_LIBRARY_PATH=/Users/roboos/matlab/liblsl-maca64/bin
time.sleep(3) # wait a bit to allow LSL to synchronize
info = pylsl.StreamInfo(
name='My marker stream',
type='Marker',
channel_count=1,
nominal_srate=0, # should be zero for markers
channel_format=pylsl.cf_string,
source_id='',
)
# Create a new outlet
outlet = pylsl.StreamOutlet(info)
# Print the stream info
print("Now publishing stream:", info.name(), info.type(), info.channel_count(), "channels at", info.nominal_srate(), "Hz")
# markers = ['left_button', 'right_button', 'visual_stimulus', 'auditory_stimulus']
markers = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']
print("now sending markers...")
while True:
# pick a sample to send an wait for a bit
thismarker = [random.choice(markers)]
print("Pushing marker:", thismarker)
outlet.push_sample(thismarker)
time.sleep(random.random()*2)
Transmitting EEG data
import pylsl
import time
import random
# on my M3 MacBook I had to recompile the LSL library and then this is needed
# export DYLD_LIBRARY_PATH=/Users/roboos/matlab/liblsl-maca64/bin
time.sleep(3) # wait a bit to allow LSL to synchronize
nchan = 8
fsample = 250
blocksize = 25
info = pylsl.StreamInfo(
name='My EEG stream',
type='EEG',
channel_count=nchan,
nominal_srate=fsample,
channel_format=pylsl.cf_float32,
source_id='',
)
# Create a new outlet
outlet = pylsl.StreamOutlet(info)
# Print the stream info
print("Now publishing stream:", info.name(), info.type(), info.channel_count(), "channels at", info.nominal_srate(), "Hz")
# Continuously send blocks of data
while True:
# Generate a block: 25 samples x 8 channels
block = []
for i in range(blocksize):
row = [round(random.random(), 2) for ch in range(nchan)]
block.append(row)
# Push the entire block
outlet.push_chunk(block)
# Print the block for debugging
print("Pushed block:", block)
# Sleep for the duration of the block (100 ms)
time.sleep(blocksize / fsample)
Receiving markers
import pylsl
import time
# on my M3 MacBook I had to recompile the LSL library and then this is needed
# export DYLD_LIBRARY_PATH=/Users/roboos/matlab/liblsl-maca64/bin
time.sleep(3) # wait a bit to allow LSL to synchronize
# first resolve a marker stream on the lab network
print("Looking for a marker stream...")
streams = pylsl.resolve_byprop('type', 'Marker')
print(streams)
if len(streams)==0:
raise Exception("There are no marker streams")
else:
# Create an inlet for the first stream
inlet = pylsl.StreamInlet(streams[0])
while True:
sample, timestamp = inlet.pull_sample()
if len(sample):
print(f"Received marker {sample}")
Receiving EEG data
import pylsl
import time
# on my M3 MacBook I had to recompile the LSL library and then this is needed
# export DYLD_LIBRARY_PATH=/Users/roboos/matlab/liblsl-maca64/bin
time.sleep(3) # wait a bit to allow LSL to synchronize
# first resolve a marker stream on the lab network
print("Looking for an EEG data stream...")
streams = pylsl.resolve_byprop('type', 'EEG')
print(streams)
if len(streams)==0:
raise Exception("There are no marker streams")
else:
# Create an inlet for the first stream
inlet = pylsl.StreamInlet(streams[0])
while True:
sample, timestamp = inlet.pull_sample()
if len(sample):
print(f"Received data {sample}")
Plotting an EEG data stream
With some modifications to the code above, we can visualize the data stream that we are receiving by repeatedly plotting the last N seconds. For that you can make use of a circular buffer (also known as a ring buffer), i.e. a matrix where for every sample that you receive you remove the first column and then add the latest sample as the last column.
Filtering an EEG data stream
An important difference between filtering data in real-time versus reading continuous data in memory and then filtering it, is that real-time data arrives in small blocks. First of al, we cannot use a two-pass filter. Furthermore, we don’t want filter edge artifacts and hence have to maintain the filter state between the blocks.
In MATLAB this can be achieved with the low-level ft_preproc_online_filter_init and the ft_preproc_online_filter_apply functions from the FieldTrip preproc module (see also this architecture diagram).
In Python you should be able to use functions from the low-level SciPy package or from MNE-Python.
Stimulus presentation in MATLAB
With MATLAB you can use Psychtoolbox (PTB) for stimulus presentation and experimental control. Although Psychtoolbox started as an open source project, at a certain point the developers switched to a paid license model for their ongoing efforts in maintenance and support. The latest version of Psychtoolbox that is completely free is 3.0.19.16 (“Last free desert”).
% this implements a very basic visual oddball task using Psychtoolbox-3
% see https://en.wikipedia.org/wiki/Oddball_paradigm for more details
clear
clc
% Configuration
nTrials = 150;
stimulusDuration = 0.200;
fixationDuration = 0.850;
jitterRange = 0.300;
initialFixationDuration = 2.0;
% Initialize Psychtoolbox
PsychDefaultSetup(2);
% the sync tests are important for precise timing, but require all drivers and settings to be correct
% we can skip the test to allow running this example on non-optimized laptops
Screen('Preference', 'SkipSyncTests', 1);
screens = Screen('Screens');
screenNumber = max(screens);
black = BlackIndex(screenNumber);
white = WhiteIndex(screenNumber);
green = [0 1 0];
red = [1 0 0];
[window, windowRect] = PsychImaging('OpenWindow', screenNumber, black);
Screen('BlendFunction', window, 'GL_SRC_ALPHA', 'GL_ONE_MINUS_SRC_ALPHA');
[xCenter, yCenter] = RectCenter(windowRect);
Screen('TextFont', window, 'Arial');
% Prepare stimuli
fixation = '+';
standard = 'STANDARD';
deviant = 'DEVIANT';
% Keyboard configuration
KbName('UnifyKeyNames');
spaceKey = KbName('space');
escapeKey = KbName('ESCAPE');
% Wait for SPACE to start
DrawFormattedText(window, 'Press SPACE to start', 'center', 'center', white);
Screen('Flip', window);
while true
[keyIsDown, ~, keyCode] = KbCheck;
if keyIsDown && keyCode(spaceKey)
break;
end
if keyIsDown && keyCode(escapeKey)
sca;
return;
end
WaitSecs(0.01);
end
KbReleaseWait;
% Initial fixation
DrawFormattedText(window, fixation, 'center', 'center', white);
Screen('Flip', window);
WaitSecs(initialFixationDuration);
% Generate randomized trial sequence with 80% standard and 20% deviant
conditions = [ ...
repmat({'s'}, 1, round(nTrials * 0.8)), ...
repmat({'d'}, 1, round(nTrials * 0.2)) ...
];
conditions = conditions(randperm(nTrials));
fprintf('Starting oddball task...\n');
for trial = 1:nTrials
condition = conditions{trial};
% Select stimulus
if strcmp(condition, 's')
stimulus = standard;
color = green;
textSize = 50;
else
stimulus = deviant;
color = red;
textSize = 100;
end
% Present stimulus
fprintf('Trial %d/%d: %s\n', trial, nTrials, condition);
Screen('TextSize', window, textSize);
DrawFormattedText(window, stimulus, 'center', 'center', color);
onset = Screen('Flip', window);
% Prepare fixation screen
Screen('TextSize', window, 50);
DrawFormattedText(window, fixation, 'center', 'center', white);
% Schedule fixation onset 200 ms after stimulus onset
ifi = Screen('GetFlipInterval', window);
offset = Screen('Flip', window, onset + stimulusDuration - 0.5 * ifi);
% Wait until next trial
jitter = rand() * jitterRange;
WaitSecs(fixationDuration + jitter);
% Check for ESC
[keyIsDown, ~, keyCode] = KbCheck;
if keyIsDown && keyCode(escapeKey)
fprintf('Pressed ESC\n');
break;
end
end
% Finish
DrawFormattedText(window, 'Done, press SPACE to finish', 'center', 'center', white);
Screen('Flip', window);
while true
[keyIsDown, ~, keyCode] = KbCheck;
if keyIsDown && keyCode(spaceKey)
break;
end
WaitSecs(0.01);
end
% Clean up
sca;
fprintf('Oddball task finished.\n');
Stimulus presentation in Python
With Python you can use PsychoPy for stimulus presentation and experimental control. PsychoPy can be installed in a virtual environment using pip. Since PsychoPy comes with an (optional) graphical user interface, there is also a stand-alone version that you can download and run as a GUI. Since with the code above we have already been scripting in MATLAB and Python, I recommend not to use the GUI but rather also use scripting for PsychoPy.
The following example implements a simple visual experiment and can be used to get started.
# this implements a very basic visual oddball task using PsychoPy
# see https://en.wikipedia.org/wiki/Oddball_paradigm for more details
from psychopy import visual, core
from utilities import wait_for_space
import random
import pylsl
# Prepare the LSL stream info
info = pylsl.StreamInfo(
name='Stimulus computer',
type='Markers',
channel_count=1,
nominal_srate=0, # should be zero for markers
channel_format='string',
source_id='',
)
# Create a new outlet
outlet = pylsl.StreamOutlet(info)
# Print the stream info
print("Now publishing stream:", info.name(), info.type(), info.channel_count(), "channels at", info.nominal_srate(), "Hz")
screen = visual.Window([800, 600], color='black', units='pix')
fixation = visual.TextStim(screen, text='+', color='white', height=50)
standard = visual.TextStim(screen, text='STANDARD', color='green', height=50)
deviant = visual.TextStim(screen, text='DEVIANT', color='red', height=100)
wait_for_space(screen, 'Press SPACE to start')
fixation.draw()
screen.flip()
core.wait(2)
# present 10 trials of the oddball task
# where 80% of the stimuli are standard and 20% are deviant
for trial in range(10):
# randomly choose a stimulus for each trial
condition = random.choice(['s', 's', 's', 's', 'd']) # 80% standard, 20% deviant
if condition == 's':
print('standard')
stimulus = standard
else:
print('deviant')
stimulus = deviant
stimulus.draw() # present the randomly chosen stimulus
screen.flip()
outlet.push_sample([condition]) # send marker to LSL
core.wait(0.2)
fixation.draw()
screen.flip()
jitter = random.uniform(0, 0.3)
core.wait(0.85 + jitter)
wait_for_space(screen, 'Done, press SPACE to finish')
screen.close()
core.quit()






