How we make BP work in simple terms.
Interacting with BP
BP relies on a very simple paradigm that detect attention by comparing brain responses triggered during visual stimulation. For this, Brain Painting records the cerebral activity from scalp electrodes and provides control and feedback over an interface. This system is referred to as a brain-computer Interface (BCI). It only requires two monitors, an electroencephalogram (EEG) and the user's attention to use Brain Painting.
Simplified setup of Brain Painting. A: the end-user with the EEG cap. B: EEG amplifier. C: stimulation monitor. D: canvas monitor
Brain Painting matrices
This specific setup was initially used to spell letters to form words and sentences. In BP, we replaced letters by actions on a virtual canvas that result in paintings. By concentrating for a few seconds on the target, the user can send selections to the canvas. Meanwhile the lines and columns of the matrix are flashing (5 stimulations per second)
BP version 1 matrix with face flashes (text within the matrix is in german).
A standard spelling matrix during flashing
Event related potentials
Flash stimulations elicit distinctive brain responses called event-related potentials (ERPs), that are discriminated using EEG. To elicit stronger responses, we instead use a face overlay to trigger additional face-evoked potentials (Kaufmann et al., 2011).
Plotted difference between target and non-target averaged ERPs. A positivity in the amplitude is visible at about 300ms, peaking at 400ms.
The human cortex show different ERPs whether the event (the flash) occurs on the chosen target, or not. The so-called "P300 ERP" is a positive activation observable between 200ms to 500ms.
Electrodes locations for BP, according to the "10-20" system representation.
We use a g.usbamp (gtec.at) with g.gammabox and 8 active electrodes, but most amplifiers would do the job as long as their electrodes are properly positionned. The most important ones are placed on the Frontal-Central-Parietal and Occipital midline on which the P300 is most prominent. Plus some more electrodes covering the parieto-occipital area, related to vision.
Five seconds of filtered EEG signals acquired while using BP at home
Signal processing and classification
During signal processing, we acquire a time resolution between 2000Hz to 256Hz. After removing high (>60Hz), and low (.01Hz) frequencies from the EEG (and the dreadly 50Hz electromagnetic noise that our power lines emit) the signal is both human and machine readable.
Projection of a linear classifier for two classes (i.e. target and non-target). Unsurprisingy, there is an overlap between those classes, leading to a chance of classification errors.
During calibration, the EEG signals over time are cut into small pieces, called features. They are needed to train a classifier using stepwise linear discriminant analysis (SWLDA). The most helpful electrodes and timings are kept and weighted, returning a linear classifier.
As soon as the linear classifier is trained and returns a good performance, you can start using BP. In the last version we included an fast calibration process that allow to calibrate or check the performance within 10 minutes. The calibration process is integrated in the interface, so that any operator can easily do it within a few clicks.
Disclaimer: This section contains technical information about how BP works. Those were made readable for a broad audience. If you look for exactness, please refer to the publication list. If you cannot access one them, don't hesitate to contact us to request an author's version. Alternatively, the wikipedia page for BCIs might give you a larger picture of the topic.