Le Wu, Aiping Liu, Rabab K. Ward, Z. Jane Wang, Xun Chen
©SHUTTERSTOCK.COM/TRIFF
Brain–computer interfaces (BCIs) employ neurophysiological signals derived from the brain to control computers or external devices. By enhancing or replacing human peripheral functioning capacity, BCIs offer supplementary degrees of freedom, significantly improving individuals’ quality of life, particularly offering hope for those with locked-in syndrome (LIS). Moreover, BCI applications have expanded across medical and nonmedical domains, including rehabilitation, clinical diagnosis, cognitive and affective computing, and gaming. Over the past decades, with a wealth of brain signals captured invasively or noninvasively, BCI has made spectacular progress. However, this also poses new challenges for signal processing techniques, such as characterization and classification. In this review, we first introduce signal enhancement and characterization methods to mine inherent patterns of nonstationary and time-varying brain signals. Then, we highlight widely adopted classification methods in BCI and the challenges they face. This article aims to comprehensively overview crucial signal processing techniques in BCI and provide suggestions for future directions.
BCIs are designed to obtain brain signals, decode specific patterns, and transmit user intents to external devices, establishing a direct communication channel between the brain and a computer without peripheral nerve and muscle involvement. BCIs hold the potential to replace or supplement peripheral functions, such as spelling words, controlling neuroprostheses, and moving cursors, benefiting individuals with neuromuscular disorders. Particularly valuable for LIS patients, BCIs enable basic communication with caregivers, significantly impacting individuals, the economy, and society.
Over the last 25 years, in-depth brain function investigation, powerful real-time hardware systems, and advanced signal processing technologies have contributed to the exponential growth and evolution of the BCI field. Signal processing technologies are essential for recent improvements in BCI decoding accuracy and efficiency. While several review articles exist on BCI, a comprehensive survey addressing the evolution of key signal processing technologies over the past decades remains absent. This review aims to provide a systematic overview of recent BCI advancements in signal processing technologies, discussing future directions in response to the growing demands of diverse applications, inspiring further progress in both BCI technologies and applications.
In 1929, Hans Berger [1] pioneered noninvasive recording of neuroelectrical signals from the scalp, later known as electroencephalography (EEG). However, early acquisition technology struggled with electrical and physiological disturbances. Extracting intentions from spontaneous EEG remained a fantasy, but EEG’s potential in examining brain functions and clinical applications grew. Over time, the relationship between brain physiology and actual or imagined behavior became clearer, inspiring attempts to control brain signals. In 1968, Wyrwicka and Sterman [2] demonstrated that, with specific instrumental learning, cats could control sensorimotor cortex slow-wave rhythms for food rewards. Concurrently, researchers showed that humans and monkeys could control brain rhythms with sensory feedbacks, fueling interest in using brain signals for device control [3]. By 1973, Vidal [4] introduced the term BCI and developed the first vision-based BCI system. Since then, numerous BCI systems have been developed, with strategies to improve communication performance. Modern BCIs have evolved from limited capacities like “on” and “off” to complex functions, such as controlling robotic arms, synthesizing speech, and manipulating cursors. This progress can be attributed to at least three factors.
First, BCI advancements have been driven by a growing number of individuals dedicated to improving human health and understanding the brain. BCIs hold significant potential for enhancing communication in patients with LIS due to neuromuscular diseases. Furthermore, BCI progress relies on voluntary participation from patients in exploratory experiments. Second, the rapid development of acquisition hardware supports collecting various physiological signals from the brain at different depths and spatial resolutions and allows online processing. This wealth of brain signals enables accurate recognition of user intent. Finally, recent BCI improvements have benefited from the contributions of the signal processing community. While simple brain signal characteristics can be visually discerned, subtle information requires advanced signal processing technologies. These technologies facilitate faster signal transmission, extraction, and amplification of desired brain activity information from redundant signals, and conversion into accurate commands for controlling external devices. Consequently, advancements in signal processing technology have played a crucial role in BCI development.
The initial step in developing a BCI system involves brain signal acquisition, an area that has seen diverse methodologies over the years, each with its own advantages and tradeoffs in terms of safety, clarity, portability, and cost. Broadly, these methods fall into two categories: invasive and noninvasive. Invasive methods, such as electrocorticography (ECoG) and intracortical recording, require surgical implantation of electrodes within the skull. While ECoG captures electrical oscillations from the cortical surface, intracortical recording delves deeper into the gray matter, collecting single-unit activity (SUA), multiunit activity (MUA), and local field potentials (LFPs). Stereotactic EEG (sEEG) is another invasive technique that uses depth electrodes for precise targeting and recording from deeper brain structures. On the other hand, noninvasive methods, including EEG, magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS), record brain signals using sensors placed on or near the scalp.
Acquired brain signals can be classified as electrophysiological (EEG, MEG, ECoG, sEEG, intracortical recording) or hemodynamic (fMRI, fNIRS). Hemodynamic methods indirectly record neuronal activity through changes in venous blood composition, while electrophysiological methods capture voltage changes resulting from interneuronal electrochemical transmission. The wide use of electrophysiological methods in real-time BCI systems notwithstanding, EEG stands out due to its noninvasive nature, high temporal resolution, portability, and affordability. However, it faces limitations in signal quality compared to invasive methods, attributed to signal attenuation as neuron potentials traverse the skull and tissues.
BCI paradigms generally encompass two categories: exogenous and endogenous. Exogenous paradigms decode brain signals elicited by external stimuli, while endogenous paradigms rely on spontaneous brain signals generated by mental tasks without external triggers. The well-explored exogenous paradigm includes the P300 event-related potential-based (ERP) BCI. Recorded using EEG, P300 responses are usually elicited approximately 300 ms after an oddball stimulus in the parietal lobe. The first P300 ERP-based BCI system, a word spelling system controlled by brain signals, was proposed by Farwell and Donchin in 1988 [5]. A screen displays a letter matrix to the user, and each letter flashes randomly. When the desired letter is highlighted, the user’s gaze elicits a P300 ERP, which is used for selection. Despite its effectiveness as a communication tool for those with motor neuron diseases, single-trial P300 ERP detection is challenging due to potential noise sources. Techniques for preprocessing, feature extraction, and classification are necessary to interpret brain signals effectively. However, the P300 ERP’s efficiency is limited as it can output only a few characters per minute. Alternatively, steady-state visual evoked potentials (SSVEPs)-based BCI has gained popularity. This method involves the occipital cortex detecting sinusoidal-like oscillations in response to high-frequency visual stimuli. It generates high signal-to-noise SSVEP signals, enabling efficient target selection. SSVEPs and P300 ERPs are favored due to their experimental convenience and adaptability, requiring minimal training. Advanced classification algorithms and error-related negative evoked potentials further enhance their performance. However, they might not be ideal for patients with severe neurological deficits who have difficulty gazing, or for prolonged use due to visual fatigue.
In contrast to exogenous paradigms, endogenous BCI paradigms are a promising research avenue, centering on self-regulated brain signal patterns evoked by mental tasks. Slow cortical potentials (SCPs), with frequencies ranging from 0.1 to 1 Hz, are a prime example. SCPs present as positive or negative fluctuations in cerebral electrical activity, lasting from several hundred milliseconds to seconds. Negative SCPs relate to behavioral or cognitive preparation, while positive SCPs indicate reduced neural excitation and behavioral inhibition. SCP-based BCIs enable control of external devices by modulating brain signals for both healthy individuals and paralyzed patients. However, SCPs’ slow dynamics challenge real-time performance [6]. Another prominent endogenous paradigm involves sensorimotor rhythms (SMRs), encompassing electrical oscillations in the mu (8–13 Hz), beta (14–26 Hz), low gamma (30–50 Hz), and high gamma (50–200 Hz) frequency bands. SMR amplitude changes, linked to movement or motor intent, are characterized by decreased mu and beta bands and increased low and high gamma bands. High gamma bands, particularly, relate to critical cognitive and motor functions, making them vital in SMR-based BCI systems. Modulating these bands allows users to control cursors or operate neural prostheses, powered wheelchairs, and orthoses through SMR-based BCIs, assisting those with motor impairments in regaining mobility [7]. Despite requiring more focused training and having lower efficiency compared to exogenous paradigms, endogenous paradigm-based BCIs offer essential benefits for patients with complete motor function loss, as no external stimuli gazing is needed.
Apart from the four well-established paradigms, emerging BCI approaches aim to enhance user comfort and system performance. Hybrid BCIs, combining different control signals or modalities, have been proposed to augment control efficiency. For instance, Allison et al. utilized a hybrid BCI that leverages EEG signals from motor imagery and visual stimuli, significantly surpassing standalone BCI paradigms [8]. Similarly, Kwak et al. integrated EEG and fNIRS signals to circumvent limitations in solo EEG- and fNIRS-based systems, yielding higher recognition accuracy for mental arithmetic and motor imagery tasks [9]. Despite the superior performance of hybrid paradigms, concerns persist about the intuitiveness of their control processes. To address this, Min et al. emphasized the importance of cognitive signal-based BCIs that decode goal-oriented intentions from the prefrontal cortex [6]. Prefrontal ECoG signals, for example, have been shown to predict hand grasping and elbow flexion movements before initiation [10]. Additionally, EEG signals decoding error-related potentials have also potentially enhanced motor imagery tasks [11]. Such cognitive brain signals present promising prospects for enhancing BCI performance and user acceptance. Furthermore, Gao et al. championed fusing human and artificial intelligence to establish advanced BCIs with collaborative intelligence [12].
As depicted in Figure 1, both invasive and noninvasive brain signals undergo several processing stages: digitization, transmission, preprocessing, feature extraction, pattern classification, and finally, target command generation to operate external devices. This comprehensive BCI framework encompasses various signal processing techniques. To improve BCI usability in practical environments, several considerations are necessary at each stage. These include enhancing the signal-to-noise ratio through noise suppression, developing signal representations that highlight target rhythms, and creating efficient pattern classification methods. In the sections that follow, we will review relevant signal processing techniques—such as spike sorting, signal filtering, blind source separation (BSS), time–frequency analysis, and classification algorithms—with an emphasis on their advancements over the past two decades.
Figure 1. Framework of a common BCI system (adapted from [3]). As shown in the figure, the acquired neurophysiological activity signals are preprocessed, feature extracted, and converted into control commands by pattern classification methods. The brown texts represent the signal processing techniques that may be used.
The process of spike sorting plays a pivotal role in the analysis of electrophysiological recordings, particularly those from intracortical sources, by accurately identifying and isolating action potentials generated by individual neurons. This is crucial as raw neural signals typically comprise a blend of action potentials from various closely situated neurons, in addition to background noise, making the accurate extraction and attribution of these action potentials essential for the interpretation of neural activity and the enhancement of BCI performance [13].
Addressing these challenges, the spike sorting workflow commences with preprocessing, a step that filters data, removes artifacts, and normalizes signal amplitude, thereby emphasizing the pertinent frequency range (typically 300 Hz to 3 kHz for spikes). This is followed by spike detection, where preprocessed signals are scanned for potential neural action events. The evolution of techniques here, from simple fixed-voltage thresholds to adaptive thresholding and wavelet-based methods [14], has greatly improved detection accuracy and reliability. The feature extraction phase then follows, a stage where methodologies have seen substantial advancement. From the basic extraction of waveform parameters like spike amplitude, width, and area, the process has evolved to employ principal component analysis (PCA) for its ability to highlight key variations in spike shapes and minimize redundant information. More recently, wavelet-based methods have emerged, providing a time–frequency signal representation, thereby capturing transient characteristics of spike waveforms, and further improving classification accuracy [15].
Following feature extraction, spike clustering and classification occur, where feature vectors are grouped based on their similarity. Clustering algorithms such as k-means, hierarchical clustering, and supervised learning techniques like support vector machines (SVMs) and artificial neural networks [16], have been employed to improve classification accuracy and automation. Finally, the validation phase has transitioned from manual inspection to more objective and automated methods. Measures, such as Victor–Purpura distances and van Rossum distances, assess spike sorting performance by comparing the similarity or dissimilarity between spike trains [17]. Objective metrics like precision, recall, and F1 score have also been developed to enable standardized comparisons of different sorting techniques.
The efficacy of spike sorting techniques in invasive BCI studies is well-documented, with significant advances in recognizing user intent, controlling digital and mechanical devices, and improving communication efficiency for paralyzed users [18]. However, spike sorting technology encounters significant hurdles, notably overlapping spikes, which result from simultaneous or near-simultaneous neuronal firing, complicating the sorting process and segregation of concurrent action potentials. Currently, template matching and Bayesian techniques offer promising solutions to manage these overlaps and improve spike separation accuracy [13]. Furthermore, the variability in spike waveforms, induced by factors like electrode drift, neural activity shifts, or recording conditions, poses a challenge to the consistent classification and separation of spikes from identical neurons. Adaptive spike sorting algorithms have emerged as potential solutions, capable of monitoring and adjusting to spike waveform changes over time, thereby ensuring precise spike separation despite waveform variability.
Despite its potential, current technology has inherent limitations. For instance, accurately determining the number of clusters, or neurons, in unsupervised clustering algorithms is a challenge due to the indeterminacy of the true neuron count. To address this, model selection techniques or information-theoretic criteria, such as the Akaike Information Criterion or the Bayesian Information Criterion, are employed to approximate optimal cluster numbers, enhancing sorting accuracy [19]. Furthermore, some spike sorting algorithms are computationally intensive, complicating real-time processing, particularly with high-density electrode arrays or large-scale recordings. Evaluating the accuracy and reliability of spike sorting algorithms also poses a challenge due to the lack of ground truth data, the true identity of spike-generating neurons. One possible solution is using simulated data or ground truth data from simultaneous intracellular and extracellular recordings to assess and compare algorithm performance [13]. The establishment of standardized evaluation metrics and benchmarks could further facilitate the comparison and validation of different spike sorting methodologies, propelling the field’s progression.
Currently, spike sorting techniques, to a certain extent, require human intervention for optimization. However, given the rapid escalation in demand for massive-channel and portable electrode arrays, the necessity for automated approaches becomes overwhelmingly clear [13]. Moreover, developing hardware implementation strategies that facilitate efficient deployment of BCI technology holds significant value for future advancements.
Neurophysiological signals are prone to contamination from physiological and nonphysiological artifacts during acquisition. Nonphysiological artifacts, such as powerline interference, along with physiological artifacts like electrooculogram, electrocardiogram (ECG), and electromyogram signals, often emerge during EEG data collection. Various filtering techniques have been explored to counteract these issues. Classical frequency filters, for instance, effectively eliminate narrow band noise, such as powerline interference. Advanced parametric filters, including adaptive, Wiener, Kalman, and Bayes filters, optimize filtering parameters using reference signals or signal state information, promoting the extraction of clean brain signals. However, these methods often presuppose distinct spectral characteristics for brain signals and artifacts, a condition that may not always be met due to frequency overlap, thereby constraining their efficacy [20]. This article will spotlight two prevalent filters, smoothing and spatial filters, and discuss their successful applications in the BCI field.
Smoothing filters utilize adjacent data points to refine signal values without distorting their overall tendencies. Representative methods include moving average, median, and Savitzky–Golay (SG) filters. The moving average filter, a simple low-pass filter, calculates the average value within a fixed-length signal window. Meanwhile, the median filter, a nonlinear filter, outputs the median value of a sorted signal window. Both filters reduce noise in BCI applications. However, moving average filters may flatten and widen peaks in brain signals, potentially causing erroneous analyses. The SG filter, a more suitable alternative, has been extensively studied for the past two decades. Originally developed in 1964 for noise reduction in chemistry spectra [21], the SG filter has since gained popularity for enhancing various signal types. It employs local least-square polynomial regression to determine each point’s smoothed value, retaining waveform peak shapes and heights essential for brain signal analysis. Figure 2(a) displays the original and filtered EEG signals, illustrating noise reduction and signal smoothness. Since the early 2000s, SG filters have improved noise reduction in EEG and ECoG, thereby directly augmenting BCI system performance [22]. Acharya et al. [23] and Gajbhiye et al. [24] developed adaptive strategies to optimize the order and frame size selection for the SG filter, aiming to maximize correlation coefficients and improve denoising performance. In a different approach, Agarwal et al. [25] enhanced signal quality using a cascaded version of the SG filter.
Figure 2. (a) Example outputs processed by a smoothing filter, i.e., SG filter. This filter smooths the raw EEG signal while ensuring that the tendency is not distorted. (b) Example outputs of spatial filters (adapted from [28]), where the motor imagery of the left (right) hand can activate the Region around the right (left) motor cortex.
Spatial filters effectively recover essential information distributed across channels, thereby enhancing the extraction of discriminative features. Figure 2(b) illustrates how well-established spatial filters can intuitively highlight task-related regions. Generally, spatial filters are categorized into reference filters and data-driven filters. Reference filters, including ear-reference, bipolar, common average reference (CAR), and Laplacian, are predefined using electrode position information and neurophysiological plausibility. Conversely, data-driven filters are established and optimized using user-specific signals, with representative filter construction algorithms including common spatial pattern (CSP), independent component analysis (ICA), PCA, and canonical correlation analysis (CCA).
Reference filters are prevalent in BCI system construction because of their efficiency and implementation simplicity. These filters, including bipolar, CAR, and Laplacian filters, work by subtracting the average brain signal from specific electrodes from the electrode of interest, thereby focusing on signal fluctuations in the BCI system. McFarland et al. [26] highlighted the superior performance of CAR and Laplacian methods over the ear-reference method in EEG-based BCI systems. They also introduced large Laplacian filters that better match the topographical extent of the EEG signal. However, the efficacy of reference filters heavily depends on judicious reference selection. To address this, some researchers have proposed strategies that enhance reference signal generation, such as linearly combining several reference channels [27].
Data-driven spatial filters optimize parameters using brain signals in either supervised (CSP) or unsupervised (ICA, PCA, CCA) manners. These filters identify correlations between electrode channels to suppress noise and weight each channel according to its relevance to specific brain patterns. CSP filters enhance differences between two classes of signals by maximizing variance differences. In practice, high variance often reflects a strong brain rhythm of interest, while low variance indicates an attenuated rhythm. This property enables the filters to enhance brain pattern distinctions. ICA and PCA decompose multichannel brain signals into independent or linearly uncorrelated components. CCA maximizes the correlation coefficient between two filtered signals. Notably, CCA and ICA methods are frequently used for removing locomotion-induced artifacts, with ICA-based methods being the most prevalent. The CSP algorithm has garnered considerable attention in BCI applications over the past decades [29], with numerous extensions developed for more efficient spatial filtering. These extensions broadly fall into two categories: spatiospectral joint analysis and regularization. In this section, we review these CSP algorithm improvements, which may also inspire further advancements in other spatial filter algorithms.
The CSP algorithm is effective in extracting spatial information, but it does not consider spectral characteristics. Given that BCI signals often exhibit notable variations within specific frequency bands, such as the mu-band variations in SMR-based BCI, ignoring spectral information can lead to inaccurate focusing on redundant details. To address this, several CSP extensions have been proposed to jointly analyze both spatial and spectral characteristics. For instance, the common spatiospectral patterns approach optimizes spatial filters alongside frequency filters for each channel, allowing for an additional capture of spectral information [30]. Another method, subband CSP (SBCSP), decomposes signals into subbands using a filter bank, with the final decision based on fusing CSP features extracted from each band [31]. The filter bank CSP (FBCSP) goes a step further by obtaining correlation information between different subbands through mutual learning [32], thereby improving BCI performance. However, while filter bank-based methods provide more detailed spatiospectral information, they also introduce a greater computational burden. As a result, additional FBCSP variants have been proposed to adaptively select fewer targeted frequency components based on their importance to the BCI task.
In addition, CSP has been reported to be highly sensitive to noise, outliers, and prone to overfitting. To address these issues, several extended studies of CSP have demonstrated that adding regularization or prior information to the algorithm can help mitigate these problems. Regularization can be applied either at the level of covariance matrix estimation or at the level of the objective function. For example, Kang et al. [33] proposed a composite covariance matrix obtained through linearly combining covariance matrices of multiple subjects, which proved more robust than traditional CSP, especially with limited training data. Additionally, Lotte and Guan [34] introduced a unified theoretical framework for regularized CSP and developed four improved regularized algorithms aimed at either constraining covariance matrix estimation or optimizing the objective function according to brain signal characteristics. Subsequent studies have sought to design better regularized CSP algorithms from various perspectives, such as stationary subspace analysis, divergence maximization, and probabilistic analysis.
In this section, we have examined a pivotal signal processing technique in BCI, namely, signal filtering. The establishment of suitable filters aids in the removal of extraneous information from recorded signals, and the extraction of physiologically relevant features, both of which are critical for building efficient BCI systems. Compared to reference filters, data-driven spatial filters usually provide superior BCI performance due to their capacity to extract task-specific and subject-specific information under diverse brain conditions. However, data-driven methods often necessitate a significant amount of training data and are susceptible to interference. Furthermore, their performance can degrade significantly across trials and subjects. Therefore, future research might consider the development of robust spatial filters capable of extracting highly discriminative and generalizable features from limited data.
In BCI, observations are typically collected from a group of sensors, each receiving a mix of source signals. BSS aims to recover target sources from observed signals, with only these observations available for the separation procedure. For instance, PCA uses an orthogonal linear transformation to decompose observations into a set of uncorrelated principal components that maximize variance. It is assumed that artifacts and underlying neurophysiological activities are represented by different components with distinct topographies and power spectra. As a result, artifact removal and dimensionality reduction can be achieved by eliminating insignificant components considered as noise. Another conventional BSS algorithm is ICA, which assumes that sources are non-Gaussian and mutually independent signals. By decomposing brain signals using higher-order statistics, ICA can detect small nonbrain artifacts and has been extensively employed in EEG denoising [35].
Although BSS techniques like PCA and ICA have demonstrated utility in artifact removal, they may face some limitations in practical usage. These techniques are fundamentally linear models and may struggle with nonlinearly mixed noise. Moreover, they only utilize spatial information, ignoring temporal correlations, which restricts their focus to isolated artifacts, such as ECGs and eye movements, limiting broader applications. Additionally, assumptions like spatial orthogonality and statistical independence of source signals may be unrealistic constraints for real-world sources. Consequently, complex artifacts may be distributed among multiple components, making their effective removal challenging. Recently, joint BSS (JBSS) methods have demonstrated improved performance in removing complex noise. JBSS methods can exploit dependence relationships of sources across multiple datasets, potentially achieving performance enhancements beyond those attained by single-set BSS approaches. For example, Clercq et al. [36] used CCA to suppress muscle artifacts in EEG signals. Their study constituted two datasets: the original signal and its temporally delayed version. CCA was then employed to identify sources that were maximally autocorrelated between datasets and mutually uncorrelated within each. It was assumed that muscle artifacts exhibited relatively low autocorrelation compared to neurophysiological activity. Experiments on simulated data also demonstrated that CCA’s denoising performance surpassed that of ICA. Furthermore, various effective JBSS methods have been developed, such as the ensemble empirical mode decomposition-CCA (EMD-CCA) method for noise reduction in single-channel signals, and the multiset CCA method capable of processing more than two datasets simultaneously. Chen et al. [35] systematically reviewed numerous JBSS methods.
CCA has also been successfully applied in SSVEP-based BCI. Lin et al. [37] first utilized CCA to find the maximal correlation coefficient between EEG signals and manually designed reference signals, representing the SSVEP frequency by the reference signal with the largest coefficient. They demonstrated that the CCA-based method achieved higher identification accuracy than power spectral density-based analysis. Several CCA extensions have been proposed to improve recognition accuracy, including multiway CCA and regularized multiway CCA. However, these methods relied on predefined sine–cosine waves for constructing reference signals, which may be problematic due to intersubject or intertrial EEG variations. Zhang et al. [38] addressed this by using multiset CCA to optimize reference signals from multiple sets of EEG data at the same stimulus frequency. To fully exploit SSVEP-related signals, researchers have also developed various approaches, such as filter bank CCA and task-related component analysis. However, these models only consider linear relationships, neglecting the nonlinearities in real EEG signals. Recently, kernel CCA, deep CCA, and deep multiset CCA have been proposed to learn nonlinear EEG signal representations using kernel- and neural network-based methods. In summary, BSS methods, particularly JBSS methods, have been successfully employed in BCI, offering promising solutions to real-world problems.
Brain signals, characterized by their nonstationary and time-varying nature, are subject to variations due to experimental conditions and mental states. For example, SMRs dynamically change in response to an individual’s imagined movement. Although frequency-based features offer high spectral resolution, they fail to capture essential time domain information needed for decoding dynamic brain signals. Consequently, time–frequency analysis, which examines time-varying spectral content, has gained prominence in recent years (refer to Figure 3). Time–frequency analysis methods are divided into two categories: time–frequency distribution (TFD) estimation and time–frequency decomposition. TFD estimation techniques represent power and phase changes in brain signals over time, while time–frequency decomposition methods dissect signals into components with distinct time–frequency characteristics, enabling the reconstruction of the original signal via inverse transform. Numerous studies have shown that incorporating time–frequency information significantly improves BCI system performance when compared to frequency analysis alone.
Figure 3. The wavelet method was used to calculate the time–frequency spectrum of channels C3 (a) and C4 (b) when a subject executed left-hand movement imagery, as adapted from [39]. This analysis revealed that the energy in the mu-band of the ipsilateral sensorimotor cortex initially increases and subsequently decreases during task performance. Conversely, the energy in the corresponding contralateral brain region exhibits an inverse pattern. Hence, time–frequency analysis offers an intuitive representation of the temporal changes in brain rhythms.
The short-time Fourier transform (STFT) is a notable TFD estimation method. By dividing the signals into short consecutive windows and performing Fourier transforms, STFT provides insights into the time–frequency domain. However, the choice of window size poses a tradeoff between time and frequency resolutions. Wide windows excel at distinguishing between similar or low-frequency components, providing high frequency resolution. This advantage comes at the expense of reduced temporal resolution, hindering the accurate identification of frequency changes’ timing. Narrower windows, conversely, offer excellent temporal resolution for precise timing detection but struggle with closely spaced frequency components, resulting in limited frequency resolution. Since brain signals encompass both high- and low-frequency components, an adaptable strategy is necessary to optimize temporal resolution across varying frequencies. The continuous wavelet transform (CWT) presents an alternative approach, employing narrow windows for high frequencies and wide windows for low frequencies, addressing the need for flexibility in time–frequency analysis.
CWT enables a more flexible analysis through a series of bases (wavelets) with different resolutions both on time and frequency. The CWT of the signal x(t) at the scale s (associated with the frequency) and the time t is defined as: \[{X}{(}{t},{s}{)} = \mathop{\int}\nolimits_{{-}\infty}\nolimits^{{+}{\infty}}{\frac{1}{\sqrt{s}}}{x}{(}{\tau}{)}{\psi}{\left(\frac{{\tau}{-}{t}}{s}\right)}{d}{\tau}{.} \tag{1} \]
Here, ${\psi}{((}{\tau}{-}{t}{)} / {s}{)}$ represents the dilated and shifted version of the mother wavelet ${\psi}{(}{\tau}{)}$. Changing the scale factor s can simultaneously raise (lower) the center frequency of the wavelet and compress (stretch) the waveform in the time domain. This property grants the CWT the ability to employ flexible window lengths at varying frequencies. For BCIs, the wavelet waveforms should bear biologically plausible components relative to the target signal. Typical wavelet bases employed include Morlet and Haar wavelets. However, CWT’s deployment of multiple mother wavelet variations across a plethora of frequencies introduces redundancy. In addressing this, modifications, such as the discrete wavelet transform [40] and the wavelet packet transform [41], have been proposed. These methodologies adopt discrete values for the dilation and time-shifting of the mother wavelet, offering a more computationally efficient alternative while maintaining satisfactory frequency resolution.
Another promising research direction is to use time–frequency analysis to locate subject-specific frequency bands to enhance motor intention recognition. For instance, Delisle-Rodriguez et al. [42] applied STFT to automatically locate the subject-specific bands that pack the highest power during pedaling motor imagery and achieved satisfactory classification results, demonstrating that these specific narrow bands improve both performance and computation efficiency. Similarly, Kumar et al. [43] and Yang et al. [44] showed that optimizing subject-specific frequency bands can further enhance motor imagery classification performance. These studies indicate that BCI systems can improve accuracy by pinpointing the subject-specific frequency bands with target fluctuations in brain signals. However, careful design of time–frequency analysis or optimization methods is crucial, as inaccurate localization of narrow bands may lead to omitted information and significantly degraded performance [42].
Despite their merits, wavelet-based techniques also possess limitations. First, the selection of the basis significantly influences the analysis outcome. Second, a tradeoff between temporal and frequency resolution persists. In response to these challenges, EMD and its derivatives have emerged as promising alternatives within the BCI domain. EMD decomposes signals iteratively and adaptively into simpler, data-driven bases, called intrinsic mode functions, unveiling diverse neurophysiological patterns. Due to its data-driven operation and the employment of Hilbert transform-derived instantaneous frequencies, EMD surpasses traditional Fourier or wavelet-based methods in terms of both temporal and frequency resolutions. As evidenced by Huang et al. [45], EMD offers superior frequency resolution compared to CWT in SSVEP-based BCI tasks. Moreover, EMD extensions, such as ensemble EMD [46] and multivariate EMD [47], proficiently analyze intricate brain signals across multiple channels.
This section outlines popular time–frequency analysis methods, with STFT and wavelet-based techniques being parameter-driven methods reliant on predefined bases. Though intuitive and efficient, determining critical parameters remains challenging. Conversely, EMD, a data-driven method, offers a flexible representation with accurate instantaneous frequency, but at a higher computational cost. Consequently, a tradeoff between computational speed and decomposition accuracy must be considered in practical applications.
In the architecture of a BCI, classification algorithms play a pivotal role by interpreting user intentions and converting them into control instructions for external devices. These algorithms traditionally optimize a function to weigh features and partition classes based on the features extracted from brain signals. As per [48], these conventional classifiers can be divided into distinct categories based on their design approach. First, linear classifiers, such as linear discriminant analysis (LDA) and SVM construct linear decision boundaries. Second, neural networks, like multilayer perceptrons, are composed of artificial neurons structured into layers, creating approximations of linear or nonlinear decision boundaries. Next, Bayesian classifiers, including Bayes quadratic classifiers and hidden Markov models, develop probabilistic models for each class, using Bayes’ theorem to classify feature vectors. Nearest neighbor classifiers, such as k nearest neighbor and Mahalanobis distance, label based on proximity to neighbors. Last, hybrid classifiers amalgamate multiple classifiers to generate more robust predictions, examples of which include boosting, voting, and stacking techniques.
Classifiers, in conjunction with extracted features, efficiently decipher signal patterns produced during specific brain activities. For instance, linear classifiers have been successfully applied in P300-based spelling systems and upper limb movements decoding, showcasing impressive recognition accuracy with both LDA and SVM [49]. Moreover, an enhanced self-paced BCI system has also been introduced, leveraging multiple SVMs and multiple classifier systems [50]. However, the performance of these traditional classifiers is largely determined by feature representation accuracy, and human expertise may be inadequate for general circumstances. To address this issue, recent research has pivoted toward deep neural networks (DNNs) as a means to bolster BCI performance. DNNs operate by learning hierarchical representations of brain signals via multiple stacked layers, thereby concurrently capturing high-level spatial–temporal features and latent dependencies. By merging feature representation with classification, DNNs streamline the BCI pipeline. Numerous DNN architectures, such as convolution neural network (CNN), deep belief network, recurrent neural network (RNN), and their derivatives, have been implemented in the BCI domain. A hybrid model of CNN and RNN, where the former extracts spatial information while the latter learns temporal information, is most frequently employed in the BCI field. While DNN-based methodologies have attained state-of-the-art performance in a range of BCI tasks, they are not without issues, including model overfitting and distribution mismatch.
One major challenge faced by existing classification methods is the small size of training samples. In practice, it is unfeasible to have users complete a multitude of tasks prior to BCI usage, resulting in a scarcity of training samples. This limitation can lead to overfitting and a subsequent decline in BCI performance. An intuitive solution to this problem is data augmentation, a strategy that artificially enriches and diversifies the training sample, thereby enhancing the characterization of the target distribution. Data augmentation methods in BCI are typically divided into data modification and generative model-based methods. Data modification techniques produce artificial samples by transforming, adding noise to, and blending raw signals or extracted feature vectors. For instance, Zhang et al. [51] transformed STFT features of EEG signals to augment training samples, improving the robustness of their CNN model. Another method involved the creation of artifact samples through time and time–frequency domain segmentation and recombination. Generative model-based methods, including variational autoencoders and generative adversarial networks, generate more realistic and diverse samples by leveraging distribution information learned from the training samples. These approaches have demonstrated significant improvements in the training process and optimized BCI classification performance in scenarios with limited sample sizes [52].
In addition, certain classifiers have been specifically developed to tackle limited sample sizes. Lotte et al. [53] suggested using three classifiers—a shrinkage LDA classifier, random forest, and Riemannian classifier—when confronted with scarce training samples. These classifiers construct robust models by enhancing distribution modeling accuracy, employing ensemble learning, and mapping data to geometric space, respectively. Furthermore, novel machine learning paradigms, such as few-shot learning and semisupervised learning, have been introduced to combat the issue of small sample sizes [54]. Few-shot learning improves the prediction of new sample classes by comparing the similarity of new samples with available training samples. This method allows for performance enhancement through the use of other relevant datasets, thus reducing dependency on training data. Semisupervised learning utilizes a small number of labeled samples and a large number of unlabeled samples. This paradigm alleviates the difficulty of constructing effective classification models by capturing underlying distribution information from unlabeled samples, reducing the need for a large number of labeled training samples.
Another challenge faced is the distribution mismatch between training samples and testing samples. Brain signals are inherently nonstationary, meaning the brain patterns observed during training may differ from those recorded during testing. Factors like intersubject variability, changes in mental or physical states, and surrounding noise can significantly alter the distributions within the collected brain signals. If the classifiers cannot adapt to these distribution changes in a timely manner, BCI performance will inevitably decrease. Existing methods have attempted to solve this problem through transfer learning, which aims to apply the knowledge gained from tasks of classifying training samples to better identify testing samples with mismatched distributions. These classifiers re-estimate and update their parameters using labeled or unlabeled samples from the testing session, enabling them to adapt to distribution changes and effectively classify nonstationary brain signals. For instance, an adaptive version of the LDA classifier uses Kalman filtering to track changes in the coefficient and update the classifier according to the properties of the input samples [55]. Additionally, Vidaurre et al. [56] proposed an unsupervised adaptation method for the LDA classifier that effectively updates class means and the global covariance matrix by considering the nonstationary nature of brain signals. Both studies have shown that adaptive classifiers can better classify motion imagery brain signals compared to original methods. However, traditional adaptive classifiers only use shallow feature representations to update the classifier, potentially limiting their adaptability.
Recently, DNN-based transfer learning methods have gained attention due to their powerful ability to extract transferable features and align mismatched distributions. A common method is to fine-tune a pretrained DNN model using newly inputted calibration samples. The pretrained model, which has been optimized with a large amount of data, is assumed to provide beneficial knowledge for further supervised learning from the limited samples [57]. Another representative case of transfer learning is domain adaptation (DA), which assumes that training samples and calibration samples are collected from the source and target domains, respectively. DA aligns the marginal or conditional distributions between source and target domains through optimization of DNN parameters, with measurement metrics of distribution differences including maximum mean discrepancy, Kullback–Leibler divergence, and Jensen–Shannon divergence [54]. However, both fine-tuning and DA methods require users to provide new samples to calibrate the classifier, which may be inconvenient. Alternatively, efforts have been made to address the problem of distribution mismatch from the perspective of domain generalization, which aims to extract invariant features across domains, thus eliminating the need for target domain data.
In this section, we have discussed traditional and DNN-based classification methods commonly employed in BCI. These classifiers can achieve promising performance when combined with ample training samples and robust feature representations. However, two major challenges in practical BCI applications, namely small training sample size and distribution mismatch, significantly impede accurate classification. We have also examined some promising solutions to these issues, but further efforts are required to mitigate the disturbances caused by the nonstationary and variable nature of brain signals. Another well-established approach to enhance BCI performance is the coadaptation strategy. Coadaptive calibration is a dynamic and interactive learning process in which both the user and the BCI system adapt to each other over time. This process enables continuous improvement in BCI performance as the system learns from the user’s brain activity patterns while the user learns to modulate their brain activity more effectively. Studies have demonstrated the effectiveness of coadaptive BCI systems [58]. In summary, by addressing the challenges of small sample sizes, distribution mismatches, and the nonstationary nature of brain signals, researchers can develop more effective and user-friendly BCI systems capable of better adapting to a diverse range of real-world applications.
In this article, we have examined recent advancements in signal processing for BCI technology. BCIs offer a novel mode of communication that is not only convenient but also provides significant benefits for individuals with neuromuscular diseases. The general framework of a BCI system includes signal acquisition, preprocessing, feature extraction, and pattern classification, with each of these steps posing distinct challenges for signal processing technologies. We have discussed a variety of signal enhancement and characterization methods. Given the inherently weak magnitude of brain signals and their susceptibility to contamination by artifacts, accurately representing neurophysiological activity is critical for effective decoding processes. Various methods, including spike sorting, filters, and BSS have been developed to enhance features of interest; these methods take into account the unique properties of brain signals, such as spatial distribution and rhythm bands. Data-driven methods generally tend to outperform parametric ones, which could be attributed to the fact that brain signals recorded in real-world scenarios often do not exactly conform to the assumptions of parametric models. In contrast, data-driven methods are capable of fitting the inherent correlations in the current data, thus enhancing accuracy. However, they are often sensitive to data variability caused by cross trials or cross subjects, which makes building a robust method a significant challenge. This overview of signal processing techniques and their advancements in BCI over the past two decades are summarized in Table 1.
Table 1. Fundamental signal processing techniques and their advancements in BCI over the past two decades.
We also introduced the commonly used classification methods in BCI. While most classification methods can achieve promising performance under ideal conditions, practical issues, such as small training sample size and distribution mismatch, can significantly degrade BCI performance. Data augmentation-based methods enhance the number and diversity of training samples to some extent, but the artificially created samples often involve simple morphological changes in existing samples, making it difficult to simulate the actual sample distribution. Therefore, how to find the possible regularities of brain signals for simulating the samples with more physical significance is demanded. In addition, a number of works try to utilize knowledge learned from other users to facilitate the training of target users. Despite their improvement in classification performance, they inevitably require the acquisition of labeled or unlabeled calibration samples from the target user. Further attempts may be made on domain generalization and online DA methods without imposing any calibration burden on the target user. In recent years, DNNs have gained significant attention due to their excellent feature representation capabilities. However, they also bring additional challenges in architecture design and interpretability.
It is worth noting that despite the surge in BCI-related publications over the past two decades aimed at enhancing the quality of life for those with LIS, definitive improvements in quality of life have not yet been convincingly demonstrated. Although published reports indicate relatively rapid spelling capabilities, very few works have demonstrated this in patients with completely LIS. Consequently, whether the absence of contingent thinking and intention in completely LIS impedes BCI performance remains an open question that warrants further investigation [59].
This work was supported in part by the National Key Research and Development Program of China under Grant 2021YFF1200600; in part by the National Natural Science Foundation of China under Grants 32271431, 82272070, and 61922075; in part by the Natural Sciences and Engineering Research Council of Canada under Grant RGPIN-2019-03981; in part by the China Postdoctoral Science Foundation under Grant 2022M723055; and in part by the Fundamental Research Funds for the Central Universities under Grants WK2100000028 and KY2100000123. All future correspondence should be sent to the corresponding author, Xun Chen (xunchen@ustc.edu.cn).
Le Wu (lewu@ustc.edu.cn) received his Ph.D. degree in biomedical engineering from the University of Science and Technology of China, Hefei, China, in 2021. He is with the Epilepsy Centre, Department of Neurosurgery, The First Affiliated Hospital of University of Science and Technology of China, Division of Life Sciences and Medicine, and also with the School of Information Science and Technology, University of Science and Technology of China, Hefei 230001, China. His current research interests include artificial intelligence in medicine and brain–computer interface. He is a Member of IEEE.
Aiping Liu (aipingl@ustc.edu.cn) received her B.Sc. degree in electrical engineering from the University of Science and Technology of China in 2009, and M.S. and Ph.D. degrees in electrical and computer engineering from the University of British Columbia in 2011 and 2016, respectively. She has been a postdoctoral research fellow at the Pacific Parkinson’s Research Center at the University of British Columbia. Currently, she is an associate professor at the Department of Electronic Engineering and Information Science at the University of Science and Technology of China, Hefei 230027, China. Her research interests include biomedical signal processing, neuroimaging analysis, brain stimulation, and brain connectivity modeling. She is a Member of IEEE.
Rabab K. Ward (rababw@ece.ubc.ca) received her Ph.D. degree in electrical engineering and computer science from the University of California, Berkeley, USA in 1972. She is a professor emeritus in the Department of Electrical and Computer Engineering at the University of British Columbia, Vancouver, BC V6T 1Z4, Canada. Her research interests are mainly in the areas of signal, image, and video processing. She was the past president of the IEEE Signal Processing Society (2016–2017) and was the past IEEE director (Division IX). She is a Fellow of the Royal Society of Canada, the IEEE, the National Academy of Engineering, the Canadian Academy of Engineers, and the Engineering Institute of Canada.
Z. Jane Wang (zjanew@ece.ubc.ca) received her Ph.D. degree in electrical engineering from the University of Connecticut. She is a full professor in the Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC V6T 1Z4, Canada. Her research interests include statistical signal processing and machine learning, with applications in digital media and biomedical data analytics. She is a Fellow of IEEE and the Canadian Academy of Engineers.
Xun Chen (xunchen@ustc.edu.cn) received his B.Sc. degree in electrical engineering from the University of Science and Technology of China in 2009 and his Ph.D. degree in biomedical engineering from the University of British Columbia, Canada in 2014. He has been a research scientist in the Department of Electrical and Computer Engineering at the University of British Columbia. Currently, he is a full professor and the head of the Department of Electrical Engineering and Information Science, University of Science and Technology of China, Hefei 230027, China. His research interests include the broad areas of statistical signal processing and machine learning in biomedical applications. He is a Senior Member of IEEE.
[1] H. Berger, “Über das elektroenkephalogramm des menschen,” Archiv Psychiatrie Nervenkrankheiten, vol. 87, no. 1, pp. 527–570, Apr. 1929, doi: 10.1007/BF01797193.
[2] W. Wyrwicka and M. B. Sterman, “Instrumental conditioning of sensorimotor cortex EEG spindles in the waking cat,” Physiol. Behav., vol. 3, no. 5, pp. 703–707, Sep. 1968, doi: 10.1016/0031-9384(68)90139-X.
[3] U. Chaudhary, N. Birbaumer, and A. Ramos-Murguialday, “Brain–computer interfaces for communication and rehabilitation,” Nature Rev. Neurol., vol. 12, no. 9, pp. 513–525, Sep. 2016, doi: 10.1038/nrneurol.2016.113.
[4] J. J. Vidal, “Toward direct brain-computer communication,” Annu. Rev. Biophys. Bioeng., vol. 2, no. 1, pp. 157–180, Jun. 1973, doi: 10.1146/annurev.bb.02.060173.001105.
[5] L. A. Farwell and E. Donchin, “Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials,” Electroencephalogr. Clin. Neurophysiol., vol. 70, no. 6, pp. 510–523, Dec. 1988, doi: 10.1016/0013-4694(88)90149-6.
[6] B.-K. Min, R. Chavarriaga, and J. D. R. Millán, “Harnessing prefrontal cognitive signals for brain-machine interfaces,” Trends Biotechnol., vol. 35, no. 7, pp. 585–597, Jul. 2017, doi: 10.1016/j.tibtech.2017.03.008.
[7] H. Yuan and B. He, “Brain–computer interfaces using sensorimotor rhythms: Current state and future perspectives,” IEEE Trans. Biomed. Eng., vol. 61, no. 5, pp. 1425–1435, May 2014, doi: 10.1109/TBME.2014.2312397.
[8] B. Z. Allison, C. Brunner, V. Kaiser, G. R. Müller-Putz, C. Neuper, and G. Pfurtscheller, “Toward a hybrid brain–computer interface based on imagined movement and visual attention,” J. Neural Eng., vol. 7, no. 2, Apr. 2010, Art. no. 026007, doi: 10.1088/1741-2560/7/2/026007.
[9] Y. Kwak, W.-J. Song, and S.-E. Kim, “FGANet: fNIRS-guided attention network for hybrid EEG-fNIRS brain-computer interfaces,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 30, pp. 329–339, Feb. 2022, doi: 10.1109/TNSRE.2022.3149899.
[10] S. Ryun, J. S. Kim, S. H. Lee, S. Jeong, S.-P. Kim, and C. K. Chung, “Movement type prediction before its onset using signals from prefrontal area: An electrocorticography study,” BioMed Res. Int., vol. 2014, Jul. 2014, Art. no. 783203, doi: 10.1155/2014/783203.
[11] P. W. Ferrez and J. D. R. Millán, “Simultaneous real-time detection of motor imagery and error-related potentials for improved BCI accuracy,” in Proc. 4th Int. Brain-Comput. Interface Workshop Training Course, 2008, pp. 197–202.
[12] X. Gao, Y. Wang, X. Chen, and S. Gao, “Interface, interaction, and intelligence in generalized brain–computer interfaces,” Trends Cogn. Sci., vol. 25, no. 8, pp. 671–684, Aug. 2021, doi: 10.1016/j.tics.2021.04.003.
[13] K. D. Harris, R. Q. Quiroga, J. Freeman, and S. L. Smith, “Improving data quality in neuronal population recordings,” Nature Neurosci., vol. 19, no. 9, pp. 1165–1174, Sep. 2016, doi: 10.1038/nn.4365.
[14] Z. Nenadic and J. W. Burdick, “Spike detection using the continuous wavelet transform,” IEEE Trans. Biomed. Eng., vol. 52, no. 1, pp. 74–87, Jan. 2005, doi: 10.1109/TBME.2004.839800.
[15] H. G. Rey, C. Pedreira, and R. Q. Quiroga, “Past, present and future of spike sorting techniques,” Brain Res. Bull., vol. 119, no. Pt B, pp. 106–117, Oct. 2015, doi: 10.1016/j.brainresbull.2015.04.007.
[16] M. Radmanesh, A. A. Rezaei, M. Jalili, A. Hashemi, and M. M. Goudarzi, “Online spike sorting via deep contractive autoencoder,” Neural Netw., vol. 155, pp. 39–49, Aug. 2022, doi: 10.1016/j.neunet.2022.08.001.
[17] M. C. van Rossum, “A novel spike distance,” Neural Comput., vol. 13, no. 4, pp. 751–763, Apr. 2001, doi: 10.1162/089976601300014321.
[18] F. R. Willett, D. T. Avansino, L. R. Hochberg, J. M. Henderson, and K. V. Shenoy, “High-performance brain-to-text communication via handwriting,” Nature, vol. 593, no. 7858, pp. 249–254, May 2021, doi: 10.1038/s41586-021-03506-2.
[19] C. Rossant et al., “Spike sorting for large, dense electrode arrays,” Nature Neurosci., vol. 19, no. 4, pp. 634–641, Apr. 2016, doi: 10.1038/nn.4268.
[20] X. Chen, X. Xu, A. Liu, M. J. McKeown, and Z. J. Wang, “The use of multivariate EMD and CCA for denoising muscle artifacts from few-channel EEG recordings,” IEEE Trans. Instrum. Meas., vol. 67, no. 2, pp. 359–370, Feb. 2018, doi: 10.1109/TIM.2017.2759398.
[21] A. Savitzky and M. J. Golay, “Smoothing and differentiation of data by simplified least squares procedures,” Anal. Chem., vol. 36, no. 8, pp. 1627–1639, Jul. 1964, doi: 10.1021/ac60214a047.
[22] U. Schmitt, A. K. Louis, F. Darvas, H. Buchner, and M. Fuchs, “Numerical aspects of spatio-temporal current density reconstruction from EEG-/MEG-data,” IEEE Trans. Med. Imag., vol. 20, no. 4, pp. 314–324, Apr. 2001, doi: 10.1109/42.921480.
[23] D. Acharya, A. Rani, S. Agarwal, and V. Singh, “Application of adaptive Savitzky–Golay filter for EEG signal processing,” Perspectives Sci., vol. 8, pp. 677–679, Sep. 2016, doi: 10.1016/j.pisc.2016.06.056.
[24] P. Gajbhiye, N. Mingchinda, W. Chen, S. C. Mukhopadhyay, T. Wilaiprasitporn, and R. K. Tripathy, “Wavelet domain optimized Savitzky–Golay filter for the removal of motion artifacts from EEG recordings,” IEEE Trans. Instrum. Meas., vol. 70, pp. 1–11, 2021, doi: 10.1109/TIM.2020.3041099.
[25] S. Agarwal, A. Rani, V. Singh, and A. P. Mittal, “EEG signal enhancement using cascaded S-Golay filter,” Biomed. Signal Process. Control, vol. 36, pp. 194–204, Jul. 2017, doi: 10.1016/j.bspc.2017.04.004.
[26] D. J. McFarland, L. M. McCane, S. V. David, and J. R. Wolpaw, “Spatial filter selection for EEG-based communication,” Electroencephalogr. Clin. Neurophysiol., vol. 103, no. 3, pp. 386–394, Sep. 1997, doi: 10.1016/S0013-4694(97)00022-2.
[27] A. T. Sözer and C. B. Fidan, “Novel spatial filter for SSVEP-based BCI: A generated reference filter approach,” Comput. Biol. Med., vol. 96, pp. 98–105, May 2018, doi: 10.1016/j.compbiomed.2018.02.019.
[28] Z. Y. Chin, K. K. Ang, C. Wang, C. Guan, and H. Zhang, “Multi-class filter bank common spatial pattern for four-class motor imagery BCI,” in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., 2009, pp. 571–574, doi: 10.1109/IEMBS.2009.5332383.
[29] H. Ramoser, J. Muller-Gerking, and G. Pfurtscheller, “Optimal spatial filtering of single trial EEG during imagined hand movement,” IEEE Trans. Rehabil. Eng., vol. 8, no. 4, pp. 441–446, Jan. 2001, doi: 10.1109/86.895946.
[30] S. Lemm, B. Blankertz, G. Curio, and K.-R. Muller, “Spatio-spectral filters for improving the classification of single trial EEG,” IEEE Trans. Biomed. Eng., vol. 52, no. 9, pp. 1541–1548, Oct. 2005, doi: 10.1109/TBME.2005.851521.
[31] Q. Novi, C. Guan, T. H. Dat, and P. Xue, “Sub-band common spatial pattern (SBCSP) for brain-computer interface,” in Proc. 3rd Int. IEEE/EMBS Conf. Neural Eng., 2007, pp. 204–207, doi: 10.1109/CNE.2007.369647.
[32] K. K. Ang, Z. Y. Chin, H. Zhang, and C. Guan, “Filter bank common spatial pattern (FBCSP) in brain-computer interface,” in Proc. IEEE Int. Joint Conf. Neural Netw. (IEEE World Congr. Comput. Intell.), 2008, pp. 2390–2397, doi: 10.1109/IJCNN.2008.4634130.
[33] H. Kang, Y. Nam, and S. Choi, “Composite common spatial pattern for subject-to-subject transfer,” IEEE Signal Process. Lett., vol. 16, no. 8, pp. 683–686, Aug. 2009, doi: 10.1109/LSP.2009.2022557.
[34] F. Lotte and C. Guan, “Regularizing common spatial patterns to improve BCI designs: Unified theory and new algorithms,” IEEE Trans. Biomed. Eng., vol. 58, no. 2, pp. 355–362, Feb. 2011, doi: 10.1109/TBME.2010.2082539.
[35] X. Chen, Z. J. Wang, and M. McKeown, “Joint blind source separation for neurophysiological data analysis: Multiset and multimodal methods,” IEEE Signal Process. Mag., vol. 33, no. 3, pp. 86–107, May 2016, doi: 10.1109/MSP.2016.2521870.
[36] W. De Clercq, A. Vergult, B. Vanrumste, W. Van Paesschen, and S. Van Huffel, “Canonical correlation analysis applied to remove muscle artifacts from the electroencephalogram,” IEEE Trans. Biomed. Eng., vol. 53, no. 12, pp. 2583–2587, Nov. 2006, doi: 10.1109/TBME.2006.879459.
[37] Z. Lin, C. Zhang, W. Wu, and X. Gao, “Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs,” IEEE Trans. Biomed. Eng., vol. 53, no. 12, pp. 2610–2614, May 2007, doi: 10.1109/TBME.2006.886577.
[38] Y. Zhang, G. Zhou, J. Jin, X. Wang, and A. Cichocki, “Frequency recognition in SSVEP-based BCI using multiset canonical correlation analysis,” Int. J. Neural Syst., vol. 24, no. 4, Jun. 2014, Art. no. 1450013, doi: 10.1142/S0129065714500130.
[39] B. Xu et al., “Wavelet transform time-frequency image and convolutional network-based motor imagery EEG classification,” IEEE Access, vol. 7, pp. 6084–6093, Dec. 2018, doi: 10.1109/ACCESS.2018.2889093.
[40] E. Donchin, K. M. Spencer, and R. Wijesinghe, “The mental prosthesis: Assessing the speed of a P300-based brain-computer interface,” IEEE Trans. Rehabil. Eng., vol. 8, no. 2, pp. 174–179, Jun. 2000, doi: 10.1109/86.847808.
[41] W. Ting, Y. Guo-Zheng, Y. Bang-Hua, and S. Hong, “EEG feature extraction based on wavelet packet decomposition for brain computer interface,” Measurement, vol. 41, no. 6, pp. 618–625, Jul. 2008, doi: 10.1016/j.measurement.2007.07.007.
[42] D. Delisle-Rodriguez et al., “System based on subject-specific bands to recognize pedaling motor imagery: Towards a BCI for lower-limb rehabilitation,” J. Neural Eng., vol. 16, no. 5, Jul. 2019, Art. no. 056005, doi: 10.1088/1741-2552/ab08c8.
[43] S. Kumar, A. Sharma, and T. Tsunoda, “Subject-specific-frequency-band for motor imagery EEG signal recognition based on common spatial spectral pattern,” in Proc. Trends Artif. Intell., 16th Pacific Rim Int. Conf. Artif. Intell., Cham, Switzerland: Springer-Verlag, 2019, pp. 712–722.
[44] Y. Yang, S. Chevallier, J. Wiart, and I. Bloch, “Subject-specific time-frequency selection for multi-class motor imagery-based BCIS using few Laplacian EEG channels,” Biomed. Signal Process. Control, vol. 38, pp. 302–311, Sep. 2017, doi: 10.1016/j.bspc.2017.06.016.
[45] M. Huang, P. Wu, Y. Liu, L. Bi, and H. Chen, “Application and contrast in brain-computer interface between Hilbert-Huang transform and wavelet transform,” in Proc. 9th IEEE Int. Conf. Young Comput. Scientists, 2008, pp. 1706–1710, doi: 10.1109/ICYCS.2008.537.
[46] Z. Wu and N. E. Huang, “Ensemble empirical mode decomposition: A noise-assisted data analysis method,” Adv. Adaptive Data Anal., vol. 1, no. 1, pp. 1–41, 2009, doi: 10.1142/S1793536909000047.
[47] N. Rehman and D. P. Mandic, “Multivariate empirical mode decomposition,” Proc. Roy. Soc. A, Math., Physical Eng. Sci., vol. 466, no. 2117, pp. 1291–1302, 2010, doi: 10.1098/rspa.2009.0502.
[48] F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, and B. Arnaldi, “A review of classification algorithms for EEG-based brain–computer interfaces,” J. Neural Eng., vol. 4, no. 2, Jan. 2007, Art. no. R1, doi: 10.1088/1741-2560/4/2/R01.
[49] D. J. Krusienski et al., “A comparison of classification techniques for the P300 speller,” J. Neural Eng., vol. 3, no. 4, Jan. 2007, Art. no. 299, doi: 10.1088/1741-2560/3/4/007.
[50] M. Fatourechi, R. Ward, and G. Birch, “A self-paced brain–computer interface system with a low false positive rate,” J. Neural Eng., vol. 5, no. 1, Dec. 2007, Art. no. 9, doi: 10.1088/1741-2560/5/1/002.
[51] K. Zhang et al., “Data augmentation for motor imagery signal classification based on a hybrid neural network,” Sensors, vol. 20, no. 16, Aug. 2020, Art. no. 4485, doi: 10.3390/s20164485.
[52] F. Fahimi, S. Dosen, K. K. Ang, N. Mrachacz-Kersting, and C. Guan, “Generative adversarial networks-based data augmentation for brain–computer interface,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 9, pp. 4039–4051, Sep. 2021, doi: 10.1109/TNNLS.2020.3016666.
[53] F. Lotte et al., “A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update,” J. Neural Eng., vol. 15, no. 3, Jun. 2018, Art. no. 031005, doi: 10.1088/1741-2552/aab2f2.
[54] X. Chen, C. Li, A. Liu, M. J. McKeown, R. Qian, and Z. J. Wang, “Toward open-world electroencephalogram decoding via deep learning: A comprehensive survey,” IEEE Signal Process. Mag., vol. 39, no. 2, pp. 117–134, Mar. 2022, doi: 10.1109/MSP.2021.3134629.
[55] C. Vidaurre, A. Schlogl, R. Cabeza, R. Scherer, and G. Pfurtscheller, “Study of on-line adaptive discriminant analysis for EEG-based brain computer interfaces,” IEEE Trans. Biomed. Eng., vol. 54, no. 3, pp. 550–556, Mar. 2007, doi: 10.1109/TBME.2006.888836.
[56] C. Vidaurre, M. Kawanabe, P. von Bünau, B. Blankertz, and K.-R. Müller, “Toward unsupervised adaptation of LDA for brain–computer interfaces,” IEEE Trans. Biomed. Eng., vol. 58, no. 3, pp. 587–597, 2011, doi: 10.1109/TBME.2010.2093133.
[57] S. Raghu, N. Sriraam, Y. Temel, S. V. Rao, and P. L. Kubben, “EEG based multi-class seizure type classification using convolutional neural network and transfer learning,” Neural Netw., vol. 124, pp. 202–212, Jan. 2020, doi: 10.1016/j.neunet.2020.01.017.
[58] C. Vidaurre, C. Sannelli, K.-R. Müller, and B. Blankertz, “Co-adaptive calibration to improve BCI efficiency,” J. Neural Eng., vol. 8, no. 2, Mar. 2011, Art. no. 025009, doi: 10.1088/1741-2560/8/2/025009.
[59] U. Chaudhary, N. Mrachacz-Kersting, and N. Birbaumer, “Neuropsychological and neurophysiological aspects of brain-computer-interface (BCI) control in paralysis,” J. Physiol., vol. 599, no. 9, pp. 2351–2359, May 2021, doi: 10.1113/JP278775.
Digital Object Identifier 10.1109/MSP.2023.3278074