Selin Aviyente, Alejandro F. Frangi, Erik Meijering, Arrate Muñoz-Barrutia, Michael Liebling, Dimitri Van De Ville, Jean-Christophe Olivo-Marin, Jelena Kovačević, Michael Unser
©SHUTTERSTOCK.COM/TRIFF
The Bio Image and Signal Processing (BISP) Technical Committee (TC) of the IEEE Signal Processing Society (SPS) promotes activities within the broad technical field of biomedical image and signal processing. Areas of interest include medical and biological imaging, digital pathology, molecular imaging, microscopy, and associated computational imaging, image analysis, and image-guided treatment, alongside physiological signal processing, computational biology, and bioinformatics.
BISP has 40 members and covers a wide range of Editors Information Classification Scheme, including CIS-MI: medical imaging; BIO-MIA: medical image analysis; BIO-BI: biological imaging; BIO: biomedical signal processing; BIO-BCI: brain/human-computer interfaces; and BIO-INFR: bioinformatics. BISP plays a central role in the organization of the IEEE International Symposium on Biomedical Imaging (ISBI) and contributes to the technical sessions at the ICASSP and the ICIP. In this article, we provide a brief history of the TC, review the technological and methodological contributions its community delivered, and highlight promising new directions we anticipate.
Until 2002, the signal processing activities related to biomedical imaging were overseen by the Image and Multidimensional Digital Signal Processing Committee of the SPS and typically presented in topical sessions at ICIP and ICASSP. The SPS also cosponsored IEEE Transactions on Medical Imaging. Yet, at the turn of the century, the importance of imaging in medicine and biology was becoming increasingly apparent. At the same time, advanced signal processing played an ever increasing role in the reconstruction and analysis of the vast volume of images produced. This realization was reinforced by the creation of the National Institute of Bioimaging and Bioengineering (NIBIB) by the U.S. National Institutes of Health (NIH) and U.S. Congress in December 2000 as an agency solely dedicated to the advancement of imaging technology and bioengineering. The latter was an official recognition of the crucial role of engineering in biomedical research and of the necessity to fund such research activities. This motivated the SPS and IEEE Engineering in Medicine and Biology Society (EMBS) to join forces and demonstrate leadership in biomedical imaging research.
Accordingly, it was decided to launch a new regular meeting on biomedical imaging: the ISBI (Figure 1), in close collaboration with NIBIB. The task of organizing this conference was given to Michael Unser (SPS representative) and Zhi-Pei Liang (EMBS representative). The fact that Prof. Unser had spent the larger part of his career at the NIH facilitated the interaction with NIBIB, which committed to supporting the first edition of ISBI that took place in Washington, DC, USA, in July 2002. The unique aspect of ISBI was to cover the whole spectrum and range of imaging, from nano (electron and optical microscopy) to macro (medical imaging modalities) [1].
Figure 1. The original ISBI logo designed by Annette Unser (graphic artist and sister of the founding chair), with the eye projecting a distinctive vision for ISBI. Observers have suggested that the central motif illustrates the Fourier slice theorem, or for the more pessimistic ones, the typical artifacts of the filtered back-projection reconstruction algorithm.
With the creation of ISBI and its establishment as the IEEE flagship conference in biomedical imaging, the next step was to put in place a structure to promote the conference and ensure its scientific quality. Since Prof. Unser with his team had formulated the vision for ISBI, he was instructed to form the SPS BISP TC and to make suggestions for its initial membership. In addition to its strategic role in bioimaging, BISP was given the mission to oversee the SPS activities in biomedical signal processing (e.g., the analysis of physiological signals) and bioinformatics—in short, to be responsible for all signal processing activities in medicine and biology and to maintain a liaison with its sister TC in EMBS, the Technical Community on Biomedical Imaging and Image Processing. Since the inception of the TC, BISP members have also actively participated in cross-Society activities, such as the IEEE Life Science Technical Community and the IEEE Brain Technical Community.
The inaugural ISBI was held between 7 and 10 July 2002, at the Ritz-Carlton Hotel, Washington, DC. The meeting was organized jointly by the SPS and the EMBS. Significant support was provided by both the NIH and NIBIB (US$40,000 in grants and approximately 50 paid registrants). The conference was a huge success, providing a venue for interdisciplinary exchange with researchers from both medical and biological imaging areas. It was also well attended by NIH representatives. Dr. Elias Zerhouni gave the opening address as the then newly appointed NIH director. This attracted many observers, including members of the press. Dr. Roderic Pettigrew also delivered a speech—the very first in his new function as the director of NIBIB. Both directors expressed a strong interest in the conference and commented on the need to strengthen the links between the engineering and biomedical research communities.
ISBI 2002 had two parts to the scientific program: 1) the contributed papers reviewed by the technical program committee and 2) the invited papers. Out of 355 submitted papers, 73 were accepted for oral presentation and 142 for poster presentation. The invited program consisted of 10 special sessions that were organized by leading researchers in the field.
Despite a very active and growing community, BISP membership has increased only moderately, yet the TC has always strived for a well-balanced representation across the broad range of subfields it covers. A key task for BISP members was to ensure that papers submitted to ISBI (or to dedicated tracks at ICASSP and ICIP) would benefit from the availability of a highly qualified pool of reviewers and editors. BISP members also participated in the many activities related to ISBI as members of the organizing committee. Since 2006, ISBI has been held regularly as an annual four-day conference. Figure 2 summarizes with a word cloud the keywords from the keynote titles since ISBI’s inception. Outstanding clinical and technical speakers delivered their visions for the field, relevant trends, or challenges ahead. Among our distinguished speakers, there were Nobel Prize winners and top NIH officers.
Figure 2. A word cloud with the most frequent keywords from the titles of all the ISBI keynotes in the past 20 years. Clinicians, Nobel Prize winners, and NIH officials have given these talks, among other contributors.
In 2022, ISBI was held for the first time as a fully hybrid conference in Kolkata, India, with every session having both physical and online speakers and audiences. Out of 785 submitted papers, 309 were accepted. In addition to the regular paper sessions, there were five special sessions, five plenary talks, six challenges, and six tutorials. In addition to ISBI, BISP has been an active contributor to ICASSP since 2006, with the number of submitted papers increasing from 100 in 2006 to 222 in 2020.
Biomedical data come in many shapes and forms. BISP focuses on digital images and signals, which can be automatically processed and analyzed by advanced computational methods. To study biological processes in health and disease, many images and signal acquisition techniques have been developed in the past century, reflecting the fact that biological phenomena occur at different spatial and temporal scales (Figure 3). Before discussing methodological advances, we survey some of the most prominent modalities for molecular and cellular imaging, tissue imaging, anatomical and functional medical imaging, neuroimaging, physiological signal recording, and several data types in bioinformatics.
Figure 3. Examples of the many data acquisition modalities in biomedical image and signal processing operating at various spatial scales. BFM, bright-field microscopy; CT, computed tomography; ECG, electrocardiography; EEG, electroencephalography; EM: electromagnetic; EMG, electromyography; FCM, flow cytometry; FM, fluorescence microscopy; GE, gene expression; MA, microarray; MRI, magnetic resonance imaging; US, ultrasound; WSI, whole-slide imaging.
Molecular and cellular imaging has undergone multiple revolutions in the past three decades, moving from a mainly qualitative to a mostly quantitative field thanks to advances in molecular probes as well as imaging modalities [2], [3], [4]. With the advent of the green fluorescent protein, pioneered by Osamu Shimomura, Martin Chalfie, and Roger Y. Tsien (Nobel Prize in Chemistry, 2008), microscopy has become one of the key tools in biological research [3]. Fluorescence microscopy became a fast-growing field to study (quantitatively and often within a high-throughput content setup) processes and organelles within living cells and organisms. More recently, a vast leap has been made with the invention of superresolution techniques, based on seminal work by Eric Betzig, Stefan W. Hell, and William E. Moerner (Nobel Prize in Chemistry, 2014).
Another recent development is selective plane illumination (light sheet) microscopy, which allows long-term biological studies of living organisms with rapid acquisition, high resolution, and minimal phototoxicity. Classical image and signal processing methods as well as modern deep learning-based methods are increasingly used not only for reconstruction and deconvolution of the data produced by advanced microscopy imaging modalities but also for enabling downstream tasks such as segmentation, classification, tracing, and tracking [4], [5], [6]. Fluorescence microscopy has enabled the study of dynamic processes within cells and complements structural and static imaging modalities, such as scanning probe microscopy, electron microscopy (Nobel Prize in Physics, 1986), and cryo-electron microscopy (Nobel Prize in Chemistry, 2017), which have become part of the vast arsenal of tools for the life sciences. In recent years, several new journals or sections in established publications, e.g., Biological Imaging, Frontiers in Bioinformatics, and Cell Reports Methods, have been launched to host the increasing number of publications in this domain.
Microscopy can also be employed to study biological phenomena at the tissue level. Rather than imaging individual cells (molecular/cellular imaging) for basic research or anatomical structures and entire organs (medical imaging) for clinical diagnostics, the imaging of tissue slides prepared from biopsies enables characterizing and grading disease processes ex vivo as revealed by abnormal cell arrangements and tissue architectures. This is especially important in researching and diagnosing pathologies, notably the many types of cancer (the field of oncology), known to manifest themselves first at the cell and tissue level (histopathology). Recent advances in digital whole-slide imaging (WSI) systems (sometimes referred to as virtual microscopy) have created unprecedented opportunities for computer-aided diagnosis in histopathology. Both image and signal processing play a prominent role in histopathological image analysis, especially for breast cancer, prostate cancer, lung cancer, tumor pathology in many other forms of cancer, and cancer prognosis.
Review papers have summarized and commented on the challenges and opportunities in this domain [7], [8]. Typical tasks include the detection and segmentation of cell nuclei, glands, and lymphocytes and computing various quantitative morphological features for classification. This, in turn, requires effective techniques for image normalization as well as feature selection and dimensionality reduction. Analysis of the spatial arrangements of tissues is often facilitated by graph-based representation and topological modeling. The challenges in histopathological image analysis are not only due to the high complexities of the image structures but also to the typically large image sizes, on the order of tens of thousands by tens of thousands of pixels, at multiple magnifications.
Traditionally, tissue classification has been performed using handcrafted features and machine learning methods, such as support vector machines and random forests. Still, there is now growing evidence that deep artificial neural networks (NNs) provide fast and reliable image analysis on a par with seasoned pathologists and can serve as a synergistic tool for the latter to improve accuracy and throughput. However, the full adoption of deep learning methods in pathology is hindered by the lack of large and reliably annotated image cohorts documenting the large diversity of diseases and the high variability of disease traits, calling for efficient automated annotation methods.
Medical imaging refers to the imaging techniques and processes to gain insights into the interior of a body for clinical diagnosis or medical intervention as well as visual representations of the function of organs or tissues. Medical imaging can be divided into structural or anatomical imaging and functional or physiological imaging. Many medical imaging techniques have been invented since the discovery of X-rays by Wilhelm Conrad Röntgen in 1895, which form the basis of projection X-rays and computed tomography (CT). In 1946, Bloch and Purcell (Nobel Prize in Physics, 1952) independently discovered nuclear magnetic resonance, which formed the basis of MRI (Paul Lauterbur and Sir Peter Mansfield in the 1970s, Nobel Prize in Physiology or Medicine, 2003). MRI developed into a platform technology with many specialized techniques, providing insights into anatomy, perfusion, diffusion, and deformation. Ultrasound (US) was used in medicine since World War II, but it was not until the late 1970s that US imaging was popularized as a clinical imaging modality.
In 1963, David Kuhl and Roy Edwards introduced emission reconstruction tomography, a method that later became single-photon emission CT (SPECT). Sir Godfrey Hounsfield (Nobel Prize in Physiology or Medicine, 1979) developed the first prototype of a CT scanner in 1963, thanks to the availability of modern computers to solve the complex image reconstruction problems that ensued. Michael Hoffman, Michel Ter-Pogossian, and Michael E. Phelps built the first positron emission tomography (PET) camera in 1974. Underpinning these techniques, there are considerable signal and image processing problems, ranging from image reconstruction, image deconvolution, image denoising and restoration, image transformation, and multimodal image coregistration. Over the last three to four decades, several signal processing developments had major impacts on medical imaging. For example, wavelets and splines played a major role in medical image interpolation, denoising, and filtering. Mutual information and other information-theoretic metrics revolutionized multimodal image registration. Compressed sensing (CS) provided a novel approach to find solutions to underdetermined linear equations with a major impact on image reconstruction from projections (CT), from k-space (MRI), or from sensors (US), and, of course, the latest developments of deep learning and their impact across the board.
Another flourishing outlet for biomedical signal and image processing has been the processing of structural and functional MRI (fMRI) data [9]. The concept of establishing connectivity between brain regions has been fundamental in many emerging methodologies [10] (Figure 4). Structural connectivity is defined by the strength of interregional axonal fiber pathways that can be revealed using tractography methods applied to diffusion-weighted MRI (dMRI) data. Functional connectivity relates to the statistical interdependency between two time series of blood oxygenation level-dependent (BOLD) activity. When established for all possible pairs of regions, defined by a brain atlas, this leads to the structural and functional connectomes, respectively. While hemodynamic imaging has been the most commonly used modality for constructing the functional connectome, neurophysiological signals such as magneto/electroencephalography (M/EEG) have also been adopted thanks to their high temporal resolution.
Figure 4. Structural and functional connectomes play a key role in representing relationships between brain regions. (a) From diffusion-weighted MRI (dMRI), the orientation of axonal bundles in white matter can be extracted and processed by tractography to obtain the strength of structural connectivity between all pairs of regions. (b) Functional MRI (fMRI) provides a series of volumes where the blood oxygenation level-dependent (BOLD) signal is related to neuronal activity. Time series analysis exists in large diversity, but the functional connectome that reflects the statistical interdependencies between pairs of time series is one of them.
Both model-based and data-driven methods have been developed to quantify functional connectivity, including multivariate autoregressive models, graphical models, phase synchrony, and information-theoretic metrics. Functional connectivity is also intimately related to blind source separation. This, in turn, relies on decomposing the data matrix into components driven by maximizing covariance [using techniques such as principal component analysis, singular value decomposition (SVD), or higher order SVD] or statistical independence (using independent component analysis), which has become part of the pipeline in fMRI software suites. During the past decade, dynamic functional connectivity has been introduced to acknowledge the changing patterns of cofluctuations, either by sliding-window functional connectivity, instantaneous activation patterns, or autoregressive models.
The resulting connectomes are commonly represented by graphs and analyzed to reveal organizational principles using, for instance, local clustering coefficients, efficiency, small-worldness, centrality, and phenotype behavior and disorder. These approaches have been applied to other species’ connectomes obtained using different modalities, leading to a new field, network neuroscience, which further branched out to machine learning, information theory, and computational neuroscience. Finally, the emergence of graph signal processing has found its way into the neuroimaging field [11], providing a way to combine brain structure (i.e., a graph defined by the structural connectome) and brain function (i.e., graph signals obtained by fMRI snapshots of brain activity).
With the advent of wearable sensors, physiological signals are collected for various applications and are central to multiple new technologies, including brain-computer interfacing/human-machine interfacing (HMI), neurorehabilitation, and neuroprosthetics, in addition to medical diagnosis and monitoring [12]. The most employed physiological signals include electromyography (EMG), respiration, speech, heart rate variability, photoplethysmography (PPG), electrocardiography (ECG), and M/EEG. Some prominent application areas that routinely rely on physiological signals include emotion recognition, autonomous driving, mental health, and assistive technologies. For example, physiological changes such as heart rate, skin conductance, and PPG signals are monitored for measuring human emotions as they are more reliable and harder to alter compared to explicit behaviors such as facial expressions and speech.
Similarly, the design of HMI systems requires the consistent and accurate decoding of motor intent with minimal training and calibration. The multimodal high-density sensing technology coupled with the nonstationary and nonlinear nature of biological signals requires the development of innovative signal processing and machine learning techniques to process, decompose, and decode these signals. Some methodologies employed in this area of research include blind source separation, time-frequency analysis, multimodal data fusion, supervised (or semisupervised) learning, and deep learning. Different tasks, such as event detection, prediction, and diagnosis, have been addressed using these tools.
Since the turn of the century, major advances in molecular biology, along with advances in genomic data acquisition technologies, led to the growth of biological data generated and shared by the scientific community; e.g., The Cancer Genome Atlas (TCGA). These data bring with themselves significant challenges in the identification of gene expression mechanisms; the determination of proteins encoded by the genes; understanding how these interact, i.e., gene regulatory networks; and marker identification.
BISP has contributed to this area by introducing a new line of research: genomic and proteomic signal processing [13]. While biomolecular sequence analysis has been addressed by computer scientists, physicists, and mathematicians, it was only at the turn of the century that signal processing started to play a role in this area. Genomic and proteomic data can be modeled as noisy, continuous, or discrete signals that represent the molecular structure and activities in cells. The high dimensionality, variability, and complexity of these data require the development of new signal processing methodologies that effectively deal with these challenges. By mapping the character strings corresponding to gene sequences into numerical sequences, signal processing offers a set of tools for solving highly relevant problems. For example, the magnitude and the phase of properly defined Fourier transforms can be used to predict important properties of protein-coding regions in DNA.
Similarly, concepts from digital filtering can be employed to analyze the mapping of DNA into proteins and the interdependence of two sequences. These and other signal processing methodologies, such as frequency domain analysis, high-dimensional data analysis, CS, and network inference, have played important roles in the advancement of this field. Genomic and proteomic signal processing both have had a substantial impact on different application areas, including sequence analysis, microarray analysis, structure identification, and regulatory networks.
From 2005 to 2013, the IEEE International Workshop on Genomic Signal Processing and Statistics (GENSiPS) was organized annually and sponsored by the SPS. These workshops covered topics related to high-dimensional genomic data analysis, gene regulatory network inference, marker identification, drug screening, and proteomics.
The field of biomedical image and signal processing has seen major methodological advances not only in how data are recorded, stored, and transmitted but also in how they are best represented, processed, analyzed, and modeled, depending on the application domain. Many paradigms have been proposed in recent decades by various schools of thought, resulting in a wide range of theories and methods for challenging problems, such as image and signal restoration, reconstruction, detection, segmentation, classification, pattern recognition, and statistical analysis, as documented in numerous textbooks and reviews. Given the limited space in this article, we only briefly discuss some of the most impactful developments in recent years, including methods for computational imaging and deep learning-based image and signal analysis and efforts to stimulate reproducible research.
Most biomedical imaging modalities have a strong computational component as they systematically rely on signal processing to reconstruct the images from the raw imaging data. The data can take the form of 1) 2D projections of a 3D object, as in X-ray tomography, PET, and cryo-electron microscopy; 2) a series of blurred 2D slices of a specimen, as in fluorescence microscopy; or 3) samples of the Fourier transform of an object, as in MRI and optical diffraction tomography. By capitalizing on the knowledge of the imaging physics (linear forward model), the reconstruction task can then be formulated as an inverse problem. Until recently, classical imaging (MRI and CT) relied on a direct inversion of this forward model. This is achieved, for instance, by inverse Fourier transformation in MRI (with uniform sampling in k-space) or by inverse radon transformation (the celebrated filtered back projection algorithm) in CT. This works well when the measurements are sufficiently numerous and diverse and when the noise is negligible.
Besides the streamlining of the reconstruction process itself (improved nonuniform fast Fourier transform, optimization of sampling parameters, etc.), the earlier involvement of the signal processing community was to combat the effect of noise with the help of advanced statistical methods. One notable example of such success is the method of ordered subsets in PET and SPECT [15]. Another fruitful approach inspired by Wiener filtering is to inject prior information in a stochastic model (e.g., generalized Gaussian in a transformed domain), which makes a direct link between maximum a posteriori (MAP) reconstruction and regularization/energy minimization techniques [16].
The more significant revolution in imaging came with CS with theorists [17], [18] and then experimentalists [19], [20] showing the feasibility of image reconstruction from a reduced set of measurements. A milestone in this line of research was the development of efficient minimization methods under sparsity constraints, in particular the (fast) iterative soft thresholding algorithm and alternating direction method of multipliers [21]. The main benefit of CS is to enable faster imaging, which reduces not only cost but also radiation exposure (in the case of X-ray or PET/SPECT). This has led to a major revolution in MRI, with fast (CS-based) imaging protocols now offered by most vendors of MRI technology. While CS kept SPS researchers busy from 2005 to 2017, another wave then overtook the field—the incorporation of NNs in the image reconstruction pipeline. This led to further significant improvement in image quality (Figure 5), especially in extreme scenarios, e.g., low signal-to-noise ratio (SNR) and CS [14].
Figure 5. A comparison of tomographic reconstruction algorithms for CS with a reduction of the number of views by seven. (a) Ground truth (high-quality reconstruction from 1,000 views). (b) Conventional reconstruction (filtered back projection) from a subset of 143 views. (c) CS reconstruction using total variation regularization. (d) CS reconstruction using a CNN (FBPConvNet). The middle panel displays the image residuals with the same contrast. The magnified images in the lower panel represent the corresponding region of interest overlaid in (a). (a) Ground truth. (b) Signal-to-noise ratio (SNR) = 24.06 dB. (c) SNR = 29.06 dB. (d) SNR = 35.38 dB. (Source: The figure is adapted from [14].)
While image reconstruction based on convolutional NNs (CNNs) still has shortcomings—CNNs are poorly understood and can behave erratically (lack of stability and hallucination)—they demonstrate the potential for better reconstruction quality [22]. It is noteworthy that it took signal processing pioneers less than a year to tune their new CNN-based methods to the point where they would outperform sparsity-based methods for CS in public imaging challenges by the same margin (typically >4 dB) that the latter had achieved over classical reconstruction during a whole decade of intense research activity. CNNs and learning-based techniques are presently at the center of attention of the research community. Recent trends include the development of more sophisticated iterative reconstruction schemes that rely on CNNs to regularize the solution—as enabled by the plug-and-play framework [23]—as well as the use of deep learning for the resolution of more challenging nonlinear inverse problems such as diffuse optical tomography [24] and diffraction tomography.
Traditionally, methods for image and signal processing have been based on carefully designed mathematical models of the phenomena and anomalies of interest and their translation into efficient rules-based computational algorithms. Illustrative examples of this are mathematical point-spread function models of widefield or confocal microscopes based on physical (optical) principles, serving as the basis for various image restoration methods (in particular, deconvolution) [25] and object detection methods (such as single-molecule localization) [26]. However, as in many other fields, the demand for new and better methods from practitioners in biology and medical diagnostics outstrips the supply of researchers and developers in image and signal processing. That is, there are many more biologists and physicians in the world looking for tools to facilitate their data processing workflows than there are scientists and engineers looking to develop mathematical models and image/signal processing algorithms specifically for biomedical applications.
Moreover, especially in the biomedical field, many image and signal analysis tasks are notoriously difficult to model mathematically due to the complex nature of the problem, the high ambiguity of the data, and the subjectivity of human experts who define the gold standards for interpreting the data. Thus, as imaging and measurement devices improved over the years and the number of potentially automated data processing tasks grew, the need for more generic, data-driven, and learning-based methods also increased.
In the past decade, the rapidly growing availability of large datasets, powerful computing capabilities, and open-access software libraries and frameworks has accelerated the development and adoption of machine learning and deep learning methods in biomedical image and signal processing [6], [27], [28], [29]. These methods show increasingly superior performance in benchmarking studies for various tasks, including reconstruction, restoration, detection, segmentation, classification, and tracking. In particular, deep learning of artificial NNs has become a popular approach for solving data analysis problems where multimodal, multidimensional, and multiparametric datasets need to be jointly processed, posing a clear challenge to traditional analysis methods. For the processing of biomedical images, CNNs in particular have become mainstream, a prominent example of this being the U-Net architecture [30], of which many variants exist for various tasks and applications, such as segmentation (Figure 6).
Figure 6. Examples of cell segmentation using deep NNs in diverse types of microscopy images. From (a) to (h), the images were captured using bright-field microscopy, phase-contrast microscopy (2×), differential interference contrast microscopy, and fluorescence microscopy (4×) and contain distinct types of cells in different spatial arrangements (densities and confluences). The segmentation results are the overlaid colored cell contours (arbitrary colors). These results were produced using a single deep learning framework with a U-Net-like macro-architecture consisting of various layers/blocks whose microarchitectures were optimized automatically using a neural architecture search approach [31]. The examples illustrate the power of deep learning and the level of automation that can be achieved nowadays in optimizing image segmentation results without requiring expert user input, other than manual annotations, to learn from.
For biomedical signal processing, especially for dealing with time series, recurrent NNs such as the long short-term memory unit have seen widespread adoption. However, despite promising results, many challenges remain to be addressed before deep learning solutions can be integrated with full confidence and accountability into the workflows of biomedical practitioners, such as developing ways to incorporate expert knowledge and improving the explainability and generalizability of the models (see the discussion of future directions in the last section).
Reproducing the results presented in a research work can be very challenging. For a computational algorithm, details such as the exact dataset, initialization or termination procedures, and precise parameter values are often omitted in the publication for various reasons. This makes it difficult, if not impossible, for someone else to obtain the same results [32]. In the early 2000s, the need to boost research by implementing reproducible research practices became apparent. Vandewalle et al. [32] published a seminal manuscript in IEEE Signal Processing Magazine in 2009, which defines reproducible research as follows:
“A research work is called reproducible if all information relevant to the work, including, but not limited to text, data, and code, is made available, so that an independent researcher can reproduce the results.”
The authors also distinguish six levels of reproducibility, from Level 5 (an independent researcher can easily reproduce results with at most 15 min of user effort, requiring only standard freely available tools—C compiler, etc.) to Level 0 (an independent researcher cannot reproduce results).
The issue of reproducibility has been raised many times in the past few decades. In the 1980s, there was a growing awareness of poor building on the previous work of others [33]. Published algorithms were frequently evaluated with only select data and ad hoc metrics. The comparison of algorithms and software performance was difficult. In the 1990s, the first benchmark initiative in biomedical imaging, the Retrospective Image Registration Experiment, appeared at Vanderbilt University [34]. We needed to wait until the early 2010s for bioimaging reference datasets and challenges (benchmarks associated with competitions) to appear. ISBI 2012 in Barcelona was the first edition of the symposium to hold challenges on the following topics:
At ISBI 2015, Prof. Ronneberger’s team won the Cell Tracking Challenge (third edition) [37] and the dental X-ray image segmentation challenge with their U-Net [30]. Figure 7 shows a word cloud of the challenge titles over the years; detection, images, and tracking occupy a prominent place in the cloud.
Figure 7. The ISBI is the premier scientific venue for the BISP TC. Since 2012, our community has organized more than 60 challenges, where open datasets, well-specified tasks, and evaluation metrics have been made available for multiple groups to participate, compete, and learn from each other. Challenges have covered many imaging modalities and scales, image computing tasks, and organ systems.
In the bioimaging community, the push for reproducibility led to several open software platforms, such as Cell Profiler (https://cellprofiler.org/), Fiji (https://fiji.sc/), and Icy (https://icy.bioimageanalysis.org/). They were made available to the community in the early 2000s to share the then state-of-the-art analytical methods, which are now used for integrated deep learning framework deployment [38].
Finally, imaging challenges foster collaboration between institutions and continents. Since 2012, more than 60 challenges have been organized, led by multiple institutions. Of these, 31 were organized by European institutions, 12 by organizations in the Americas, 10 by Asia or Oceania, and six involved cross-continental collaboration from the Americas, Asia, and Europe. These collaborations drove our community to learn from the strengths and pitfalls [39] in organizing challenges and interpreting their results [40] and thus developed best-practice guidelines for transparent reporting [41].
Advanced technologies for capturing biomedical images and signals have made a growing and lasting positive impact on clinical diagnostics and therapeutics, medical research, and life sciences. They will continue to help improve our understanding of the conditions underlying human health and how to prevent and treat disease. Modern biomedical image and signal acquisition systems are based on a wide range of physical phenomena (electricity, magnetism, light, sound, force, etc.) capable of providing complementary information about the anatomical and functional properties of the human body and living organisms in general. Also, the sensitivity, resolution, and quality of these systems have improved dramatically over the years to the point where automated image and signal processing are now indispensable in virtually all clinical and biomedical research applications.
At this point in time, unlike in the past century, the chances of discovering totally new physical principles that could ultimately be used in biomedical practice have diminished, yet the challenges of fully exploiting existing technologies are far from having been solved. One of the main problems for the image and signal processing community in the years ahead will be to develop effective methods for data fusion and integration [42] to maximize the potential of multimodal and correlative imaging as well as combining imaging and nonimaging (e.g., “omics”) data. This requires finding solutions to dealing with the fundamentally different nature of different data sources and the inevitable imbalances in the data but also with the huge volumes (terabytes and no doubt soon petabytes) of multimodal datasets.
Despite being comparatively young, the BISP community has already seen and contributed to major paradigm shifts in biomedical image and signal processing. Still, in addition to the data challenges mentioned previously, many fundamental technical challenges remain. Examples include some of the problems caused by the increasing emphasis on learning-based approaches. For starters, these approaches are typically very data hungry, while the human and time resources to produce high-quality annotated datasets are usually severely limited, especially in the biomedical domain, not to mention additional limiting factors due to privacy regulations. This requires the development of semi/unsupervised learning approaches, data modeling and simulation methods that can generate high-fidelity ground-truth data for training, and ways to integrate expert domain knowledge into the learning framework.
Furthermore, even if sufficient annotated data can be collected to train a machine or deep learning-based method for a given application, the resulting model is considered a black box in the eyes of practitioners, who remain fully accountable for any decisions based on the model’s predictions. Hence, there is a great need for explainable and interpretable machine and deep learning solutions. This is a fantastic opportunity for BISP researchers, many of whom traditionally are used to developing mathematical models based on sound physical principles, which by design are much more explainable and interpretable. Another challenge stemming from limited training data is the typically poor generalizability of the learned models. While organized competitions in the field have done a great service by providing public datasets and benchmarks, it is now well known that models based on them do not always work on private datasets. This calls for continuing efforts to make public datasets less selective and more representative.
Given these and many other open challenges, the BISP TC will continue to play an important role in developing ever more advanced image and signal processing methodologies underpinning the next-generation technologies needed to improve the efficacy of biomedical practice and research. In this endeavor, we believe future advances will come not only from continuing research efforts but also from innovations in education and how we train the next generation of scientists and engineers in our field. Clearly, biomedical image and signal processing has become increasingly multidisciplinary, requiring a deep understanding of not only the mathematics and algorithms of how to model and process digital images and signals but also of the underlying physical principles and limitations of data acquisition using various systems; the biomedical knowledge to properly interpret the data; the data science and informatics expertise to handle large datasets; and the experimental and statistical know-how to validate methods thoroughly. To this end, we envision the BISP TC strengthening ties with the relevant bodies in the respective disciplines and becoming more multidisciplinary in the future.
The authors are the present and past BISP TC Chairs in reverse chronological order.
Selin Aviyente (aviyente@egr.msu.edu) is with the Department of Electrical and Computer Engineering, Michigan State University, East Lansing, MI 48824 USA. She is a Senior Member of IEEE.
Alejandro F. Frangi (a.frangi@leeds.ac.uk) is with the Centre for Computational Imaging and Simulation Technologies in Biomedicine, Schools of Computing and Medicine, University of Leeds, LS2 9JT Leeds, U.K.; the Alan Turing Institute, LS2 9BW London, U.K.; and the Departments of Electrical Engineering and Cardiovascular Sciences, KU Leuven, 3000 Leuven, Belgium. He is a Fellow of IEEE.
Erik Meijering (erik.meijering@unsw.edu.au) is with the School of Computer Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia. He is a Fellow of IEEE.
Arrate Muñoz-Barrutia (mamunozb@ing.uc3m.es) is with the Bioengineering Department, Universidad Carlos III de Madrid, Leganés, 28912 Madrid, Spain. She is a Senior Member of IEEE.
Michael Liebling (michael.liebling@idiap.ch) is with the Idiap Research Institute, 1920 Martigny, Switzerland. He is a Member of IEEE.
Dimitri Van De Ville (dimitri.vandeville@epfl.ch) is with the Neuro-X Institute, School of Engineering, EPFL, 1202 Geneva, Switzerland, and the Faculty of Medicine, University of Geneva, 1202 Geneva, Switzerland. He is a Fellow of IEEE.
Jean-Christophe Olivo-Marin (jcolivo@pasteur.fr) is with the Cell Biology and Infection Department, Institut Pasteur, F-75724 Paris, France. He is a Fellow of IEEE.
Jelena Kovačević (jelenak@nyu.edu) is with the New York University Tandon School of Engineering, Brooklyn, NY 11201 USA. She is a Fellow of IEEE.
Michael Unser (Michael.Unser@epfl.ch) is with the School of Engineering, EPFL, CH-1015 Lausanne, Switzerland. He is a Fellow of IEEE.
[1] M. Unser and Z.-P. Liang, “Guest editorial: First IEEE symposium on biomedical imaging,” IEEE Trans. Med. Imag., vol. 21, no. 8, pp. 850–851, Aug. 2002, doi: 10.1109/TMI.2002.803604.
[2] R. F. Murphy, E. Meijering, and G. Danuser, “Guest editorial: Molecular and cellular bioimaging,” IEEE Trans. Image Process., vol. 14, no. 9, pp. 1233–1236, Sep. 2005, doi: 10.1109/TIP.2005.855701.
[3] J. Kovačević and R. F. Murphy, “Guest editorial: Molecular and cellular bioimaging,” IEEE Signal Process. Mag., vol. 23, no. 3, p. 19, May 2006, doi: 10.1109/MSP.2006.1628874.
[4] A. Muñoz-Barrutia, J. Kovačević, M. Kozubek, E. Meijering, and B. Parvin, “Guest editorial: Quantitative bioimaging: Signal processing in light microscopy,” IEEE Signal Process. Mag., vol. 32, no. 1, pp. 18–19, Jan. 2015, doi: 10.1109/MSP.2014.2359691.
[5] C. Kervrann, S. T. Acton, J.-C. Olivo-Marin, C. Ó. S. Sorzano, and M. Unser, “Introduction to the issue on advanced signal processing in microscopy and cell imaging,” IEEE J. Sel. Topics Signal Process., vol. 10, no. 1, pp. 3–5, Feb. 2016, doi: 10.1109/JSTSP.2015.2511299.
[6] E. Meijering, V. Calhoun, G. Menegaz, D. Miller, and J. Ye, “Guest editorial: Deep learning in biological image and signal processing,” IEEE Signal Process. Mag., vol. 39, no. 2, pp. 24–26, Mar. 2022, doi: 10.1109/MSP.2021.3134525.
[7] M. N. Gurcan, L. Boucheron, A. Can, A. Madabhushi, N. Rajpoot, and B. Yener, “Histopathological image analysis: A review,” IEEE Rev. Biomed. Eng., vol. 2, pp. 147–171, Oct. 2009, doi: 10.1109/RBME.2009.2034865.
[8] X. Zhou et al., “A comprehensive review for breast histopathology image analysis using classical and deep neural networks,” IEEE Access, vol. 8, pp. 90,931–90,956, May 2020, doi: 10.1109/ACCESS.2020.2993788.
[9] T. Adali et al., “Introduction to the issue on fMRI analysis for human brain mapping,” IEEE J. Sel. Topics Signal Process., vol. 2, no. 6, pp. 813–816, Dec. 2008, doi: 10.1109/JSTSP.2008.2009263.
[10] D. Van De Ville, V. Jirsa, S. Strother, J. Richiardi, and A. Zalesky, “Introduction to the issue on advanced signal processing for brain networks,” IEEE J. Sel. Topics Signal Process., vol. 10, no. 7, pp. 1131–1133, Oct. 2016, doi: 10.1109/JSTSP.2016.2602945.
[11] W. Huang, T. A. W. Bolton, J. D. Medaglia, D. S. Bassett, A. Ribeiro, and D. Van De Ville, “A graph signal processing perspective on functional brain imaging,” Proc. IEEE, vol. 106, no. 5, pp. 868–885, May 2018, doi: 10.1109/JPROC.2018.2798928.
[12] D. Farina, A. Mohammadi, T. Adali, N. V. Thakor, and K. N. Plataniotis, “Signal processing for neurorehabilitation and assistive technologies,” IEEE Signal Process. Mag., vol. 38, no. 4, pp. 5–7, Jul. 2021, doi: 10.1109/MSP.2021.3076280.
[13] D. Schonfelda, J. Goutsias, I. Shmulevich, I. Tabus, and A. H. Tewfik, “Introduction to the issue on genomic and proteomic signal processing,” IEEE J. Sel. Topics Signal Process., vol. 2, no. 3, pp. 257–260, Jun. 2008, doi: 10.1109/JSTSP.2008.925864.
[14] K. H. Jin, M. T. McCann, E. Froustey, and M. Unser, “Deep convolutional neural network for inverse problems in imaging,” IEEE Trans. Image Process., vol. 26, no. 9, pp. 4509–4522, Sep. 2017, doi: 10.1109/TIP.2017.2713099.
[15] S. Ahn and J. Fessler, “Globally convergent image reconstruction for emission tomography using relaxed ordered subsets algorithms,” IEEE Trans. Med. Imag., vol. 22, no. 5, pp. 613–626, May 2003, doi: 10.1109/TMI.2003.812251.
[16] C. Bouman and K. Sauer, “A generalized Gaussian image model for edge-preserving MAP estimation,” IEEE Trans. Image Process., vol. 2, no. 3, pp. 296–310, Jul. 1993, doi: 10.1109/83.236536.
[17] D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289–1306, Apr. 2006, doi: 10.1109/TIT.2006.871582.
[18] E. J. Candès and M. B. Wakin, “An introduction to compressive sampling,” IEEE Signal Process. Mag., vol. 25, no. 2, pp. 21–30, Mar. 2008, doi: 10.1109/MSP.2007.914731.
[19] M. Lustig, D. L. Donoho, and J. M. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med., vol. 58, no. 6, pp. 1182–1195, Dec. 2007, doi: 10.1002/mrm.21391.
[20] M. Guerquin-Kern, M. Häberlin, K. Pruessmann, and M. Unser, “A fast wavelet-based reconstruction method for magnetic resonance imaging,” IEEE Trans. Med. Imag., vol. 30, no. 9, pp. 1649–1660, Sep. 2011, doi: 10.1109/TMI.2011.2140121.
[21] M. A. T. Figueiredo and R. D. Nowak, “An EM algorithm for wavelet-based image restoration,” IEEE Trans. Image Process., vol. 12, no. 8, pp. 906–916, Aug. 2003, doi: 10.1109/TIP.2003.814255.
[22] G. Wang, J. C. Ye, K. Mueller, and J. A. Fessler, “Image reconstruction is a new frontier of machine learning,” IEEE Trans. Med. Imag., vol. 37, no. 6, pp. 1289–1296, Jun. 2018, doi: 10.1109/TMI.2018.2833635.
[23] U. S. Kamilov, C. A. Bouman, G. T. Buzzard, and B. Wohlberg, “Plug-and-play methods for integrating physical and learned models in computational imaging: Theory, algorithms, and applications,” IEEE Signal Process. Mag., vol. 40, no. 1, pp. 85–97, Jan. 2023, doi: 10.1109/MSP.2022.3199595.
[24] J. Yoo et al., “Deep learning diffuse optical tomography,” IEEE Trans. Med. Imag., vol. 39, no. 4, pp. 877–887, Apr. 2020, doi: 10.1109/TMI.2019.2936522.
[25] P. Sarder and A. Nehorai, “Deconvolution methods for 3-D fluorescence microscopy images,” IEEE Signal Process. Mag., vol. 23, no. 3, pp. 32–45, May 2006, doi: 10.1109/MSP.2006.1628876.
[26] C. Vonesch, F. Aguet, J.-L. Vonesch, and M. Unser, “The colored revolution of bioimaging,” IEEE Signal Process. Mag., vol. 23, no. 3, pp. 20–31, May 2006, doi: 10.1109/MSP.2006.1628875.
[27] H. Greenspan, B. van Ginneken, and R. M. Summers, “Deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1153–1159, May 2016, doi: 10.1109/TMI.2016.2553401.
[28] E. Meijering, “A bird’s-eye view of deep learning in bioimage analysis,” Comput. Struct. Biotechnol. J., vol. 18, pp. 2312–2325, Jan. 2020, doi: 10.1016/j.csbj.2020.08.003.
[29] S. K. Zhou et al., “A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises,” Proc. IEEE, vol. 109, no. 5, pp. 820–838, May 2021, doi: 10.1109/JPROC.2021.3054390.
[30] T. Falk et al., “U-Net: Deep learning for cell counting, detection, and morphometry,” Nature Methods, vol. 16, no. 1, pp. 67–70, Jan. 2019, doi: 10.1038/s41592-018-0261-2.
[31] Y. Zhu and E. Meijering, “Automatic improvement of deep learning-based cell segmentation in time-lapse microscopy by neural architecture search,” Bioinformatics, vol. 37, no. 24, pp. 4844–4850, Dec. 2021, doi: 10.1093/bioinformatics/btab556.
[32] P. Vandewalle, J. Kovačević, and M. Vetterli, “Reproducible research in signal processing,” IEEE Signal Process. Mag., vol. 26, no. 3, pp. 37–47, May 2009, doi: 10.1109/MSP.2009.932122.
[33] K. Price, “Anything you can do, I can do better (No you can’t)…,” Comput. Vision, Graph., Image Process., vol. 36, nos. 2–3, pp. 387–391, Nov./Dec. 1986, doi: 10.1016/0734-189X(86)90083-6.
[34] J. West et al., “Comparison and evaluation of retrospective intermodality brain image registration techniques,” J. Comput. Assisted Tomogr., vol. 21, no. 4, pp. 554–568, Jul. 1997, doi: 10.1097/00004728-199707000-00007.
[35] N. Chenouard et al., “Objective comparison of particle tracking methods,” Nature Methods, vol. 11, no. 3, pp. 281–289, Mar. 2014, doi: 10.1038/nmeth.2808.
[36] R. D. Rudyanto et al., “Comparing algorithms for automated vessel segmentation in computed tomography scans of the lung: The VESSEL12 study,” Med. Image Anal., vol. 18, no. 7, pp. 1217–1232, Oct. 2014, doi: 10.1016/j.media.2014.07.003.
[37] M. Maška et al., “A benchmark for comparison of cell tracking algorithms,” Bioinformatics, vol. 30, no. 11, pp. 1609–1617, Jun. 2014, doi: 10.1093/bioinformatics/btu080.
[38] E. Gómez-de Mariscal et al., “DeepImageJ: A user-friendly environment to run deep learning models in Imagej,” Nature Methods, vol. 39, no. 2, pp. 73–86, 2022.
[39] A. Reinke, M. D. Tizabi, M. Eisenmann, and L. Maier-Hein, “Common pitfalls and recommendations for grand challenges in medical artificial intelligence,” Eur. Urol. Focus, vol. 7, no. 4, pp. 710–712, Jul. 2021, doi: 10.1016/j.euf.2021.05.008.
[40] L. Maier-Hein et al., “Why rankings of biomedical image analysis competitions should be interpreted with care,” Nature Commun., vol. 9, no. 1, Dec. 2018, Art. no. 5217, doi: 10.1038/s41467-018-07619-7.
[41] L. Maier-Hein et al., “BIAS: Transparent reporting of biomedical image analysis challenges,” Med. Image Anal., vol. 66, Dec. 2020, Art. no. 101796, doi: 10.1016/j.media.2020.101796.
[42] D. Lahat, T. Adali, and C. Jutten, “Multimodal data fusion: An overview of methods, challenges, and prospects,” Proc. IEEE, vol. 103, no. 9, pp. 1449–1477, Sep. 2015, doi: 10.1109/JPROC.2015.2460697.
Digital Object Identifier 10.1109/MSP.2023.3242833