Rui Guo, Tianyao Huang, Maokun Li, Haiyang Zhang, Yonina C. Eldar
©SHUTTERSTOCK.COM/PAPAPIG
Electromagnetic (EM) imaging is widely applied in sensing for security, biomedicine, geophysics, and various industries. It is an ill-posed inverse problem whose solution is usually computationally expensive. Machine learning (ML) techniques and especially deep learning (DL) show potential in fast and accurate imaging. However, the high performance of purely data-driven approaches relies on constructing a training set that is statistically consistent with practical scenarios, which is often not possible in EM-imaging tasks. Consequently, generalizability becomes a major concern. On the other hand, physical principles underlie EM phenomena and provide baselines for current imaging techniques. To benefit from prior knowledge in big data and the theoretical constraint of physical laws, physics-embedded ML methods for EM imaging have become the focus of a large body of recent work.
This article surveys various schemes to incorporate physics in learning-based EM imaging. We first introduce background on EM imaging and basic formulations of the inverse problem. We then focus on three types of strategies combining physics and ML for linear and nonlinear imaging and discuss their advantages and limitations. Finally, we conclude with open challenges and possible ways forward in this fast-developing field. Our aim is to facilitate the study of intelligent EM-imaging methods that will be efficient, interpretable, and controllable.
EM fields and waves have long been used as a sensing method. This is because the EM field can penetrate various media, interact with materials, and alter its distribution in both space and time. Hence, electric and magnetic properties of materials, such as permittivity, permeability, and conductivity, can be inferred from field samples. EM imaging refers to reconstructing the value distribution of electric or magnetic parameters from measured EM fields, through which a better understanding of the domain of investigation (DoI) can be obtained. EM-imaging techniques have been widely applied in security, biomedicine, geophysics, and various industries. In security, for example, EM imaging using radars can help locate targets that are invisible to optical imaging. In biomedicine, microwave imaging can detect anomalies in the permittivity distribution caused, e.g., by a cerebral hemorrhage. As a final example, images of conductivity distribution reconstructed from low-frequency EM fields may reveal deep structures in the earth.
A theoretical model of EM imaging is illustrated in Figure 1, where EM sensors, i.e., antennas, are deployed around the DoI. When an external source illuminates the DoI, sensors record the EM field. Given the transmitting waveform as well as locations of the transmitting antennas, the EM field propagates according to Maxwell’s equations [1]. In the frequency domain, EM propagation can be described by the following partial differential equation (PDE): \[\nabla\times\nabla\times{\bf{E}}{(}{\bf{r}}{)}\,{-}\,{\omega}^{2}{\mu}{\epsilon}{(}{\bf{r}}{)}{\bf{E}}{(}{\bf{r}}{)} = {i}{\omega}{\mu}{\bf{J}}{(}{\bf{r}}{)} \tag{1} \]
Figure 1. The EM-imaging setup. EM imaging converts measured data to the spatial distribution of electric parameters in the DoI.
where E is the vector electric field, r is the spatial position, $\mu$ is permeability, $\epsilon$ is complex permittivity, J is the electric current source, $\omega$ is the angular frequency, and $\nabla\times$ is the curl operator. Complex permittivity is expressed as ${\epsilon} = {\epsilon}_{R} + {i}{\sigma}{/}{\omega},$ where its real part ${\epsilon}_{R}$ is permittivity and its imaginary part is related to conductivity ${\sigma}{.}$ Equation (1) can describe both wave physics ${(}{\epsilon}_{R}\gg{\sigma}{/}{\omega}{)}$ and diffusion physics ${(}\sigma{/}\omega\gg{\epsilon}_{R}{),}$ depending on the settings of investigation. Here permeability is assumed to be constant, which is reasonable in most imaging scenarios. In EM imaging, we usually have information about the sources; therefore, $\omega$ and J are both known. Once we measure the electric field at the receiver locations, complex permittivity can be recovered by solving (1). This process defines EM inverse problems in which EM parameters are solved given measured electric fields.
The EM inverse problem is nonlinear and often ill-posed due to the following several challenges:
Recent advances in big data storage, massive parallelization, and optimization algorithms have facilitated the development of ML and its applications in EM imaging [2], [3], [4], [5]. ML is attractive for overcoming the aforementioned limitations due to the following aspects. First, the time-consuming operations of modeling and inversion can be surrogated by data-driven models to make imaging faster. Second, prior knowledge that is difficult to describe with rigorous forms can be recorded after the learning process, which helps improve imaging accuracy. Finally, DL software frameworks provide user-friendly interfaces to fully exploit computing power without low-level programming on heterogeneous platforms, which largely reduces the complexity of algorithm implementation for high-performance imaging.
Training a surrogate model for data-image mappings such as deep neural networks (DNNs) has shown promising results [3]. However, success relies on constructing a training dataset that is statistically consistent with practical scenarios. Due to multiple scattering effects, simply establishing the mapping from EM data to electric properties by black-box regression may lead to implausible predictions, even when the measured data are not highly out of distribution. On the other hand, physical laws provide baselines for EM imaging. The relationship between EM fields and electric properties is inherent in Maxwell’s equations.
Recent trends show that a hybrid of physics- and data-driven methods can analyze and predict data more effectively [5]. Such techniques can be grouped into learning-assisted physics-driven techniques and physics-embedded ML approaches. The former category solves the inverse problem in physics-based frameworks, where learning approaches are applied to augment performance, such as generating better initial guesses [6], improving the bandwidth of measured data [7], or encoding prior knowledge [8]. The latter category performs imaging mainly in data-driven manners, where algorithms are designed according to physical laws, such as tailoring inputs and labels [9], [10], [11], [12], loss functions [13], [14], [15], and neural network structures [16], [17], [18]. Although frameworks of physics-based techniques have been well studied, the learning methods for EM imaging vary widely.
This article aims to review recent frontiers in physics-embedded ML for EM-imaging techniques and shed light on designing efficient and interpretable ML-based imaging algorithms. Existing approaches include modeling Maxwell’s equations into the learning process and combining trainable parameters with full-wave EM solvers or differential/integral operators [13], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25]. These ML models not only describe physical principles but also record the prior knowledge gained from training data. Compared with purely data-driven models, physics-embedded approaches possess higher generalizability and can learn effectively from fewer training data [26]. In addition, as there is no need to train the physics part, both the memory and computation complexity of DNNs can be reduced.
We begin in the “Formulations and Challenges of EM Imaging” section by introducing basic formulations of the EM imaging problem and stating some of the challenges with conventional methods. We then categorize existing physics-embedded ML approaches into three kinds: learning after physics processing, learning with physics loss, and learning with physics models in the “Learning After Physics Processing,” “Learning With Physics Loss,” and “Learning With Physics Models” sections, respectively. The “Challenges and Opportunities” section discusses open challenges and opportunities in this fast-developing field. We draw conclusions in the “Conclusions and Outlooks” section.
EM imaging is an inverse problem that calculates electric parameters of the DoI from measured EM fields. This process incorporates EM modeling, which simulates “measured” data based on numerical models. It can be described as minimizing the “misfit” between the observed and simulated data [27] \[{L}{(}{\bf{\epsilon}}{)} = {\Vert}{\bf{d}}_{\text{obs}}{-}{F}{(}{\bf{\epsilon}}{)}{\Vert}^{2} + {{\lambda}{\phi}}_{r}{(}{\bf{\epsilon}}{)} \tag{2} \] where ${\bf{d}}_{\text{obs}}$ is the field observed by receivers, $\bf{\epsilon}$ is complex permittivity, ${F}{(}\bf{\epsilon}{)}$ represents the EM-modeling function, ${\phi}_{r}$ is the regularization term, and $\lambda$ is a regularization factor. Note that “complex permittivity” is usually simplified to “conductivity” in low-frequency EM methods. This article uses “complex permittivity” to represent the unknown in both high- or low-frequency methods.
EM modeling, ${F}{(}\bf{\epsilon}{),}$ computes the EM field in space given the permittivity distribution in the DoI and the information on the sources. It is usually achieved by numerical means, e.g., the finite-element and finite-difference techniques and the methods of moments [28]. These procedures partition the DoI into thousands or millions of subdomains and convert the wave equation into a matrix equation. EM fields in space are obtained by solving the matrix equation that involves thousands or millions of unknowns. The solution process can take minutes or hours. This computationally expensive process is usually called full-wave simulation. To accelerate the modeling process, one can make approximations so that the EM field is linear in the electric parameters, such as the Born or Rytov approximations [1]. In this linearized process, the EM field is computed by simple matrix operations, such as matrix-vector multiplications or the Fourier transform.
The regularization, ${\phi}_{r}{(}\bf{\epsilon}{),}$ is used to incorporate prior knowledge into the imaging process, and it varies in different tasks. For instance, in geophysical or biomedical imaging, to emphasize the sharpness or smoothness of material boundaries, ${\ell}_{1}$ and ${\ell}_{2}$ norms of the spatial gradient of $\bf{\epsilon}$ are usually adopted [29], [30], [31], [32], [33]. In radar imaging, the sparsity of the observed scene is often exploited to improve imaging quality by incorporating sparsity regularization, such as the ${\ell}_{1}$ norm given by ${\phi}_{r}{(}{\bf{\epsilon}}{)} = {\left|{\bf{\epsilon}}\right|}_{1}$ [34], [35], [36].
Equation (2) is usually minimized by iterative gradient descent methods, but some challenges still exist. For instance, each iteration requires computing the forward problem and its Fréchet derivative. Many forward problems are computationally intensive, and computing the Fréchet derivative of ${F}{(}{\bf{\epsilon}}{)}$ with respect to ${\bf{\epsilon}}$, i.e., ${\bf{S}} = \partial{F}{(}{\bf{\epsilon}}{)/}\partial{\bf{\epsilon}}$, needs to call the forward solver ${F}{(}\bf{\epsilon}{)}$ many times, which exacerbates computational burden. Some fast algorithms that compute approximations of Fréchet derivatives have been proposed [27], [37], [38], but the solution process still needs to be accelerated. Furthermore, when ${F}{(}\bf{\epsilon}{)}$ is rigorously solved from Maxwell’s equations, the objective function is nonconvex and has numerous local minima due to nonlinearity between EM responses and permittivity. Finally, gradient descent methods lack flexibility in exploiting the prior knowledge that is not described by simple regularization. On the other hand, stochastic inversion schemes have been developed to cope with the necessity of reliable uncertainty estimation and to have a natural way to inform imaging with realistic and complex prior information [39], [40], [41]; however, stochastic sampling is often computationally intensive.
Physics-embedded ML models provide potential solutions to the challenges mentioned in this section. In the following, we present three types of physics-embedded models for EM imaging, as depicted in Figure 2. The first class processes EM data using conventional physical methods and ML models sequentially [9], [10], [11], [42], [44]. The second type optimizes network parameters with physics constraints, for example, solving forward problems in the training loss function [13], [15], [19]. The third category unrolls the physical methods with neural networks [16], [17], [18], [20], [21], [22], [23], [24], [25]. These three approaches are discussed in detail in the following three sections.
Figure 2. The three ways of incorporating physics into the ML model. (a) Learning after physics processing: the physics model is employed to initialize the input of ML models. (b) Learning with physics loss: physics knowledge is incorporated into the loss functions. (c) Learning with physics models: physics knowledge is used to guide design of the ML architecture.
The learning after physics processing approach consists of two sequential steps: first, a roughly estimated image is recovered using classical qualitative or quantitative methods; second, the rudimentary image is polished using a DNN trained with the ground truth as labels. In this approach, the tasks of DNNs become image processing, such as eliminating artifacts and improving resolution. Next we introduce several classical works that utilize DNNs to enhance image quality in this direction.
In conventional EM imaging, permittivity is iteratively refined by updating in the descent directions of the objective function. This motivates employing multiple convolutional neural network (CNN) modules to progressively improve image resolution, starting from some rudimentary images. These images can come from conventional imaging methods, including linear backprojection [9], subspace optimization [10], one-step Gauss–Newton [11], and contrast source-inversion techniques [42]. Finally, the CNN can output superresolution images close to the ground truth. To construct the training dataset, many researchers convert the handwritten digits dataset to permittivity models, then perform full-wave simulations to obtain the scattered electric data. To train the DNN, the rudimentary image is taken as the input, while the corresponding true-permittivity image is the label. After training with handwritten letters, the DNN predicts targets with more complex shapes and permittivity.
Advanced DNN architectures may improve the performance of image enhancement. The U-Net [5], [74], one of the most widely used architectures, is built on the encoder-decoder architecture and has skip connections bringing encoded features to the decoder. This ensures feature similarity between the input and output and is especially suitable for superresolution. For example, the authors in [10] and [42] use U-Nets to achieve superresolution for 2D microwave imaging. In [44], the 3D inverse scattering problem is solved by a 3D U-Net, where the input is the preliminary 3D model recovered by Born approximation inversion and the Monte Carlo method.
Another architecture for superresolution is a generative adversarial network (GAN). In [11] and [12], a GAN with cascaded, object-attentional superresolution blocks is applied to imaging with an inhomogeneous background. The authors use a GAN with an attention scheme to improve the resolution by highlighting scatterers and inhibiting the artifacts. In [11], after training with 6,000 handwritten digit scatterers, the GAN reconstructs U-shape plexiglass scatterers in a through-wall imaging test within 1 s. Structural similarity improves by more than 50% compared to conventional algorithms.
It should be noted that the more the input is processed by physics, the better the generalizability will be. For instance, the Wei and Chen [10] compare performances with various network inputs, including raw, scattered data; permittivity from backprojection; and permittivity from the dominant current scheme (DCS). The network behaves the poorest when directly inputting raw data, and best when inputting the image from the DCS. This is because preprocessing in the DCS involves more physics and thus reduces the network’s burden. A similar conclusion is drawn in [45], where the input and output of neural networks are preprocessed with the wave-propagation operator, i.e., Green’s function, to a deeper degree, achieving better performance on accuracy and robustness against noise than the DCS [10] when recovering high-permittivity targets.
The sequential workflow provides great flexibility in borrowing well-developed DL techniques in image processing, and most of the mentioned works can achieve real-time imaging. Recent works have extended this approach to uncertainty quantification of imaging results [46]. However, although a DNN can generate a plausible image, the recovered permittivity values may significantly differ from true ones. This is because the DNN is trained without the supervision of the EM field. In the following section, we introduce another group of methods that takes into account fitness between the computed and measured data.
This section presents several approaches that impose additional physical constraints on the loss function when training the network weights, different from conventional ML models, which use only the difference between predicted and labeled images as a loss function. The advantage of additional constraints in loss is demonstrated in “Incorporating Forward Modeling in Loss: A Mathematical Example.” In the following, we consider different types of physics losses: rigorous measurement, learned measurement, and PDE-constrained.
Incorporating Forward Modeling in Loss: A Mathematical Example
Optimizing deep neural network (DNN) parameters with the constraint of a forward problem can alleviate the ambiguity caused by nonuniqueness of the inverse problem.
We demonstrate it using a toy problem [19], where the forward process has analytical solutions: \[{m}\,{:=}\,{F}{(}{p}{)} = {p}^{2}{.} \tag{S1} \]
The inversion has two branches of solutions, ${p} = + {\sqrt{m}}\,{\text{and}}\,{p} = {-}{\sqrt{m}}$, see the black lines in Figure S1. To solve the inverse problem with a DNN, the training dataset is constructed such that for each sample $\left({m},{\sqrt{m}}\right)$, there is another one $\left({m},{-}{\sqrt{m}}\right)$ in it. The point that simultaneously minimizes the distance between the two solutions is zero. When training is supervised by labels ${p}\left({\pm}{\sqrt{m}}\right)$, the predictions are zeros [see Figure S1(a)], which are fake answers caused by the nonuniqueness. On the other hand, when training is supervised by labels ${p}^{2}$, the correct branch can be predicted by controlling the signs of solutions [see Figure S1(b)]. In EM imaging, the forward problems are described by Maxwell’s equations.
Figure S1. Incorporating forward modeling into training to reduce nonuniqueness of the inverse problem [19]. (a) When training is supervised by $p$ ($\pm\sqrt{m}$), the predictions are zeros and (b) when training is supervised by labels ${p}^{2},$ the correct branch can be predicted by controlling the signs of solutions.
Consider the inverse problem solved by a DNN with the measured data ${\bf{d}}$ as the input and the permittivity $\bf{\epsilon}$ as the output. When the EM data at the receivers can be numerically computed as in many applications, one can embed the data fitness, which involves physical rules, in the training loss function [13].
Let ${\bf{\epsilon}}_{T}$ and ${\bf{d}}_{T}$ denote the labeled permittivity and EM data, respectively, for training. Purely data-driven imaging uses permittivity loss ${L}_{\epsilon} = {\Vert}{\bf{\epsilon}}{-}{\bf{\epsilon}}_{T}{\Vert}^{2}$ for training. The physics-embedded one further incorporates the measurement (data) loss ${L}_{d},$ given by \[{L} = \alpha{L}_{\epsilon} + \beta{L}_{d} = \alpha{\Vert}{\bf{\epsilon}}{-}{\bf{\epsilon}}_{T}{\Vert}^{2} + \beta{\Vert}{F}\left({\bf{\epsilon}}\right){-}{\bf{d}}_{T}{\Vert}^{2} \tag{3} \] where ${\alpha}$ and ${\beta}$ are weighting coefficients. If $\alpha = {0},$ the DNN training can be regarded as unsupervised learning. In the geosteering EM data inversion [13], this scheme achieves a two-orders-of-magnitude-lower data misfit compared to training with permittivity loss only ${(}{\beta} = {0}{)}{.}$
Training such a DNN requires backpropagating the gradients of the data misfit, where the Fréchet derivative $\partial{F}{/}\partial\bf{\epsilon}$ needs to be computed outside the DL framework. The methods for estimating the derivative have been addressed in traditional deterministic inversion, such as the finite-difference or adjoint-state technique [47].
Computing the Fréchet derivative $\partial{F}{/}\partial\bf{\epsilon}$ is time consuming. An idea to accelerate it is to surrogate the numerical forward solver ${F}{(}{\cdot}{)}$ with a DNN ${\Theta}_{F}{(}{\cdot}{)}$ [19]. The training contains two stages: 1) training the forward solver ${\Theta}_{F}^{\ast}$ and 2) training the inverse operator ${\Theta}_{I}^{\ast},$ given by \begin{align*}\begin{gathered}{{\Theta}_{F}^{\ast} = \arg\mathop{\min}\limits_{{\Theta}_{F}}{{\Vert}{{\Theta}_{F}{(}{\bf{\epsilon}}_{T}{)}{-}{\bf{d}}_{T}}{\Vert}}^{2},}\\{{\Theta}_{I}^{\ast} = \arg\mathop{\min}\limits_{{\Theta}_{I}}{{\Vert}{{\Theta}_{F}^{\ast}{(}{\Theta}_{I}{(}{\bf{d}}_{T}{))}{-}{\bf{d}}_{T}}{\Vert}}^{2}{.}}\end{gathered} \tag{4} \end{align*}
Both stages take the measurement misfit as the loss function, which involves physical rules.
This scheme successfully solves the logging-while-drilling inverse problem for borehole imaging [19], (see Figure 3) where the purely data-driven approach did not achieve satisfactory reconstructions due to the severe nonlinearity and ill-posedness in logging-while-drilling inversion [48].
Figure 3. Underground resistivity predicted from logging-while-drilling EM data [19]. The gray curve represents drilling trajectory, along which EM fields are transmitted and collected by a logging instrument. Resistivity around the trajectory is recovered by a DNN trained with a learned measurement loss.
The PDE-constrained loss inserts PDEs into the loss function. The representative work is the physics-informed neural network (PINN) [14], [49], which is designed for both forward and inverse problems. It is a mesh-free method and can seamlessly fuse knowledge from observations and physics. We present an example of a PINN for the inverse problem in “A Physics-Informed Neural Network for the Inverse Problem.”
A PINN is applied to electrical impedance tomography in [15]. Electrical impedance tomography measures the voltages on a body surface after electric currents are injected. The imaging recovers conductivity distribution inside the body. It usually contains multiple transmitting and receiving sensors. In “A Physics-Informed Neural Network for the Inverse Problem,” we show that a PINN contains one forward network ${\Theta}_{F}$ and one inverse network ${\Theta}_{I}$ for one transmitting source. When J sources illuminate the domain, the PINN will contain J + 1 networks, corresponding to J forward networks that output voltages generated by different sources and one inverse network that outputs conductivity. Furthermore, the loss function should be modified to ${L} = {\Sigma}_{i}^{J}{L}_{i},$ where ${L}_{i}$ represents the loss function for the ith source. Simultaneously training all networks can satisfy both PDEs and boundary conditions (measurements).
A Physics-Informed Neural Network for the Inverse Problem
Consider the 1D time-domain electromagnetic wave equation \[\frac{{\partial}^{2}{E}{(}{x},{t}{)}}{\partial{x}^{2}}{-}{\mu}{\epsilon}{(}{x}{)}\frac{{\partial}^{2}{E}{(}{x},{t}{)}}{\partial{t}^{2}} = {0} \tag{S2} \] where $E$ is the electric field, $\epsilon$ is permittivity, $\mu$ is permeability, and $t$ and $x$ are the time and spatial coordinate, respectively. Together with some boundary conditions, the equation can be analytically or numerically solved to yield $E$ (forward problem) or ${\epsilon}$ (inverse problem) given $t$ and $x.$
Take the inverse problem with one-source multiple receivers as an example. A physics-informed neural network (PINN) specifies two separate deep neural networks (DNNs), namely, ${\Theta}_{F}$ and ${\Theta}_{I}{.}$ The input of ${\Theta}_{F}$ is $x$ and $t$ and its output is the electric field $\tilde{E},$ denoted by $\tilde{E} = {\Theta}_{F}{(}{x},{t}{)}{.}$ Similarly, the input of ${\Theta}_{I}$ is $x$ and its output is permittivity $\tilde{\epsilon},$ denoted by $\tilde{\epsilon} = {\Theta}_{I}{(}{x}{)}{.}$ The two separate DNNs are simultaneously trained with a shared loss function $L,$ which includes a supervised measurement loss of $E$ regarding initial and boundary conditions \[{L}_{\text{data}} = \frac{1}{{N}_{\text{data}}}\mathop{\sum}\limits_{{i} = {1}}^{{N}_{\text{data}}}\left({\tilde{E}}\left({x}_{i},{t}_{i}\right){-}{E}_{T}\left({x}_{i},{t}_{i}\right)\right)^{2} \tag{S3} \] and an unsupervised loss of partial differential equation constructed according to (S2) \[{L}_{\text{PDE}} = \frac{1}{{N}_{\text{PDE}}}\mathop{\sum}\limits_{{j} = {1}}\limits^{{N}_{\text{PDE}}}{{\left({\frac{{\partial}^{2}\tilde{E}\left({{x}_{j},{t}_{j}}\right)}{\partial{x}^{2}}{-}{\mu}\tilde{\epsilon}{(}{x}_{j}{)}\frac{{\partial}^{2}\tilde{E}\left({{x}_{j},{t}_{j}}\right)}{\partial{t}^{2}}}\right)}^{2}} \tag{S4} \] given by ${L} = {\alpha}_{\text{data}}{L}_{\text{data}} + {\alpha}_{\text{PDE}}{L}_{\text{PDE}}$. Here $\left({x}_{i}{,}{t}_{i}\right)$ and $\left({x}_{j}{,}{t}_{j}\right)$ are sampled at the initial/boundary position and in the domain of investigation (DoI), respectively. In addition, ${E}_{T}$ is the labeled measurement, ${N}_{\text{data}}$ is the number of labeled samples, ${N}_{\text{PDE}}$ is the number of unlabeled samples in the DoI, and ${\alpha}_{\cdot}$ are weights. The partial differentiations are achieved by the automatic differentiation in the deep learning framework. After training, one can use ${\Theta}_{I}$ to predict permittivity at arbitrary location ${x}$. Therefore, the PINN is mesh free.
The smoothness of conductivity and known conductivity on the boundary are represented as regularizations in the loss function of a PINN to stabilize the inverse process [15]. In numerical simulations, the authors set ${J} = {8},$ ${N}_{\text{data}} = {10},{000},$ and ${N}_{\text{PDE}} = {8},{000}$ and achieve better results than two conventional methods. However, one should note that we seldom have so many measurements in reality, so its performance on experimental data imaging needs to be further investigated.
When inverting limited-aperture EM data, insufficient measurements may lead to the instability of training a PINN. In this case, it would be better to use the first two approaches that explicitly define the measurement loss at the receivers. A PINN outperforms the two approaches when simulating the EM response is prohibitive, for example, due to the high computational cost or complex EM environment. Finally, when recovering diverse targets, the former two approaches can make predictions without retraining the neural network, while a PINN needs to be trained for each target.
Following the use of unrolling in other domains [17], [50], [51], unrolling has also been used in EM-imaging models, yielding physics-embedded neural networks. We group this type into three subtypes: unrolling the measurement-to-image (inverse), unrolling the image-to-measurement (forward), and simultaneously unrolling both mappings.
We demonstrate the unrolling of linear inverse problems through radar imaging. Here, the electric parameters of interest are intensities of scatterers in the DoI, denoted by $\bf{\epsilon}$ with a slight abuse of notation. Then, linear approximation is usually applied in the forward model for high computational efficiency, yielding ${F}{(}\bf{\epsilon}{)} = \bf{\Phi}\bf{\epsilon},$ where $\bf{\Phi}$ is a matrix determined by the radar waveform and the geometry of the DoI. Conventional radar imaging can be formulated as a compressed sensing problem: ${\min}_{\bf{\epsilon}}{\Vert}{\bf{d}}_{\text{obs}}{-}{\bf{\Phi}}{\bf{\epsilon}}{\Vert}^{2} + {\lambda}{\vert}{\bf{\epsilon}}{\vert}_{1}$. There are myriad methods proposed to solve such problems. A well-known technique is the iterative shrinkage thresholding algorithm (ISTA) that iteratively performs proximal gradient descent [52]. Specifically, the solution is updated by \[{\bf{\epsilon}}_{k} = {\cal{S}}_{\frac{\lambda}{L}}\left({\frac{1}{L}{\bf{\Phi}}^{H}{\bf{d}}_{\text{obs}} + \left({{\bf{I}}{-}\frac{1}{L}{\bf{\Phi}}^{H}\bf{\Phi}}\right){\bf{\epsilon}}_{{k}{-}{1}}}\right) \tag{5} \] where ${L} = {\lambda}_{\max}{(}{\bf{\Phi}}^{H}\bf{\Phi}{)}$ is the Lipschitz constant, and ${\lambda}_{{\text{max}}}{(}{\cdot}{)}$ represents the maximum eigenvalue of a Hermitian matrix, and ${\cdot}^{H}$ denotes the conjugate transpose. The elementwise soft-threshold operator ${\cal{S}}_{\theta}$ assigns those elements below the threshold $\theta$ to zeros, defined as ${\left[{{\cal{S}}_{\theta}\left({\bf{u}}\right)}\right]}_{i} = {\text{sign}}\left({[\bf{u}{]}_{i}}\right){\left({\left|{{\left[{\bf{u}}\right]}_{i}}\right|{-}{\theta}}\right)}_{ + },$ where ${\text{sign}}{(}{\cdot}{)}$ returns the sign of a scalar, ${(}{\cdot}{)}_{+}$ means $\max(·,0),$ and $\theta$ is the threshold. The ISTA shows high accuracy but requires thousands of iterations for convergence.
To accelerate the solution process, the learned ISTA (LISTA) with only several neural network layers is proposed in [53], where each layer unfolds ISTA iterations. Particularly, the LISTA treats ${\lambda}{/}{L},{(}{1}{/}{L}{)}{\bf{\Phi}}^{H}$ and ${(}{\bf{I}}{-}{(}{1}{/}{L}{)}{\bf{\Phi}}^{H}\bf{\Phi}{)}$ in (5) as variables to learn from training data with a backprojection algorithm, disregarding their physics structures. The numerical results in [53] show that the LISTA can achieve virtually the same accuracy as the ISTA using nearly two-order fewer iterations and does not require knowledge of $\bf{\Phi}$. Nevertheless, a challenge in the LISTA is that there are many variables to learn, requiring careful tuning of hyperparameters to avoid overfitting and gradient vanishing.
Embedding physics models into the neural networks reduces the number of variables while maintaining fast convergence rate [23], [24]. For example, the mutual inhibition matrix ${\bf{I}}{-}{(}{1}{/}{L}{)}{\bf{\Phi}}^{H}\bf{\Phi}$ has a Toeplitz or a doubly block Toeplitz structure due to the nature of radar-forward models. The degrees of freedom with such a Toeplitz structure are reduced to O(N) from $O({N}^{2}),$ the counterpart without this structure. By incorporating such a structure, the proposed method in [23] significantly reduces the dimension of neural networks, thereby reducing the amount of training data, memory requirements, and computational cost, while maintaining comparable imaging quality as the LISTA. A similar approach is also adopted in [24], which explores the coupling structure between different blocks in the radar-forward model $\bf{\Phi}$.
The objective function of nonlinear EM imaging, ${L}{(}{\bf{\epsilon}}{)} = {\Vert}{\bf{d}}_{\text{obs}}{-}{F}{(}\bf{\epsilon}{)}{\Vert}^{2}$, where ${F}{(}\bf{\epsilon}{)}$ is numerically solved from PDEs, is conventionally minimized through gradient descent methods. With the Gauss–Newton method, permittivity is updated according to \[{\bf{\epsilon}}_{{k} + {1}} = {\bf{\epsilon}}_{k} + {(}{\bf{S}}^{H}{\bf{S}}{)}^{{-}{1}}{\bf{S}}^{H}{(}{\bf{d}}_{\text{obs}}{-}{F}{(}{\bf{\epsilon}}_{k}{))} \tag{6} \] where $\text{S}$ is the Fréchet derivative of F at ${\bf{\epsilon}}_{0}.$ Note that $\text{S}$ contains only the local property of the objective function, and computing ${F}{(}{\bf{\epsilon}}{),}\,{\bf{S}}$ and $({\bf{S}}^{H}{\bf{S}}{)}^{{-}{1}}$ is usually expensive.
By unrolling, a set of descent directions ${\bf{K}}{s}$ can be learned, instead of computing ${\bf{S}}$ and $({\bf{S}}^{H}{\bf{S}}{)}^{{-}{1}}{\bf{S}}^{H}$ online. This is called the supervised descent method (SDM), which was first proposed for solving nonlinear least-squares problems in computer vision [54]. In online imaging, permittivity is updated by \[{\bf{\epsilon}}_{{k} + {1}} = {\bf{\epsilon}}_{k} + {\bf{K}}_{k}{(}{\bf{d}}_{\text{obs}}{-}{F}{(}{\bf{\epsilon}}_{k}{))}{.} \tag{7} \]
In training, the EM response is taken as the input, while the corresponding ground truth of complex permittivity is the label. The interested reader is referred to [21] for more details on the training.
The SDM shows high generalizability in EM imaging. The imaging process is mainly governed by physical law, with a soft constraint imposed by learned descent directions. The experiments show that the descent directions trained with simple scatterers can be applied to predict complex targets in inhomogeneous media for various applications, such as geophysical inversion [55], [56], [57], [58], microwave imaging [21], [59], and biomedical imaging [60], [61]. Figure 4 shows its applications and generalizability.
Figure 4. Imaging with the SDM for geophysics [55], microwave [21] and biomedical data [61]. The SDM is able to reconstruct complex inhomogeneous media while the training scenarios are simple. CT: computerized tomography.
The SDM may be flexibly combined with techniques in conventional gradient-based inversion. For example, using regularizations on the objective function of prediction, images can be predicted with either smooth [55] or sharp [62] interfaces without retraining descent directions. Furthermore, the SDM and conventional methods can be flexibly switched to satisfy different requirements of speed and accuracy [55].
A limitation of the SDM is its lower speed of prediction compared with end-to-end DNNs because the online forward modeling is, in general, computationally intensive. Efforts have been made to unroll nonlinear forward modeling to accelerate imaging, which will be discussed in the following section.
Accelerating the forward process also improves the efficiency of EM imaging. The next sections introduce two methods where the frequency- and time-domain forward modeling is accelerated by unrolling integral and differential operations, respectively.
In this scheme, the differential equation (1) is first converted into an integral form, and then the forward modeling ${F}{(}\bf{\epsilon}{)}$ involving integral operations is unrolled as a physics-embedded network ${\Theta}_{F}{.}$ After the networks are trained, they are combined with generic networks ${\Theta}_{I}$ that perform inverse mappings to achieve full-wave EM imaging; hence, the cascaded neural networks become a physics-embedded DNN (PE-Net) [22].
The schematic architecture of the PE-Net is shown in Figure 5, where ${\bf{\epsilon}}_{0}$ is the initial permittivity, ${\Theta}_{F}$ represents the DNN-based forward-modeling solver, and ${\{}{\Theta}_{I}^{1},{\Theta}_{I}^{2},\ldots{\}}$ represent neural networks that predict the update of complex permittivity. Let K be a predefined maximum number of iterations. The final output ${\bf{\epsilon}}_{K}$ is then \[{\bf{\epsilon}}_{K} = {\Theta}_{I}\left({\bf{\epsilon}}_{0},{\bf{d}}_{\text{obs}}\right) = {\bf{\epsilon}}_{0} + \mathop{\sum}\limits_{{k} = {1}}^{K}{{\Theta}_{I}^{k}}\left({\bf{d}}_{\text{obs}}{-}{\Theta}_{F}\left({\bf{\epsilon}}_{{k}{-}{1}}\right)\right) \tag{8} \]
Figure 5. Physics-embedded DNNs for microwave imaging [22], where the forward modeling is unrolled into a neural network. The parameters of the forward solver ${\Theta}_{F}$ are fixed when training the inverse networks ${\Theta}_{I}\text{s}.$
where k is the index of iterations.
The integral form of the wave equation is \[{\bf{E}}{(}{\bf{r}}{)} = {\bf{E}}^{\text{inc}}{(}{\bf{r}}{)} + {\omega}^{2}{\mu}\mathop{\int}\nolimits_{V}{\overrightarrow{\overrightarrow{\bf{G}}}}_{0}\left({\bf{r}},{\bf{r'}}\right)\left[{\epsilon}{(}{\bf{r'}}{)}{-}{\epsilon}_{0}\right]{\bf{E}}{(}{\bf{r'}}{)}{d}{\bf{r'}} \tag{9} \] where ${\bf{E}}^{\text{inc}}$ is the incident field generated by the source, ${\overrightarrow{\overrightarrow{\bf{G}}}}_{0}$ is the Green’s function describing wave propagation, ${\epsilon}_{0}$ is the permittivity of the background, and V is the DoI. Here, ${\Theta}_{F},$ which essentially solves (9), is established by unfolding the conjugate gradient method. Conventionally, solving (9) is simplified as calculating ${\bf{x}}$ (representing the unknown E) from ${\bf{A}}{(}{\bf{\epsilon}}{)}{\bf{x}} = {\bf{b}}$, where ${\bf{A}}{(}{\bf{\epsilon}}{)}$ is a matrix related to wave physics and target permittivity, and ${\bf{b}}$ is a constant vector. Generally, $\text{A}$ is a full matrix with millions of elements, making solving using iterative matrix equation solvers very time consuming. In [18], inspired by the conjugate gradient method, the matrix equation is solved by alternately predicting the conjugate direction ${\bf{p}}$ and the solution update $\Delta{\bf{x}}$ iteratively. The details can be found in “The Conjugate Gradient and Update-Learning Methods.” The experiments show that ${\Theta}_{F}$ needs many fewer iterations than the conjugate gradient method.
The Conjugate Gradient and Update-Learning Methods
To solve the matrix equation ${\bf{A}}{\bf{x}} = {\bf{b}}$ whose ${\bf{A}}$ is positive definite, the conjugate gradient method computes the conjugate direction ${\bf{p}}$ and updates the solution in an iterative manner. The solution process may take hundreds to thousands iterations.
In electromagnetic modeling, incident waves at a certain frequency behave similarly, which leads to a limited diversity of the right-hand term ${\bf{b}}$. In addition, the background Green’s function does not change with different targets. Thus, characteristics of matrix equations derived from (9) are similar given a certain scenario. This enables the machine to learn how to update ${\bf{p}}$ and ${\bf{x}}$ from training data, rather than computing them online, as in the conjugate gradient method, to obtain a faster convergence speed.
In the update-learning method [22], iterations in conjugate gradient approach are unrolled by $N$ neural network blocks, which contain cascaded neural network ${\Theta}_{p}$ and ${\Theta}_{dx},$ which predict the quasi-conjugate direction and the solution update, respectively. The following detailed algorithms of the two approaches are presented:
Conjugate gradient method.
1: Input ${\bf{x}}_{0}$
2: ${\bf{r}}_{0} = {\bf{b}}{-}{\bf{Ax}}_{0},{\bf{\bf{p}}}_{1} = {\bf{r}}_{0}$
3: ${\alpha}_{1} = {(}{\bf{r}}_{0}^{T}{\bf{r}}_{0}{)}{/}{\bf{p}}_{1}^{T}{(}{\bf{Ap}}_{1}{)}$
4: ${\bf{x}}_{1} = {\bf{x}}_{0} + {\alpha}_{1}{\bf{p}}_{1}$
5: for ${k} = {1},{2},\ldots$ until ${\Vert}{\bf{\bf{r}}}_{k}{\Vert}\leqq{\varepsilon}$
6: ${\bf{r}}_{k} = {\bf{r}}_{{k}{-}{1}}{-}{\alpha}_{k}{(}{\bf{Ap}}_{k}{)}$
7: ${\beta}_{{k} + {1}} = {(}{\bf{r}}_{k}^{T}{\bf{r}}_{k}{)}{/}{(}{\bf{r}}_{{k}{-}{1}}^{T}{\bf{r}}_{{k}{-}{1}}{)}$
8: ${\bf{p}}_{{k} + {1}} = {\bf{r}}_{k} + {\beta}_{{k} + {1}}{\bf{p}}_{k}$
9: ${\alpha}_{{k} + {1}} = {(}{\bf{r}}_{k}^{T}{\bf{r}}_{k}{)}{/}{\bf{p}}_{{k} + {1}}^{T}{(}{\bf{Ap}}_{{k} + {1}}{)}$
10: ${\bf{x}}_{{k} + {1}} = {\bf{x}}_{k} + {\alpha}_{{k} + {1}}{\bf{p}}_{{k} + {1}}$
Update-learning method.
2: ${\bf{r}}_{0} = {\bf{b}}{-}{A}{\bf{x}}_{0},{\bf{p}}_{1} = {\bf{r}}_{0}$
3: ${\bf{x}}_{1} = {\bf{x}}_{0}$
4: for ${k} = {1},{2},\ldots,{N},$
5: ${\bf{r}}_{k} = {\bf{b}}{-}{A}{\bf{x}}_{k}$
6: ${\bf{p}}_{{k} + {1}} = {\Theta}_{\bf{p}}^{k}{(}{\bf{p}}_{k},{\bf{r}}_{k},{\bf{r}}_{{k}{-}{1}}{)}$
7: ${\bf{x}}_{{k} + {1}} = {\bf{x}}_{k} + {\Theta}_{dx}^{k}{(}{\bf{p}}_{{k} + {1}},{Ap}_{{k} + {1}},{\bf{r}}_{k}{)}$
Aside from fast convergence, ${\Theta}_{F}$ possesses high generalizability thanks to the incorporation of wave physics. Note that the $\text{A}$ matrix is constructed inside the network by integral operations, which involves Green’s function that describes the interactions of electric fields in the entire domain. The explicit field integration brings global information of wave propagation to the receptive field of convolutional layers. After training with 32,000 samples, the network can predict the field of targets statistically different from the training ones in real time, which builds the foundation for the following imaging problem.
The weights of the ${\Theta}_{F}$ are fixed during training ${\Theta}_{I}{s}{.}$ The ${\Theta}_{I}\text{s}$ are achieved by generic networks, which perform nonlinear mapping from the measurement domain to the permittivity domain. In training, the EM fields generated by synthetic targets are taken as the input, while the target images are taken as labels. In prediction, the network achieves superresolution reconstruction of targets that is quite different from the training ones. At the same time, the simulated field of the predicted target is in good agreement with the observed field.
Attempts have also been made to unroll the time-domain wave equation with recurrent neural networks (RNNs) [20], [63], where differential operations are represented by network layers. In the 2D case, Maxwell’s equations can be written as \begin{align*}\begin{cases}\begin{array}{l}{{\epsilon}_{R}\frac{\partial{E}_{z}}{\partial{t}} = \frac{\partial{H}_{y}}{\partial{x}}{-}\frac{\partial{H}_{x}}{\partial{y}}{-}{\sigma}{E}_{z},}\\{{\mu}\frac{\partial{H}_{x}}{\partial{t}} = {-}\frac{\partial{E}_{z}}{\partial{y}},}\\{{\mu}\frac{\partial{H}_{y}}{\partial{t}} = \frac{\partial{E}_{z}}{\partial{x}}}\end{array}\end{cases} \tag{10} \end{align*} where E and H are the electric and magnetic fields, respectively, that are coupled with each other; the subscripts represent spatial components of the vector field; ${\epsilon}_{R}$ is permittivity; ${\sigma}$ is conductivity; and x, y, z, and t is the spatial and time coordinates, respectively.
After discretization, for instance, the ${\mu}{(}\partial{H}_{x}{/}\partial{t}{)} = {-}{(}\partial{E}_{z}{/}$ $\partial{y}{)}$ term becomes \begin{align*}\begin{gathered}{{\mu}\frac{{H}_{x}^{{n} + {1}{/}{2}}\left({{i},{j} + \frac{1}{2}}\right){-}{H}_{x}^{{n}{-}{1}{/}{2}}\left({{i},{j} + \frac{1}{2}}\right)}{\Delta{t}} = }\\{{-}\frac{{E}_{z}^{n}{(}{i},{j} + {1}{)}{-}{E}_{z}^{n}{(}{i},{j}{)}}{\Delta{y}}}\end{gathered} \tag{11} \end{align*} where ${(}{i},{j} + {1}{/}{2}{),}{(}{i},{j}{)}$ and ${(}{i},{j} + {1}{)}$ represent the discrete spatial coordinate, ${n}{-}{1}{/}{2},$ n and ${n} + {1}{/}{2}$ represent discrete time; and $\Delta{t}$ and $\Delta{y}$ are time and space intervals, respectively. The spatial differentiation in the right side can be represented by convolutional kernels. In the time domain, the H-field at time ${n} + {1}{/}{2}$ is computed by ${H}_{x}^{{n} + {1}{/}{2}} = {H}_{x}^{{n}{-}{1}{/}{2}} + {\Theta}_{F}{(}{E}_{z}^{n}{),}$ where ${\Theta}_{F}$ represents basic operations on ${E}_{z}^{n}$ realized by neural networks. Therefore, fields at the next time step can be updated from fields at current time steps. The updating process can be described by an RNN.
Considering the couplings between the E- and H-fields, the architecture unrolled from (10) is presented in Figure 6. In the forward problem, after the target material is specified, EM data are generated by running the RNN without training. In the inverse problem, the material is represented by trainable parameters that are optimized by minimizing the misfit between simulated and labeled data. Training such a network and updating its weights is equivalent to gradient-based EM imaging. The use of automatic differentiation greatly improves the accuracy of gradient computation and achieves three-orders-of-magnitude acceleration compared with the conventional finite-difference method [20].
Figure 6. The cell architecture of an RNN for simulating wave propagation [20]. At each time step, the RNN outputs the ${E}{-}{\text{field}}\,{E}_{z}\,{\text{and}}\,{H}{-}{\text{fields}}\,{H}_{x},\,{H}_{y}$ in the entire DoI, which are computed from their values in the previous time step, according to Maxwell’s equations. The partial derivatives are approximated with finite differences. Taking the permittivity as a trainable layer, training this network and updating its weights is equivalent to gradient-based EM imaging.
Many approaches, such as Born iterative [1] or contrast source inversion [64], solve the inverse scattering problem from a physics point of view: the target is progressively refined by simulating the physics more accurately. To be specific, they first reconstruct permittivity from a linear process by approximating the electric field E in the integration of (9) to the incident field ${\bf{E}}^{\text{inc}}{.}$ Intermediate parameters, e.g., total field and contrast source, can be estimated with this permittivity. Then, a more accurate permittivity model is computed from the intermediate parameters and measurements, which are used to better approximate the intermediate parameters in the next iteration. When the intermediate parameters lead to scattered fields that fit measurements, the iteration stops and outputs the final estimated permittivity.
The aforementioned process can be unfolded into neural networks [16], [25]. Here, for instance, the cell architecture of a physical model-inspired neural network (PM-Net) [16] is presented in Figure 7. Each cell contains approximated forward and inverse processes. In the forward process, the total field E is computed from the contrast source J, and a better contrast source is predicted from the total field and permittivity. In the inverse process, permittivity is predicted from the contrast source and the total field. The network is trained using 300 handwritten letter profiles. In prediction, it achieves real-time imaging within 1 s while the alternating direction method of multipliers (ADMMs) needs 300 s. The resolution is also largely improved [16].
Figure 7. Simultaneously unrolling forward and inverse processes into neural networks [16]. The forward process (in green) computes the contrast source ${\bf{J}}$ and total field ${\bf{E}}^{t}$ given permittivity, while the inverse process (in blue) infers permittivity from measurements ${\bf{J}}$ and ${\bf{E}}^{t}$.
Based on [16] and our previous work [18], it is possible to compare different strategies of deep unrolling when they solve the same inverse scattering problem. We note that some simulation setups, e.g., electric scale, number of measurements, random noises, and training set, are not exactly the same. However, these differences are at an acceptable level and do not affect the conclusions of the comparison. The test data are simulated from the “Austria” model, which is often used as a benchmark model in inverse scattering. It is challenging due to strong scatterings inside and among the rings. The reconstructed images with different approaches are presented in Figure 8, where the ADMMs is taken as the benchmark. The result in contrast source network (CS-Net) is recovered by gradient-based optimization, whose initial guess is provided by a DNN [6]. Imaging with back projection scheme (BPS) is in the scope of learning after physics processing, where the neural network performs image enhancement after traditional qualitative imaging [10]. The SDM unrolls the inverse mapping [21], the PE-Net unrolls the forward mapping [18], and the PM-Net unrolls both mappings [16]. In general, learning-governed methods can achieve higher resolution than physics-governed ones thanks to incorporating prior knowledge through offline training. BPS has lower accuracy than the other three approaches belonging to learning with physics models. The PM-Net obtains the best accuracy by elaborately tailoring the neural network according to EM theory.
Figure 8. Comparisons of the reconstructed “Austria” model with different methods [16], [18], [21]. The true relative permittivity ${(}{\epsilon}_{r} = {\epsilon}{/}{\epsilon}_{0}{)}$ is 1.5. The imaging results are plotted in the same color scale. The ADMMs is taken as the benchmark. CS-Net [6] uses a neural network to provide initial guesses for gradient-based optimization. BPS [10] belongs to learning after physics processing. The SDM, PE-Net, and PM-Net are learning with physics models. Specifically, the SDM [21] learns the inverse mapping, the PE-Net [18] unfolds the forward modeling, and the PM-Net unfolds both forward and inverse mappings [16].
The comparisons of imaging time and memory of network parameters are depicted in Table 1. Note that the SDM and PE-Net are trained for two- and three-frequency imaging, respectively, while other methods are trained for single-frequency imaging. Therefore, the time/memory cost in the SDM and PE-Net for single-frequency imaging can be less than the shown values. The imaging time of the PE-Net and the PM-Net is at the same level, which is much faster than other methods. The CS-Net sequentially performs network inference and gradient-based optimization; hence, it takes the most time. BPS improves imaging speed by avoiding the EM-modeling process. We note that the speed of BPS in the original article [10] is faster than the presented one [16]. The SDM needs to rigorously solve Maxwell’s equations in the prediction stage, hence, it is slower than BPS, PE-Net, and PM-Net. In addition, the memory of neural network parameters is also compared. Smaller networks imply that fewer training data are needed. The SDM is the most memory consuming as it records several descent directions represented by full matrices. The PE-Net requires the second-largest amount of memory because it unrolls a number of complete EM-modeling processes into the neural network. In contrast, according to the inverse scattering theory, the PM-Net finds a better way to unroll approximated forward and inverse problems, which leads to a much smaller network size.
Table 1. Imaging time and memory of network parameters in different methods [16].
Using trainable layers to unroll linear imaging problems is time and memory efficient but only suitable for simple backgrounds because linear approximation neglects the interactions of EM fields in complex media. In highly inhomogeneous media, nonlinear EM imaging is required to quantitatively evaluate the target. The SDM provides a way that seamlessly combines ML and EM modeling, but the numerical forward modeling process limits its online prediction speed. The strategies unrolling PDE-based forward and inverse processes on neural networks for higher speed are proposed.
The methods in [22] and [16] are applicable when the background medium is known a priori (so that the Green’s function is known). They originate from different views, i.e., the mathematical and physical views. The former mimics the optimization process, while the latter progressively retrieves the permittivity by predicting more accurate electric fields, which is equivalent to taking more multiple scatterings into account. It is shown that partially unfolding forward and inverse problems according to the inverse scattering theory leads to a time- and memory-efficient neural network.
When the background medium is unknown, describing wave equations using partial differential operations is more appropriate. The work in [20] shows the feasibility of embedding partial differential operations into the network. This direction has potentially many applications for reconstructing complex media, such as biomedicine engineering and geophysical exploration.
We tentatively discuss some open challenges and opportunities in this field from three aspects: data, physics, and algorithm.
In detection, it is usually challenging to obtain the exact electric properties of targets, and the images in historical datasets often suffer from nonunique interpretations. Synthetically generating training data is time consuming for large-scale imaging problems. Therefore, public datasets that cover various typical applications are required so that researchers can train, test, and compare among different methods. The datasets should contain scenarios where the inverse problem is highly ill-posed, e.g., highly inhomogeneous media, multiple strong scatterers, and limited observations, to test the performance limit of imaging algorithms. In addition, the methodology used for evaluating the completeness of training datasets should be investigated [65].
Incorporating physics theory into data-driven methods is challenging. The DoI is partitioned into triangle (2D) or tetrahedral (3D) elements for accurate EM modeling in many applications. The element’s size, number, and topology are different case by case, which means that the number of neurons and their connections vary with training samples. A graph neural network may provide a solution [66], [67] to this challenge but needs further investigation. For large-scale EM problems, the number of elements may be millions or billions. The training cost will be a critical issue when the elements are expressed as neurons. Finally, DL techniques also provide new perspectives of integrating EM methods with other imaging modalities to achieve better resolution [68], [69], [70], but how to embed different physical principles in a unified neural network remains open.
The credibility of predictions needs to be improved. Current strategies mainly rely on statistical analyses of test datasets. An alternative may be checking data fitness after image reconstruction. Other methods, such as uncertainty analysis, are in urgent need [46], [71]. In addition, performance guarantees can be further investigated. Physics-embedded DL structures increase interpretability, which may benefit the progress of theoretical guarantees and thus release the burden on parameter tuning. Furthermore, although mean-squared error is mainly used as the optimization target in EM imaging, loss functions that minimize structure similarity [72], image features [73], or probability distribution [43] can be flexibly applied in the DL framework.
We surveyed three types of physics-embedded, data-driven imaging methods. Learning after physics processing is straightforward, and it allows for borrowing advanced DL techniques in image processing at a minimal cost. Its shortcoming is that the DNN training does not follow wave physics, leading to difficulties in processing out-of-distribution data. Therefore, it is suitable for fast estimation of target shapes and properties but has risks in quantitative imaging.
Learning with physics loss ensures training to obey wave physics and reduces the ill-posedness of imaging so that the neural networks are more robust than learning after physics processing. The solution process for the PDE and the inverse problem can be simultaneously trained using a PINN; however, it requires a large volume of data that may be difficult to collect in practice. Using a measurement loss function based on a predefined forward problem can perform better for inverting limited-aperture data. One limitation of this method is that physics cannot guide image reconstruction in online prediction. Compared to learning after physics processing, techniques of this type require more computation resources in the training stage.
Learning with physics models incorporates physics operators in both training and prediction. The imaging mainly relies on physics computation, while trained parameters provide the necessary prior knowledge for image reconstruction. It has better generalizability than former methods and can achieve real-time imaging. However, the neural network architecture needs to be tailored for different problems, increasing complexity in the design.
Artificial intelligence has blossomed in recent years due to the advancement of modern hardware and software, which builds the foundations of these progresses. Although the EM community is happy to embrace these changes, the reliability of data-driven imaging remains an issue. EM theory provides baselines for EM sensing and imaging. Embedding physics in DNNs can improve interpretability and generalizability and thus improves safety in real-world applications. Recent research in EM imaging has proven its feasibility, and we are sure this path will continue to expand in the next few years.
This work is supported in part by the Institute for Precision Medicine, Tsinghua University; National Natural Science Foundation of China (61971263 and 62171259); and Shuimu Tsinghua Scholar Program of Tsinghua University (2021SM031). Maokun Li is the corresponding author.
Rui Guo (guor93@tsinghua.edu.cn) received his Ph.D. degree in electrical engineering from Tsinghua University in 2021. He is currently a postdoctoral researcher with the Department of Electronic Engineering, Tsinghua University, Beijing 100084, China. His research interests include the theory and algorithms of machine learning-based multiphysical data inversion. He received the IECAS Outstanding Student Award, PIERS Best Student Paper Award, and Excellent Doctoral Dissertation Award from China Education Society of Electronics. He is also among the recipients of Shuimu Scholars at Tsinghua University. He is a Member of IEEE.
Tianyao Huang (huangtianyao@tsinghua.edu.cn) received his Ph.D. degree in electronics engineering from Tsinghua University in 2014. Since July 2017, he has been an assistant professor with the Intelligent Sensing Lab, Department of Electronic Engineering, Tsinghua University, Beijing 100084, China. His research interests include compressed sensing, radar signal processing, waveform design, and joint design of radar and communication. He is a Member of IEEE.
Maokun Li (maokunli@tsinghua.edu.cn) received his Ph.D. degree in electrical engineering from the University of Illinois at Urbana-Champaign, Champaign, Illinois, 61820, USA, in 2007. In 2014, he joined the Department of Electronic Engineering, Tsinghua University, Beijing, 100084, China. His research interests include fast algorithms in computational electromagnetics and their applications in antenna modeling, electromagnetic compatibility analysis, geophysical exploration, and biomedical imaging. He is an associate editor of IEEE Transactions on Antennas and Propagation and IEEE Transactions on Geoscience and Remote Sensing. He was among the recipients of the 2017 IEEE Ulrich L. Rohde Innovative Conference Paper Award and the 2019 PIERS Young Scientist Award. He is a Senior Member of IEEE.
Haiyang Zhang (haiyang.zhang@weizmann.ac.il) received his Ph.D. degree in information and communication engineering from Southeast University in 2017. He is currently an assistant professor in the School of Communication and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210000, China. His research interests include 6G near-field communications, learning and sampling theory, and physical-layer security. He is a Member of IEEE.
Yonina C. Eldar (yonina.eldar@weizmann.ac.il) received her Ph.D. degree in electrical engineering and computer science from the Massachusetts Institute of Technology (MIT) in 2002. She is a professor in the Department of Math and Computer Science at the Weizmann Institute of Science, Rehovot, 7610001, Israel, where she heads the center for Biomedical Engineering and Signal Processing. She is also a visiting professor at MIT, Cambridge, Massachusetts, 02412, USA, and the Broad Institute of MIT, and is an adjunct professor at Duke University, Durham, North Carolina. She is a member of the Israel Academy of Sciences and Humanities and a fellow of the European Association for Signal Processing. She has received the IEEE Signal Processing Society Technical Achievement Award, IEEE/AESS Fred Nathanson Memorial Radar Award, IEEE Kiyo Tomiyasu Award, Michael Bruno Memorial Award from the Rothschild Foundation, Weizmann Prize for Exact Sciences, and Wolf Foundation Krill Prize for Excellence in Scientific Research. She is the editor-in-chief of Foundations and Trends in Signal Processing and serves IEEE on several technical and award committees. She is a Fellow of IEEE.
[1] W. Chew, Waves and Fields in Inhomogeneous Media. New York, NY, USA: Springer-Verlag, 1990.
[2] A. Massa, D. Marcantonio, X. Chen, M. Li, and M. Salucci, “DNNs as applied to electromagnetics, antennas, and propagation—A review,” IEEE Antennas Wireless Propag. Lett., vol. 18, no. 11, pp. 2225–2229, Nov. 2019, doi: 10.1109/LAWP.2019.2916369.
[3] G. Wang, M. Jacob, X. Mou, Y. Shi, and Y. C. Eldar, “Deep tomographic image reconstruction: Yesterday, today, and tomorrow—Editorial for the 2nd special issue ‘machine learning for image reconstruction’,” IEEE Trans. Med. Imag., vol. 40, no. 11, pp. 2956–2964, Nov. 2021, doi: 10.1109/TMI.2021.3115547.
[4] M. Li et al., “Machine learning in electromagnetics with applications to biomedical imaging: A review,” IEEE Antennas Propag. Mag., vol. 63, no. 3, pp. 39–51, Jun. 2021, doi: 10.1109/MAP.2020.3043469.
[5] X. Chen, Z. Wei, M. Li, and P. Rocca, “A review of deep learning approaches for inverse scattering problems (Invited Review),” Prog. Electromagn. Res., vol. 167, pp. 67–81, Jan. 2020, doi: 10.2528/PIER20030705.
[6] Y. Sanghvi, Y. Kalepu, and U. K. Khankhoje, “Embedding deep learning in inverse scattering problems,” IEEE Trans. Comput. Imag., vol. 6, pp. 46–56, May 2019, doi: 10.1109/TCI.2019.2915580.
[7] Z. Lin, R. Guo, M. Li, A. Abubakar, T. Zhao, F. Yang, and S. Xu, “Low-frequency data prediction with iterative learning for highly nonlinear inverse scattering problems,” IEEE Trans. Microw. Theory Techn., vol. 69, no. 10, pp. 4366–4376, Oct. 2021, doi: 10.1109/TMTT.2021.3098769.
[8] A. Bora, A. Jalal, E. Price, and A. G. Dimakis, “Compressed sensing using generative models,” in Proc. Int. Conf. Mach. Learn., PMLR, 2017, pp. 537–546.
[9] L. Li, L. G. Wang, F. L. Teixeira, C. Liu, A. Nehorai, and T. J. Cui, “DeepNIS: Deep neural network for nonlinear electromagnetic inverse scattering,” IEEE Trans. Antennas Propag., vol. 67, no. 3, pp. 1819–1825, Mar. 2019, doi: 10.1109/TAP.2018.2885437.
[10] Z. Wei and X. Chen, “Deep-learning schemes for full-wave nonlinear inverse scattering problems,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 4, pp. 1849–1860, Apr. 2018, doi: 10.1109/TGRS.2018.2869221.
[11] X. Ye, Y. Bai, R. Song, K. Xu, and J. An, “An inhomogeneous background imaging method based on generative adversarial network,” IEEE Trans. Microw. Theory Techn., vol. 68, no. 11, pp. 4684–4693, Nov. 2020, doi: 10.1109/TMTT.2020.3015495.
[12] X. Ye, D. Yang, X. Yuan, R. Song, S. Sun, and D. Fang, “Application of generative adversarial network-based inversion algorithm in imaging two-dimensional lossy biaxial anisotropic scatterer,” IEEE Trans. Antennas Propag., early access, 2022, doi: 10.1109/TAP.2022.3164198.
[13] Y. Jin, Q. Shen, X. Wu, J. Chen, and Y. Huang, “A physics-driven deep-learning network for solving nonlinear inverse problems,” Petrophysics, vol. 61, no. 1, pp. 86–98, 2020, doi: 10.30632/PJV61N1-2020a3.
[14] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Comput. Phys., vol. 378, pp. 686–707, Feb. 2019, doi: 10.1016/j.jcp.2018.10.045.
[15] L. Bar and N. Sochen, “Strong solutions for PDE-based tomography by unsupervised learning,” SIAM J. Imag. Sci., vol. 14, no. 1, pp. 128–155, 2021, doi: 10.1137/20M1332827.
[16] J. Liu, H. Zhou, T. Ouyang, Q. Liu, and Y. Wang, “Physical model-inspired deep unrolling network for solving nonlinear inverse scattering problems,” IEEE Trans. Antennas Propag., vol. 70, no. 2, pp. 1236–1249, Feb. 2022, doi: 10.1109/TAP.2021.3111281.
[17] H. K. Aggarwal, M. P. Mani, and M. Jacob, “MoDL: Model-based deep learning architecture for inverse problems,” IEEE Trans. Med. Imag., vol. 38, no. 2, pp. 394–405, Feb. 2019, doi: 10.1109/TMI.2018.2865356.
[18] R. Guo, Z. Lin, T. Shan, X. Song, M. Li, F. Yang, S. Xu, and A. Abubakar, “Physics embedded deep neural network for solving full-wave inverse scattering problems,” IEEE Trans. Antennas Propag., early access, 2021, doi: 10.1109/TAP.2021.3102135.
[19] M. Shahriari, D. Pardo, J. A. Rivera, C. Torres-Verdín, A. Picon, J. Del Ser, S. Ossandón, and V. M. Calo, “Error control and loss functions for the deep learning inversion of borehole resistivity measurements,” Int. J. Numer. Methods Eng., vol. 122, no. 6, pp. 1629–1657, Mar. 2021, doi: 10.1002/nme.6593.
[20] Y. Hu, Y. Jin, X. Wu, and J. Chen, “A theory-guided deep neural network for time domain electromagnetic simulation and inversion using a differentiable programming platform,” IEEE Trans. Antennas Propag., vol. 70, no. 1, pp. 767–772, Jan. 2022, doi: 10.1109/TAP.2021.3098585.
[21] R. Guo, Z. Jia, X. Song, M. Li, F. Yang, S. Xu, and A. Abubakar, “Pixel- and model-based microwave inversion with supervised descent method for dielectric targets,” IEEE Trans. Antennas Propag., vol. 68, no. 12, pp. 8114–8126, Dec. 2020, doi: 10.1109/TAP.2020.2999741.
[22] R. Guo, T. Shan, X. Song, M. Li, F. Yang, S. Xu, and A. Abubakar, “Physics embedded deep neural network for solving volume integral equation: 2D case,” IEEE Trans. Antennas Propag., early access, 2021, doi: 10.1109/TAP.2021.3070152.
[23] R. Fu, Y. Liu, T. Huang, and Y. C. Eldar, “Structured LISTA for multidimensional harmonic retrieval,” IEEE Trans. Signal Process., vol. 69, pp. 3459–3472, Jun. 2021, doi: 10.1109/TSP.2021.3086593.
[24] R. Fu, T. Huang, L. Wang, and Y. Liu, “Block-sparse recovery network for two-dimensional harmonic retrieval,” Electron. Lett., vol. 58, no. 6, pp. 249–251, Mar. 2022, doi: 10.1049/ell2.12409.
[25] T. Shan, Z. Lin, X. Song, M. Li, F. Yang, and S. Xu, “Neural born iteration method for solving inverse scattering problems: 2D cases,” 2021, arXiv:2112.09831.
[26] N. Shlezinger, J. Whang, Y. C. Eldar, and A. G. Dimakis, “Model-based deep learning,” 2020, arXiv:2012.08405.
[27] T. Habashy and A. Abubakar, “A general framework for constraint minimization for the inversion of electromagnetic measurements,” Prog. Electromagn. Search, vol. 46, pp. 265–312, Jun. 2004, doi: 10.2528/PIER03100702.
[28] J.-M. Jin, Theory and Computation of Electromagnetic Fields. Hoboken, NJ, USA: Wiley, 2011.
[29] T. Klose, J. Guillemoteau, G. Vignoli, and J. Tronicke, “Laterally constrained inversion (LCI) of multi-configuration EMI data with tunable sharpness,” J. Appl. Geophys., vol. 196, p. 104,519, Jan. 2022, doi: 10.1016/j.jappgeo.2021.104519.
[30] M. S. Zhdanov, “New advances in regularized inversion of gravity and electromagnetic data,” Geophys. Prospecting, vol. 57, no. 4, pp. 463–478, Jun. 2009, doi: 10.1111/j.1365-2478.2008.00763.x.
[31] G. Vignoli, G. Fiandaca, A. V. Christiansen, C. Kirkegaard, and E. Auken, “Sharp spatially constrained inversion with applications to transient electromagnetic data,” Geophys. Prospecting, vol. 63, no. 1, pp. 243–255, Jan. 2015, doi: 10.1111/1365-2478.12185.
[32] A. Abubakar, P. M. Van den Berg, and J. J. Mallorqui, “Imaging of biomedical data using a multiplicative regularized contrast source inversion method,” IEEE Trans. Microw. Theory Techn., vol. 50, no. 7, pp. 1761–1771, Aug. 2002, doi: 10.1109/TMTT.2002.800427.
[33] S. Zhong, Y. Wang, Y. Zheng, S. Wu, X. Chang, and W. Zhu, “Electrical resistivity tomography with smooth sparse regularization,” Geophys. Prospecting, vol. 69, nos. 8–9, pp. 1773–1789, Oct./Nov. 2021, doi: 10.1111/1365-2478.13138.
[34] G. Vignoli and L. Zanzi, “Focusing inversion technique applied to radar tomographic data,” in Proc. Near Surf. 2005 11th Eur. Meeting Environ. Eng. Geophys., European Association of Geoscientists and Engineers, p. 13.
[35] L. C. Potter, E. Ertin, J. T. Parker, and M. Cetin, “Sparsity and compressed sensing in radar imaging,” Proc. IEEE, vol. 98, no. 6, pp. 1006–1020, Feb. 2010, doi: 10.1109/JPROC.2009.2037526.
[36] V. M. Patel, G. R. Easley, D. M. Healy, and R. Chellappa, “Compressed synthetic aperture radar,” IEEE J. Sel. Topics Signal Process., vol. 4, no. 2, pp. 244–254, Feb. 2010, doi: 10.1109/JSTSP.2009.2039181.
[37] M. Zhdanov and G. Hursan, “3D electromagnetic inversion based on quasi-analytical approximation,” Inverse Problems, vol. 16, no. 5, p. 1297, Oct. 2000, doi: 10.1088/0266-5611/16/5/311.
[38] A. V. Christiansen, E. Auken, C. Kirkegaard, C. Schamper, and G. Vignoli, “An efficient hybrid scheme for fast and accurate inversion of airborne transient electromagnetic data,” Explor. Geophys., vol. 47, no. 4, pp. 323–330, Aug. 2016, doi: 10.1071/EG14121.
[39] Q. Shen, J. Chen, and H. Wang, “Data-driver interpretation of ultradeep azimuthal propagation resistivity measurements: Transdimensional stochastic inversion and uncertainty quantification,” Petrophysics, vol. 59, no. 6, pp. 786–798, Dec. 2018, doi: 10.30632/PJV59N6-2018a4.
[40] T. M. Hansen, “Efficient probabilistic inversion using the rejection sampler—Exemplified on airborne EM data,” Geophys. J. Int., vol. 224, no. 1, pp. 543–557, Jan. 2021, doi: 10.1093/gji/ggaa491.
[41] G. De Pasquale, N. Linde, J. Doetsch, and W. S. Holbrook, “Probabilistic inference of subsurface heterogeneity and interface geometry using geophysical data,” Geophys. J. Int., vol. 217, no. 2, pp. 816–831, May 2019, doi: 10.1093/gji/ggz055.
[42] K. Xu, L. Wu, X. Ye, and X. Chen, “Deep learning-based inversion methods for solving inverse scattering problems with phaseless data,” IEEE Trans. Antennas Propag., vol. 68, no. 11, pp. 7457–7470, Nov. 2020, doi: 10.1109/TAP.2020.2998171.
[43] Z. Hu et al., “DPIR-Net: Direct PET image reconstruction based on the Wasserstein generative adversarial network,” IEEE Trans. Radiat. Plasma Med. Sci., vol. 5, no. 1, pp. 35–43, Jan. 2021, doi: 10.1109/TRPMS.2020.2995717.
[44] J. Xiao, J. Li, Y. Chen, F. Han, and Q. H. Liu, “Fast electromagnetic inversion of inhomogeneous scatterers embedded in layered media by Born approximation and 3-D U-Net,” IEEE Geosci. Remote Sens. Lett., vol. 17, no. 10, pp. 1677–1681, Oct. 2020, doi: 10.1109/LGRS.2019.2953708.
[45] Z. Wei and X. Chen, “Physics-inspired convolutional neural network for solving full-wave inverse scattering problems,” IEEE Trans. Antennas Propag., vol. 67, no. 9, pp. 6138–6148, Jun. 2019, doi: 10.1109/TAP.2019.2922779.
[46] Z. Wei and X. Chen, “Uncertainty quantification in inverse scattering problems with Bayesian convolutional neural networks,” IEEE Trans. Antennas Propag., vol. 69, no. 6, pp. 3409–3418, Oct. 2020, doi: 10.1109/TAP.2020.3030974.
[47] A. Abubakar, T. Habashy, V. Druskin, L. Knizhnerman, and D. Alumbaugh, “2.5 D forward and inverse modeling for interpreting low-frequency electromagnetic measurements,” Geophysics, vol. 73, no. 4, pp. F165–F177, Jul. 2008, doi: 10.1190/1.2937466.
[48] M. Shahriari, D. Pardo, A. Picón, A. Galdran, J. Del Ser, and C. Torres-Verdín, “A deep learning approach to the inversion of borehole resistivity measurements,” Comput. Geosci., vol. 24, no. 3, pp. 971–994, Apr. 2020, doi: 10.1007/s10596-019-09859-y.
[49] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, “Physics-informed machine learning,” Nature Rev. Phys., vol. 3, no. 6, pp. 422–440, May 2021, doi: 10.1038/s42254-021-00314-5.
[50] Y. B. Sahel, J. P. Bryan, B. Cleary, S. L. Farhi, and Y. C. Eldar, “Deep unrolled recovery in sparse biological imaging: Achieving fast, accurate results,” IEEE Signal Process. Mag., vol. 39, no. 2, pp. 45–57, Mar. 2022, doi: 10.1109/MSP.2021.3129995.
[51] V. Monga, Y. Li, and Y. C. Eldar, “Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,” IEEE Signal Process. Mag., vol. 38, no. 2, pp. 18–44, Mar. 2021, doi: 10.1109/MSP.2020.3016905.
[52] A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM J. Imag. Sci., vol. 2, no. 1, pp. 183–202, Mar. 2009, doi: 10.1137/080716542.
[53] K. Gregor and Y. Lecun, “Learning fast approximations of sparse coding,” in Proc. Int. Conf. Mach. Learn., 2010, pp. 399–406.
[54] X. Xiong and F. De la Torre, “Supervised descent method and its applications to face alignment,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 532–539, doi: 10.1109/CVPR.2013.75.
[55] R. Guo, M. Li, F. Yang, S. Xu, and A. Abubakar, “Application of supervised descent method for 2D magnetotelluric data inversion,” Geophysics, vol. 85, no. 4, pp. WA53–WA65, Jun. 2020, doi: 10.1190/geo2019-0409.1.
[56] Y. Hu, R. Guo, Y. Jin, X. Wu, M. Li, A. Abubakar, and J. Chen, “A supervised descent learning technique for solving directional electromagnetic logging-while-drilling inverse problems,” IEEE Trans. Geosci. Remote Sens., vol. 58, no. 11, pp. 8013–8025, Nov. 2020, doi: 10.1109/TGRS.2020.2986000.
[57] S. Lu, B. Liang, J. Wang, F. Han, and Q. H. Liu, “1-D inversion of GREATEM data by supervised descent learning,” IEEE Geosci. Remote Sens. Lett., vol. 19, pp. 1–5, 2022, doi: 10.1109/LGRS.2021.3053247.
[58] P. Hao, X. Sun, Z. Nie, X. Yue, and Y. Zhao, “A robust inversion of induction logging responses in anisotropic formation based on supervised descent method,” IEEE Geosci. Remote Sens. Lett., vol. 19, pp. 1–5, 2022, doi: 10.1109/LGRS.2021.3078756.
[59] Z. Jia, R. Guo, M. Li, G. Wang, Z. Liu, and Y. Shao, “3-D model-based inversion using supervised descent method for aspect-limited microwave data of metallic targets,” IEEE Trans. Geosci. Remote Sens., vol. 60, 2022, Art. no. 2001110, doi: 10.1109/TGRS.2021.3078542.
[60] Z. Lin, R. Guo, K. Zhang, M. Li, F. Yang, S. Xu, and A. Abubakar, “Neural network-based supervised descent method for 2D electrical impedance tomography,” Physiol. Meas., vol. 41, no. 7, p. 074003, Aug. 2020, doi: 10.1088/1361-6579/ab9871.
[61] K. Zhang, R. Guo, M. Li, F. Yang, S. Xu, and A. Abubakar, “Supervised descent learning for thoracic electrical impedance tomography,” IEEE Trans. Biomed. Eng. (1964–present), vol. 68, no. 4, pp. 1360–1369, Apr. 2021, doi: 10.1109/TBME.2020.3027827.
[62] R. Guo, M. Li, F. Yang, S. Xu, and A. Abubakar, “Regularized supervised descent method for 2-D magnetotelluric data inversion,” in Proc. SEG Tech. Program Expanded Abstr., Society of Exploration Geophysicists, 2019, pp. 2508–2512, doi: 10.1190/segam2019-3215540.1.
[63] L. Guo, M. Li, S. Xu, F. Yang, and L. Liu, “Electromagnetic modeling using an FDTD-equivalent recurrent convolution neural network: Accurate computing on a deep learning framework,” IEEE Antennas Propag. Mag., early access, 2021, doi: 10.1109/MAP.2021.3127514.
[64] P. M. V. Den Berg and R. E. Kleinman, “A contrast source inversion method,” Inverse Problems, vol. 13, no. 6, p. 1607, Dec. 1997, doi: 10.1088/0266-5611/13/6/013.
[65] M. Salucci, N. Anselmi, G. Oliveri, P. Calmon, R. Miorelli, C. Reboud, and A. Massa, “Real-time NDT-NDE through an innovative adaptive partial least squares SVR inversion approach,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 11, pp. 6818–6832, Aug. 2016, doi: 10.1109/TGRS.2016.2591439.
[66] W. Herzberg, D. B. Rowe, A. Hauptmann, and S. J. Hamilton, “Graph convolutional networks for model-based learning in nonlinear inverse problems,” IEEE Trans. Comput. Imag., vol. 7, pp. 1341–1353, Dec. 2021, doi: 10.1109/TCI.2021.3132190.
[67] A. Al-Saffar, L. Guo, and A. Abbosh, “Graph attention network for microwave imaging of brain anomaly,” 2021, arXiv:2108.01965.
[68] Y. Sun, B. Denel, N. Daril, L. Evano, P. Williamson, and M. Araya-Polo, “Deep learning joint inversion of seismic and electromagnetic data for salt reconstruction,” in Proc. SEG Tech. Program Expanded Abstr., Society of Exploration Geophysicists, 2020, pp. 550–554, doi: 10.1190/segam2020-3426925.1.
[69] P. Mojabi, M. Hughson, V. Khoshdel, I. Jeffrey, and J. LoVetri, “CNN for compressibility to permittivity mapping for combined ultrasound-microwave breast imaging,” IEEE J. Multiscale Multiphys. Comput. Tech., vol. 6, pp. 62–72, Apr. 2021, doi: 10.1109/JMMCT.2021.3076827.
[70] R. Guo, H. M. Yao, M. Li, M. K. P. Ng, L. Jiang, and A. Abubakar, “Joint inversion of audio-magnetotelluric and seismic travel time data with deep learning constraint,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 9, pp. 7982–7995, Sep. 2021, doi: 10.1109/TGRS.2020.3032743.
[71] S. Oh and J. Byun, “Bayesian uncertainty estimation for deep learning inversion of electromagnetic data,” IEEE Geosci. Remote Sens. Lett., vol. 19, pp. 1–5, 2022, doi: 10.1109/LGRS.2021.3072123.
[72] Y. Huang, R. Song, K. Xu, X. Ye, C. Li, and X. Chen, “Deep learning-based inverse scattering with structural similarity loss functions,” IEEE Sensors J., vol. 21, no. 4, pp. 4900–4907, Feb. 2021, doi: 10.1109/JSEN.2020.3030321.
[73] A. Zheng, K. Liang, L. Zhang, and Y. Xing, “A CT image feature space (CTIS) loss for restoration with deep learning-based methods,” Phys. Med. Biol., vol. 67, no. 5, p. 055010, Feb. 2022, doi: 10.1088/1361-6560/ac556e.
[74] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention (Lecture Notes in Computer Science, vol. 9351), N. Navab, J. Hornegger, W. Wells, and A. Frangi, Eds., Cham, Switzerland: Springer, 2015, pp. 234–241, doi: 10.1007/978-3-319-24574-4_28.
Digital Object Identifier 10.1109/MSP.2022.3198805