James B. Aimone, Shashank Misra
©SHUTTERSTOCK.COM/ANDREY SUSLOV
Achieving brain-like efficiency in computing requires a co-design between the development of neural algorithms, brain-inspired circuit design, and careful consideration of how to use emerging devices. The recognition that leveraging device-level noise as a source of controlled stochasticity represents an exciting prospect of achieving brain-like capabilities in probabilistic neural algorithms, but the reality of integrating stochastic devices with deterministic devices in an already-challenging neuromorphic circuit design process is formidable. Here, we explore how the brain combines different signaling modalities into its neural circuits as well as consider the implications of more tightly integrated stochastic, analog, and digital circuits. By acknowledging that a fully CMOS implementation is the appropriate baseline, we conclude that if mixing modalities is going to be successful for neuromorphic computing, it will be critical that device choices consider strengths and limitations at the overall circuit level.
Neuromorphic computing seeks to achieve the brain’s unparalleled energy efficiency and computational capabilities by emulating many aspects of the brain’s unique approach to computation [1], [2]. One aspect of the brain that has become widely cited in neuromorphic efforts, but in practice has not often been systematically explored, is noise. The fact that neural computation is noisy is taken almost as a defining feature of the brain. It is something that we perceive about ourselves as humans and is something that we can readily find evidence for in experimental observations of the brain’s activity and behaviors. This in turn has implications for what people view as the long-term value proposition of neuromorphic computing [3].
There is, of course, an appeal to the idea of embracing, rather than fighting, noise in electronics [4]. The need for high reliability in modern microelectronics drives inefficiency; we typically operate transistors at many orders of magnitude of energy more than the Landauer limit to ensure reliability at the device level. Also, our computing systems typically consist of multiple layers of error correction that are applied in circuits, architectures, and software. The costs of today’s microelectronics fabrication are also heavily weighted by the immense requirements of minimal device-to-device variability and other noise suppression considerations. Afterward, despite all of these efforts to eliminate it, we then very often turn around and introduce noise back into computing systems, at great expense, through pseudorandom number generation at the software level [5] to perform tasks ranging from Monte Carlo sampling [6] to stochastic optimization [7].
So, is noise in neuromorphic computing a good thing? On the one hand, it promises a path toward cost and energy efficiency while also providing desired capabilities for probabilistic computations. On the other hand, the conflation of neuromorphic computing with noise has biased the neuromorphic community away from applications where numerical precision is valued.
Here, we seek to step back and look at what noise means in the brain and how this is applicable to neuromorphic computing. We do this with a self-interested engineering perspective. While the brain clearly leverages stochasticity in its own computations, it is by no means necessary to include this into an engineered system. But given the perceived value of noise in both capability and efficiency, we need to ask what type of noise we want. To do this, we start by looking at the brain itself.
While noise is widely accepted both inside and outside neuroscience as a fundamental property of the brain, the reality is less obvious. Like other complex physical systems, behaviors in the brain that appear to be due to noise often, upon deeper inspection, have deterministic explanations. In 1995, Mainen and Sejnowski demonstrated that spike generation in neurons was in fact highly reliable if neuron inputs are controlled for no variation [8]. The perceived noise in a neuron’s output is largely explained by the variation of the neuron’s input. This classic finding has rather large implications in neuroscience; the fact that spike generation is itself reliable is critical in most neuroscience theories of learning and brain function.
What is clear is that the brain exhibits stochasticity at the synaptic scale, with synaptic vesicle release being best described as stochastic for many decades [9]. While this stochasticity has been dismissed by some as just an implication of the biological substrate, such rationalization rarely proves to be valid in neuroscience, and it is worth noting that evolution has gone out of its way to minimize the effects of noise in many other aspects of neural computation, such as in the spike generation example described previously. Furthermore, this synaptic release noise is highly regulated as some synapses are more reliable than others. This variation of synaptic release is one of the key knobs of synaptic plasticity, particularly in short-term facilitation and depression of synapses.
Nevertheless, in the fields of neuromorphic computing and theoretical neuroscience, weights are typically viewed as deterministic, and any stochasticity, if present, is attributed to a neuron’s spike generation. In large part this is due to the pressing need to abstract away some aspect of neural complexity. Neurons receive inputs from thousands of upstream neurons; to consider each of those inputs as a random variable introduces a level of complexity most techniques in theoretical neuroscience are ill suited to handle, whereas mean-field averaging over synapses and neurons is a well-accepted technique for describing the brain.
The stochasticity of the brain’s synapses highlights a rather stark disconnect between how brains operate and how neuromorphic computing has evolved as a field. Biological synapses are believed to be low precision in their impact, but the probability that a response will be generated is a continuous-valued Bernoulli probability. In contrast, efforts to develop analog synapse devices have often emphasized a need for high reliability and precision in synapses, while digital CMOS neuromorphic hardware, such as the Intel Loihi, IBM TrueNorth, and SpiNNaker platforms, have focused on deterministic calculations, with limited randomness included through simple circuit pseudorandom number generators (PRNGs).
It should not be surprising that the brain leverages stochasticity at the most numerous neuronal device type (synapses) and emphasizes determinism at the relatively less numerous and physically larger neuron level. Any error correction is going to incur costs, and for spike generation, these costs are reflected in the incredibly high density of sodium channels in the active zone of somas, which ensure that any local variation in channel number or membrane geometry will have minimal, if any, effect on action potential generation. Further, the rapid nonlinear dynamics of the action potential provides an amplifier effect that ensures that the voltage of the action potential is considerably above any noise sensitivities that would be due to lower channel densities (Figure 1).
Figure 1. Neuron axons (shown in beige) behave digitally, silent with the exception of discrete spike events (shown in red), while neuron dendrites are analog in their behavior (shown in green). (a) Synapses transduce digital spikes from the axon with a probabilistic transmission of information across the synapse due to the stochastic release of neurotransmitters (shown in blue), leading to an analog signal in the postsynaptic dendrite. (b) The cell body (or soma) of the neuron converts the analog information in the dendrites to a digital spiking output that is transmitted, potentially over long distances, by the axon. Illustrative voltage traces show that voltage evolves differently over time at different locations of a neuron in response to an input neuron spiking.
In the case of the synapse, the stochasticity is driven by small number effects (only a low integer number of vesicles are available) and thermal effects on the biophysics of synaptic molecules. Presumably, biology could build extremely reliable synapses that would be correspondingly larger and have more reliable synaptic machinery to ensure release if triggered. Indeed, there are cases in the brain where such reliability exists, such as in the early auditory system [10], where sensory information about auditory frequencies must be transmitted reliably. This suggests that in most areas of the brain the computational advantages of retaining stochasticity outweigh the advantages of precision.
What does this mean for computing? From an engineering perspective, it is instructive about where in a neuromorphic system it is critical to prioritize precision. Without much effort, we can construct Bernoulli devices with much higher event-to-event reliability than biological synapses [11], [12]; however, we should still expect that the analog accumulation of current from many synapses will be a source of noise. As a result, from a neuromorphic perspective, the value of high-precision synaptic devices is limited by the reality of noise in the downstream analog computation. However, the reverse is likely also true; if the analog computation is summing over random variates, the value of increasing precision of the analog synaptic weights will be limited.
This observation that the use of stochastic synapses throttles the need for high precision in either the stochastic or analog synaptic devices is a convenient outcome for neuromorphic computing, but it is not a free lunch. The cost incurred by having a stochastic synaptic integration process is the requirement for neural sampling algorithms, which has increasingly been recognized in the neuroscience and neuromorphic communities as one of the most promising paths forward for the neural computing field [13], [14].
There are two immediate downsides to using sampling algorithms other than the deterministic neural algorithms for which the community is most familiar. The most obvious is that we would be moving away from mainstream applications, particularly those algorithms in modern artificial intelligence (AI) that primarily leverage high-precision deterministic arithmetic in GPUs. The second downside is time; as the name implies, sampling algorithms require multiple samples, and typically the efficiency of those samples leaves much to be desired. In Monte Carlo sampling, for example, the accuracy of a solution scales with the square of the number of samples, making it a rather inefficient estimator if an efficient deterministic approach is available.
Despite these downsides, embracing a probabilistic perspective of neuromorphic computing offers real advantages [15], [16]. While most programming paradigms assume determinism, this is not due to a lack of need for or interest in probabilistic algorithms. Monte Carlo sampling is the go-to method for a number of computational applications, such as finance and particle physics. Similarly, the modern AI community would very much like to have uncertainties and confidence directly reflected in neural network algorithms, and early approaches, such as restricted Boltzmann machines and deep belief networks, were in part intended to compute uncertainty-aware artificial neural networks (ANNs) [17]. Although these approaches have taken a back seat to deterministic algorithms, such as convolutional neural networks, because of the latter’s suitability for GPUs [18], they provide an opportunity for a probabilistic neuromorphic computing approach [19]. Likewise, sampling algorithms often rely on simpler arithmetic than deterministic alternatives, opening the door for embarrassingly parallel variants that can take advantage of the extreme parallelism of neuromorphic hardware [20] (Figure 2).
Figure 2. (a) Deterministic algorithms produce a single answer that may require considerable computational work but only needs to be done once. (b) Sampling algorithms ideally use a less computationally intensive algorithm to produce estimates of a problem’s answer, but many samples are required.
For these reasons, exploring a computer that embraces stochasticity in a brain-like ubiquitous fashion would appear to be desirable. Furthermore, noise is ubiquitous in all devices, so there is an unconstrained space to select from to match computational opportunities. Of course, simply using noisy devices instead of deterministic devices does not do anything useful—it only provides a lower quality computer. Rather, we need to engineer a computing system that uses stochasticity when we need stochasticity and relies on deterministic components when we need determinism. Herein lies the challenge.
While every device is noisy, the aspects that make one noise source more desirable than the next are not intuitively obvious. Fundamentally, there are three general physics origins for noise: number fluctuations, thermodynamic fluctuations, and quantum fluctuations, and quantum superposition [15]. A more useful categorization is in terms of the randomness the devices produce. Thermal fluctuations on a continuum, like resistor noise, produce Gaussian fluctuations, while photon counting is a Poisson process. A particularly useful abstraction comes from devices that produce weighted coinflips, which are in effect Bernoulli random variables. This can be produced from carefully thresholding devices with thermal or quantum fluctuations, like a magnetic tunnel junction, or from precisely designed superposition states. A device providing a random source taking the shape of a Bernoulli coinflip—1 with a probability p, 0 with probability 1 − p—is mathematically straightforward and powerful yet is conceptually simple to control from a circuit level. This device can be reconfigured to sample a wide range of useful distributions, while only requiring the transversal of a tree structure with branch conditions and storage.
Using stochastic devices outside of a purely analog context requires them to be connected to other computing elements, most typically devices contributing to deterministic logic. The instant these stochastic devices become part of a larger computing system we have to consider the costs of the overall circuit—the efficient analog computation, the transduction and communication required to move the information to the traditional computing piece, and the digital computing piece. Because conventional PRNGs are software generated, they effectively fall victim to the von Neumann bottleneck. Moving the stochasticity out of software to a physics-derived hardware source may provide benefits in efficiency, but to realize the advantages of hardware stochasticity, it is necessary to have algorithms that push more of the workload onto the tailored solution. If pushed to a stand-alone accelerator, this will simply create a communication bottleneck. This is part of the appeal of a neuromorphic framework with ubiquitous stochasticity.
Interestingly, this puts device-level stochasticity into the same challenging situation that emerging devices for synapses face [21]. To realize the benefits of these approaches, both stochastic devices and emerging memory devices need to be used at large scale, but to be used at large scale requires that these devices be tightly integrated with logic to avoid losing all of their benefit to communication costs.
One implication of this need to integrate stochastic devices closely with other neural logic devices is that it becomes critical to assess a device’s potential value with the overall system in mind. Standard device measures, such as device-to-device variability, speed, energy, and lifetime, are still important, but equal attention needs to be paid to the cost of integrating the device with other logic. It becomes irrelevant if one device is faster than another if the faster device has a larger impact on the performance of the integrated traditional logic. Similarly, just as important as stand-alone device characteristics are where it can be used—a device that can only be used in crossbar arrays is more constrained than one that can be interspersed arbitrarily with conventional CMOS devices. The former limits acceleration to vector–matrix situations, while the latter can also be used to do things that do not make sense in a vector–matrix context, like conditional traversal of the branches of a tree.
Thus, from an architectural standpoint, there is a strong incentive to do this integration at as low a level as possible. However, information movement from conventional CMOS devices nonconventional devices incurs some signal transduction costs, like discrimination and latching. These incur significant costs in efficiency, even to the extent that the efficiency of the stochastic device itself is an insignificant part of the efficiency of the associated circuitry. Pushing integration to lower levels requires paying the transduction cost more frequently.
We believe that there are only two paths forward. First, we can push more of the calculation onto specialized devices that can be fabricated together and directly perform more of a calculation. By considering the brain’s example of linking stochastic synapses with the analog integration and deterministic spike generation, we can perhaps realize a similar compromise that pairs the strengths of different devices together while shielding the overall circuit from each device’s respective drawbacks. Alternatively, we should also consider looking back to CMOS, which provides the highest degree of scaling and control today, to inquire how much of those nonconventional goals can be accomplished using established fabrication capabilities. We will explore both of these here in turn.
As explained previously, if it is unrealistic to have a stochastic non-CMOS device tightly integrated with CMOS logic, one possibility is to construct the full neural logic from non-CMOS devices (Figure 3). Taking this path poses two separate challenges. The first is the diversity of functions that can be performed using the non-CMOS elements. Many of the devices proposed for neuro-inspired computing—resistive materials, magnetic tunnel junctions, and so on—can be used to generate Bernoulli coinflips and also have the proposed application as memory devices. A significant opportunity lies in adding a range of elementary operations—logical functions, conditional logic, addition, and so on—using none other than the non-CMOS elements. Taking this route requires evaluating the tradeoff in accuracy and efficiency of nontraditional implementations of these functions, and of the complexities of tuning the same device to perform different computational functions side by side.
Figure 3. (a) Analog crossbars implement vector–matrix multiplications natively by Kirchhoff’s law summation of currents weighted by device conductances. While efficient, this fully analog implementation can be challenged by the accumulation of noise across different parts of the circuit. (b) A brain-like mixture of stochastic communication, analog dendrites, and spiking outputs potentially allows the noise of different devices to be directed toward making sampling algorithms efficient.
A second limitation comes from considering how the devices are organized at the circuit level. There have been extensive studies explaining how resistive materials, magnetic tunnel junctions, carbon nanotubes, and other devices can be used to emulate the biophysics of neurons. Above the single device level, these have been investigated extensively in crossbar architectures.
While crossbars are convenient for engineering and research purposes, it is likely that the device community will need to look beyond crossbars if nonconventional devices are to make a larger impact on the neuromorphic field. Fully analog crossbars face significant challenges in scaling and from sneak paths between devices that complicate their programming. For instance, memristors were originally proposed as a scalable two-terminal synapse [22], but their use in resistive memories has pushed toward including transistors to isolate them for training and programming, eliminating much of the scaling advantage. Algorithmically they are limiting as well; while dense vector–matrix multiplication is a key kernel for ANNs, it is less efficient for many other tasks as well as being the highly optimized target of both conventional systolic array accelerators and modern GPUs.
These limitations are not necessarily of the devices themselves, but rather the architecture that has become somewhat synonymous with their exploration. This illustrates a rather important point—if one is not careful, it is relatively easy to get stuck in a local research minimum with a circuit architecture that is not well suited for either the devices or the algorithms being considered. It is useful to compare today’s analog crossbars to how the brain’s circuit combines its devices. While biological neural circuits may look like crossbars in their mappings, the digital axon " stochastic chemical synapse " analog dendrite process described previously is quite different from a resistive memory and provides a separation between inputs and outputs that enables more robust scaling and reliability. Expanding the range of device capabilities being considered can also allow exotic approaches to match the expected performance of conventional algorithms. For example, hybrid circuits that blend several device-level capabilities have been proposed for tasks such as stochastic optimization that achieve state-of-the-art approximation algorithm performance [23], [24]. The recognition that there is value in integrating stochastic devices alongside analog devices thus presents both device communities with an opportunity to revisit some of the circuit assumptions that have heretofore defined their devices’ use.
Neuro-inspired computation using conventional CMOSs has recently mostly happened at the level of a general-purpose computer, after a penalty to convert everything to deterministic computation has already been paid. Here, we suggest an underexplored opportunity to implement a different functionality at the few-device level itself. Can a CMOS device, which is mostly used as a switch, be used to generate stochastic signals? Taking this approach has many obvious payoffs considering the last section—the CMOS itself can obviously already be arranged into a flexible range of computational elements and cascaded in ways that are already well established.
At some level of abstraction, non-CMOS devices merely offer some transfer function—they sweep out some range of output voltages given an input voltage. All the devices mentioned earlier require thermal fluctuations and a hysteretic transfer function to generate Bernoulli coinflips. Put this way, an SRAM cell or a CMOS Schmitt trigger provides a similar mapping of input–output signals, but without the added burden of needing to transduce signals to integrate with CMOS logic. However, there is also the matter of how the devices are used, which conceptually requires analog tuning—the probability p. In general, using a CMOS platform still provides the benefit that the exploration of these circuit-level problems is more accessible. Furthermore, there is an opportunity to interleave fixed-point computation concepts in low-level circuits, which has the effect of moving between digital and analog domains without transduction.
While our discussion has focused on probabilistic computing, many of the opportunities and limitations apply more generally to neuro-inspired computing. Generally, different approaches are often put in opposition to one another, but we suggest a more holistic approach to solving some standout questions that have emerged from our exploration of stochastic approaches to computing. The path to bringing new device capabilities into neuromorphic systems needs to be more deliberately taken, and the success of those trajectories may depend on whether these walks are taken together or in isolation.
What device technologies and materials can be integrated with CMOS without degrading the performance of conventional logic (Figure 4)? In the absence of this kind of integration, the only solutions are stand-alone accelerators, which suffer from the von Neumann bottleneck. Often a significant computational payoff needs to be identified before direct integration (an expensive problem to solve) is addressed. We suggest that as new functionality is identified in novel devices, translating that functionality to few-transistor CMOS proxies needs to be rewarded as a research activity precisely because it enables the discovery of system-level opportunities. Equally important is identifying simple proxies for key aspects of integration of new technologies, e.g., material compatibility, which is also often an underappreciated research area.
Figure 4. Recent trends in heterogeneous integration have often emphasized physically separating circuit components to enable fabrication, but this can introduce a different form of communication bottleneck. Horizontal axis: Mixing modalities into a single chip will eliminate bottlenecks. Vertical axis: Proper integration of computational modalities (stochastic, digital, analog) minimizes conversion inefficiencies. Green is efficient, orange is inefficient.
Next, there is the question of where the boundary between analog and digital should be. Efficiency improvement in transduction at different levels—the single device level (e.g., a latch) or the small circuit level (e.g., a crossbar)—may lack the flashy appeal of demonstrating the greater efficiency of a novel device, but it is likely to be just as important and impacts entire classes of devices. In addition to understanding what the transduction efficiency is at different levels, we also need to understand the ways in which analog computing elements can be cascaded. This is a general problem that can either be addressed in exotic devices or translated to analog CMOS devices and made available to a much broader range of researchers. This is a common problem in the design of analog circuits like amplifiers and needs to be treated like a significant research challenge in the neuro-inspired computing area.
Finally, what new computations can be performed in the analog and digital domains? An obvious opportunity is that device-level computational functions need to be stated simply in terms of input–output transfer curves. This can be useful in moving logic into the analog domain, but also in finding CMOS proxies of novel device functions. There also needs to be innovation in assembling stand-alone computational circuits, whether it comes from cascading analog functions or from finding CMOS circuits that perform functions outside of traditional logic and interleaving fixed-point representations to mimic analog functionality.
All of this brings us back to the brain. We should look to the brain not only for inspiration for new devices, but also for how best to combine multiple types of imperfect devices that leverage different physics for computing. This leads to the counterintuitive conclusion that future devices do not have to be engineered to be perfect as the benefits of such optimization will likely be lost upon integration anyway. Rather, the brain shows us that an assembly of imperfect parts, if appropriately constructed, can show incredible robustness. While this clashes with the last half century of microelectronics engineering, it gets to the heart of what biology tells us. A neuromorphic system does not need to rely on perfect electronic devices, but that system should be constructed to leverage its devices’ imperfections.
This material is based upon work supported by the U.S. Department of Energy (DOE) Office of Science Office of Basic Energy Science and Advanced Scientific Computing Research as part of the Co-Design in Microelectronics Program. We thank Lindsey Aimone for editorial assistance.
This article has been authored by an employee of National Technology & Engineering Solutions of Sandia, LLC under Contract DE-NA0003525 with the DOE. The employee owns all rights, title, and interest in and to the article and is solely responsible for its contents. The U.S. Government retains and the publisher, by accepting the article for publication, acknowledges that the U.S. Government retains a nonexclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this article or allow others to do so, for U.S. Government purposes. The DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan at https://www.energy.gov/downloads/doe-public-access-plan.
This article describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the article do not necessarily represent the views of the DOE or the U.S. Government.
James B. Aimone (jbaimon@sandia.gov) is a Distinguished Member of Technical Staff in the Center for Computing Research at Sandia National Laboratories, Albuquerque, NM 87185-5820 USA, where he is a lead researcher in leveraging computational neuroscience to advance artificial intelligence and in using neuromorphic computing platforms for future scientific computing applications. He currently leads a multi-institution U.S. Department of Energy Office of Science Microelectronics Co-Design project titled COINFLIPS (which stands for CO-designed Influenced Neural Foundations Inspired by Physical Stochasticity), which is focused on developing a novel probabilistic neuromorphic computing platform, and he currently leads several other research efforts on designing neural algorithms for scientific computing applications and neuromorphic machine learning implementations. Prior to joining the technical staff at Sandia, he was a postdoctoral research associate at the Salk Institute for Biological Studies, with a Ph.D. in computational neuroscience from the University of California, San Diego and bachelor’s and master’s degrees in chemical engineering from Rice University. He is a Senior Member of IEEE.
Shashank Misra (smisra@sandia.gov) is a Distinguished Member of Technical Staff in the Microsystems Engineering Systems and Applications center at Sandia National Laboratories, Albuquerque, NM 87185-5820 USA. He earned a doctorate in physics from the University of Illinois at Urbana-Champaign in 2005, and since 2013, has been a member of the research staff at Sandia National Laboratories. His research interests revolve around using chemical vapor deposition and scanning tunneling microscopy-based lithography to fabricate atomically precise dopant devices in semiconductors. This includes the discovery of novel metastable materials based on atomic-scale modification and understanding chemistry at the atomic scale at surfaces. Application areas for the devices span sensing, quantum demonstrations, and stochastic devices for probabilistic and neuromorphic computing.
[1] J. B. Aimone, “Neural algorithms and computing beyond Moore’s law,” Commun. ACM, vol. 62, no. 4, pp. 110–110, Apr. 2019, doi: 10.1145/3231589.
[2] J. B. Aimone, “A roadmap for reaching the potential of brain-derived computing,” Adv. Intell. Syst., vol. 3, no. 1, 2021, Art. no. 2000191, doi: 10.1002/aisy.202000191.
[3] K. Meier, “Special report: Can we copy the brain? - The brain as computer,” IEEE Spectr., vol. 54, no. 6, pp. 28–33, Jun. 2017, doi: 10.1109/MSPEC.2017.7934228.
[4] T. Hylton, T. M. Conte, and M. D. Hill, “A vision to compute like nature: Thermodynamically,” Commun. ACM, vol. 64, no. 6, pp. 35–38, Jun. 2021, doi: 10.1145/3431282.
[5] J. Von Neumann, “Various techniques used in connection with random digits,” J. Res. Nat. Bur. Standards Appl. Math Ser., vol. 12, pp. 36–38, 1951. [Online] . Available: https://mcnp.lanl.gov/pdf_files/InBook_Computing_1961_Neumann_JohnVonNeumannCollectedWorks_VariousTechniquesUsedinConnectionwithRandomDigits.pdf
[6] N. Metropolis and S. Ulam, “The Monte Carlo method,” J. Amer. Statist. Assoc., vol. 44, no. 247, pp. 335–341, Sep. 1949, doi: 10.1080/01621459.1949.10483310.
[7] M. X. Goemans and D. P. Williamson, “Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming,” J. ACM, vol. 42, no. 6, pp. 1115–1145, Nov. 1995, doi: 10.1145/227683.227684.
[8] Z. F. Mainen and T. J. Sejnowski, “Reliability of spike timing in neocortical neurons,” Science, vol. 268, no. 5216, pp. 1503–1506, Jun. 1995, doi: 10.1126/science.7770778.
[9] K. M. Franks, C. F. Stevens, and T. J. Sejnowski, “Independent sources of quantal variability at single glutamatergic synapses,” J. Neuroscience, vol. 23, no. 8, pp. 3186–3195, Apr. 2003, doi: 10.1523/JNEUROSCI.23-08-03186.2003.
[10] E. G. Krächan, A. U. Fischer, J. Franke, and E. Friauf, “Synaptic reliability and temporal precision are achieved via high quantal content and effective replenishment: Auditory brainstem versus hippocampus,” J. Physiol., vol. 595, no. 3, pp. 839–864, Feb. 2017, doi: 10.1113/JP272799.
[11] S. Liu et al., “Random bitstream generation using voltage-controlled magnetic anisotropy and spin orbit torque magnetic tunnel junctions,” IEEE J. Explor. Solid-State Computat., vol. 8, no. 2, pp. 194–202, Dec. 2022, doi: 10.1109/JXCDC.2022.3231550.
[12] L. Rehm et al., “Stochastic magnetic actuated random transducer devices based on perpendicular magnetic tunnel junctions,” Physical Rev. Appl., vol. 19, no. 2, Feb. 2023, Art. no. 024035, doi: 10.1103/PhysRevApplied.19.024035.
[13] L. Buesing, J. Bill, B. Nessler, and W. Maass, “Neural dynamics as sampling: A model for stochastic computation in recurrent networks of spiking neurons,” PLoS Comput. Biol., vol. 7, no. 11, Nov. 2011, Art. no. e1002211, doi: 10.1371/journal.pcbi.1002211.
[14] J. D. Smith et al., “Neuromorphic scaling advantages for energy-efficient random walk computations,” Nature Electron., vol. 5, no. 2, pp. 102–112, Feb. 2022, doi: 10.1038/s41928-021-00705-7.
[15] S. Misra et al., “Probabilistic neural computing with stochastic devices,” Adv. Mater., early access, 2022, doi: 10.1002/adma.202204569.
[16] K. Y. Camsari, R. Faria, B. M. Sutton, and S. Datta, “Stochastic p-bits for invertible logic,” Physical Rev. X, vol. 7, no. 3, Jul. 2017, Art. no. 031014, doi: 10.1103/PhysRevX.7.031014.
[17] G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, vol. 313, no. 5786, pp. 504–507, Jul. 2006, doi: 10.1126/science.1127647.
[18] S. Hooker, “The hardware lottery,” Commun. ACM, vol. 64, no. 12, pp. 58–65, Dec. 2021, doi: 10.1145/3467017.
[19] E. Neftci, “Stochastic neuromorphic learning machines for weakly labeled data,” in Proc. IEEE 34th Int. Conf. Comput. Des. (ICCD), 2016, pp. 670–673, doi: 10.1109/ICCD.2016.7753355.
[20] J. D. Smith et al., “Solving a steady-state PDE using spiking networks and neuromorphic hardware,” in Proc. Int. Conf. Neuromorphic Syst., 2020, pp. 1–8, doi: 10.1145/3407197.3407202.
[21] M. Lanza, G. Molas, and I. Naveh, “The gap between academia and industry in resistive switching research,” Nature Electron., vol. 6, no. 4, pp. 1–4, Apr. 2023, doi: 10.1038/s41928-023-00954-8.
[22] S. H. Jo, T. Chang, I. Ebong, B. B. Bhadviya, P. Mazumder, and W. Lu, “Nanoscale memristor device as synapse in neuromorphic systems,” Nano Lett., vol. 10, no. 4, pp. 1297–1301, Apr. 2010, doi: 10.1021/nl904092h.
[23] B. H. Theilman, Y. Wang, O. D. Parekh, W. Severa, J. D. Smith, and J. B. Aimone, “Stochastic neuromorphic circuits for solving MAXCUT,” in Proc. IEEE Int. Parallel Distrib. Process. Symp. (IPDPS), 2023, pp. 779–787, doi: 10.1109/IPDPS54959.2023.00083.
[24] W. A. Borders, A. Z. Pervaiz, S. Fukami, K. Y. Camsari, H. Ohno, and S. Datta, “Integer factorization using stochastic magnetic tunnel junctions,” Nature, vol. 573, no. 7774, pp. 390–393, Sep. 2019, doi: 10.1038/s41586-019-1557-9.
Digital Object Identifier 10.1109/MED.2023.3298873
Date of current version: 15 September 2023