JP7824554B2 - Machine Learning Systems - Google Patents

Description

The present invention relates to a machine learning system that uses optical circuits to perform calculations for machine learning using deep neural networks.

Information processing using machine learning with deep neural networks (DNNs) is gaining attention. DNNs are large-scale nonlinear networks in which numerous neurons with nonlinear responses are connected by synapses. Deep learning techniques, particularly those using networks with multi-layered neuron configurations, have been widely applied. This technology has been reported to demonstrate superior performance in a wide range of fields, including image and speech recognition, robot control, and artificial data generation. However, as the number of layers and neurons (nodes) increases, the scale of the required nonlinear matrix operations explodes, posing a major challenge: the increased time and power required for inference and learning. In recent years, information processing techniques using optical circuits as computing elements have attracted attention as a method to solve these issues.

In general machine learning based on DNN, an output is obtained by applying the transformation of the following equation (1) to the signal x ^(L) in the Lth layer.
x ^(L+1) = f(ω ^(L) x ^(L) ) (1)

where f is a nonlinear function such as tanh or relu, x ^(L) and x ^(L+1) are M- and N-dimensional input/output vectors, respectively, and ω ^(L) is an M-row, N-column transformation matrix called a weight matrix. In machine learning, the desired final output is obtained by learning and updating the weight matrix ω ^(L) .

In a DNN based on optical computation using optical circuits, part or all of this nonlinear matrix multiplication is performed optically. A known example of a conventional optical computation device is the optical computation device described in Non-Patent Document 1. Figure 1 illustrates a conventional optical computation device. This optical computation device 10 is composed of a digital circuit 11, an optical interferometer array 12, a high-speed digital-to-analog converter (hereinafter referred to as "DAC") 13 for an input signal x that converts a digital signal from the digital circuit into an analog signal, a DAC 14 for a weight w, a high-speed analog-to-digital converter (hereinafter referred to as "ADC") 15 that converts an analog output signal y from the optical interferometer array into a digital signal, an E/O converter 16 that converts an input electrical signal into an optical signal, and an O/E converter 17 that converts an output optical signal into an electrical signal. By arranging multiple optical interferometer arrays as the optical interferometer array, the intensity of the input signal x is converted according to the phase value of the interferometer. The output optical signal y can be expressed as shown in Equation (2).

Equation (2) corresponds to optically executing ω ^(L) x ^(L) in the equation (1) in the aforementioned DNN. Regarding nonlinear conversion, in the case of Non-Patent Document 1, the received light is converted into an electrical signal and then digitally processed. However, by placing some kind of nonlinear optical element downstream of the interference system, it is possible to fully execute equation (1), including nonlinear operations. In equation (1), the matrix product ω ^(L) x ^(L) determines the number of operations, so even if the nonlinear operation is performed after the output optical signal is photoelectrically converted, it is still useful. In this embodiment, learning the weights in equation (1) is achieved by learning and updating the set phase value of the interference system. The phase value of the interference system can be dynamically changed, for example, by changing the refractive index of the waveguide using the thermo-optic effect.

The operating bandwidth of the optical circuit described above is extremely wide (> THz), so in principle it is possible to calculate equation (2) at a higher frequency than with electronic circuits. Furthermore, by taking advantage of the parallelism of light's space, wavelength, etc., it is possible to perform calculations in parallel. Therefore, optical calculation circuits can exhibit features such as superior power consumption and calculation speed compared to calculation elements that use existing electronic circuits.

In practical applications, it is desirable for input x and output y to be input/output to a digital electronic circuit, as shown in Figure 1. This is because, in the optical signal state, it is difficult to store data and perform general-purpose calculations and data manipulation, including pre- and post-processing. Therefore, as shown in Figure 1, a DAC, ADC, EO converter, and OE converter are required. As such, in conventional optical calculation technology, optical signals are generated from digital circuits via DACs and EO converters, so the calculation speed of the calculation circuit is limited by the operating bandwidth of the DAC/ADC and EO/OE converter. In general, the operating bandwidth of DACs/ADCs is slower than that of EO/OE converters, so the operating bandwidth of the DAC/ADC in particular was the main limiting factor in calculation speed.

In addition, conventional optical computing devices were configured as a type of in-memory computer, in which the memory was transferred to a separate memory called the phase value of the optical interference system, and calculations were performed on that interference system.

Y. Shen et al., "Deep learning with coherent nanophotonic circuits", Nature photonics, Vol.11, P441-447, June 2017. D. P. Lopes et al., "Multipurpose self-configuration of programmable photonic circuits", NATURE COMMUNICATIONS, December 11, 2020, (2020)11:6359, p.1-11.＜URL: https://www.nature.com/articles/s41467-020-19608-w＞ Kaname Jinguji and Manabu Oguma, "Optical Half-Band Filters", JOURNAL OF LIGHTWAVE TECHNOLOGY, April 2000, VOL.18, NO.2, p.252-259. M. Nakajima et al. "Optical Arbitrary Waveform Processing of Over 100 Spatial Channels for Optical Performance Monitoring", JOURNAL OF LIGHTWAVE TECHNOLOGY, January 15, 2019, VOL. 37, NO. 2, p.291-299. <URL:https://ieeexplore.ieee.org/abstract/document/8465965>

The present disclosure is made to solve these problems of the prior art, and one of its objectives is to provide a machine learning system that can eliminate the bottleneck of optical computation caused by DAC/ADC. Another objective is to provide a machine learning system in which the optical weight generator and the optical computation core are arranged separately, and the weighted optical signal generated by the optical weight generator is distributed to the optical computation core via optical fiber.

To achieve this objective, one embodiment of the present disclosure is a machine learning system having the following configuration:

This machine learning system includes an optical weight generator that generates one or more weighted wavelength-multiplexed lights, the optical weight generator comprising a memory in which weights are stored and one or more optical arbitrary waveform shapers that generate weighted wavelength-multiplexed lights by mapping the weights stored in the memory in the wavelength direction and the time direction, and an optical calculation core comprising an electronic circuit and an optical circuit unit. The optical weight generator and the optical calculation core are connected by one or more optical fibers. The optical circuit unit is configured to perform a product-sum operation between the value indicated by an input signal from the electronic circuit and the weight indicated by the weighted wavelength-multiplexed light from the optical weight generator. Signals are input and output between the electronic circuit and the optical circuit unit via an analog-to-digital converter and/or a digital-to-analog converter with an operating bandwidth lower than the signal bandwidth of the weighted wavelength-multiplexed light input to the optical circuit unit.

FIG. 1 is a diagram showing a conventional optical computing device. FIG. 2 is a diagram illustrating a configuration of a machine learning system according to an embodiment of the present disclosure. FIG. 3 is a diagram illustrating a first configuration example of an optical AWG included in the optical weight generator according to the embodiment of the present disclosure. FIG. 4 is a diagram illustrating a second configuration example of an optical AWG included in the optical weight generator according to the embodiment of the present disclosure. FIG. 5 is a diagram illustrating a third configuration example of an optical AWG included in the optical weight generator according to the embodiment of the present disclosure. FIG. 6 is a diagram showing a schematic configuration of an up-clocked optical arithmetic circuit according to an embodiment of the present disclosure. FIG. 7 is a diagram showing a schematic configuration of a down-clocked optical arithmetic circuit according to an embodiment of the present disclosure. FIG. 8 is a diagram illustrating a configuration of an optical circuit unit that implements a two-layer fully connected perceptron (MLP) according to an embodiment of the present disclosure. FIG. 9 is a diagram illustrating a configuration of an optical circuit unit that implements a recurrent neural network (RNN) model according to an embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of the layout configuration of a machine learning system according to an embodiment of the present disclosure.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
An embodiment of a machine learning system according to the present disclosure will be described as an example with reference to Figure 2. Figure 2 is a diagram showing the configuration of a machine learning system 20 according to an embodiment of the present disclosure. In this embodiment, as shown in Figure 2, an optical weight generator 21 that multiplexes and distributes the weights ω _ij in the above equation (2) in terms of wavelength, time, and space, and an element 22 (hereinafter referred to as an "optical calculation core") that executes equation (1) are separate and connected by one or more optical fibers 23.

The optical weight generator 21 includes a memory 24 that stores and holds the weights ω _ij and one or more optical arbitrary waveform generators 25 (hereinafter referred to as "optical AWGs") that generate a weighted wavelength-division multiplexed (WDM) optical signal by mapping the weighted signal _{ω ij} stored in the memory in the wavelength λ direction and the time t direction. The optical weight generator 21 also includes a processor and an input/output interface that control the memory 24 and the optical AWG 25. The optical weight generator distributes the weights as a wavelength-division multiplexed (WDM) optical signal via an optical fiber 23. As described below, the optical computation core 22 may perform multiple matrix operations depending on the network topology. Therefore, it is desirable for the optical weight generator 21 to have the function of distributing multiple weighted wavelength-division multiplexed optical signals in parallel. This can also be rephrased as the function of distributing a spatially multiplexed (SDM) wavelength-division multiplexed (WDM) optical signal. In this embodiment, the modulation speed of the weighted multiplexed optical signal is desirably faster than the sampling rate of the DAC/ADC of the optical computation core, which will be described later. The ratio of the modulation speed (i.e., operating speed) of the optical AWG to the sampling rate (i.e., operating speed) of the DAC/ADC at this time is hereinafter referred to as the oversampling ratio (OS ratio).

The optical calculation core 22 constituting the machine learning system of this embodiment is composed of, for example, a PC 26 including a memory and processor for storing input and output data, a DAC 27 that converts digital input data into an analog signal, an ADC 28 that converts output data from the optical circuit unit into a digital signal, and an optical circuit unit 30 that performs photoelectric calculations using the analog RF signal from the DAC. The optical circuit unit 30 and PC 26 are connected via an FPGA or DAC/ADC board 29 including the DAC 27 and DAC 28, and a communication interface 31 composed of, for example, PCIE or Ethernet (registered trademark). The optical circuit unit 30 is configured to perform a product-sum calculation between the value indicated by the input signal input via the DAC 27 from an electronic circuit such as the PC 26 and the weight indicated by the weighted wavelength-multiplexed light from the optical weight generator.

In the optical calculation core 22 of this embodiment, the DAC 27 must parallelize M vector dimensions of the input signal, and the ADC 28 must parallelize K vector dimensions of the output signal, but their respective sampling rates (hereinafter also referred to as "operating bandwidth") may be slower than the weighting signal of the optical AWG described above. As will be described later, this low-speed input signal is upconverted to the optical signal bandwidth by the optical circuit unit of this embodiment, and calculations based on equation (1) are performed. The final output of the calculations is then downconverted again to a low-speed bandwidth signal. This provides excellent functionality, such as enabling calculations in the optical signal bandwidth without being limited by the operating bandwidths of the DAC and ADC.

(Light Weight Generator)
Next, a specific example of the configuration of the optical AWG 25 included in the optical weight generator that constitutes the machine learning system of this embodiment will be described with reference to FIGS.

A first example of an optical AWG configuration is a system using an SDM-WDM optical modulator. The SDM-WDM optical modulator 35 is realized, for example, by parallelizing Mach-Zehnder modulators for the number of spatial channels x wavelength channels. The example shown in Figure 3 includes optical modulators 36 ₁ , 36 ₂ , ... 36 _M that modulate the phase of an optical signal according to the input weight, and a wavelength MUX 37 that multiplexes the output light from each optical modulator. While this example shows a configuration in which optical modulators are parallelized for the number of wavelength channels, multiple configurations such as those shown in Figure 3 can be used to parallelize for the number of spatial channels. This first example configuration has excellent ease of implementation, as it can be implemented using commercially available WDM modulators, etc. However, it has disadvantages such as the fact that the signal bandwidth of the optical multiplexed signal depends on the operating bandwidth of the optical modulator, making it difficult to sufficiently increase the OS ratio, and that an optical modulator array for the number of spatial channels x wavelength channels is still required.

Next, a second configuration example of an optical AWG will be described with reference to FIG. 4 . The second configuration example of an optical AWG is a system in which an optical pulse δ(t) is introduced into a waveguide-type optical filter. The second configuration example 40 of the optical AWG shown in FIG. 4 includes a wavelength demultiplexer 42 that wavelength-divides pulse light input from a pulse light source 41, waveguide-type optical filters 43 ₁ , 43 ₂ , ... 43 _M to which the wavelength-divided wavelength light is input, and a wavelength multiplexer 44 that multiplexes the output light from each waveguide-type optical filter. FIG. 4 shows an example of a configuration in which optical filters are parallelized for the number of wavelength channels. For example, a waveguide-type optical filter can generate any designed impulse response by configuring it as shown in the optical filter 43 in the bubble in FIG. 4 . The optical filter 43 is composed of optical waveguides 47 ₁ , 47 ₂ , 47 ₃ , ... 47 _i that are branched by an optical coupler 46 and coupled by an optical coupler 49. The optical waveguides 47 ₁ , 47 ₂ , 47 ₃ , ..., 47 _i each constitute a delay line in which the delay difference increases sequentially by θ. Furthermore, each optical waveguide is provided with a phase shifter 48 ₁ , 48 ₂ , 48 ₃ , ..., 48 _i . Note that each optical waveguide may be provided with a variable attenuator as necessary. Weighting is performed by the phase shifter of each of the optical waveguides 47 ₁ to 47 _i . With this configuration, the optical pulse δ(t) undergoes the following transformation:

In Equation (3), θ represents the delay time difference of the delay interference system, _and ^ω'i represents the weighting component due to the i-th delay line. Assuming that δ(t) is a signal with an intensity of 1 only at a repetition period of ^N'θ , the configuration of Figure 4 can generate any weighted signal over time according to the weighting settings of the optical filter. Furthermore, by parallelizing the configuration of Figure 4 as many times as the number of spatial channels, weights for spatial channels x wavelength channels can be multiplexed and generated. This method has the advantage of not using an optical modulator and can be configured using passive waveguides. Furthermore, although the OS ratio depends on θ, θ can be set arbitrarily by adjusting the delay length of the waveguide, which allows the OS ratio to be set arbitrarily. On the other hand, this method has the disadvantage of requiring an optical filter array equal to the number of spatial channels x wavelength channels. For simplicity's sake, the optical filter in Figure 4 is described using an example of a finite impulse response (FIR) filter. However, the optical circuit configuration is not limited to the illustrated example, as long as the impulse response can be designed. Another method for obtaining an arbitrary impulse response can be configured using an infinite impulse response (IIR) filter. As an example, a configuration such as that shown in Non-Patent Document 2 can be used. Furthermore, as another configuration using an FIR filter, a lattice type filter can be used. As an example, a configuration in which an optical interference system is formed in multiple stages can be used, as shown in Non-Patent Document 3. Such a configuration is also included in the second configuration example.

Furthermore, a third configuration example of an optical AWG will be described with reference to FIG. 5 . The third configuration example of an optical AWG is a method in which an optical pulse δ(t) is introduced into a spatial optical system-type optical filter. The calculation principle is similar to that of the second configuration example, but this method uses a spatial optical system, making it possible to perform the calculation of Equation (3) for the required number of spatial channels × wavelength channels in a single device. For example, this can be realized by a configuration in which an input optical pulse signal is introduced into a spatial optical system, as shown in FIG. 5 . FIG. 5 shows a third configuration example 50. The third configuration example 50 includes a pulse light source 51 and a spatial optical filter 52 to which an input pulse signal from the pulse light source and a weight ω _ij are input. For example, the spatial optical filter 52 can be realized by the configuration shown in the balloon in FIG. 5 . The spatial optical filter 52 includes an input optical fiber 521, a collimating lens 522, a diffraction grating 523, a Fourier lens 524, a spatial light modulator (SLM) 525, a Fourier lens 526, a diffraction grating 527, a collimating lens 528, and an output optical fiber 529. Pulsed light from the input optical fiber 521 passes through the collimating lens 523 and is split by the diffraction grating 523 to form different beam spots on the SLM 525 for each spatial channel and wavelength channel. The SLM 525 can provide different impulse responses for each channel by setting a phase pattern for each channel. The transmitted (or reflected) light from the SLM 525 is combined again by the diffraction grating 527, passes through the collimating lens 528, and is output from the output optical fiber 529. This configuration example has the excellent advantage of being able to perform the calculation of the above-described equation (3) all at once using a single device. Note that by using a reflective element (e.g., reflective liquid crystal on silicon, LCOS, or digital mirror device, DMD) in the SLM, the optical system can be folded, and therefore the input optical fibers 521 and 529, collimating lenses 522 and 528, diffraction gratings 523 and 527, and Fourier lenses 524 and 526 can each be configured using the same components. This contributes to achieving excellent functions, such as reducing the number of parts and the housing size of the spatial optical filter 52. Note that for the spatial optical filter 52 in FIG. 5, a configuration in which only wavelength channels are manipulated collectively is shown for simplicity of explanation. However, as in Non-Patent Document 4, for example, spatial multiplexing can be achieved by spatially separating multiple pulse input light beams in the y-axis direction, and then dispersing each input pulse light beam separated in the y-axis direction in the x-axis direction, thereby integrating spatial channels into a single device.

(Optical computing core)
Next, the configuration of the optical computation core constituting the machine learning system of this embodiment will be described with reference to Figures 6 and 7. For simplicity, the following description will be given assuming M = K, but the scope of the present invention is not limited to this condition. First, the main components of the optical computation core will be described. To optically perform a calculation such as that shown in Equation (1), it is necessary to be able to perform the matrix multiplication operation and nonlinear operations shown in Equation (2) above. The calculation operation shown in Equation (2) above can be decomposed into a vector dot product and its sum. In this embodiment, the optical circuit unit is configured by combining optical computation circuits that perform vector dot product and sum operations using two different methods, making it possible to upconvert low-speed signals from a DAC to the signal bandwidth of an optical signal and perform the calculation. Hereinafter, the two different types of optical computation circuits will be referred to as the up-clocked type and the down-clocked type, respectively.

FIG. 6 is a diagram illustrating the outline of the configuration of an arithmetic element constituting an up-clocked optical arithmetic circuit. FIG. 6( a) shows an example of an arithmetic element 60 that performs a vector dot product and constitutes an up-clocked optical arithmetic circuit. The arithmetic element that performs a vector dot product and constitutes this up-clocked optical arithmetic circuit is called a Type I vector dot product element. The Type I vector dot product element 60 includes a wavelength demultiplexer 61, optical modulators 62 ₁ , 62 ₂ , ... 62 _M , and a wavelength multiplexer 63. In the Type I vector dot product element 60, the wavelength demultiplexer 61 separates the weighted wavelength-multiplexed light from the optical weight generator into wavelengths λ ₁ , λ ₂ , ... λ _M , and each separates it into optical modulators 62 ₁ , 62 ₂ , ... 62 _M. RF signals X ₁ , X ₂ ... X _M input from the DAC are input to the optical modulators as RF signals. Here, the signal band of the RF signal is slower than the signal band of the weighted wavelength multiplexed light in accordance with the OS ratio.

In each optical modulator, as shown by the signal waveform in Fig. 6(b), the weighted optical signal from the optical weight generator is modulated by the RF signal X _i for each wavelength λ _i , thereby performing a multiplication operation for each time domain. As a result, an optical signal in which the input value indicated by the RF signal and the weight indicated by the weighted optical signal are subjected to a vector dot product operation is arranged in the time/wavelength space and output.

This output can be described as the matrix in Figure 6(c). As shown in Figure 6(c), the column direction of this matrix is the time axis direction and the row direction is the wavelength axis direction. As explained above, with the Type I vector dot product calculation element 60 in Figure 6(a), M modulators can perform MN calculations for each clock of the low-speed RF signal, so the signal bandwidth of the output light output as a result of this calculation is faster by the OS ratio than the signal bandwidth of the original RF signal. This output signal is multiplexed by the wavelength MUX 63 and output as wavelength-multiplexed light. This Type I vector dot product calculation element 60 can speed up the signal bandwidth of the output light by the OS ratio than the signal bandwidth of the original RF signal.

FIG. 6( d ) is a diagram showing the schematic configuration of a summing operation element 64 constituting an up-clocked optical arithmetic circuit. FIG. 6( d ) shows an example of an arithmetic element 64 constituting an up-clocked optical arithmetic circuit that performs summing. This arithmetic element 64 that performs summing is called a Type I summing operation element. The Type I summing operation element 64 shown in FIG. 6( d ) can be configured with a light-receiving element, for example, a photodetector (PD) 65. Generally, PDs do not have significant wavelength dependence in the bands used for WDM communications (e.g., C band). Therefore, reception of the wavelength-multiplexed optical signal output from the vector dot product operation circuit by the PD 65 can be considered as a summing operation in the wavelength direction. Specifically, as shown by the circled area of the matrix in FIG. 6( e ), addition is performed in the row direction, which is the wavelength axis direction, to output an RF signal y _i .

In this way, the matrix arithmetic operation of equation (2) can be performed by an up-clocked optical arithmetic circuit that combines the arithmetic elements shown in Figures 6(a) and 6(d). Because the signal bandwidth of the RF signal output from this up-clocked optical arithmetic circuit increases in speed according to the OS ratio, an ADC with a low operating bandwidth cannot resolve the output signal.

Therefore, in the optical circuit unit of this embodiment, this up-clocked optical arithmetic circuit is combined with the down-clocked optical arithmetic circuit shown in Figure 7.

Next, the downclocked optical arithmetic circuit will be described with reference to FIG. 7 . FIG. 7( a) shows an arithmetic element 70 that constitutes the downclocked optical arithmetic circuit and that calculates a vector dot product. This arithmetic element 70 that calculates a vector dot product is called a Type II vector dot product arithmetic element. The Type II vector dot product arithmetic element 70 shown in FIG. 7( a) includes an optical modulator 71 to which weighted wavelength-multiplexed light delivered from an optical weight generator and a high-speed RF signal output from an upclocked optical arithmetic circuit are introduced as an RF input signal. The RF signal output from the upclocked optical arithmetic circuit is sped up in accordance with the OS ratio, and a signal y _N corresponding to the dimension N of the vector is multiplexed in the time series direction. This sped-up RF signal becomes the RF input signal to the optical modulator 71. Here, it is desirable that the weight introduced into the optical modulator 71 be different from the weight introduced in the upclocked optical arithmetic circuit. The vector dot product calculation by this optical modulator 71 can also be described by the matrix shown in FIG. 7( b). As shown in the matrix of Fig. 7(b), the relationship between wavelength and time is reversed compared to the matrix of Fig. 6(b). In this way, the Type II vector dot processor 70 can use the optical modulator 71 to calculate the vector dot product of the input value indicated by the RF signal and the weight indicated by the weighted optical signal.

Next, referring to FIG. 7( c), the arithmetic element 72 that performs summation and constitutes the down-clocked optical arithmetic circuit will be described. This arithmetic element 72 is referred to as a Type II summation element. The Type II summation element 72 is configured to demultiplex the wavelength-multiplexed light output from the vector dot product circuit using a wavelength demultiplexer 73 and introduce each wavelength light into different PDs 74 ₁ , 74 ₂ , ..., 74 _M for each wavelength channel. The operating bandwidths of these PDs are set lower than the signal bandwidth of the optical signal output from the vector dot product element. For example, the relationship between the optical signal bandwidth and the PD operating bandwidth can be set as shown in FIG. 7( e). This effectively averages the signals for each wavelength channel in the time direction. This averaging operation can be interpreted as effectively performing a summation operation in matrix calculations along the time direction. Specifically, in the matrix of FIG. 7( e), as shown by the boxes, the values of each row are added in the row direction corresponding to the time axis. Instead of setting the operating bandwidth of the PD low, an RF filter or RF amplifier having a bandwidth lower than the signal bandwidth of the optical signal may be inserted downstream of the PD. With this configuration, it becomes possible to output an RF signal that is down-converted from the high-speed wavelength-multiplexed light in the optical signal bandwidth output from the vector dot product optical calculation element to a low signal bandwidth that can be resolved by a low-speed ADC.

Therefore, by combining the up-clocked optical arithmetic circuit shown in Figure 6 with the down-clocked optical arithmetic circuit shown in Figure 7, an optical circuit unit can be realized that has a low signal bandwidth for the input and output RF signals and can perform vector dot product operations at a speed that is increased by the OS ratio.

In addition, by using an optical unit circuit that combines an up-clocked optical arithmetic circuit and a down-clocked optical arithmetic circuit as the optical unit circuit 30 of the optical arithmetic core of the machine learning system shown in Figure 2, it is possible to provide a machine learning system that can eliminate the bottleneck of optical arithmetic caused by the DAC/ADC.

Figure 8(a) illustrates the configuration of an optical circuit unit 80 that combines the up-clocked optical arithmetic circuit shown in Figure 6 and the down-clocked optical arithmetic circuit shown in Figure 7 to implement a two-layer fully connected perceptron (MLP). Figure 8(b) shows an equivalent network model of the MLP realized by this optical circuit unit 80. Combining a Type I vector dot product arithmetic element 81, a Type I summing arithmetic element 82, a Type II vector dot product arithmetic element 83, and a Type II summing arithmetic element 84 as shown in Figure 8(a) enables two-layer MLP calculations. The RF signal input to the Type I vector dot product arithmetic element 81 and the RF signal output from the Type II summing arithmetic element 84 are low-speed RF signals that are input to and output from the electronic circuit via a low-speed DAC 85 or ADC 86. A high-speed RF signal is input from the Type I summation element 82 to the Type II vector dot product element 83. A nonlinear activation function is required in the hidden layer. For example, as shown by f in FIG. 8( a), by introducing a Mach-Zehnder modulator at the Type II vector dot product element 83, it is possible to apply a sine-shaped nonlinear function. While the example shown here is a two-layer MLP, the same operation can be extended to accommodate any number of layers. Furthermore, by imposing constraints on the weights delivered by the optical weight signal input by the optical weight generator, it is possible to perform operations such as convolution operations with the same configuration.

The optical circuit unit 80 in FIG. 8( a) described above executes feedforward neural network calculations without recursive coupling, but by configuring it as shown in FIG. 9( a), it is possible to implement an optical circuit unit that calculates a recursive neural network (RNN) model described by equations (4) and (5).

Here, Ω, M, and ω represent the connections of the input layer, intermediate layer (RNN layer), and output layer, respectively.

Figure 9(a) illustrates the configuration of an optical circuit unit 90 that implements an RNN by combining the up-clocked optical arithmetic circuit shown in Figure 6 and the down-clocked optical arithmetic circuit shown in Figure 7. Figure 9(b) shows an equivalent network model of the RNN realized by this configuration. As shown in Figure 9(a), RNN calculations are possible by combining a Type I vector dot-product calculation element 91, a Type I addition calculation element 92, a Type II vector dot-product calculation element 93, and a Type II addition calculation element 94. The RF signal input to the Type I vector dot-product calculation element 91 and the RF signal output from the Type II addition calculation element 94 are low-speed RF signals that are input to and output from the electronic circuit via a low-speed DAC 95 or ADC 96. High-speed RF signals are input from the Type I addition calculation element 92 to the Type II vector dot-product calculation element 93. In this case, a nonlinear activation function is still required in the hidden layer. However, for example, by introducing a Mach-Zehnder modulator at the location of the Type II vector dot product optical processor 93 indicated by f in FIG. 9(a), it is possible to apply a sinusoidal nonlinear function.

In the configuration shown in Figure 9(a), the input to the optical circuit unit is an RF signal input from a low-speed DAC, and the output signal is an RF signal that can be resolved by a low-speed ADC.

Next, let us consider the operation speed of the configurations shown in Figures 8 and 9. The optical circuit unit 80 that realizes the MLP shown in Figure 8 is considered to be equivalent to decomposing Equation (2) into two layers per unit time. The operation speed (operations per sec [OPS]) is expressed by the following Equation (6):
OPS = (2MN+(M-1)+(N-1)) f _da/ad (6)

Here, M and N represent the number of dimensions of the input/output layer and the number of dimensions of the hidden layer, respectively. However, it is assumed here that the number of dimensions of the input/output layer is the same. M corresponds to the number of parallel DAC/ADCs, and N corresponds to the OS ratio. For example, assume that the operating band of the optical AWG of the optical weight generator and the sampling rate of the DAC/ADC are f _awg = 50 GSa/s and f _da/ad = 100 MSa/s, respectively. In this case, the OS ratio of equation (2) is 500. In this case, assuming that the number of parallel DACs is M = 100, it is estimated that a calculation speed of approximately 10 TOPS can be achieved.

Similarly, let us consider the calculation speed of the configuration that realizes the RNN in Fig. 9. The configuration that realizes the RNN in Fig. 9 is considered to be equivalent to solving equations (4) and (5) per unit time. The calculation speed is expressed by the following equation (7).
OPS = (2MN+N ² +(M-1)+(N-1)) f _da/ad (7)

Assuming the same sampling rate and parallelism as in the case of Figure 6(a), it is estimated that a calculation speed of approximately 250 TOPs can be achieved. These calculation speed values are performance indices that exceed the theoretical calculation performance of a typical CPU (approximately 500 TOPs).

(Form of use)
Conventional configurations (e.g., Figure 1) are a type of in-memory computer in which the memory is transferred to a separate memory for storing the phase values of an optical interferometer and calculations are performed on that interferometer. In contrast, the machine learning system disclosed herein separates the optical weight generator, which has the memory function for storing weights and the distribution function, from the optical calculation core that performs the calculations, resulting in a configuration similar to that of general-purpose computing. Furthermore, in conventional general-purpose computers, weight distribution is performed via electronic cables, so it is desirable to place the memory and calculation circuit (e.g., CPU) as close as possible from the perspective of power and wiring. In contrast, the machine learning system disclosed herein distributes weights from the optical weight generator to the optical calculation core via optical fiber, allowing weight information to be transferred to the optical calculation core with almost no power consumption (no loss during propagation). Furthermore, by utilizing the wavelength, time, and spatial multiplicity of light, weight information can be transferred using several optical fibers (or a single multi-core fiber). Therefore, unlike conventional computing units, the optical weight generator 101 and the optical computation cores 102, 102 can be located far apart, as shown in FIG. 10, and the optical weight generator 101 and the optical computation core 102 can be connected by an optical fiber 105. Note that FIG. 7 also shows the optical circuit unit 103 and digital circuit 104 included in the optical computation core 102. In this way, in the machine learning system of this embodiment, the optical weight generator and the optical computation core can be located far apart, allowing them to be placed in different racks or different buildings. For example, a usage scenario in which the client side, where the optical computation core is located, and the server side, where the weights are distributed, can be separated is possible. In this case, the client side only requires memory consumption for input and output [O(N)], and can outsource the storage and distribution of weight information [O(N ² )] to the server side. Furthermore, the client side can perform calculations upconverted to the high-speed optical signal bandwidth on the server side via a low-speed DAC/ADC. In addition, by distributing the signal of the optical weight generator to multiple clients, it is possible to learn in a distributed manner.

As described above, the machine learning system disclosed herein enables the separation of computational load between the client side and the server side as described above, enabling low-load, memory-saving calculations on the client side.

It is possible to provide a machine learning system that can eliminate the bottleneck of optical calculations caused by DAC/ADC.

It is also possible to provide a machine learning system in which the optical weight generator and the optical calculation core are arranged separately, and the weight optical signal generated by the optical weight generator is distributed by the optical calculation core via optical fiber.

Claims (9)

an optical weight generator for generating one or more weighted wavelength multiplexed lights,
a memory in which weights are stored;
an optical weight generator including one or more optical arbitrary waveform shapers that generate weighted wavelength-multiplexed light by mapping the weights stored in the memory in the wavelength direction and the time direction;
an optical calculation core including an electronic circuit and an optical circuit unit;
A machine learning system comprising:
the optical weight generator and the optical calculation core are connected by one or more optical fibers;
the optical circuit unit is configured to perform a product-sum operation on a value indicated by an input signal from the electronic circuit and a weight indicated by the weighted wavelength-multiplexed light from the optical weight generator,
A machine learning system characterized in that signal input and output between the electronic circuit and the optical circuit unit is performed via an analog-to-digital converter and/or a digital-to-analog converter with an operating bandwidth lower than the signal bandwidth of the weighted wavelength multiplexed light input to the optical circuit unit. The optical circuit unit of the optical calculation core comprises:
an up-clocked optical arithmetic circuit that performs a product-sum operation on an input value indicated by an RF signal in the operating band of the analog-to-digital converter input from the electronic circuit and a weight indicated by the weighted wavelength multiplexed light from the optical weight generator, and outputs the result as an RF signal in the signal band of the weighted wavelength multiplexed light;
The machine learning system according to claim 1, characterized in that it is configured in combination with a down-clocked optical arithmetic circuit that outputs the result of a product-sum operation between the value indicated by the RF signal in the signal band of the weighted wavelength multiplexed light and the weight indicated by the weighted wavelength multiplexed light from the optical weight generator as an RF signal in the operating band of the digital-to-analog converter. The up-clocked optical arithmetic circuit comprises:
a plurality of optical modulators;
a wavelength demultiplexer that demultiplexes the weighted wavelength multiplexed light and inputs the demultiplexed light to each of the optical modulators;
a wavelength MUX that multiplexes the output light from each of the optical modulators and outputs it as wavelength-multiplexed output light; and
a summing operation element including a light receiving element that converts the wavelength multiplexed output light into an RF signal,
The machine learning system of claim 2, wherein each of the optical modulators is configured to modulate each input wavelength light using an RF signal input via the analog-to-digital converter. The down-clocked optical arithmetic circuit comprises:
a vector dot product calculation element including one optical modulator that modulates and outputs the weighted wavelength multiplexed light using the RF signal in the signal band of the weighted wavelength multiplexed light;
A plurality of light receiving elements;
The machine learning system according to claim 2, characterized in that it is configured to include a wavelength DEMUX that demultiplexes input wavelength-multiplexed light into each wavelength and outputs it to each light receiving element, and a summation calculation element that includes The optical arbitrary waveform shaper of the optical weight generator includes:
a plurality of optical modulators that modulate light of different wavelengths according to input weights;
The machine learning system according to claim 1, further comprising a wavelength MUX that multiplexes the wavelength lights output from each of the optical modulators. The optical arbitrary waveform shaper of the optical weight generator includes:
a pulsed light source;
a wavelength demultiplexer that demultiplexes the impulse light from the pulse light source to generate light of a plurality of wavelengths;
a plurality of waveguide-type optical filters that generate impulse responses according to the weights of the wavelength lights output from the wavelength demultiplexer;
The machine learning system according to claim 1, further comprising a wavelength MUX that multiplexes the wavelength lights output from each of the optical filters. The optical arbitrary waveform shaper of the optical weight generator includes:
a pulsed light source;
The machine learning system of claim 1, further comprising a spatial filter that receives impulse light from the pulsed light source and generates one or more weighted wavelength multiplexed lights. A machine learning system as described in any one of claims 1 to 7, characterized in that the optical weight generator and the optical calculation core are located in separate racks or separate buildings. A machine learning system as described in any one of claims 1 to 7, characterized in that it comprises multiple optical calculation cores, and the weight wavelength multiplexed light from the optical weight generator is distributed to each optical calculation core.