JP7779028B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

In order to overcome the limitations of Moore's Law, the entire IT (Information Technology) industry is searching for new computer architectures that are not the conventional von Neumann type.
Many non-von Neumann machines have been proposed, ranging from quantum computing, which will likely take many years to develop, to relatively easy-to-get-started technologies such as FPGAs (Field Programmable Gate Arrays) and GPGPUs (General-Purpose Computing on Graphics Processing Units). All of these avoid the von Neumann bottleneck and efficiently solve problems in specific fields that require computational resources.

Machine learning is one problem that can be combined with these. In particular, today's multi-layered deep learning requires a large amount of computing resources, so many companies are researching and developing ways to make it more efficient. Thanks to the success of deep learning, new machine learning models are being developed and improved every day at research institutes and companies.

RNNs (Recurrent Neural Networks) are known as a machine learning technique suitable for handling time-series data. Because RNNs have a structure that includes loops within the network, they can use the correlation between past data and current data as weighting. RNNs are expected to be used in applications such as dynamic decision-making in video processing and natural language processing.

However, it is known that updating the interlayer weights due to backpropagation through time takes a very long time, especially in high-performance RNNs such as LSTM (Long Short Term Memory) and GRU (Gated Recurrent Unit), and various methods for improving this have been studied.In recent years, an RNN algorithm called Reservoir Computing (RC), which updates the weights only in the output stage, has attracted attention.
RC is a special type of RNN that has a structure called a "reservoir" with random, fixed connections, rather than multi-layer connections like deep learning.

The presence or absence of connections between nodes in RC, as well as the connection direction and weights, are determined prior to machine learning (training). Typically, when RC is implemented in a program, the presence or absence of connections and the numerical weights for each connection are determined randomly, and a reservoir is generated using the connection probability between nodes and the weight variance as parameters. Training proceeds while the connections and weights between nodes in the reservoir remain fixed, and only the weights in the output layer are updated by the output weight calculation unit shown in the diagram. Once the training process is complete, the output weights are fixed and output data is obtained in the inference process.

Because reservoirs have random and fixed connections, physical entities can be used as reservoirs. Examples of reservoirs that can be used include random network circuits with nodes made up of elements with nonlinear properties, blocks of electrically nonlinear materials, and materials with optical nonlinearity.

Japanese Patent Application Laid-Open No. 2019-101635 U.S. Pat. No. 1,039,5168

Machine learning using RC makes it possible to predict time series data. For long-term predictions, it is desirable to be able to predict not only periodic data that occurs frequently, but also events that occur infrequently.

Analysis using long-term time series data can be used, for example, to predict temperature changes throughout the year or to predict failures from vibrations during machine operation. Long-term temperature changes include both daily temperature changes and seasonal fluctuations, and it is necessary to learn about waveforms that combine two waveforms with significantly different occurrence frequencies. Furthermore, if unusual vibrations that can lead to failures occasionally appear among the vibrations that occur during normal machine operation, the failure prediction system can become more accurate by effectively learning about these special vibrations.

However, with a single RC, it is difficult to predict realistic data with different occurrence periods, even when the scale is increased. Therefore, there is a need to combine multiple RCs to improve the prediction accuracy of complex data.

Therefore, this information processing device comprises a first reservoir computing, a second reservoir computing, a third reservoir computing, and a fourth reservoir computing, each having an input layer, a reservoir, and an output layer, the first reservoir computing, the third reservoir computing, the fourth reservoir computing, and the second reservoir computing being connected in series in this order, and a first data processing unit having the first reservoir computing and the second reservoir computing, and a second data processing unit having the third reservoir computing and the fourth reservoir computing, wherein first periodic data is given as teacher data to the first reservoir computing, and second periodic data having a longer period than the first periodic data is given as teacher data to the third reservoir computing, and in the first data processing unit time series data is input to the first reservoir computing, which outputs first output data; the first output data and fourth output data from the fourth reservoir computing are input to the second reservoir computing; in the second data processing unit, a prediction error signal indicating a difference between the first output data and the first periodic data is input to the third reservoir computing; third output data from the third reservoir computing is input to the fourth reservoir computing; the prediction error signal is provided to the fourth reservoir computing as teacher data; the fourth reservoir computing outputs the fourth output data based on the third output data and the prediction error signal; in the first data processing unit, the second reservoir computing outputs second output data based on the fourth output data and the first output data .

According to one embodiment, prediction accuracy can be improved.

FIG. 1 is a diagram illustrating a configuration of a computer system according to an embodiment. FIG. 1 is a conceptual diagram of reservoir computing in a computer system as an example of an embodiment. 1 is a diagram illustrating reservoir computing in a computer system according to an exemplary embodiment. FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device of a computer system according to an embodiment. 10 is a flowchart illustrating a process of a training phase in a computer system as an example of an embodiment. 10 is a flowchart illustrating processing of an inference phase in a computer system as an example of an embodiment.

The following describes embodiments of the present information processing device, information processing method, and information processing program with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude the application of various modifications and technologies not explicitly stated in the embodiments. In other words, this embodiment can be implemented with various modifications within the scope of its intent. Furthermore, each figure does not intend to include only the components shown in the figure, but may also include other functions, etc.

(A) Configuration FIG. 1 is a diagram illustrating a schematic configuration of a computer system 1 as an example of an embodiment.

As shown in Figure 1, the computer system 1 includes reservoir computing (RC) units 20-1 to 20-4, a prediction error calculation circuit 213, a data buffer 2141, and a management unit 101.

The reservoir computing units 20-1 to 20-4 have the same configuration. Hereinafter, when there is no need to distinguish between the reservoir computing units 20-1 to 20-4, they will be referred to as reservoir computing units 20. Furthermore, reservoir computing units may be referred to as RC units.
FIG. 2 is a conceptual diagram of reservoir computing 20 in a computer system 1 according to an embodiment.
The RC 20 includes a reservoir 201 , an input layer 202 and an output layer 203 .

In reservoir 201, multiple nonlinear nodes are randomly fixed and connected. The presence or absence of connections between nodes in reservoir 201, as well as the connection direction and weights, are determined prior to machine learning (training).

When implementing RC in a program, the presence or absence of connections and the weight values for each connection are decided randomly, and the reservoir is generated using the connection probability between nodes and the weight distribution as parameters.

Training of the RC 20 is performed while the inter-node connections and weights in the reservoir 201 are fixed. The output layer 203 includes an output weight calculation unit 2031. The output weight calculation unit 2031 updates the output weights Wo during the training process. When the training process is completed, the output weights Wo in the output layer 203 are fixed. In the inference process, output data is obtained from the output layer 203.
The output layer 203 of each RC 20 is realized by the information processing device 10 (management unit 101) described later.

In reservoir 201, nonlinear nodes are randomly fixed and connected. Therefore, a physical entity may be used as reservoir 201. In other words, RC20 may be physical reservoir computing. Physical reservoir computing may be configured as a circuit network of nonlinear elements.

To realize RC20 as physical reservoir computing, reservoir 201 may be a random network circuit with nodes made of elements with nonlinear characteristics, a mass of electrically nonlinear material, or a material with optical nonlinearity.
FIG. 3 is a schematic diagram showing the RC 20 in the computer system 1 as an example of an embodiment.
The RC 20 may be represented by the diagram illustrated in FIG.
In the physical reservoir computing 20 exemplified in FIG. 3, the reservoir 201 is configured as a network of physical entities having a plurality of nonlinear nodes.

RC20 may be physical reservoir computing in which reservoir 201 is a network of physical entities with multiple nonlinear nodes.

In the RC 20, input data is input to the input layer 202. The data processed in the input layer 202 is input to the reservoir 201, and after being processed in this reservoir 201, is input to the output layer 203.
Hereinafter, in FIG. 1, the direction from the input layer 202 to the output layer 203 via the reservoir 201 in the RC 20-1 will be referred to as the forward direction.

As shown in Figure 1, in this computer system 1, multiple (four in the example shown in Figure 1) RCs 20-1 to 20-4 are connected in series.

A prediction error calculation circuit 213 is located downstream of RC20-1 in the forward direction. RC20-2 is also located downstream of this prediction error calculation circuit 213 in the forward direction.

Furthermore, RC20-3 is located downstream of RC20-2, and RC20-4 is located downstream of RC20-3.

In addition, in the computer system 1 illustrated in Figure 1, the continuum formed by four RCs 20-1 to 20-4 connected in series is bent 180 degrees between RCs 20-2 and RCs 20-3. As a result, the route direction from RCs 20-3 to RCs 20-4 is opposite to the route direction (forward direction) from RCs 20-1 to RCs 20-2. Hereinafter, the direction opposite to the forward direction will be referred to as the reverse direction.

Hereinafter, RC20-1 may be referred to as forward RC#1, and RC20-2 may be referred to as forward RC#2. RC20-3 may be referred to as reverse RC#2, and RC20-4 may be referred to as reverse RC#1.

Time-series data (input data) is input to the forward RC #1, which infers (predicts) a time-series periodic signal. That is, the forward RC #1 infers a periodic signal a predetermined time ahead (in the future).
The output data (inference result) output from the forward RC#1 is input to the prediction error calculation circuit 213 and the backward RC#1.

During the training phase, a first periodic signal (sine signal or cosine signal) is input to forward RC#1 as training data, and the weights of the output layer 203 are updated. The first periodic signal can also be called first periodic data.

The prediction error calculation circuit 213 calculates the difference (prediction error) between the output data (inference result) of the forward RC#1 and the teacher data (first periodic signal) of the forward RC#1. That is, the prediction error calculation circuit 213 calculates the prediction error of the forward RC#1. The prediction error of the forward RC#1 calculated by the prediction error calculation circuit 213 is input to the data buffer 214 and stored in a predetermined storage area of the data buffer 214.
The prediction error in forward RC#1 calculated by the prediction error calculation circuit 213 is also input to forward RC#2.
A signal indicating the prediction error in the forward RC#1 is input to the forward RC#2, and a time-series periodic signal is inferred.
The output data (inference result) output from the forward RC#2 is input to the backward RC#2.

In the training phase, a second periodic signal (sine signal or cosine signal) is input to the forward RC#2, and the weights of the output layer 203 are updated. The second periodic signal may be referred to as second periodic data. The second periodic signal has a different period from the first periodic signal, and may be, for example, longer than the period of the first periodic signal.
Furthermore, it is desirable that the second periodic signal has a waveform that is detected less frequently in the input data than the first periodic signal.

The output data of the forward RC#2 is input to the backward RC#2, which infers the difference (prediction error) between the output signal (inference result) of the forward RC#1 and the teacher data (first periodic signal) of the forward RC#1. In other words, the backward RC#2 infers the prediction error of the forward RC#1 a predetermined time ahead (in the future).

In the training phase, the actual values of the prediction errors calculated by the prediction error calculation circuit 213 are read from the data buffer 214 and then input as training data to the backward RC#2, and the weights of the output layer 203 are updated.
The output data (inference result) output from the backward RC#2 is input to the backward RC#1.

The output data of the backward RC #2 and the output data of the forward RC #1 are input to the backward RC #1, and data one step ahead (future) of the input data (time series data) input to the forward RC #1 is inferred (time series prediction).
In the training phase, input data input to the forward RC#1 is input as training data to the backward RC#1, and the weights of the output layer 203 are updated.

1, a forward RC #1 and a reverse RC #1 are arranged vertically side by side. The pair of the forward RC #1 and the reverse RC #1 may be referred to as a first data processing unit 211. Data processing in the first data processing unit 211 may also be referred to as first-stage data processing.
In the first data processing unit 211, the RC 20-1 is arranged in the forward direction, and the RC 20-4 is arranged in the reverse direction.

A waveform that occurs frequently and repeatedly is predicted in the first data processing unit 211. When irregular data that cannot be predicted by the first data processing unit 211 is input, the irregular data is passed to the second data processing unit 212 (described later) as a prediction error of the forward direction RC#1.
It can be said that irregular data is data that occurs infrequently. The first data processing unit 211 predicts time series data that occurs infrequently.

1, a forward RC#2 and a reverse RC#2 are arranged vertically side by side. The pair of the forward RC#2 and the reverse RC#2 may be referred to as a second data processing unit 212. The second data processing unit 212 includes a prediction error calculation circuit 213 and a data buffer 214. The data processing performed by the second data processing unit 212 may be referred to as second-stage data processing.
In the second data processing unit 212, the RC 20-2 is arranged in the forward direction, and the RC 20-3 is arranged in the reverse direction.

A signal indicating the prediction error of forward RC#1 generated by a prediction error calculation circuit 213 is input to the second data processing unit 212. Therefore, when a waveform that occurs infrequently is input to forward RC#1, a large signal is input to the second data processing unit 212, which processes the signal.
That is, when data that cannot be predicted by the first data processing unit 211 is input, a signal is passed to the second data processing unit 212, and the second data processing unit 212 makes a prediction.

In the second data processing unit 212, a second periodic signal different from the first periodic signal is input as training data to the forward RC#2. Furthermore, a signal indicating the prediction error of the forward RC#1 generated by the prediction error calculation circuit 213 is input as training data to the reverse RC#2. In this way, the second data processing unit 212 is not trained on the frequently occurring (most frequent) data (first time series data) handled by the first data processing unit 211, and is only responsible for predictions regarding events that occur less frequently than this first time series data.

In the example shown in Figure 1, the second data processing unit 212 is located downstream in the forward direction relative to the first data processing unit 211. As a result, the computer system 1 is configured as a multi-stage reservoir computing system with the second data processing unit 212 downstream of the first data processing unit 211.

The management unit 101 realizes the output layer 203 of each RC20. For example, the management unit 101 performs calculations as the output weight calculation unit 2031 in the output layer 203 of each RC20.

Furthermore, the management unit 101 initializes various variables in the RC 20 during the training phase of the computer system 1. The management unit 101 also controls the input of input data and teacher data to each RC 20.
Furthermore, the management unit 101 controls the input of input data to each RC 20 in the inference phase of the computer system 1 .
FIG. 4 is a diagram illustrating an example of a hardware configuration of the information processing device 10 of the computer system 1 according to an embodiment.

The information processing device 10 is a computer and includes, for example, a processor 11, memory 12, storage device 13, graphics processing device 14, input interface 15, optical drive device 16, device connection interface 17, and network interface 18 as components. These components 11 to 18 are configured to be able to communicate with each other via a bus 19.

The processor (control unit) 11 controls the entire information processing device 10. The processor 11 may be a multiprocessor. The processor 11 may be, for example, any one of a CPU, MPU (Micro Processing Unit), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), and FPGA (Field Programmable Gate Array). The processor 11 may also be a combination of two or more elements from the CPU, MPU, DSP, ASIC, PLD, and FPGA.

The processor 11 then executes a control program (information processing program, not shown) for the information processing device 10, thereby realizing the functions of the management unit 101 illustrated in Figure 1.

The information processing device 10 realizes the functions of the management unit 101 by executing programs (information processing programs, OS programs) recorded on, for example, a computer-readable non-transitory recording medium.

Programs describing the processing to be executed by the information processing device 10 can be recorded on various recording media. For example, the programs to be executed by the information processing device 10 can be stored in the storage device 13. The processor 11 loads at least a portion of the program in the storage device 13 into the memory 12 and executes the loaded program.

In addition, the program to be executed by the information processing device 10 (processor 11) can also be recorded on a non-transitory portable recording medium such as an optical disk 16a, memory device 17a, or memory card 17c. A program stored on a portable recording medium becomes executable after being installed in the storage device 13, for example, under control of the processor 11. The processor 11 can also read and execute the program directly from the portable recording medium.

Memory 12 is a storage memory that includes ROM (Read Only Memory) and RAM (Random Access Memory). The RAM of memory 12 is used as the main storage device of information processing device 10. The RAM temporarily stores at least part of the program to be executed by processor 11. Memory 12 also stores various data necessary for processing by processor 11.

The storage device 13 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a storage class memory (SCM), and stores various data. The storage device 13 is used as an auxiliary storage device for the information processing device 10.
An OS program, a control program, and various data are stored in the storage device 13. The control program includes an information processing program.

The auxiliary storage device may be a semiconductor storage device such as an SCM or a flash memory. A plurality of storage devices 13 may be used to configure a redundant array of inexpensive disks (RAID).
The storage device 13 may store various data generated by the management unit 101 .

A monitor 14a is connected to the graphics processing unit 14. The graphics processing unit 14 displays images on the screen of the monitor 14a in accordance with instructions from the processor 11. Examples of the monitor 14a include a display device using a CRT (Cathode Ray Tube) or a liquid crystal display device.

A keyboard 15a and a mouse 15b are connected to the input interface 15. The input interface 15 transmits signals sent from the keyboard 15a and mouse 15b to the processor 11. Note that the mouse 15b is an example of a pointing device, and other pointing devices can also be used. Examples of other pointing devices include a touch panel, tablet, touchpad, and trackball.

The optical drive device 16 uses laser light or other light sources to read data recorded on an optical disc 16a. The optical disc 16a is a portable, non-transitory recording medium on which data is recorded so that it can be read by reflected light. Examples of optical discs 16a include DVDs (Digital Versatile Discs), DVD-RAMs, CD-ROMs (Compact Disc Read Only Memory), and CD-Rs (Recordable)/RWs (Rewritable).

The device connection interface 17 is a communications interface for connecting peripheral devices to the information processing device 10. For example, a memory device 17a or a memory reader/writer 17b can be connected to the device connection interface 17. The memory device 17a is a non-transitory recording medium, such as a USB (Universal Serial Bus) memory, that has the ability to communicate with the device connection interface 17. The memory reader/writer 17b writes data to or reads data from the memory card 17c. The memory card 17c is a card-type non-transitory recording medium.

The network interface 18 is connected to a network. The network interface 18 sends and receives data via the network. Other information processing devices, communication devices, etc. may also be connected to the network.

(B) Operation The processing of the training phase in the computer system 1 as an example of the embodiment configured as described above will be described with reference to the flowchart (steps A1 to A7) shown in FIG.
In step A1, the management unit 101 initializes the weight of the reservoir 201 of each RC 20.
In step A2, the management unit 101 causes the forward direction RC #1 to input, as input data, time series data one step earlier (time t-1) than the latest time series data (time t).

In addition, the management unit 101 inputs the first periodic signal to the forward RC#1 as training data and causes the output layer 203 (output weight calculation unit 2031) to perform training (machine learning).

In forward RC#1, the output weight calculation unit 2031 optimizes the weights of the output layer 203 by minimizing the loss function using, for example, the least squares method based on the inference results performed on the input data and the training data.

The inference result (output data) by the forward RC#1 is input to the prediction error calculation circuit 213 and the backward RC#1. The prediction error calculation circuit 213 calculates the difference (prediction error) between the output data (inference result) of the forward RC#1 and the teacher data (first periodic signal) of the forward RC#1 (step A5). The prediction error of the forward RC#1 calculated by the prediction error calculation circuit 213 is input as input data to the forward RC#2. The prediction error of the forward RC#1 calculated by the prediction error calculation circuit 213 is also input as teacher data to the backward RC#2 via the data buffer 214.
Furthermore, the management unit 101 causes the second periodic signal to be input to the forward direction RC#2 as training data.

In forward RC#2, the output weight calculation unit 2031 optimizes the weights of the output layer 203 by minimizing the loss function using, for example, the least squares method based on the inference results performed on the input data (prediction error) and the training data (step A6).

By inputting the prediction error of forward RC#1 to the input layer 202 of forward RC#2, the data series that can be inferred in the first stage data processing (first data processing unit 211) is not transmitted to the second data processing unit 212, and only the data that cannot be inferred is received by the second data processing unit 212, and training proceeds. The output data of forward RC#2 is input to backward RC#2.
Furthermore, the management unit 101 inputs the prediction error of the forward RC#1 calculated by the prediction error calculation circuit 213 to the forward RC#2 via the data buffer 214 as training data.

In the backward RC#2, the output weight calculation unit 2031 optimizes the weights of the output layer 203 by minimizing a loss function using, for example, the least squares method based on the inference results performed on the input data (output data of the forward RC#2) and the teacher data (step A7). The output data of the backward RC#2 is input to the backward RC#1.
The management unit 101 inputs the latest (time t) time series data (input data) to the backward RC#1 as training data.

In backward RC#1, the output weight calculation unit 2031 optimizes the weights of the output layer 203 by minimizing the loss function using, for example, the least squares method based on the inference results performed on the input data (output data of backward RC#2) and the training data (step A3).

In step A4, the management unit 101 checks whether training has been performed for all of the teacher data that was planned in each RC 20. If the result of the check shows that training has not been performed for all of the teacher data (see the NO route in step A4), the process returns to step A2.
On the other hand, if training has been performed for all the teaching data (see the YES route in step A4), the process ends.
Next, the processing of the inference phase in the computer system 1 as an example of the embodiment will be described with reference to the flowchart (steps B1 to B6) shown in FIG.
In step B1, the management unit 101 causes the forward direction RC #1 to input the latest time series data (at time t) as input data.

The forward RC#1, to which the time-series data is input, performs inference. The inference result (output data) is input to the prediction error calculation circuit 213 and the backward RC#1. The prediction error calculation circuit 213 calculates the difference (prediction error) between the output data (inference result) of the forward RC#1 and the teacher data (first periodic signal) of the forward RC#1 (step B2). The prediction error of the forward RC#1 calculated by the prediction error calculation circuit 213 is input as input data to the forward RC#2.
In step B3, the forward RC#2 performs inference on the input data (prediction error). The output data of the forward RC#2 is input to the backward RC#2.

In step B4, backward RC#2 performs inference on the input data (output data of forward RC#2). The inference results are input to backward RC#1.

In step B5, backward RC#1 performs inference on the input data (output data of backward RC#2) and predicts the time series data one step ahead (future: time t+1) of the latest time series data input in step B1. The inference result is output as output data of the computer system 1. The output data may be stored in a specified storage area such as memory 12 or storage device 13.

In step B6, the management unit 101 checks whether inference has been performed for all input data. If the result of the check shows that inference has not been performed for all input data (see the NO route in step B6), the process returns to step B1.
On the other hand, if inference has been performed for all input data (see YES route in step B6), the process ends.

(C) Effects As described above, according to the computer system 1 as an example of an embodiment, the first data processing unit 211, which includes a forward RC#1 and a backward RC#1, infers frequently occurring time series data, and when data that cannot be inferred by the first data processing unit 211 is input, a signal is passed as a prediction error of the forward RC#1 to the second data processing unit 212. As a result, the second data processing unit 212 does not perform machine learning on the most frequently occurring periodic data, and can be responsible for inference only regarding infrequent events, thereby improving the overall prediction accuracy.
Here, the results of a simulation using a physical reservoir with 100 nodes as the reservoir 201 will be shown.

The reservoir 201 was implemented as a network (a crossbar structure with elements and switches at the intersections) composed of elements (memristors) with nonlinear characteristics. Each reservoir 201 had 10 inputs and 10 outputs, with 100 node elements at the intersections. The connections between the nodes were determined randomly, and the strength of the connections (reservoir 201 connection weights) were also determined randomly. The computer system 1 shown in Figure 1 was constructed using four RCs 20 equipped with reservoirs 201 configured in this way, and training and inference were performed.
In addition, a single RC with 400 nodes was created in the program as a comparison source, and the prediction accuracy was compared.
The task was a general nonlinear function prediction problem called NARMA10, which is a method for evaluating RNNs.

Learning was performed using a data sequence from the NARMA10 function, and the error (NRMSE: normalized root mean square error) was calculated from the correct data sequence output from the NARMA10 function and the numerical sequence predicted by the RC after learning. Learning was performed under the same conditions using a conventional RC and this computer system 1, and the prediction error was evaluated.

As a result, while the prediction error with conventional single-structure RC was 0.1982, the prediction error with this computer system 1 was 0.1276, which reduced the error and enabled more accurate function prediction.

(D) Others The configurations and processes of this embodiment can be selected as needed, or can be combined as appropriate.
The disclosed technology is not limited to the above-described embodiment, and can be implemented in various modifications without departing from the spirit of the present embodiment.

For example, in the above-described embodiment, four RCs 20 are provided, and prediction processing is performed in two stages using the first data processing unit 211 and the second data processing unit 212. However, this is not limited to this. Six or more RCs 20 may be provided, multiple second data processing units 212 may be provided, and prediction processing may be performed in three or more stages.
In this case, the input irregular data may be propagated sequentially to the second data processing units 212 in the second stage and thereafter, and prediction may be performed.

In the above-described embodiment, each RC 20 may be realized by hardware having a physical reservoir, or the information processing device 10 may realize the functions of each RC 20 by executing software.
Furthermore, the above disclosure enables those skilled in the art to implement and manufacture the present embodiment.

(E) Supplementary Notes The following supplementary notes are further disclosed regarding the above-described embodiments.
(Appendix 1)
a first data processing unit having a first reservoir computing arranged in a forward direction and a second reservoir computing arranged in a reverse direction to the first reservoir computing;
a second data processing unit having a third reservoir computing arranged in the forward direction and a fourth reservoir computing arranged in the reverse direction;
Equipped with
In the first data processing unit, first time series data is predicted;
In the second data processing unit, second time series data having a lower occurrence frequency than the first time series data is predicted;
An information processing apparatus characterized in that, when data that cannot be predicted by the first data processing unit is input to the first data processing unit, the second data processing unit generates a signal indicating a prediction error of the first data processing unit and makes a prediction based on the signal.

(Appendix 2)
In the first data processing unit, an output of the first reservoir computing is input to the second reservoir computing;
The information processing device described in Appendix 1, characterized in that in the second data processing unit, a signal indicating the prediction error of the first reservoir computing is generated as the signal, the signal is input to the third reservoir computing, and the output of the third reservoir computing is input to the fourth reservoir computing.

(Appendix 3)
a first periodic signal is input to the first reservoir computing as training data;
3. The information processing device according to claim 1, wherein a second periodic signal different from the first periodic signal is input to the third reservoir computing as training data.

(Appendix 4)
The information processing device according to any one of appendices 1 to 3, wherein time series data is input to the fourth reservoir computing as training data.

(Appendix 5)
The information processing device according to any one of appendices 1 to 4, wherein a signal indicating a prediction error of the first reservoir computing is input as training data to the third reservoir computing.

(Appendix 6)
An information processing device described in any one of appendices 1 to 5, characterized in that at least one of the first reservoir computing, the second reservoir computing, the third reservoir computing, and the fourth reservoir computing has a circuit network of nonlinear elements as a reservoir.

(Appendix 7)
a first data processing unit having a first reservoir computing arranged in a forward direction and a second reservoir computing arranged in a reverse direction to the first reservoir computing;
a second data processing unit having a third reservoir computing arranged in the forward direction and a fourth reservoir computing arranged in the reverse direction;
In an information processing device comprising:
A process of predicting first time series data in the first data processing unit;
a process of predicting second time series data that occurs less frequently than the first time series data, in the second data processing unit;
an information processing method comprising: when data that cannot be predicted by the first data processing unit is input to the first data processing unit, the second data processing unit generates a signal indicating a prediction error of the first data processing unit and makes a prediction based on the signal.

(Appendix 8)
a process in the first data processing unit inputting an output of the first reservoir computing into the second reservoir computing;
The information processing method of claim 7, further comprising the steps of: generating, in the second data processing unit, a signal indicating the prediction error of the first reservoir computing; inputting the signal to the third reservoir computing; and inputting the output of the third reservoir computing to the fourth reservoir computing.

(Appendix 9)
inputting a first periodic signal as training data into the first reservoir computing;
The information processing method described in Appendix 7 or 8, further comprising a process of inputting a second periodic signal different from the first periodic signal as training data to the third reservoir computing.

(Appendix 10)
An information processing method described in any one of appendices 7 to 9, characterized in that the fourth reservoir computing includes a process of inputting time series data as training data.

(Appendix 11)
An information processing method according to any one of appendices 7 to 10, characterized in that a signal indicating a prediction error of the first reservoir computing is input to the third reservoir computing as training data.

(Appendix 12)
An information processing method according to any one of appendices 7 to 11, wherein at least one of the first reservoir computing, the second reservoir computing, the third reservoir computing, and the fourth reservoir computing comprises a circuit network of nonlinear elements as a reservoir.

(Appendix 13)
a first data processing unit having a first reservoir computing arranged in a forward direction and a second reservoir computing arranged in a reverse direction to the first reservoir computing;
a second data processing unit having a third reservoir computing arranged in the forward direction and a fourth reservoir computing arranged in the reverse direction;
In an information processing device comprising:
In the first data processing unit, first time series data is predicted;
In the second data processing unit, second time series data having a lower occurrence frequency than the first time series data is predicted;
An information processing program characterized in that, when data that cannot be predicted by the first data processing unit is input to the first data processing unit, the second data processing unit generates a signal indicating a prediction error of the first data processing unit, and causes a processor to execute a process of making a prediction based on the signal.

(Appendix 14)
In the first data processing unit, an output of the first reservoir computing is input to the second reservoir computing;
The information processing program of claim 13, characterized in that the processor is caused to execute a process in the second data processing unit to generate a signal indicating a prediction error of the first reservoir computing as the signal, input the signal to the third reservoir computing, and input the output of the third reservoir computing to the fourth reservoir computing.

(Appendix 15)
inputting a first periodic signal as training data into the first reservoir computing;
The information processing program described in Appendix 13 or 14, characterized in that the processor is caused to execute a process of inputting a second periodic signal different from the first periodic signal as teacher data to the third reservoir computing.

(Appendix 16)
The information processing program according to any one of appendices 13 to 15, characterized in that the processor is caused to execute a process of inputting time series data as teacher data into the fourth reservoir computing.

(Appendix 17)
The information processing program according to any one of appendices 13 to 16, characterized in that the processor is caused to execute a process of inputting a signal indicating a prediction error of the first reservoir computing as teacher data into the third reservoir computing.

(Appendix 18)
The information processing program according to any one of appendices 13 to 17, wherein at least one of the first reservoir computing, the second reservoir computing, the third reservoir computing, and the fourth reservoir computing comprises a circuit network of nonlinear elements as a reservoir.

1 Computer system 10 Information processing device 11 Processor 12 Memory 13 Storage device 14 Graphics processing device 14a Monitor 15 Input interface 15a Keyboard 15b Mouse 16 Optical drive device 16a Optical disk 17 Device connection interface 17a Memory device 17b Memory reader/writer 17c Memory card 18 Network interface 18a Network 19 Bus 20-1 to 20-4, 20 Reservoir computing 101 Management unit 201 Reservoir 202 Input layer 203 Output layer 2031 Output weight calculation unit 211 First data processing unit 212 Second data processing unit

Claims (4)

a first reservoir computing, a second reservoir computing, a third reservoir computing, and a fourth reservoir computing, each having an input layer, a reservoir, and an output layer;
The first reservoir computing, the third reservoir computing, the fourth reservoir computing, and the second reservoir computing are connected in series in this order, and
a first data processing unit having the first reservoir computing and the second reservoir computing;
a second data processing unit having the third reservoir computing and the fourth reservoir computing;
First periodic data is provided as training data to the first reservoir computing, and second periodic data having a longer period than the first periodic data is provided as training data to the third reservoir computing,
In the first data processing unit, time series data is input to the first reservoir computing, the first reservoir computing outputs first output data, and the first output data and fourth output data from the fourth reservoir computing are input to the second reservoir computing;
In the second data processing unit, a prediction error signal indicating a difference between the first output data and the first periodic data is input to the third reservoir computing, third output data from the third reservoir computing is input to the fourth reservoir computing, the prediction error signal is provided to the fourth reservoir computing as teacher data, and the fourth reservoir computing outputs the fourth output data based on the third output data and the prediction error signal;
In the first data processing unit, the second reservoir computing outputs second output data based on the fourth output data and the first output data.
1. An information processing device comprising: 2. The information processing apparatus according to claim 1, wherein at least one of the first reservoir computing, the second reservoir computing, the third reservoir computing, and the fourth reservoir computing comprises a circuit network of nonlinear elements as a reservoir. a first reservoir computing, a second reservoir computing, a third reservoir computing, and a fourth reservoir computing, each having an input layer, a reservoir, and an output layer;
The first reservoir computing, the third reservoir computing, the fourth reservoir computing, and the second reservoir computing are connected in series in this order, and
a first data processing unit having the first reservoir computing and the second reservoir computing;
a second data processing unit having the third reservoir computing and the fourth reservoir computing;
In an information processing device comprising:
First periodic data is provided as training data to the first reservoir computing, and second periodic data having a longer period than the first periodic data is provided as training data to the third reservoir computing,
a process in which, in the first data processing unit, time series data is input to the first reservoir computing, the first reservoir computing outputs first output data, and the first output data and fourth output data from the fourth reservoir computing are input to the second reservoir computing ;
a process in which, in the second data processing unit, a prediction error signal indicating a difference between the first output data and the first periodic data is input to the third reservoir computing, third output data from the third reservoir computing is input to the fourth reservoir computing, the prediction error signal is provided to the fourth reservoir computing as teacher data, and the fourth reservoir computing outputs the fourth output data based on the third output data and the prediction error signal;
An information processing method, characterized in that, in the first data processing unit, the second reservoir computing comprises a process of outputting second output data based on the fourth output data and the first output data . a first reservoir computing, a second reservoir computing, a third reservoir computing, and a fourth reservoir computing, each having an input layer, a reservoir, and an output layer;
The first reservoir computing, the third reservoir computing, the fourth reservoir computing, and the second reservoir computing are connected in series in this order, and
a first data processing unit having the first reservoir computing and the second reservoir computing;
a second data processing unit having the third reservoir computing and the fourth reservoir computing;
In an information processing device comprising:
First periodic data is provided as training data to the first reservoir computing, and second periodic data having a longer period than the first periodic data is provided as training data to the third reservoir computing,
In the first data processing unit, time series data is input to the first reservoir computing, the first reservoir computing outputs first output data, and the first output data and fourth output data from the fourth reservoir computing are input to the second reservoir computing;
In the second data processing unit, a prediction error signal indicating a difference between the first output data and the first periodic data is input to the third reservoir computing, third output data from the third reservoir computing is input to the fourth reservoir computing, the prediction error signal is provided to the fourth reservoir computing as teacher data, and the fourth reservoir computing outputs the fourth output data based on the third output data and the prediction error signal;
In the first data processing unit, the second reservoir computing outputs second output data based on the fourth output data and the first output data.
An information processing program that causes a processor to execute a process.