JP3180334B2 - Information processing device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus that stores information in a self-organizing manner by imitating a neural network.

2. Description of the Related Art Conventionally, an information processing apparatus that stores information in a self-organizing manner by imitating a neural network is configured by combining a large number of pseudo neural elements described below. Figure 5 is a neural is a conceptual diagram of simulated action element comprising constituent units of the signal conversion circuit which simulates, simulated neural element gives one output receives N input x ₁ ~x _n. i-th input signal x _i is transformed in pseudo synaptic connections indicated by the signal converting circuit to w _i · x _i. N signals w ₁ × ₁ converted by pseudo-synaptic connection
~ W _n · x _n enter the adder indicated by the circle, the addition result y is sent to the nonlinear element indicated by the ellipse, and the final output f (y)
That is, _{f (Σw i · x i)} . The characteristics of the nonlinear element include, for example, a sigmoid function with a threshold value h, f (y, h) = 1 / (1 + exp (−y + h)), a Heaviside function, Are often used for associative memory (In the following description, an adder for adding N inputs and a non-linear element are collectively called a pseudo-neural element, and these are collectively handled.) In addition, according to a correction signal from a correction device described later, both or one of the pseudo synaptic connection strength w _i and the threshold h of the pseudo neural element changes.

FIG. 6 is a schematic diagram of a signal conversion circuit configured by combining a plurality of the above-described pseudo-neural elements. Squares represent pseudo-synaptic connections, and circles represent pseudo-neural elements. The embodiment of the pseudo-neural element and the signal conversion circuit is realized by hardware or software or a hybrid system combining them.

An information processing device that simulates a conventional neural circuit includes a signal conversion circuit and a correction circuit that changes the strength w _i of the pseudo-synapse connection and the threshold value h of the pseudo neural element according to a set learning algorithm. An algorithm for determining the amount of change in the strength w _i of the pseudo-Sipnas connection and the threshold h of the pseudo-nerve element is described in, for example, literature (PDP Dadel, P325, DE Ramelhart et al.
The error backpropagating terminal shown in the name, translated by Shunichi Amari, 1989) is well known. FIG. 7 is a block diagram of an information processing apparatus which is a constituent unit of an information processing apparatus simulating a neural circuit employing an error back propagation terminal as a learning algorithm. In the following description, this is referred to as "O ( Old-type information processing apparatus ". 7, reference numeral 1 denotes a signal conversion circuit, reference numeral 2 denotes a correction circuit, and reference numeral 15 denotes an "O-type information processing apparatus" including the signal conversion circuit and the correction circuit 2. FIG. 3 is a block diagram of an information processing apparatus configured by stacking three types of information processing apparatuses 15 in three layers, and adding an error calculation circuit 4 to one of them. Reference numerals 1x, 1y, and 1z denote signal conversion circuits. , 2x, 2y, and 2z indicate correction circuits, 4z indicates an error calculation circuit, and 19 indicates an information processing device including a signal conversion circuit 1z, a correction circuit 2z, and an error calculation circuit 4z (in the following description, respectively). Is
This information processing apparatus is referred to as an “OT (old teach) information processing apparatus”.

Next, with reference to FIG. 8, the operation of the information processing apparatus having a hierarchical structure using the non-linear characteristic of the pseudo-nerve element as a sigmoid function and employing an error back propagation terminal will be described. In an information processing device that imitates a neural circuit, a plurality of signals are mostly used. However, distinguishing all of the signals by subscripts only complicates the expression. , The suffix is omitted unless it is particularly necessary.

The operation of the information processing apparatus during learning is as follows. That is, the signal conversion circuit 1x outputs the input signal S
(I) is converted into a signal S (x) and output, but a correction circuit
2x stores the signal S (i) and the signal S (x) internally and waits for the input of the error signal D (x). Similar processing is performed in the signal conversion circuits 1y and 1z and the correction circuits 2y and 2z. Then, the final output S (z), the ideal signal corresponding to the input signal S (i), and T, also called a teacher signal, are sent to the error calculation circuit 4z, and the error relating to the j-th pseudo neural element in the third layer is obtained. The signals D _j (z) are respectively D _j (z) ≒ −μ (S _j (z) −T _j ) where μ is calculated as a positive constant, and an error signal D (z) obtained by summing them is Output to the correction circuit 2z. The correction circuit 2z receives the held signals S (y) and S (z) and the error signal D (z), sends a correction signal M (z) to the signal conversion circuit 1z, and
The pseudo-synaptic connection W _ji (z) connected to the j-th pseudo-neural circuit in 1z is expressed as W _ji (z) + D _j (z) · S _j (z) · (1-S _j (z)) -Correct to S _i (y). Hereinafter, the same procedure is performed in the signal conversion circuits 1y and 1x and the correction circuits 2y and 2x. Therefore, by repeating this procedure called learning, the information processing apparatus learns the relationship between the input signal S (i) and the ideal output T, and can infer a completely new input from past experience. Will produce appropriate output. In this way, when the information processing device performs sufficient learning and the error output from the error calculation circuit becomes smaller than a certain value, the procedure relating to the correction circuit is omitted, and only the forward processing from input to output is performed. Will be done.

The information processing apparatus described above, which is constructed in a hierarchical manner, and the information processing apparatus stacked in four or more layers can realize an arbitrary input / output function with an accuracy corresponding to the scale. It is shown in MBE88-9, and it has been reported in IEICE Research Report NC89-15 that a pseudo-neural element for extracting features in the intermediate layer can be formed.

Separately from this, associative memory device "Associtron", which is summarized in 48 pages of Bulletin of the Faculty of Engineering, The University of Tokyo, No.13, published in 1975, is known as an example of a single-layer information processing device. I have. FIG. 9 is a block diagram showing the outline of this "associatron", where 1as is a signal conversion circuit, 21as
Indicates a rewriting circuit, 22as indicates a synapse connection editing circuit, and 23as indicates a storage device. The operation of the associatron will be described. The input signal S (ia
Writing of the associative memory associating the output signal S (oas) from s) is performed as follows. Synaptic connection editing circuit
22as is a storage circuit 23as from which the input signal S (ias, k), the output signal S (oas, k), and the ideal output signal S (tas, k) are read.
And the strength Wji (as) of the pseudo-synaptic connection is given by Wij (as) = f (Σμ · (S (tas, n) −Sj ( oas, n)) · Si (ias, n)) = f (Memo), Wji (as) = f (Memo + μ · (S (tas, k) −Sj (oas, k)) · Si (ias, k)) and the correction signal M (as) to be corrected is sent to the signal conversion circuit 1as, and the strength of the pseudo-synaptic connection held in the storage device 23as is rewritten to a new one. Regarding the recall of associtronic associative memory, when an input signal S (ias) is input to the signal conversion circuit 1as, the signal passes through a pseudo-synaptic connection, thereby storing the stored signal S closest to the signal.
The ideal output S (tas, m) relating to (ias, m) is most strongly recalled, and is subjected to shaping by the pseudo-neural cell which is a nonlinear element, and finally, the signal S (tas, m) is output. It is done.

Problems to be Solved by the Invention However, the above-described conventional information processing apparatus simulating a neural circuit cannot control information expression such as feature extraction in an intermediate layer in a hierarchical information processing apparatus at all, As the scale of the network is increased to improve the performance of the information processing apparatus, the required number of times of learning increases rapidly.

In addition, in forming an information expression as described above, the empirical information expression acquisition means of learning used is used to perform information processing even if it is difficult to logically configure an information processing algorithm. Although the device can be configured, there is a problem that there is no performance evaluation means such as confirming that there is no illegal output, although the reliability is lower than the logically configured information expression (information processing algorithm). Was.

Further, there is a problem that it is difficult to form a free network structure in the conventional information processing apparatus.

On the other hand, the ideal storage device Associatron, which was pointed out earlier,
Because it is difficult to construct a powerful learning algorithm, only a correlation table between patterns is created internally, and a sophisticated table such as feature extraction is used, as in a hierarchical information processing device. There is a problem that it is not possible to carry out information processing.

SUMMARY OF THE INVENTION An object of the present invention is to provide an information processing apparatus which can control the information expression of a hidden layer and, at the same time, simulates a neural circuit having a high learning speed in a hierarchical information processing apparatus. An object of the present invention is to provide a ring-shaped information processing device corresponding to a single-layered information processing device having a powerful learning algorithm that can be used.

Means for Solving the Problems The present invention is an information processing apparatus simulating a neural network, and includes a pseudo-neural element having a pseudo-synaptic connection that performs a linear transformation by imitating a nerve cell and a nonlinear element that performs a non-linear transformation. A plurality of arranged signal conversion circuits, a correction circuit that receives an input signal to the signal conversion circuit, an output signal from the signal conversion circuit, and an error signal from the outside, and corrects pseudo-synaptic coupling of the signal conversion circuit; And a signal fixing circuit for fixing a part or all of the output signal of the signal conversion circuit to a value input from the outside. According to another aspect of the present invention, there is provided an "O-type information processing device" simulating a conventional neural circuit, the above-described information processing device according to the present invention, and a random pattern generating random input signal to the information processing device. An information processing apparatus including at least one of a convergence determination circuit that determines convergence for an output or an input and outputs a convergence determination signal; A random pattern generation circuit for generating a random input signal to the "O-type information processing apparatus", an output fixing circuit for fixing a part or all of the output, a convergence judgment for judging the convergence of the output or input and outputting a convergence judgment signal An information processing device having at least one of the circuits is used in accordance with the situation, and an information processing device having a hierarchical structure, a ring structure, or a combination thereof is used. It constitutes a device.

Operation According to the configuration of the present invention described above, it is possible to control the information expression of the intermediate layer, and at the same time, it is possible to realize an information processing apparatus that imitates a neural circuit having a high learning speed. Further, by using the output of the final output layer as the input reproduction, an information processing apparatus that simulates a neural circuit whose characteristic evaluation is simple can be configured. Furthermore, associative memory can be realized with a powerful learning algorithm by the ring information processing device.

Embodiment An embodiment of the present invention will be described below with reference to FIGS.

FIG. 1 is a block diagram schematically showing an information processing apparatus having a hierarchical structure enabling learning of an intermediate layer.
1a, 1b, 1c, 1d, 1e denote signal conversion circuits, and reference numerals 2a, 2b, 2
c, 2d, 2e are correction circuits, 3 is an error editing circuit, 4e and 4c are error calculation circuits, 20 is an intermediate learning weight correction circuit, and 10 is information that enables learning of an intermediate layer. Each of the processing devices is shown. Hereinafter, this information processing apparatus is referred to as a “T (teach) type information processing apparatus”.

Next, the operation of the information processing apparatus having a hierarchical structure capable of learning the intermediate layer will be described with reference to FIG.
Learning consists of forward processing and backward processing. In addition,
In the following description, it is assumed that the non-linear characteristic of the quasi-neural element is represented by a sigmoid function according to the back propagation method.

First, in the forward processing, the input signal S (i) is sent to the signal conversion circuit 1a and the correction circuit 2a. The signal conversion circuit 1a converts the input signal S (i) into a signal S (1a),
(1a) is sent to the correction circuit 2a and the signal conversion circuit 1b. The signal conversion circuit 1b converts the signal S (1a) into a signal S (1b),
The signal S (1b) is converted to a signal conversion circuit 1 in the information processing apparatus 10.
c, to the correction circuit 2c and the correction circuit 2c included in the intermediate learning weight correction circuit 20 in the information processing apparatus 10. The signal conversion circuit 1c of the “T-type information processing device” 10 converts the signal S (1b) into the signal S (1c), and converts the signal S (1c) into the signal conversion circuit 1d, the correction circuit 2d, and the weight correction of the intermediate academic association. Error calculation circuit in circuit 20
4c and the correction circuit 2c. The signal conversion circuit 1d and the correction circuit 2d perform the same processing as the signal conversion circuit 1b and the correction circuit 2b, but the signal conversion circuit 1e and the correction circuit 2e, together with the similar processing of the signal conversion circuit 1b and the correction circuit 2b, The output signal S (o) is sent to the error calculation circuit 4e.

In the backward signal processing, the error calculation circuit 4e compares the ideal output signal T (i, 1) corresponding to the input signal to be input with the output signal S (o) from the signal conversion circuit 1c, for example, D ( 2e) = − μ (S (o) −T (i, 1)), an error signal D (2e) is obtained according to a set procedure as shown in the following expression, and the error signal D (2e) is sent to the correction device 2e. send. The correction circuit 2e receives the error signal D (2e) and uses the signals S (o) and S (1d) received during the forward processing to generate a pseudo synapse connected to the jth pseudo neural element in the signal conversion circuit 1e. A signal M (1e) for correcting the coupling strength _Wji (1e) to, for example, _Wji (1e) + D (2e) · S (1e) (1−S (1e)) · S (1d) is converted. The error signal D (2d) is sent to the correction circuit 2d, for example, ΣDn (2e) · Sn (1e) (1-Sn (1e)) · Wn1 (1e). The correction circuit 2d performs the same processing as the correction circuit 2e and sends an error signal D (2c) to the error editing circuit 3.

On the other hand, the error calculation circuit 4c generates an ideal intermediate output T (i, 2) corresponding to part or all of the input S (i) and a signal S (1c).
And, for example, D (m) = − μm (S (1c)) − T (i, 2)), and an error signal D (m) is obtained according to a set procedure such as Error editing circuit 3 converts signal D (m)
Send to The signal S (1c) in this case is composed of three signals S1 (1c
c), S2 (1c) and S3 (1c), but the ideal intermediate output T
Taking the case where (i, 2) is two signals T1 (i, 2) and T1 (i, 2) as an example, specifically, the error signal D (m) is obtained by the procedure described above. When used, D1 (m) = − μm (S1 (1c)) − T (i, 2)), D2 (m) = − μm (S2 (1c)) − T (i, 2)), D3 (m ) = 0. The error editing circuit 3 uses the two error signals D (2c) and D (m) to convert the error signal D ′ (2c) into, for example, D ′ (2c) = αD (2c) + βD (m) α and β are edited as positive constants and sent to the correction circuit 2c. Also, the correction circuits 2c, 2b,
2a performs the same processing as the correction circuit 2e.

By repeatedly performing the above processing, the information processing apparatus having the hierarchical structure outputs meaningful outputs from the intermediate signal conversion circuit 1c and the final signal conversion circuit 1e.

An information processing apparatus simulating this neural circuit, which has an intermediate ideal signal T (i, 2)
As a result, the output according to the conventional method is given, and the ideal output T (i,
If the ideal output that we really seek is given as 1), the information processing device that could not be achieved by the conventional method can be realized with less learning times than the information processing device that imitates the conventional neural circuit. Specifically, this is a case of application to character segmentation, which is a pre-process of character recognition. In this case, image data of a document is given as input, and character segmentation of the conventional method is used as an intermediate ideal output. And the ideal output should be given to the output layer. As a result, the information processing apparatus proceeds with learning while referring to the output of the conventional method, and outputs a superior output.

FIGS. 2 and 3 are block diagrams showing an embodiment of another type of information processing apparatus according to the present invention. In the following description, in order to avoid confusion, various information processing apparatuses will be handled as follows. The information processing apparatus shown in FIG. 1 is called a "T-type information processing apparatus", and an information processing apparatus in which a random pattern generating circuit is added to the "T-type information processing apparatus" is referred to as "R (" random) type information ". Processing apparatus "and" T-type information processing apparatus "to which an output fixing circuit is added are referred to as" F (fixed) information processing apparatus "and" T-type information processing apparatus ".
A convergence judgment circuit is added to the "J (Judge) type information processing device", and an output fixed circuit and a convergence judgment circuit are added to the "T type information processing device" to the "FJ (fixed judgment) type information". Processing device ". In addition, the conventional "O
An information processing device having a random pattern generation circuit added to the "type information processing device" is referred to as an "OR (old random) information processing device", and an information processing device having the output fixing circuit added to the "O type information processing device". "OJ (old-fix) type information processing device" and "O-type information processing device" with a convergence determination circuit added thereto are referred to as "OJ (old judge) type information processing device" and "O-type information processing device". The type of information processing device to which an output fixing circuit is added is referred to as an "OFJ (old fix judge) type information processing device", respectively, and is similarly referred to.

Reference numeral 10 in FIG. 2 is an information processing apparatus having the same structure as that shown in FIG. 1, 5 is a signal fixing circuit, 6 is a convergence determining circuit, 11 is a signal fixing circuit 5 and a convergence determining circuit. 1 shows an “FJ (fixed judge) type information processing device” including a circuit 6 and an information processing device 10. The "FJ-type information processing apparatus 11" is mainly used for an information processing apparatus having a ring structure. Next, an operation when used for the information processing apparatus having a ring structure will be described.

The operations of the signal conversion circuit 1 and the intermediate learning weight correction circuit 20 are as described above. That is, the signal fixing circuit 5 converts part or all of the signal S (1) from the signal conversion circuit into a signal S ′ (1) according to the input setting signal F and outputs the signal S ′ (1), or outputs the signal S ( 1) is output as it is. The signal S (1) has three signals S1 (1), S2 (1),
This output operation will be specifically described by taking an example of the case where the setting signal S3 (1) is used.
If (1) is fixed at 0.5, the signal S (1) = [S1 (1), S2 (1), S3 (1)] is converted by the signal fixing circuit 5 into the signal S '(1). ) = [S1 (1), S2 (1), 0.5]. When the setting signal F is not input, the signal fixing circuit 5 outputs the signal S (1) as it is.

In the case where the “FJ-type information processing device” is used in an information processing device having a ring structure in which the signal fixing circuit 5 performs signal conversion in accordance with the setting signal F, the signal S (i: 1) is converted by the signal conversion circuit 1, and the signal S (1: 1) is sent to the signal conversion circuit 5 and the convergence determination circuit 6. The signal conversion circuit 5 converts a part or all of the signal S (1: 1) according to the setting signal F, and converts the converted signal S ′ (1: 1) into a convergence determination circuit 6 and other parts (not shown). Send to The convergence determination circuit 6 outputs the signal S (1:
1) is compared with the signal S '(1: 1). If the difference is smaller than the set value, the signal CJ is sent to the control circuit in the other part.
Send (convergence). If it is larger than the setting, the signal CJ (unconverged) is sent to the control circuit in the other part, but as another case, the signal is S (1: k) = S (1: k + 1) S ' (1: k) = S '(1: k + 1), and the device may enter a stable state. At this time, the convergence determination device 6 sends a signal CJ (failure) to the control circuit. In a normally conceivable setting, the control circuit stops processing when receiving the signal CJ (success) or the signal CJ (failure). On the other hand, if the signal CJ (unconverged) is received, the processing is continued, and the signal S '(1: 1) passes through the other parts to the signal SJ.
It returns to the signal conversion circuit 1 as (i: 2). The signal conversion circuit converts the signal S (i: 2) into a signal S (1: 2) and sends the signal S (i: 2) to the signal fixing circuit 5 and the convergence determination circuit 6. Thereafter, the same processing is performed, the difference between the two signals becomes smaller than the set value, the number of repetitions exceeds the set value, the signal S (1, n) and the signal S
(1, n + 1) is subtracted from the difference (or signal S '(1, n) and signal S'
Until ((difference of (1, n + 1)) becomes smaller than the set value or becomes the set value.

Reference numeral 10 in FIG. 3 denotes a "T-type information processing apparatus" having the same structure as shown in FIG. 1, and 7 denotes a random pattern for receiving a control signal CR and generating a random signal S (r).
Reference numeral 12 denotes an "R-type information processing device" comprising the random pattern generation circuit 7 and the "T-type information processing device" 10.

FIG. 4 shows an embodiment of an associative memory device according to the present invention constituted by using various information processing devices. Reference numerals 11a and 11b denote "FJ type information processing devices", and 15a, 15b and 15c denote "O type information". 19, an OT-type information processing device. Reference numeral 7 denotes a random pattern generation circuit, and reference numeral 17 denotes a random pattern generation circuit 7 and an O-type information processing device. "OR (old random) type information processing device", 8 is a control device.

The operation of the associative memory device shown in FIG. 4 will be described by taking as an example a case where the name, color and shape of an "object" are stored. The associative memory writes the signal S (i) corresponding to the name of the "object". An "OR type information processing device" 17 is input as an input signal and an "OT type information processing device" 19 is input as an ideal output signal, and a signal T (1) corresponding to the shape of an "object" is output as an "FJ type" as an ideal output signal. Signal T corresponding to the color of the "object"
"FJ-type information processing device" 11b with (2) as ideal output signal
To enter. Signal fixing circuit in "FJ type information processing apparatus" 11a, 11b (only the signal fixing circuit 5 of 11a is shown in the figure)
Is not input. In this environment, by performing the same learning as the hierarchical information processing apparatus described with reference to FIG. 2, the associative memory can be written to the strength of the pseudo-synaptic connection.

Recall of associative memory is performed as follows. First,
When recalling the name of “object” from the shape of “object”, the setting signal F (1) corresponding to the shape of “object” not to be searched is sent to the signal fixing circuit 5 in the “FJ-type information processing device” 11a. input.
Then, a signal CL is sent to the control circuit 8 to operate the random pattern generation circuit 7, and the control circuit 8 receiving the signal CL sends a signal CR (ON) to the random pattern generation circuit 7. The random pattern generation circuit 7 receives the signal CR (ON), sends a random signal S (r) to the "O-type information processing device" 15a, and stops. "O-type information processing device" 15a is signal S
(R) is converted to a signal S (1), and the "FJ-type information processing apparatus"
Send to "T-type information processing device" 10 in 11a. The “T-type information processing device” 10 converts the signal S (1) into a signal S (2) and sends the signal S (2) to the signal fixing circuit 5 and the convergence determining circuit 6. The signal fixing circuit 5 converts the signal S (2) into the setting signal F (1) as described above.
To the signal S (3), and sends it to the convergence determination circuit 6 and the “O-type information processing device” 15b. Hereinafter, the signal S (3)
Is the external output, the color of the "thing" S (5), the name of the "thing" S
While outputting (7), the signal passes through some information processing devices, becomes a signal S (7), and is input again to the “O-type information processing device” 15a. During this processing loop, the convergence determination circuit 6
Receives the signals S (2) and S (3) and, as described above,
The signal CJ ₁ (convergence / unconvergence / failure) is continuously output to the control circuit 8. The control circuit 8 outputs the signal CJ ₁ (convergence) or the signal CJ 1
Or (failure) is input, to the input circuit of the signal CJ ₁ exceeds the set value processing Le - to continue the flop. CJ ₁ (convergence)
Is input, a signal that the memory recall was successful
If the signal of SR (success) is input when the signal CJ ₁ (failure) is input or the number of processing exceeds the set figure,
A signal SR (failure) indicating that the memory recall has failed is output. Here, when the signal CJ ₁ (failure) is received, the control device 8 can send a CR (ON) to the random pattern generation circuit 7 so as to perform the same processing again.

If the search result is not satisfactory, the control circuit 8 sends the signal CL again.
And perform the same processing.

As a special example, if the ring-shaped information processing apparatus is used as a hierarchical information processing apparatus, unauthorized output confirmation of the information processing apparatus can be performed. Since the ideal output signal of the final output layer of the ring type information processing device is an input signal to the device, it is considered as an input reproduction layer, and the intermediate layer is considered as an output layer. If the output signal is fixed in the output layer and the output signal of the input reproduction layer is determined by the memory recall method of the associative memory device, the input signal corresponding to the output signal can be obtained. This makes it possible to accurately evaluate the performance of the information processing device imitating a neural circuit.

The above-described embodiment or any of the circuits described in the claims of the present invention and a part thereof can be substituted by a program. That is, the present invention can be implemented by hardware such as an apparatus, or by software such as a computer program.
It can also be configured as a hybrid system combining them.

In addition, in order to easily explain the operation of the circuit that changes the strength of the pseudo-synaptic connection, in the above description,
Although described separately for each layer, the same operation can be performed even with an integrated circuit or program in the device configuration. In addition, the signal conversion circuit and the correction circuit can be configured integrally. As described above, several circuits of the present invention can be integrated into one circuit or block to perform the same operation. Particularly, it is easy to realize the present invention in a program on a general computer.

Each of the information processing devices described above can operate by itself, may be stacked in any number of layers, or may have a more complicated structure.

Further, the description of the above-described embodiments has been made based on the error back-propagation method well-known as a learning algorithm and an extension thereof, but the present invention does not limit the learning algorithm.

Effect of the Invention As described above, the present invention provides both a signal conversion circuit simulating a neural circuit and a conventional weight correction circuit, for example, according to an evaluation function set by receiving a teacher signal and an output signal. An error calculation circuit that calculates an error of the signal, an addition circuit that adds two or more errors with a set weight and sends an error signal to a weight correction circuit, and a characteristic of the signal conversion circuit that receives an error signal from the addition circuit. The information processing apparatus simulates a neural circuit using an intermediate layer weight correction circuit including a correction circuit that outputs an error signal that corrects the error. Also, an information processing apparatus simulating a conventional neural circuit, the above-described information processing apparatus, a random pattern generation circuit for generating a random input signal to the information processing apparatus, and an output for fixing a part or all of the output An information processing apparatus including at least one of a fixed circuit, a convergence determination circuit for determining convergence of an output or an input, and outputting a convergence determination signal, and a random pattern for generating a random input signal to a conventional information processing apparatus And a data processing device having at least one of a convergence determination circuit that determines convergence of an output or an input and outputs a convergence determination signal. The information processing apparatus is also selected and configured to have a hierarchical structure, an annular structure, or a concept combining these. Therefore, according to such a configuration, it is possible to control the information expression of the intermediate layer, and at the same time, it is possible to realize an information processing device that imitates a neural circuit having a high learning speed. Further, according to the present invention, by reproducing the input of the output of the final output layer,
It is possible to configure an information processing device that imitates a neural circuit that is easy to evaluate characteristics. Further, according to the present invention, it is possible to configure an associative memory device having a powerful learning algorithm based on a ring network.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing the concept of an information processing apparatus having a hierarchical structure according to the present invention, and FIGS. 2 and 3 are blocks of another type of information processing apparatus according to the present invention. FIG. 4, FIG. 4 is a block diagram of an associative memory device according to the present invention, FIG. 5 is a conceptual diagram of a conventional pseudo-nerve element, FIG. 6 is a block diagram of a signal conversion circuit combining a conventional pseudo-nerve element, FIG. The figure is a block diagram of a conventional "O (old) type information processing apparatus",
FIG. 8 is a block diagram of a conventional information processing apparatus having a three-layer "O (old) type information processing apparatus", and FIG. 9 is a block diagram of a conventional associative memory device. 1,1a, 1b, 1c, 1d, 1e, 1x, 1y, 1z, 1as ...... signal conversion circuit, 2,2
a, 2b, 2c, 2d, 2e, 2x, 2y, 2z ... Correction circuit, 3 ... Error editing circuit, 4c, 4e, 4z ... Error calculation circuit, 5 ... Signal fixing circuit, 6 ... Convergence judgment Circuit 7, Random pattern generation circuit 8, Control device 10, T-type information processing device 11,
1a, 11b FJ-type information processing device, 12 R-type information processing device, 15, 15a, 15b, 15c, 15x, 15y O-type information processing device, 19
…… OT type information processing device, 20 …… Intermediate learning weight correction circuit,
21as ... rewriting circuit, 22as ... synapse connection editing circuit,
23as …… a storage device.

Claims (10)

(57) [Claims] An information processing apparatus simulating a neural network, comprising: a signal conversion circuit in which a plurality of pseudo neural elements each having a pseudo synaptic connection for performing a linear transformation and a nonlinear element for performing a non-linear transformation are arranged. A correction circuit that receives an input signal to the signal conversion circuit, an output signal from the signal conversion circuit, and an external error signal, and corrects a pseudo-synaptic connection of the signal conversion circuit; and an output signal of the signal conversion circuit. And a signal fixing circuit for fixing a part or all of the values to a value input from the outside. 2. An error signal for modifying the synaptic connection,
2. The information processing apparatus according to claim 1, wherein the information processing apparatus is obtained by editing a plurality of error signals. 3. An information processing apparatus simulating a neural network, comprising: a signal conversion circuit in which a plurality of pseudo neural elements having a pseudo synapse connection for performing a linear transformation and a nonlinear element for performing a non-linear transformation are arranged. A 0-type information processing apparatus comprising: a correction circuit that receives an input signal to the signal conversion circuit, an output signal from the signal conversion circuit, and an error signal from the outside and corrects a pseudo-synaptic connection of the signal conversion circuit; An information processing apparatus characterized by being combined with the information processing apparatus according to claim 2. 4. The information processing apparatus according to claim 3, wherein the 0-type information processing apparatus and the information processing apparatus according to claim 2 are connected in a hierarchical manner. 5. The information processing apparatus according to claim 3, wherein the 0-type information processing apparatus and the information processing apparatus according to claim 2 are coupled in a ring shape so that signals circulate. 6. A connection between the 0-type information processing apparatus and the information processing apparatus according to claim 2, wherein a combination of an annular connection and a connection of a hierarchical structure are combined so as to circulate a signal. An information processing apparatus according to claim 1. 7. The information processing apparatus according to claim 5, further comprising a convergence determination circuit that determines whether or not the information processing state of the information processing apparatus has converged. 8. The convergence determination according to claim 7, wherein the convergence of the state of the information processing device is determined by comparing an input signal to the signal fixing circuit with an output signal from the signal fixing circuit. Information processing device. 9. The convergence determination is performed by comparing the output signal of the signal conversion circuit with the output signal of the same signal conversion circuit in one information processing time unit.
An information processing apparatus according to claim 1. 10. The information processing apparatus according to claim 5, further comprising a random pattern generation circuit for receiving a control signal and inputting a random input signal to the information processing apparatus.