JPH0734227B2 - Hierarchical information processing circuit - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a hierarchical information processing in which a plurality of information processing layers in which a plurality of nonlinear elements each having a variable gain between input and output are arranged two-dimensionally are sequentially arranged. The circuit is particularly designed to have both an associative memory capability and a pattern recognition capability.

2. Description of the Related Art In general, when information about a certain event is input, a circuit that mutually recalls and outputs information about another event related to the input information, or incomplete information or ambiguous information about a certain event. A circuit that surely recalls and outputs complete information about the event is called an associative memory circuit, and several circuit systems have been proposed in the past.

Among the conventional associative memory circuits described above, the correlation matrix type associative memory circuit is attracting the attention of many researchers. Among the correlation matrix type associative memory circuits, in the principle configuration of the mutual associative type associative memory circuit, as shown schematically in FIG. When rows are combined in a matrix and a spatial pattern expressing a certain event is input from the left side, a spatial pattern expressing an event paired with the spatial pattern is output from the lower side. In the figure, the arrow that guides the input stimulation to each nonlinear element Indicates that there exists a variable coupling between the input of the element and the output of the preceding element, and these variable couplings exhibit variability during learning, that is, during storage of input information, and the stimulation input given from the outside It shows that the strength of the coupling between the elements changes accordingly.

Further, when the output terminal and the input terminal in the circuit configuration shown in FIG. 1 (a) are connected to form a positive feedback loop as shown in FIG. 1 (b), a self-associative associative memory circuit is formed. Can be realized.

In the associative memory circuit as described above, paying attention to the information flow between each bit of the input space pattern and each bit of the output space pattern, for example, a connection as shown in FIG. 1 (c) is formed. It will be. That is, even if attention is paid to the flow of information between any of the bits, only one stage of variable coupling is formed. Therefore,
In a conventional associative memory circuit, global information corresponding to a combination of a plurality of bits forming a spatial pattern,
That is, the function of associating high-order information corresponding to the concept of the spatial pattern and the function of associating low-order information mutually possessed by each bit are simultaneously realized in a single circuit. It was difficult to get it done.

SUMMARY OF THE INVENTION An object of the present invention is to eliminate the above-mentioned drawbacks of the prior art and to enable a circuit configuration that achieves both the association function between low-order information and the association function between high-order information that spatial patterns have. To provide a hierarchical information processing circuit.

Another object of the present invention is to automatically form a function of extracting such higher-order information in its own circuit without externally telling what the higher-order information of the spatial pattern is. Therefore, in parallel with performing the associated circuit operation, it is possible to judge which category the associated spatial pattern belongs to,
That is, it is to provide an information processing circuit having the ability to simultaneously perform pattern recognition for an associated spatial pattern.

That is, the hierarchical information processing circuit of the present invention is a hierarchical information processing circuit in which a plurality of information processing layers in which a plurality of nonlinear elements each having a variable gain between input and output are arranged two-dimensionally are sequentially arranged. Of the plurality of information processing layers, an upward signal path in which the coupling between the nonlinear elements is variable is set from the lower information processing layer to the upper information processing layer, and A downward signal path in which the coupling between the non-linear elements is variable in the opposite direction paired with the positive signal path, and the strength of the variable coupling between elements in both signal paths is changed in conjunction with each other. It is characterized by having done.

Examples Hereinafter, the present invention will be described in detail with reference to the drawings.

Now, consider a multilayer information processing circuit in which a plurality of nonlinear element layers for information processing in which a large number of nonlinear elements are two-dimensionally arranged are hierarchically arranged and combined. Each of the non-linear elements in each layer makes the gain between the input and output of the element variable according to the strength of the input signal, and is arranged between each non-linear element in each layer one step lower and one step higher than the layer. It is assumed that both have bidirectional coupling in which the strength of inter-element coupling is made variable by interposing a variable attenuator, for example.

Here, each non-linear element in the information processing layers of the second and subsequent layers has information in the previous stage, that is, immediately preceding information, as illustrated in FIG. 4 of Japanese Patent Publication No. 60-712, which is proposed by the present inventor. Each of the output terminals of the plurality of nonlinear elements in the processing layer has a plurality of input terminals coupled via a variable attenuator, and is composed of an upward-directing summing amplifier connected to the output terminals via a variable gain element, The addition of the input signals in the amplifier is performed by the calculation of the difference between the total value of all the input signals from the preceding stage applied to each positive input terminal and the average value of all the input signals from the preceding stage applied to the negative input terminal. , The total input added value of the calculation result is the spatial pattern of the coupling strength determined by the attenuation amount of each variable attenuator in which the spatial pattern of the input signal from each non-linear element in the preceding stage is interposed between each element in the preceding stage. The higher the degree of agreement with the above, the larger the value changes, and each nonlinear element produces an output when the total input added value exceeds the threshold set for each element. have.

As described above, each non-linear element has a built-in variable gain element that is connected in cascade on the output side of the summing amplifier, and as described later, this variable gain element and the variable attenuator for each input terminal The gain between the input and output of the nonlinear element is controlled according to the amplified output of the element, and the adaptive characteristic that the amplified gain of the element changes according to the input pattern is given to the reaction characteristics of each nonlinear element as described later. .

Then, the output that has passed through the upward summing amplifier and the variable gain element in each successive layer is input to the input terminal of each non-linear element in the preceding information processing layer via the on / off switch, the variable attenuator and the downward signal path. Also, the signal path is constructed from the upper layer to the lower layer in sequence. Also, by interlocking the variable attenuators in the bidirectional signal paths that make a pair between each layer, the bidirectional coupling between the nonlinear elements can be reduced by the strength of the upward variable coupling in the signal path that couples the elements and the strength of the downward variable coupling. The strength of the directional variable coupling is changed in pairs.

The strength of the variable coupling in each signal path, that is, the degree of easiness of transmission of the signal according to the attenuation amount of the variable attenuator intervening in the signal path that couples the elements, at the time of learning of the information processing circuit, that is, At the stage of storing the spatial pattern, it changes due to a change in the attenuation amount of the variable attenuator interposed in the signal path that is inversely proportional to the input signal strength. The strength of the directional variable coupling changes in conjunction with each other. That is, if the strength of the upward variable bond is increased, the strength of the downward variable bond is also increased along with the change.

Here, as shown in FIG. 2A, consider a hierarchical information processing circuit including four nonlinear element layers U ₀ , U ₁ , U ₂ , and U ₃ .
The lowest nonlinear element layer U ₀ in the information processing circuit having such a multilayer structure acts as an input layer that receives information on a spatial pattern as an input stimulation from the outside, and at the same time acts as an output layer in which a spatial pattern as an associative output appears. To do. Now, a spatial pattern in which output information appears only in every other nonlinear element 1, 3, 5, 7 in the input / output layer U ₀ and no output information appears in the other nonlinear elements 2, 4, 6 The pattern A is referred to as a pattern A, and the spatial pattern in which the output information appears only in the non-linear elements 2, 3, 4, 6 and the output information does not appear in the other non-linear elements 1, 5, 7 is called a pattern B. To do. When the patterns A and B are alternately presented and learned in the hierarchical information processing circuit described above, among the bidirectional coupling between the nonlinear elements in each layer, for example, as shown by a solid line in FIG. Only the variable coupling of the different signal paths shall be strengthened.

For example, various methods can be adopted as a learning method for storing the input space pattern so as to enhance the variable coupling as described above, and the learning method in the information processing circuit of the present invention is limited to a specific method. Not something to do.
As an example, for example, Japanese Patent Publication No. 55-22 related to the invention of the present inventors similar to that used for a so-called cognitron.
It is also possible to learn by the method described in Japanese Patent No. 974. In the case of using the cognitron type learning method, at the time of learning, the signal flow through the downward variable coupling in each signal path is stopped by the on / off switch in the signal path and the upward variable coupling is performed. The strength of each of the downward variable bonds can be strengthened in tandem with the strength of each pair of the upward variable bonds.

Next, the circuit operation of the hierarchical information processing circuit of the present invention having the above-described configuration during associative storage will be described. Then, as described above, learning by presenting a certain pattern in the information processing circuit, for example, both the pattern A and the pattern B is completed, and a specific signal path among the signal paths for coupling the elements of each layer is changed. It is assumed that only the strength of the bond is selectively strengthened to the state shown in FIG. 2 (a), for example. In addition, in FIG. 2 (a),
For simplification, the coupling between each element is shown by simply connecting them with a solid line, but in reality, the coupling strength between each element takes an arbitrary analog value according to the learning result. I shall. When, for example, only the nonlinear element 1 in the input / output layer U ₀ is externally given an input pattern forming part of the pattern A in the strengthened state of the coupling between the elements, the reaction caused by the input pattern It is assumed that the output spreads, for example, through a path indicated by a thick arrow in FIG. 2 (b). That is, the response of each element to the input pattern is, as described above, the total input path of the strength of the variable coupling of the input side signal path with respect to each non-linear element of the immediately preceding layer and the strength of the input signal transmitted through the signal path. When the total input added value determined according to the degree of coincidence of the spatial patterns over the above exceeds the threshold value of the element, the strength of the total input added value and the strength of the variable coupling to the nonlinear element at the next stage of the output side signal path are determined. According to the above, the signal is output to the output side signal path, and is transmitted through the upward variable coupling at the same time as being transmitted through the downward variable coupling. As a result, in the input / output layer U ₀ ,
As shown in FIG. 2 (b), even though only the input pattern forming a part of the pattern A is given, the response of the non-linear element to the input pattern gradually spreads through other layers, and the pattern A reaction that completely corresponds to A appears. Further, at the same time, in the uppermost layer U ₃ , the reaction pattern A appearing in the input / output layer U ₀ , that is,
The reaction output corresponding to the category of the associative output appears in the non-linear element 31, for example.

Next, as another input pattern, when the input signal is given only to the nonlinear elements 2 and 3 in the input / output layer U ₀ and the pattern of the input signal forms a part of the above-mentioned pattern B, The reaction to the input pattern spreads, for example, through the signal path in the manner shown by the thick line arrow in FIG. 2 (c), and the pattern B appears in the input / output layer U ₀ , and at the same time, in the uppermost layer U ₃ . Is a non-linear element whose reaction output corresponding to the category of the reaction pattern B is
Appears in 32.

In the second and subsequent layers, the non-linear element having a plurality of input terminals reacts with each other depending on the spatial pattern of the coupling strength with each element of the preceding stage and the spatial pattern of the input signal from each element of the preceding stage. When the total input added value of the element according to the degree of coincidence exceeds the threshold value, and the spatial pattern of the coupling strength with the preceding element and the spatial pattern of the output from the preceding element substantially match each other, the element The reaction of the non-linear element occurs due to the increase of the total input added value.
For example, regarding the nonlinear element 11 in FIG. 2A, as a result of learning, the element between element 1 and element 11 is the element 3
2 (b), since the input signal to the element 3 is absent, only the input signal to the element 1 indicates the degree of coincidence of the spatial patterns described above. , The element 11 reacts, while in FIG. 2 (c), there is no input signal to the element 1 and the element 3
The element 11 did not react because the degree of coincidence of the above-mentioned spatial patterns is low only because of the input signal to the device.

As described above, in the hierarchical information processing circuit of the present invention, even if an incomplete spatial pattern in which a part of the spatial pattern that has already been learned and stored has been omitted is given as a stimulation input, the input / output layer U ₀ is Is a complete spatial pattern without shape collapse or lack, that is, the learning pattern passes through the positive feedback loop formed by the bidirectional signal path between the above-mentioned nonlinear elements of each layer, so that time elapses. With that, it appears gradually and so-called recollection is performed. Similarly, when a learning pattern including noise is given as an input stimulation, a learning pattern in a clean form from which noise is removed appears in the input / output layer U ₀ as a recall output. At that time, among the non-linear elements in the uppermost layer U ₃ , the reaction output also appears in the non-linear element corresponding to the category of the recollection space pattern appearing in the input / output layer U _0, and the pattern recognition for the associative output pattern is also performed at the same time. Be done.

In addition, when multiple types of learning patterns are given as input stimulations in duplicate, or when partial patterns for each of multiple types of learning patterns are given as input stimulations, at first, those learning patterns are input. Although competition occurs between them, if a signal corresponding to a part of the learning pattern in a signal path begins to flow once and the strength of the variable coupling to the nonlinear elements at both ends coupled to the signal path is strengthened to some extent. , The signal easily flows in both directions through the signal path, and a kind of positive feedback loop is formed between the elements at both ends as shown in FIG. 1 (b). Strengthening each other's output,
The strength of the variable coupling in that signal path is further enhanced. As a result, with the passage of time, among these plural types of learning patterns, only a single learning pattern having the signal path as a learning path is strengthened and other learning patterns disappear, and at the same time, the top layer U _{In 3} , the input / output layer
A nonlinear element corresponding to the finally learned learning pattern responds to U _0, and the result of pattern recognition is shown.

Here, by giving an adaptation effect to the reaction characteristics of each nonlinear element,
For a nonlinear element that exhibits a large output and continues to react, the gain between its input and output decreases due to the change in the gain of the variable gain element according to the amplified output of the summing amplifier, and the output of the nonlinear element is Considering a case where the gain between the input and output of the non-linear element gradually recovers when it becomes small or becomes 0, and in such a case, a composite space in which a plurality of types of space patterns are superposed is overlapped. When a pattern is input, as described above, competition occurs between a plurality of types of spatial patterns, but over time, only a single spatial pattern becomes stronger, and other spatial patterns disappear. Note that the case under consideration is the same for the non-linear element that does not have the adaptation characteristic as described above.

Then, in the state where only a single spatial pattern becomes strong and the other spatial patterns disappear, the signal flow due to all the downward signal paths is interrupted by an on / off switch in the signal path only for a moment. Immediately after the interruption, the original state was restored, and the nonlinear element, which had been continuously showing a large output until just before the interruption of the downward signal path, had its reaction characteristics deteriorated by the adaptation effect. Therefore, immediately after the above-mentioned interruption, the spatial pattern that had been strengthened until then disappears and a reaction corresponding to another type of spatial pattern that was suppressed until then appears in the associative memory circuit. It will rise. Therefore, by repeating the instantaneous interruption of the downward signal path from the composite spatial pattern as the input information, one kind of spatial patterns of a plurality of types superposed in the composite spatial pattern is sequentially obtained. It is possible to realize an associative memory circuit having the ability to read data one by one.

Next, an example of a specific experiment showing that the information processing circuit according to the present invention has the associative memory capability of the above-described mode will be described. The experimental results described below were obtained by computer simulation.

That is, for an information processing circuit having 7 × 7 cells in the input / output layer U ₀ and having a four-layer structure U _{0 to} U ₃ , 5 types of learning patterns as shown in FIG. 3 are repeated. , And self-organized. Then, when a stimulation pattern of only a part as shown in the upper left corner of FIG. 4 is presented, the input / output layer U ₀
A time series of reaction patterns as shown in sequence in FIG. The numbers shown in the upper left corner of each pattern shown in the figure represent the elapsed time after the stimulation is presented. In this way, even if only a part of the learning pattern is presented as a stimulation pattern, an accurate spatial pattern, that is, a learning pattern, whose shape is not lost in the input / output layer U ₀ is gradually recalled over time.

Further, as shown in FIG. 5, when a plurality of types of learning patterns are input in an overlapping manner, at first, the plurality of types of learning patterns conflict with each other. Only one type of learning pattern becomes stronger, and other learning patterns disappear. At the same time, although not shown in the figure, in the final layer U ₃ , the cells corresponding to the finally learned learning pattern in the input / output layer U ₀ respond to each other, and the result of pattern recognition is shown. Note that the mark # attached to the upper right end of each pattern shown in the figure indicates that the signal flow in the downward signal path was interrupted at that time, and the recall output that lasted until that time disappeared. , Other recall output, but for some devices
It is possible to observe a mode of repetitive recollection that is emerging and gradually strengthened with a large output as shown by and a small output as indicated by □ for some other elements.

Effects As is clear from the above description, according to the present invention, it is possible to realize an information processing circuit having both associative memory capability and pattern recognition capability. It should be noted that the present invention can be applied for the purpose of utilizing only one of these two kinds of capabilities, and even in that case, since both functions act in cooperation with each other, this kind of conventional associative memory is used. A circuit or a pattern recognition circuit can exert a high level of information processing capability that has never been obtained.

The application examples of the present invention having such a special effect are listed and shown below.

1) Introduction of an associative function to pattern processing or image processing.

a) Estimation of the entire pattern from the partial pattern.

b) Removal of noise from noisy patterns.

c) Recall of the most similar patterns from ambiguous patterns.

d) Reading the constituent patterns from the composite pattern.

e) Decomposition of a composite pattern into constituent patterns.

2) Application to pattern recognition.

a) When inputting an ambiguous pattern or an incomplete pattern, it replies what a similar pattern is.

b) The constituent patterns included in the composite pattern are sequentially detected and recognized.

c) Cutting out individual patterns from the combined pattern, that is, performing segmentation, and at the same time,
Recognize what the cut out pattern is.

d) Detect and recognize a pattern buried in noise.

3) Application to various database searches.

That is, diagnosis of disease, literature search, determination of chemical structural formula,
For example, determination of gene structure.

In this application, each constituent element of the pattern is associated with an individual keyword.

For example, when diagnosing a disease, each element of the input / output layer U ₀ is made to correspond to the symptom of the disease, and each element of the uppermost layer is made to correspond to the disease name, and the information regarding the symptom may be incomplete. Even if some mistakes are made, the possible disease names are pointed out and combined, and the typical symptoms corresponding to the disease names are also output at the same time. Further, when the input information is ambiguous, it is possible to sequentially output the possible disease names.

[Brief description of drawings]

1 (a), (b) and (c) are diagrams showing the outline of the principle configuration and operation of a conventional associative memory circuit, respectively, and FIGS. 2 (a), (b) and (c) are book diagrams. FIG. 3 is a diagram sequentially showing a schematic configuration and an operation mode of the invention hierarchical information processing circuit, FIG. 3 is a diagram showing an example of a learning pattern used for computer simulation of the operation of the invention circuit, and FIG. FIG. 5 is a diagram showing an example of a mode of operation, and FIG. 5 is a diagram showing another example of a mode of operation. 1,2,3,4,5,6,7; 11,12,13,14,15; 21,22,23,24; 31,32 ……
Non-linear element, U ₀ …… Input / output layer, U ₁ , U ₂ , U ₃ …… Information processing layer.

Claims (4)

[Claims] 1. A hierarchical information processing circuit in which a plurality of information processing layers each having a plurality of nonlinear elements each having a variable gain between input and output arranged two-dimensionally are sequentially arranged. Among the processing layers, an upward signal path in which the coupling between the nonlinear elements is variable is set from the lower information processing layer toward the upper information processing layer, and is paired with the upward signal path. And a downward signal path in which the coupling between the non-linear elements is variable is set in the opposite direction, and the strength of the variable coupling between the elements in both the signal paths is changed interlockingly with each other. Hierarchical information processing circuit. 2. The at least one of the associative memory output and the pattern recognition output is obtained by extracting the associative memory output and the pattern recognition output from the information processing layers at the lower end and the upper end, respectively. The hierarchical information processing circuit according to claim 1. 3. A non-linear element that maintains a large output among the plurality of non-linear elements, and a gain between the input and the output is reduced, and a gain between the input and the output is increased in the non-linear element. The hierarchical information processing circuit according to claim 1 or 2, wherein: 4. A composite input pattern input to the information processing layer at the lower end can be sequentially decomposed and read by repeating and instantaneously disconnecting the downward signal path. A hierarchical information processing circuit according to claim 3.