JP3328953B2 - Nonlinear synaptic neuron, device thereof and prediction method using the same - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a non-linear synaptic neuron, and more particularly to a non-linear synaptic neuron for effectively executing signal prediction of voice signals, image signals, and the like.

[0002]

2. Description of the Related Art A neural network using a network imitating a human brain as an electric signal processing is used for pattern recognition, signal processing or knowledge processing, and a plurality of units corresponding to brain neurons are connected in a complicated manner. The pattern recognition function and the knowledge processing function can be executed by appropriately determining the operation of each unit and the connection form between the units.

[0003] These neural networks are classified into a pattern transmission type and an automatic transmission type based on the structure of the network. The former forms a network for converting an input pattern into an output pattern, and the latter outputs a pattern closest to the input pattern. To form a network.

In the pattern transmission type neural network, each unit (neuron) is connected to an input layer, an intermediate layer,
The output layer is hierarchized, and each unit is connected from the input layer to the output layer, so that the input unit and the output unit are independent. Then, the arithmetic function and the image information are stored as the connection form and the connection strength between the units, and the strength of the connection is arbitrarily changed using the difference between the obtained output and the desired output. The strength of the connection at this time forms a predetermined parameter as a connection coefficient or a synaptic weight.

On the other hand, in an automatic transmission type neural network, an input unit and an output unit are common, all units in the network are interconnected with each other, and the connection strength (synapse weight) is similar to that of the input unit. Used to identify patterns.

[0006] Pattern recognition is performed by such a neural network. As is well known, in order to effectively operate this neural network, a learning algorithm known as back propagation or the like is used. In order to make a neural network work in a wide range of signal prediction application fields such as recognition, speech recognition, speech synthesis, signal processing, and knowledge processing, learning such as back propagation is extremely effective.

Usually, a neural network has a hierarchical structure of an input layer, an intermediate layer, and an output layer, and this intermediate layer is known as a hidden layer. In a pattern transmission type neural network, processing of input data is performed in a learning process. Contrary to (forward), the description of the synaptic connection is changed (rearward) so that the error of the result is reduced when learning data is input, and the learning algorithm is repeated until this converges.

Therefore, conventionally, various kinds of pattern recognition and other signal predictions have been successfully achieved in a neural network to which a learning function such as a back propagation algorithm has been added.

[0009]

However, such a conventional neural network requires a large number of learning operations in order to perform an optimal prediction operation. Therefore, when the number of parameters is large, the learning itself requires a great deal of time. There was a problem that time was wasted.

That is, in the conventional general neural network, all synapses are connected between the input layer and the intermediate layer and between the intermediate layer and the output layer. As a result, the number of operations required for learning is extremely small. There was a problem of increasing.

For example, assuming a neural network having 5 elements in the input layer, 6 elements in the intermediate layer, and 7 elements in the output layer, 30 coupling combinations occur between the input layer and the intermediate layer. Forty-two bonding sets are needed between the layers and the output layer.

In the conventional network, each synapse has a linear characteristic. As a result, it is necessary to perform learning while simultaneously operating all the combinations.
As a result, even in the relatively simple network assumed above, the total number of combinations is 1260, and a large amount of computation time is required to set desired parameters for such a large number of combined sets. Was needed.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned conventional problems, and has as its object a high-speed learning type capable of setting and converging a synapse coupling coefficient to an optimum value by learning a small number of times. It is an object of the present invention to provide a nonlinear synaptic neuron with high prediction accuracy, a device thereof, and a prediction method using the same.

[0014]

According to the present invention, there is provided a nonlinear synaptic neuron and device according to the present invention, comprising a plurality of nonlinear synapse neurons for giving a desired weight to each of a plurality of input signals. Means (circuit), and an adding means (unit) for adding outputs of the respective non-linear synapses, wherein the non-linear synapse means (circuit)
Is a variable weighting coefficient (a ₀ , a ₁ , a
₂ ,..., A _n )
₀ + a ₁ x + a ₂ x ² + a ₃ x ³ ... _An x ⁿ }.

Further, the nonlinear synapse neuron and the device according to the present invention include a plurality of nonlinear synapse means (circuits) for giving a desired weight to each of a plurality of input signals, and an adding means for adding outputs of the respective nonlinear synapses. And a nonlinear synapse means (circuit).
Is connected to a plurality of complementary membership function means (circuits), which are divided into sections according to the input signal level, a part of the antecedent part is sequentially overlapped and connected in parallel, and each membership function means (circuit) And a plurality of variable weighting means (circuits) whose degree of learning is determined by the grade of the corresponding membership function.

Further, the non-linear synaptic neuron and the device according to the present invention provide delay means (circuits) connected in series for receiving a sampled time-series signal, and give a desired weight to each of the delayed time-series signals. A plurality of non-linear synapse means (circuits), and an adding means (unit) for adding outputs of the non-linear synapses, wherein the non-linear synapse means (circuit) is sectioned according to an input signal level, and sequentially A plurality of complementary membership function means (circuits) in which part of the antecedent part is overlapped and connected in parallel, and the degree of learning is determined by the grade of the corresponding membership function connected to each membership function means (circuit). And a plurality of variable weighting means (circuits) to be determined.

Further, the nonlinear synapse neuron and the device according to the present invention include a plurality of nonlinear synapse means (circuits) for giving a desired weight to each of a plurality of signals at spatially different coordinate positions, and a nonlinear synapse of each nonlinear synapse. An adding means (unit) for adding an output, wherein the non-linear synapse means (circuit) is divided into sections according to the input signal level, and a part of the antecedent part is sequentially overlapped and connected in parallel. Complementary membership function means (circuit), and a plurality of variable weighting means (circuits) connected to each membership function means (circuit) and having a degree of learning determined by the grade of the corresponding membership function. Features.

[0018]

Therefore, according to the present invention, a desired weight is given to each of a plurality of input signals by a non-linear synapse, and the outputs of the respective non-linear synapses are added.
Variable weighting factors (a ₀ , a ₁ , a ₂ ,
, A _n ), the nonlinear function {f (x) = a ₀ +
Since having a variable function shown by ^{_{a 1 x + a 2 x 2}} + a 3 x 3 ... a n x n}, the coupling coefficient of each nonlinear synapse, the other synapses are determined independently, each even during the learning Synapses are weighted very few times,
For example, convergence is achieved by learning several times when the input signal is changed, and it is possible to set a highly accurate coupling coefficient with a reduced error value.

According to the present invention, a desired weight is given to each of a plurality of input signals by a non-linear synapse, and the output of each non-linear synapse is added, and the non-linear synapse is divided into sections according to the level of the input signal. Since a part of the antecedent part is sequentially overlapped and given a variable weight according to the grade of a plurality of complementary membership functions connected in parallel, the coupling coefficient of each nonlinear synapse is independent of other synapses. The weight of each synapse is converged by an extremely small number of times, for example, several times when the input signal is changed, and a high-precision coupling coefficient is set with a reduced error value. Becomes possible.

Further, according to the present invention, a predetermined amount of delay means for receiving a sampled time-series signal is provided, a desired weight is given to each of the delayed time-series signals by a non-linear synapse, and each of the non-linear synapses is weighted. The outputs are added, and the non-linear synapse is variable weighted according to the grade of a plurality of complementary membership functions in which the input signal is sectioned according to the level and a part of the antecedent part is sequentially overlapped and connected in parallel. Therefore, the coupling coefficient of each nonlinear synapse is determined independently of the other synapses, and even during the learning, the weight of each synapse is extremely small, for example, several times when the input signal is changed. Converges, and it is possible to set a highly accurate coupling coefficient with a reduced error value.

Further, according to the present invention, a desired weight is given to each of a plurality of signals at spatially different coordinate positions by a nonlinear synapse, and the outputs of the nonlinear synapses are added,
Since the non-linear synapse, the input signal is divided into sections according to the level, and a part of the antecedent part is given a variable weight according to the grade of a plurality of complementary membership functions connected in parallel in a redundant manner, The coupling coefficient of each nonlinear synapse is determined independently of the other synapses, and even during the learning, the weight of each synapse is extremely small,
For example, convergence is achieved by learning several times when the input signal is changed, and it is possible to set a highly accurate coupling coefficient with a reduced error value.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a non-linear synaptic neuron according to the present invention, a device thereof, and a prediction method using the same will be described in detail based on embodiments.

FIG. 1 shows a preferred embodiment of a non-linear synaptic neuron according to the present invention as a schematic overall view, which comprises four synapses 10-1 to 10-.
4, wherein each of these synapses is characterized by comprising a non-linear synapse in the present invention.

The input signals x ₀ , x ₁ , x ₂ , x _{3 are} applied to input terminals 11-1 to 11-4 of each nonlinear synapse 10.
, And a desired weight is given to each of the plurality of input signals x at the nonlinear synapse 10.

The output of each nonlinear synapse 10 is
After being supplied to the adding means 12 and adding the respective outputs, it is outputted from the output terminal 19 as an output signal y. In each of the non-linear synapses 10, the content is represented by a variable function f ₀ ,
f ₁ , f ₂ , and f _3. As a result, the input / output relationship of the nonlinear synaptic neuron is given by the following equation.

Y = f ₀ (x ₀ ) + f ₁ (x ₁ ) + f ₂ (x ₂ ) + f ₃ (x ₃ ) (1) Accordingly, in the present invention, each synapse has a nonlinear input / output relationship. As a result, when a system is identified from a time-series signal or a spatial signal, even when the nonlinearity of the system is strong, highly accurate nonlinear approximation can be performed by using the nonlinear synapse of the present invention.

Therefore, in this embodiment, if the variable function f (x) of each nonlinear synapse 10 is determined by learning, a prediction signal y can be obtained with extremely high accuracy for the input signal x.

In this embodiment, the variable function f (x) of the synapse 10 is represented by the following polynomial equation (2).

F (x) = a ₀ + a ₁ x + a ₂ x ² + a ₃ x ³ +... _An ^xn (2) Therefore, _if the value of the coefficient an in the above equation (2) is determined by learning, An optimal non-linear synapse can be set for an input signal.

Hereinafter, the structure of any of the above-described nonlinear synapses 10 will be described in more detail.

When the above equation (2) is considered as a fourth-order variable function, it is expressed by the following equation (3). _{f (x) = a 0 +} a 1 x + a 2 x 2 + a 3 x 3 + a 4 x 4 = a 0 + x [a 1 + x {a 2 + x (a 3 + a 4 x)}] (3) Equation And this ( Equation 3) can be configured with the cascade-connected product-sum circuits of FIG.

That is, the non-linear synapse 10 shown in FIG. 2 includes four cascade-connected product-sum circuits 13-1 to 13-4, and each of the product-sum circuits 13 is a multiplier 14 and an adder 1 respectively.
Consists of five. In the product-sum circuit 13-1, the multiplier 14-
1 the input signal x and the preceding product-sum circuit 13-2 outputs f _-1 obtained from (x) and is supplied, the product output of the two signals is summed with the coefficient a ₀ in the adder 15-1 ing.

Similarly, each product-sum circuit 13 sequentially outputs the output f
It consists configured for adding the respective coefficients a _n to the integrated value of the input signal x _-n (x).

Therefore, it is understood that the above-mentioned equation (3) is realized by the nonlinear synapse circuit shown in FIG.

Of course, such a circuit of the non-linear synapse 10 may be constituted by an IC circuit, or may be realized by software.

As described above, when the nonlinear synapse 10 is constructed in the present invention, each synapse 1 shown in FIG.
0 is configured independently, no longer as in the conventional neural network, it is not necessary to combine the linear synapse group for all input signals to the network structure, whereby the coupling coefficient a _n of each synapse 10 described above It is very easy to learn.

As is apparent from FIG. 2, the cascade-connected product-sum circuit 13 has its downstream side, that is, the coefficient a ₀ of the product-sum circuit 13-1 is involved in the determination of the constant component. a _n is to represent the high-order components of the signal. As a result, in the present embodiment, it is preferable to determine each synapse coefficient by learning sequentially from the constant component to the higher-order component, whereby each coefficient can be easily determined to the optimum value with a small number of times. .

In order to determine the five coefficients a _{0 to} a ₄ , (x ₁ , y ₁ ), (x ₂ , y ₂ ),.
(X ₅ , y ₅ ), (x ₆ , y ₆ ),... And five or more data are required. This is because, in equation (3), a _{0 to} a
To determine all the coefficients of _4, by substituting the five data relating (x, f _(x)) in equation (3), similar to solving the 5-way simultaneous equations regarding a ₀ ~a ₄ I have. With less than five (x, f _(x) ) data, all a _{0 to} a ₄ cannot be determined.

In the following, (x ₁ , y ₁ ), (x ₂ ,
y _2), ..., describes a procedure for determining the a ₀ ~a ₄ by learning using six data (x _6, y _6).

(1) First, the coefficients a1 to a4 are set to 0.

(2) ε ² (t) ￣ ≧ 1 / 6Σ (y _i (t) ＾
−y _i ) ² (i = 1 to 6) is calculated.

[0042] ^{(3) δε 2 (t)} ¯ / δa n (t) ≧ 0 if _{a n (t + 1) =} a n (t) -G, δε 2 (t) ¯ / δa
_n (t) <0 If a _n (t + 1) = a a _n (t) + G. Here, G is a constant.

[0043] (4) (as on the curve of example FIG. _{4 B) a n (t +} 1) in the ε ² (t + 1) ¯ is epsilon ² (t) is smaller than, the a _n (t +1) as a _n (t)
Return to If ε ² (t + 1) ￣ is larger than ε ² (t) （(for example, like C on the curve in FIG. 4), the above (3)
(3) is performed by halving G at Then ε
² (t + 1) ￣ is D. A _n (t + 1) thus obtained
G is reduced by half until the root mean square error ε ² (t + 1) ￣ becomes smaller than ε ² (t) に お け る. Thus, ε ² (t
+1) continues until ￣ becomes lower than A as shown by E in FIG. Returning to (3) below.

(5) By repeating the above (3) and (4), |
^{ε 2 (t + 1) ¯} -ε 2 (t) ¯ | When there is less than the value, the learning of the a _n (t) is suspended, following a _{n + 1} (t)
Learning similar to (2) to (5) is performed.

(6) Once the values of a _{0 to} a ₄ are determined by learning, it is regarded that one learning is completed.

(7) The above (2) to (6) are repeated until ε ² (t) ￣ becomes a certain value or less.

According to experiments, the desired synaptic coefficient a
To determine the ₀ ~a _n, it was possible sufficiently high precision learning several learning. Also, five data (x ₁ , y ₁ ), (x ₂ , y ₂ ),..., (X ₅ ,
using y _5), was determined with a ₀ ~a ₄ in the manner described above, became a ₄ = 0 of course.

As described above, if each coupling coefficient of a nonlinear synapse is identified by a multi-order polynomial model, a highly accurate predicted value can be obtained for a desired input signal.

In this embodiment, the multiplier 14-1 and the adder 15- used in the product-sum circuit 13 in FIG.
1 need not be highly accurate. That is, even a non-linear highly multipliers and adders, in a form including this, by learning, set the order (number of a _n) and the coefficient a ₀ ~a _n, ε
² ￣ can be reduced. This is very convenient when configuring the device.

In the present invention, such an input signal can be handled as a time-series signal or a spatial signal. Hereinafter, another embodiment of the present invention in which a time-series signal is processed as an input signal will be described. explain.

In this embodiment, the dynamic synaptic neuron of the present invention is identified as a dynamic system by a combination with a delay element.

FIG. 5 shows a preferred embodiment of such a non-linear synaptic neuron as an overall schematic diagram.

[0053] series signal y _n when sampled in FIG. 5 in the embodiment from the input terminal 21 are connected in series are inputted to the four delay circuits 20-1 to 20-4.

[0054] Thus, the input signal y _n is and is supplied sequentially to the nonlinear synapse 10-1 to 10-4 through successively each of the delay circuits 20 for each sampling period. Therefore, according to the present embodiment, when four time-series signals are sampled and input, all the nonlinear synapses 10 are sampled.
Will work.

Now, four time-series signals X are represented by the following (4)
When given in equation, referred to as _{X = (x 0, x 1} , x 2, x 3) (4) type input vector X _i the set of input signal x ₀ ~x ₃ to each nonlinear synapse 10. This input vector X _i = (x
₀ , x ₁ , x ₂ , x ₃ ) is supplied to the input terminal of each nonlinear synapse 10 through the delay element 20 in order from the initial value of the time-series signal X.

Therefore, the learning operation and the prediction operation in the present embodiment will be described using the relationship between the input vector and the operation mode in FIG.

[0057] In the state shown by reference numeral 101, the input vector is in a state where the X ₀ to X ₂ is entered, in this three-step, with respect to each of the delay circuits 20, operates all four delay circuits 20 Cannot be performed, and is treated as a preliminary operation in the actual mode. That is, the learning operation for obtaining the coupling coefficient of each nonlinear synapse 10 in this embodiment is carried out in a plurality of times from the input vector X ₃ in FIG. 6 to X _n-1, both the four input signals at this time x _{0 to} x ₃ are supplied from each delay circuit 20 to the nonlinear synapse 10.

Accordingly, as shown in FIG. 2 described above, the coupling coefficient a of each synapse 10 depends on each of these input signals.
Can be requested. This state is denoted by reference numeral 1 in FIG.
02 is indicated.

[0059] In this embodiment, sequential learning is advanced for each input vector from X _3, indicated at 102 to the X _n-1, for k-th input vector X _k, the present embodiment The learning is performed so that the evaluation function of the following equation (5) is minimized.

Ε ² _k = (y _{k + 1} ＾ −y _{k + 1} ) ² Equation (5) In equation (5), y _{k + 1} is a time-series signal at time k + 1, and y _{k + 1 }} is This shows a predicted value when a time-series signal is input.

In this embodiment, the number of times of convergence learning until a sufficiently usable error ε _k is obtained is sufficient.
Therefore, according to the present invention, it is understood that the number of times of learning is significantly reduced as compared with the conventional neural network.

Reference numeral 103 in FIG. 6 indicates a prediction operation one sampling time ahead when the input vector X _n is supplied to the input terminal 21 after the coupling coefficient of each nonlinear synapse 10 is determined by the learning described above. The input vector X _n is given a desired weight by each of the non-linear synapses 10, and the added value is obtained as x _{n + 1} }.

Therefore, by using the input vector X _n as a sampling input signal of a time series signal in sequence, it is possible to always accurately obtain a prediction value one time ahead.

[0064] In order to further make a prediction earlier than one sampling time, as shown by reference numeral 104 in FIG. 6, which is the output for the input vector X _n y
_{n + 1 ^, y n +} 2 ^, y n + 3 ^ ... sequentially returned to the input terminal 21, by operating repeatedly nonlinear neuron according to the present invention, it becomes possible to obtain a desired future predicted value.

An example in which an actual time-series signal is identified by the nonlinear synaptic neuron according to the present embodiment will be described below.

Now, in order to perform a learning simulation on one nonlinear synapse, consider a time-series signal represented by the following equation (6).

F (x) = 2-2x ² + x ⁴ (6) That is, the above equation (6) may be composed of a multi-order polynomial in which each coefficient a of the above equation (2) has the following value. Understood.

A ₀ = 2 a ₁ = 0 a ₂ = −2 a ₃ = 0 a ₄ = 1 a ₅ = 0 In FIG. 7, the above-described polynomial polynomial is shown by a solid line. The results identified in the non-linear synaptic neurons in the examples are shown below.

FIG. 7 shows the functions identified when learning is performed using the present embodiment with the standard variable G = 0.01 and the number of learning times of 200 in the polynomial of the equation (6) shown by the solid line. Indicated by broken lines. The standard variable G indicates a standard variation width of the coefficient a at each learning. As is apparent from FIG. 7, this function almost overlaps with the time-series signal shown by the original solid line, and it is understood that the identification of the present embodiment under the above-described conditions is performed with extremely high accuracy. .

The data used for the learning is eight pieces of sampling data, and their coordinate values are shown below.

(-1.0, 1.0) (-0.8, 1.1296) (-0.5, 1.5625) (-0.1, 1.9801) (0.0, 2. 0) (0.2, 1.9216) (0.4, 1.7456) (1.0, 1.0) FIG. 8 shows the learning process in this embodiment as an error {(ε ² }).
Shows that the error is corrected along with the learning, and the square root of the mean square error decreases rapidly with the number of times of learning, indicating that 116 times of learning is sufficient to achieve an error of 1%.

FIG. 9 shows a convergence state by learning the coefficients a _{0 to} a ₅ of the multi-order polynomial used in this embodiment, and almost each coefficient is converged by learning 200 times used in the experiment. It is understood that.

Here, the convergence state of each coefficient was the same in any case of G = 0.01, 0.1, 1.0,
The learning time was, of course, faster in the order of G.

FIGS. 10 to 12 show an example in which another time-series coefficient is system-identified using this embodiment.

This experiment shows that the system identification of the non-integer exponential polynomial shown in the following equation (7) is possible.

F (x) = 2-2 × ^2.5 + x ^3.5 (7) The solid line in FIG. 10 indicates an input signal handled in this embodiment.
According to the present embodiment, even for an input signal having a non-integer exponent as described above, it is understood that a system approximation as indicated by a broken line can be performed by synapse means of a nonlinear function having an integer exponent. Is done.

According to the second experiment, learning is performed 1000 times using five sampling data to obtain the above-described system identification result indicated by the broken line.

FIG. 11 shows how the square root of the mean square error shows a decreasing behavior by learning, and shows a steep error decrease.

Further, FIG. 12 shows how the nonlinear function coefficients a _{0 to} a ₅ of the nonlinear synapse converge when learning in this embodiment. It is understood that the values converge.

In each of the embodiments described above, the weight of the non-linear synapse is determined by the polynomial polynomial. However, such a non-linear synapse can be expressed using a fuzzy rule in the present invention.

FIG. 13 shows a preferred embodiment using the fuzzy logic of the present invention, and shows an embodiment in which the above-described nonlinear synapse 10 of the neuron structure of FIG. 1 is constituted by membership function means and weighting means. Have been. In the figure, only the non-linear synapse 10-4 is illustrated, but other non-linear synapses 10-1, 10-2, and 10-3.
Is also composed of a plurality of membership function means and a plurality of weighting means provided corresponding thereto.

In FIG. 13, each nonlinear synapse 10 has a structure of a nonlinear synapse f (x) with five labels, and has five membership function circuits 3 connected in parallel.
The input signal x is supplied to 0-1 to 30-5. Each of these membership function circuit 30 is connected in parallel partially overlapped sequentially antecedent as seen in μ ₁ ~μ ₅ in FIG. 14, mutually two membership functions to further adjacent It is configured to be complementary.

Therefore, regardless of the value, the grade of the input signal x is determined only by two of the five membership function circuits 30, and the total value of both grades is always "1". The complementary type is set as follows.

Accordingly, the weight of the weighting circuits 31-1 to 31-5 connected to each membership function circuit 30 is determined by the grade of the membership function and the degree of learning is determined by the grade of the corresponding membership function. It will be.

In the embodiment, the output of the weighting circuit 31 is supplied to the above-mentioned adder 12 via the adder 32. However, in the present invention, it is not always necessary to provide the adder 32 in the nonlinear synapse 10. That is, the same effect can be obtained even if each output of the weighting circuit 31 is directly supplied to the adder 12.

In the structure shown in FIG. 13, the sum of values obtained by multiplying each grade μ with respect to the input x by the weight w indicates the input / output relationship of the nonlinear synapse 10.

Therefore, the input / output relationship of the nonlinear synapse f (x) is expressed by the following equation (8).

F (x) = Σw _j μ _j (x) (j = 0 to 4) (8) From Expression (8), each weight w depends on the grade μ of the corresponding membership function. Is understood.

The aforementioned membership function μ and input signal x
14 is shown in FIG. In FIG. 14, the input signal x fluctuates between the minimum values x _min and x _max and the time t
To form a time-series signal.

In the embodiment, the signal level x is defined as x ₀ to x
Divided into separated four intervals in _4, x _01, x ₁₂ in FIG,
be divided into x _23, x _34.

The membership functions of the membership function circuit 30 are set such that the adjacent membership functions are complementary to each other and the grade total is always "1" for each section. As a result, the membership function μ ₁ is given only for the section x ₀₁ as shown by the solid line. And the membership function μ ₂
In the section x ₀₁ and x ₁₂ are similarly membership function μ
₃ is the interval x ₁₂ and x ₂₃ , the membership function μ ₄ is the interval x ₂₃
A sign for x _34, the membership function mu ₅ the section x ₃₄
Only given about.

Therefore, at any signal level value, only the two membership functions function complementarily and their total grade is always “1”. For example, at time t ₁ , the membership functions μ ₂ and μ ₃ works.

At the time constant t ₁ , the other three membership functions μ ₁ , μ ₄ and μ ₅ are all “0”.
Thus, it is clear that the weighting determined by the grade of these membership functions does not work. Therefore, at time t ₁ , the membership function μ
_2, the weighting w ₂ and w ₃ are learned in accordance with the mu ₃ grades, learning other w _1, w _4, w ₅ is not performed,
This is similar in all the sections x _12, section x ₁₂
Then grade μ determined by each membership function
The learning of only the weights w ₂ and w ₃ according to the degrees of ₂ and μ ₃ is performed. Similar effects can be obtained in other sections.

In the present invention, the membership functions adjacent to each other are overlapped in the antecedent part and are used in a complementary manner, so that the total number of grades becomes "1" at all levels as described above. There is an advantage that the data after the above can be used as an output of the nonlinear synapse 10 as it is. That is, if these grade statistics are not "1", it is necessary to perform division or other defuzzification after addition, which significantly complicates a circuit or a processing system. Such inconvenience can be reliably prevented by using the membership function.

[0095] Figure 15 is a procedure for determining the weighting coefficient of the nonlinear synapses 10-3 at the time t ₁ is shown.

The input signal value at time t ₁ is x (t ₁ )
And the membership function excited at this time is μ
₂ and μ _3. These membership functions μ ₂ , μ ₃
Are grades μ ₂ (t ₁ ) and μ ₃ (t ₁ )
Indicated by Therefore, the ratio of the weights w ₂ and w ₃ is determined according to these grades, and the average is w
(T ₁ ).

Equation (9) shows the weight w (t ₁ ) obtained on average from the complementary membership function.

W (t ₁ ) = {w ₂ × μ ₂ (t ₁ ) + w ₃ × μ ₃ (t ₁ )} / {μ ₂ (t ₁ ) + μ ₃ (t ₁ )} = w ₂ × μ ₂ (T ₁ ) + w ₃ × μ ₃ (t ₁ ) (9) According to the present invention by using the complementary membership function as is apparent from the equation (9), the denominator is always “1”.
Thus, there is no need for defuzzification such as a divider in the subsequent stage.

As described above, according to the present invention, the weight of a nonlinear synapse can be determined using fuzzy inference, and each nonlinear synapse is independent as in the above-described polynomial polynomial. The number of times of learning for determining these weights is extremely small, and optimal synapse weight determination can be easily performed.

FIG. 16 shows a chaotic signal produced by a third-order dynamic system, and this time-series signal is represented by the following equation (10).

[0102] _{y n + 1 = f (y} n) + ay n-1 + by n-2 (10) Equation _{i.e., y n + 1 = f (} y n) + p n + q n p n + 1 = ay n q n ₊₁ = having a bp _n = by _n-1 the relationship.

In FIG. 16, the time-series signal is represented by 50 data of 0 to 49, and the value of X on the vertical axis fluctuates chaotically between ± 3.0.

Therefore, in the present invention, it is shown that such a conventionally unpredictable time-series signal can be predicted by learning a small number of times using a non-linear synapse.

In an experiment using this embodiment, FIG.
Was identified in each of the first to fifth order models.

FIG. 17 shows the five non-linear synaptic neurons according to the present invention of the first to fifth orders.

FIG. 18 shows the mean square error when the chaotic signal in FIG. 16 is identified by these five neurons.

As is apparent from FIG. 18, the dynamical system of FIG. 16 is a tertiary time-series signal. However, since the mapping function of each order at the time of identification by a neuron is a polygonal line approximation, the dynamic system of FIG. Also 4th and 5th
It can be understood that better results are obtained because the secondary neuron model can correct at higher synapses. FIGS. 19 to 23 show the mapping functions of the respective orders when the third-order time-series signals are system-identified by the above-described fifth-order neuron model. In each figure, the solid line represents FIG.
The mapping function at each order of the third-order dynamical system used to create the chaotic signal of No. 6 is shown, and the broken line shows the mapping function of each order when identified by the fifth-order neuron model.

As is clear from FIGS. 19 to 23, according to the present experimental example, since the third-order dynamical system is identified by the fifth-order neuron model, the line-to-line approximation error up to the third order is reduced to the fourth and fifth order. It is shown that correction is possible with the non-linear synapse.

FIG. 24 shows the prediction operation in the fifth-order neuron model. The fifth-order neuron model, which has been weighted and learned as described above, is sequentially trained with the input signal, and the coefficients are generated in the future while updating the coefficients. Predicting time-series signals.

This prediction may be one time ahead of the present input signal or at a predetermined arbitrary future time.

The predicted values (dashed lines) shown in FIG.
1 for the true value shown by the solid line
Shows the predicted value ahead of time. Therefore, it can be understood from FIG. 24 that, according to the present embodiment, a prediction action that sufficiently follows chaotic signals that could not be predicted conventionally can be performed.

FIG. 25 shows the fifth-order neuron model in this embodiment, which has been identified by performing weighting learning as described above, by applying feedback from the output to the input to obtain a signal from five fixed time-series signals. FIG. 2 shows a state in which predicted values that occur thereafter are sequentially predicted.
Similar to 4, the solid line indicates the true value, and the dashed line indicates the predicted future value obtained from the feedback.

The prediction in FIG. 25 performs a prediction operation for the future from only the currently given five time-series signals. Unlike FIG. 24, the prediction in FIG. .

However, it is understood from FIG. 25 that a prediction action with a very high degree of coincidence is obtained with respect to such predictions at least 5 to 10 times ahead.

FIGS. 26 to 29 show the relationship between the number of times of learning and the mean square error when the gain constant α is changed at the time of learning using this embodiment.

In the embodiment, when the following four types of α were used for the experiment, α = 0.02 α = 0.2 α = 0.56 α = 0.57 When α = 0.2, extremely good results were obtained. It is understood that system identification has been performed.

That is, learning is completed earliest when the gain constant α is set to 0.2, and as is apparent from FIG.
Sufficiently accurate identification is performed with about 10 learning times,
It is shown that the nonlinear synaptic neuron of the present invention can be identified by a very small number of times, about 100 to 1000 times as compared with the related art.

The nonlinear synapse 10 determined by fuzzy inference is similar to the nonlinear synapse based on the above-mentioned polynomial polynomial, and sequentially from a plurality of input time-series signals at the next time or at a predetermined time ahead in the future. The value can be predicted sequentially, and a predicted value ahead of a predetermined amount can always be obtained by sequentially updating the input time-series signal.

If a certain number of input signals are supplied to the neuron model whose system has been identified, the time ahead can be predicted sequentially. That is, the input signal is fixed at a certain time, but when a predicted value at the next time is output, this is fed back to the input terminal, and the predicted value in the future is further used while using the fed back predicted value one after another. The values can be updated sequentially.

Even when a non-linear synapse based on fuzzy inference is used, an experiment can identify a system with sufficiently high accuracy by learning five to ten times. There is an advantage that learning can be completed in time.

The non-linear synaptic neuron according to the present invention can process not only a time-series signal but also a spatial signal as its input signal, and can also be used to obtain a predicted value outside the given space. .

FIG. 30 shows an example of such a spatial signal, wherein x ₁ to x ₅ distributed at equal intervals in the x-axis direction.
Then, x ₇ are known. In this case, the nonlinear synaptic neuron according to the present invention is (x ₁ , x ₂ , x ₃ , x ₄ ) .
Kicking the value of _{y (y 1, y 2,} y 3, y 4) enter the output y of the respective nonlinear synapses 10 so that y ₇ f (x)
Find by learning. Of course, as it is generally known data, it is preferable to perform the learning by entering not only four data y as described above, the serial multiple data y ₁ ~y ₅₀ by four.

In this way, system identification is performed,
Next, the value of y of (x ₂ , x ₃ , x ₄ , x ₅ ) (y ₂ ,
When y ₃ , y ₄ , y ₅ ) are input, the value y ₈ corresponding to the coordinate x ₈ can be predicted from the nonlinear neuron according to the present invention.

As described above, according to the present invention, unknown coordinates can be extrapolated using not only a time-series signal but also a spatial signal.

[0125]

As described above, according to the present invention,
The weights of the non-linear synapses are determined by polynomial polynomials or fuzzy inference, and since these synapses are independent of each other without interfering with each other, it is very easy to obtain the desired weights or coupling coefficients with a small number of learnings. In particular, according to the non-linear synapse using the multi-order polynomial of the present invention and the complementary membership function of the present invention, there is an advantage that extremely accurate prediction can be performed with a simple structure.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram showing an overall configuration of a nonlinear synaptic neuron according to the present invention.

FIG. 2 is a block circuit diagram showing an embodiment in which a non-linear synapse according to the present invention is constituted by a multi-order polynomial.

FIG. 3 is an explanatory diagram for minimizing a mean square error for obtaining a weighting coefficient of a non-linear synapse composed of a multi-order polynomial.

FIG. 4 shows a _n (t) on a graph showing a relationship between a weighting coefficient of a non-linear synapse composed of a polynomial polynomial and a mean square error.
FIG. 9 is an explanatory diagram for obtaining a _n (t + 1) from FIG.

FIG. 5 is an explanatory diagram showing a preferred embodiment of a neuron that inputs a time-series signal to a non-linear synapse in parallel via a plurality of delay circuits in the present invention.

FIG. 6 is an explanatory diagram illustrating a weight learning operation and a prediction operation according to the present invention.

FIG. 7 is an explanatory diagram showing an input signal when a system is identified using the nonlinear synaptic neuron according to the present invention and a result of the system identification.

FIG. 8 is an explanatory diagram showing a convergence state of a mean square error when a system is identified using this embodiment.

FIG. 9 is an explanatory diagram showing a convergence state of each coefficient of a multi-order polynomial when a system is identified using the embodiment.

FIG. 10 is an explanatory diagram showing an input signal and a system identification result when an input signal having a non-integer exponent is system-identified using this embodiment.

FIG. 11 is an explanatory diagram showing a convergence state of a mean square error at the time of system identification in this embodiment.

FIG. 12 is an explanatory diagram showing a convergence state of each coefficient of a multi-order polynomial in system identification using the present embodiment.

FIG. 13 is an explanatory diagram showing a preferred embodiment when fuzzy inference is used for the nonlinear synaptic neuron according to the present invention.

FIG. 14 is an explanatory diagram showing a complementary membership function of the embodiment shown in FIG.

15 is an explanatory diagram showing a relationship between a complementary membership function and a weighting coefficient in the embodiment shown in FIG.

FIG. 16 is an explanatory diagram showing chaotic signals for performing system identification by the nonlinear synaptic neuron according to the present invention.

FIG. 17 shows a first order to a system identification of the chaotic signal in FIG. 16 using the nonlinear synaptic neuron of the present invention.
It is explanatory drawing which shows a 5th-order neuron model.

18 is an explanatory diagram showing a mean square error at the time of system identification by each of the neuron models shown in FIG. 17;

FIG. 19 is an explanatory diagram showing a first-order mapping function at the time of system identification.

FIG. 20 is an explanatory diagram showing a second-order mapping function at the time of system identification.

FIG. 21 is an explanatory diagram showing a third-order mapping function at the time of system identification.

FIG. 22 is an explanatory diagram showing a fourth-order mapping function at the time of system identification.

FIG. 23 is an explanatory diagram showing a fifth-order mapping function at the time of system identification.

FIG. 24 is an explanatory diagram showing a relationship between a true signal and a predicted signal when a time series signal is sequentially supplied to the non-linear synaptic neuron in the embodiment identified by the system to predict one time ahead.

FIG. 25 is an explanatory diagram showing a relationship between a true signal and a predicted signal when performing future prediction while sequentially feeding back predicted values using a time-series signal at a specific time using a neuron model in the present embodiment. is there.

FIG. 26 is an explanatory diagram showing the relationship between the number of times of learning and the error when the gain constant α at the time of learning is 0.02.

FIG. 27 is an explanatory diagram showing the relationship between the number of times of learning and the error when the gain constant α during learning is 0.2.

FIG. 28 is an explanatory diagram showing the relationship between the number of times of learning and the error when the gain constant α during learning is set to 0.56.

FIG. 29 is an explanatory diagram showing the relationship between the number of times of learning and the error when the gain constant α during learning is set to 0.57.

FIG. 30 is an explanatory diagram showing an extrapolation prediction operation when the present invention is used for a spatial signal.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 Nonlinear synapse 11 Input terminal 12 Adder 13 Product-sum circuit 14 Multiplier 15 Adder 19 Output terminal 20 Delay circuit 30 Membership function circuit 31 Weighting circuit

Continuation of the front page (56) References JP-A-3-125257 (JP, A) JP-A-2-201586 (JP, A) JP-A-5-128086 (JP, A) JP-A-2-93708 (JP) JP-A-3-218558 (JP, A) JP-A-3-268055 (JP, A) JP-A-2-162459 (JP, A) JP-A-3-95664 (JP, A) 6-35707 (JP, A) Nobuhiro Murai, et al., “Extraction of Nonlinearity of Time Series Using Neural Network Prediction Model”, IEICE Technical Report, Japan, IEICE, Japan, March 18, 1992, Vol. 91, No. 529 (NC91-98-131), pp. 237-244 Junzo Watada et al., "Fuzzy Time Series Model and Its Application to Prediction Problems", Journal of Japan Society for Business Engineering, Japan, 1983, Vol. 34, No. . 3, pp. 108-186, (JST Material No .: F0241B) Gupta, M .; M. Qi, J .; , "Synaptic and somatic learning and adaptation in fuzzy neural systems", Proc. of 1991 IEEE International Joint Conference on Neural Networks, 1991, Vol. 1, pp. 875-880, (INSPEC Accession Number: 4254100, ISBN: 0-7803-0227-3) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06N 1/00-7/08 G06G 7 / 60 G06F 9/44 G05B 13/00-13/04 JST file (JOIS) CSDB (Japan Patent Office)

Claims (3)

(57) [Claims] 1. A serially connected delay means for receiving a sampled time series signal; a plurality of nonlinear synapse means for giving a desired weight to each of the delayed time series signals; and an output of each nonlinear synapse. A non-linear synapse means, wherein a plurality of complementary membership function means are divided according to an input signal level, and a part of the antecedent part is sequentially overlapped and connected in parallel. A plurality of variable weighting means connected to each membership function means, the degree of learning being determined by a grade of the corresponding membership function, and a plurality of variable weighting means. 2. A delay circuit connected in series for receiving a sampled time series signal, a plurality of nonlinear synapse circuits for giving a desired weight to each of the delayed time series signals, and an output of each nonlinear synapse. A non-linear synapse circuit, wherein the non-linear synapse circuit is divided into sections according to an input signal level, and a plurality of complementary membership function circuit rows in which a part of the antecedent part is sequentially overlapped and connected in parallel. And a plurality of variable weighting circuits connected to each membership function circuit, the degree of learning being determined by the grade of the corresponding membership function, and a non-linear synaptic neuron device. 3. A sampled time-series signal is received and given a predetermined amount of delay, a desired weight is given to each of the delayed time-series signals by a non-linear synapse, and the outputs of a plurality of non-linear synapses are added to perform sampling. Time
Predicting the value of a time-series signal at a desired time
In the measurement method, the non-linear synapse is divided into a plurality of sections in which the input signal is sectioned according to a level based on the sampled time-series signal, and a part of the antecedent part is sequentially overlapped and connected in parallel. A predictive method using a non-linear synaptic neuron, wherein variable weighting is given according to the grade of the complementary membership function of the above.