JP3264686B2 - Neural network device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network device, and more particularly to an arrangement for determining the reliability of an output from the neural network.

[0002]

2. Description of the Related Art Conventionally, as disclosed in, for example, Japanese Patent Application Laid-Open No. 1-183763, a neural network device having a neural network constructed by learning is known. The output of the network is guaranteed only for learned and tested inputs, and there is no way to predict what output will be given to unlearned inputs. For this reason, for example, in order to construct a number recognition device that stores numbers 1 to 9 of a printer print by learning using a neural network, first, learning using several types of numbers having different print qualities as learning inputs is performed. Then, a number with a print quality different from these learning inputs is given as a test input. At this time, a test is performed to determine whether or not the output expected by humans is obtained. A method is employed in which an input is added to a learning input and learning is newly repeated. This is because only humans can judge whether the output of the neural network is appropriate for an unlearned input.

However, such a conventional neural network device has the following problems. That is, humans must judge the validity of the output with respect to the test input, and the neural network must always provide some kind of input regardless of whether it is a learned input, a test input, or an unlearned input. Although the neural network device itself cannot determine whether the output is valid or not, the reliability is problematic in an environment in which an input that was not assumed during learning and testing is expected.

On the other hand, as disclosed in the specification of Japanese Patent Application No. 2-262076 filed by the present applicant, the neural network learning elements (synaptic transmission efficiency) at the time of learning establishment are different from each other. A plurality of neural networks are provided, and when the same pattern of input is given to each of the neural networks, the degree of pattern matching between outputs from the respective neural networks is detected, and based on the degree of matching, If the reliability of the output of the neural network is determined, the device itself can determine the reliability of the output of the neural network.

[0005]

However, in such a neural network device, a learning element at the time of learning establishment differs between neural networks in order to make learning elements different from each other in advance by using a random number or the like. It is necessary to create multiple types of neural networks (initialization) that differ from each other, and it is necessary to check whether the learning elements of multiple neural networks are the same before learning. , Mass production was difficult.

The present invention has been made in view of such circumstances, and a neural network which can judge the reliability of the output of the neural network by itself and which can be easily mass-produced. It is intended to provide a device.

[0007]

The neural network device according to the present invention uses a random number or the like for making learning elements different between neural networks by devising the configuration of the neural network. This eliminates the need for initialization work, thereby achieving the above object.

That is, in the neural network device according to the first aspect, a plurality of neural networks and an output from each of the neural networks when an input of the same pattern is given to each of the neural networks. A coincidence detecting means for detecting the degree of coincidence of the pattern, and a reliability judging means for judging the reliability of the output of each of the neural networks based on the degree of coincidence detected by the degree of coincidence detecting means, Each of the neural networks is provided with input information expression conversion means for converting an input pattern to the neural network in a different conversion mode for each of the neural networks.

According to a second aspect of the present invention, in place of the "input information expression conversion means", the output patterns from the neural networks are different for each of the neural networks. An output information representation conversion means for converting in a conversion mode is provided.

The above-mentioned "reliability judging means" judges that the output of each neural network is reliable when it is detected by the coincidence detecting means that all outputs from the neural networks match. In other cases, it is of course possible to adopt a configuration that makes a judgment of no reliability. However, the reliability is determined based on whether or not the pattern matching between the outputs from the respective neural networks is equal to or greater than a predetermined reference value. It may be determined whether or not there is sex. In this case, the mode of the determination of the “reliability of output of each neural network” may be such that determination of the presence or absence of reliability is performed for each neural network, or the plurality of neural networks as a whole may be determined. It is also possible to judge whether there is reliability or to judge the degree of reliability.

[0011]

As described above, the coincidence of the output when the same input is given to a plurality of neural networks is detected, and the output of each neural network is determined based on the coincidence. Because trust decisions are being made,
The neural network device itself can determine the reliability of the output from each neural network without relying on human judgment. Therefore, with respect to the final output from the neural network device, it is possible to obtain an output that is guaranteed to be reliable or an output whose degree of reliability has been clarified, thereby reducing the reliability of the neural network device. It can be dramatically improved.

According to the first aspect of the present invention, each of the neural networks has an input information representation conversion for converting an input pattern to the neural network in a different conversion mode for each of the neural networks. Since the neural network itself includes learning means, even if the learning elements before learning are the same among the neural networks, the neural network itself including the input information expression converting means does not perform the learning before learning. Are different among the neural networks. Therefore, as the neural network itself, there is no need to create different learning elements before learning, and it is not necessary to check whether the learning elements of each neural network are the same before learning.

Similarly, according to the second aspect of the present invention, each of the neural networks has an output information expression for converting an output pattern from the neural network in a different conversion mode for each of the neural networks. Since the conversion means is provided, even if the learning elements before learning are the same between the respective neural networks, the neural network itself is in a state of the neural network including the output information expression conversion means. The previous learning element will be different between each neural network. Therefore, as the neural network itself, there is no need to create different learning elements before learning, and it is not necessary to check whether the learning elements of each neural network are the same before learning.

The input information expressing means and the output information expressing means may be used in combination.

As described above, in the neural network device according to the present invention, since the configuration of each neural network is devised, a random number or the like for making learning elements different between neural networks is used. This eliminates the need for initialization work, so that not only can the device itself determine the reliability of the output of the neural network, but also the advantage of LSI technology that creates a large number of identical elements can be used. Can be easily mass-produced.

[0016]

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 1 is an overall configuration diagram showing a first embodiment of a neural network device according to the present invention.

This neural network device has four neural networks 1, 2, 3, 4 and these neural networks 1 to 4 when the same pattern of input is given to each of the neural networks 1 to 4. 4 for detecting the degree of coincidence between the outputs of the neural network 4 and the reliability for determining the reliability of the output of each of the neural networks 1 to 4 based on the degree of coincidence detected by the degree of coincidence detecting means 5 Gender judging means 6 for converting the input patterns to the neural networks 1 to 4 in different conversion modes for each of the neural networks 1 to 4 Are provided. The programmable circuits 7 to 10 (input information expression conversion means) are provided.

As shown in FIG. 2, each of the neural networks 1 to 4 is composed of three layers, an input layer, an intermediate layer, and an output layer. Each of these layers is composed of 6, 3, and 4 neurons. ing.

Each of the neural networks 1 to 4 and each of the programmable circuits 7 to 10 are constituted by one-chip elements.

As shown in FIG. 3, each of the programmable circuits 7 to 10 comprises a gate array having the same number of input / output (six) as the number of nerve cells in the input layer. The bit position replacement input information expression is programmed to perform one-to-one input information expression conversion. Therefore, even when the same pattern of input is given to each of the neural networks 1 to 4, the input after passing through each of the programmable circuits 7 to 10 is converted into a different pattern for each of the neural networks 1 to 4. It will be.

For example, taking the neural networks 1 and 2 as representatives, a conversion Fa is programmed in the gate array of the neural network 1, and Fa: 1 → 2, 2 → 1, 3 → 4, 4 → 3 , 5 → 6,6 →
5 If the conversion Fb is programmed in the gate array of the neural network 2, Fb: 1 → 3,2 → 4,3 → 5,4 → 6,5 → 1,6 →
2 (Note: 1 → 2 means that input 1 is replaced by input 2.) The input after conversion of each neural circuit for input (1,1,1,0,0,0) is the neural network 1 (1, 1,0,1,0,0) Neural network 2 (1,0,0,0,1,1).

Alternatively, the conversion Fa is programmed in the gate array of the neural network 1 by using the operation between adjacent bits, and Fa: Xi = (Xi + Xmod (i + 1,6)) / 2 The gate of the neural network 2 If the conversion Fb is programmed in the array, Fb: Xi = (Xi + Xmod (i-1,6)) / 2 (Note: mod (j, 6) = if j = 0 then 6, j = 7 then 1 else j ) input after the conversion of the neural against the input (1,1,1,0,0,0) are neural network 1 (1,1,0.5, 0,0,0.5) neural network 2 ( 0.5, 1, 1, 0.5, 0, 0).

As described above, since each of the neural networks 1 to 4 is provided with the programmable circuits 7 to 10 to which the above-described programs are applied, even if the neural network itself has a synaptic transmission efficiency. Even if the same initial values are used, different initial values can be obtained for each of the neural networks 1 to 4 in consideration of the programmable circuits 7 to 10. Therefore, the same pattern of input is given to each of the neural networks 1 to 4 for learning, and the synapse transmission efficiency of each of the neural networks 1 to 4 when this learning is established in all the neural networks 1 to 4 is also improved. Naturally, they are different from each other.

The learning for each of the neural networks 1-4 is as follows.
Backpropagation learning method ["learning" algorithm that sequentially corrects connection weights (synaptic connection transmission efficiency) from the output side to the input side by comparing the output of the neural network with the correct answer (teacher signal)] Is performed for some input / output relations. That is, first, one input / output relationship is learned by each of the neural networks 1 to 4, and the learning is repeated until learning is achieved in all of these four neural networks 1 to 4. Then, after the learning is established, the process proceeds to learning of the next input / output relationship.

In this manner, when learning is established in all four neural networks 1 to 4 for some input / output relations, if the input given thereafter is a learned input, the four neural networks 1 4 are all correct values and coincide with each other. However, in the case of an unlearned input, the outputs of the four unit neural networks 1 to 4 are not constant. In this case, the probability that all four outputs coincide with each other is It will be extremely low.

The coincidence detecting means 5 is provided for each of the neural networks 1-4.
When the same pattern of input is given to each of the neural networks 1 to 4, the degree of pattern matching between the outputs is determined by the fact that all of the outputs of the four neural networks 1 to 4 match. The detection is performed depending on whether or not there is an error. That is, the coincidence detecting means 5 determines whether or not the outputs of the four neural networks 1 to 4 all match each other for each of the four neural cells in the output layer, and the four determinations obtained thereby. The result is output as a coincidence detection signal.

The reliability judging means 6 further judges whether or not all of the four judgment results are "the outputs of the four neural networks 1 to 4 all match". It has become. When the reliability determining means 6 determines that the outputs of the neural networks 1 to 4 match for all four nerve cells, the reliability determining means 6 determines whether the outputs of the neural networks 1 to 4 are reliable. When it is determined that there is, and the above output is output to the outside as the output of the neural network device together with the reliable signal “1”, if it is not determined that all of the above outputs are the same, Neural network 1
4 are determined to be unreliable, and all zeros are output to the outside as the output of the neural network device together with the unreliable signal "0".

As described in detail above, according to this embodiment,
The degree of coincidence is detected based on whether or not all outputs when the same input is given to a plurality of neural networks 1 to 4 having different synaptic transmission efficiencies at the time of learning coincide with each other. Since the judgment of the reliability of the output from each of the neural networks 1 to 4 is made, the neural network device itself outputs the output from each of the neural networks 1 to 4 without relying on human judgment. The reliability can be determined. Therefore, it is possible to obtain an output in which the presence or absence of the reliability is clarified also about the final output from the neural network device, and it is possible to dramatically improve the reliability of the neural network device.

Moreover, according to the present embodiment, since the neural networks 1 to 4 are provided with the programmable circuits 7 to 10, respectively, the neural networks 1 to 4 themselves have a synaptic transmission efficiency before learning. Even if the neural networks 1 to 4 are the same, in the state of the neural network including the programmable circuits 7 to 10, the synaptic transmission efficiency before learning is different between the neural networks 1 to 4. It will be different. Therefore,
As the neural networks 1 to 4 themselves, there is no need to perform an initialization operation of using random numbers or the like to generate synapse transmission efficiencies having different pre-learning synapse transmission efficiencies. There is no need to check if they are the same.

As described above, according to the present embodiment, not only the device itself can judge the reliability of the output of the neural network, but also the initialization work becomes unnecessary, so that the same element can be used in large numbers. Since the advantages of the LSI technology to be manufactured can be used, it can be easily mass-produced.

FIG. 4 is a configuration diagram of a main part showing a second embodiment of the present invention.

In this embodiment, instead of the programmable circuits 7 and 8 of the first embodiment, each of the neural networks 1 to 4 has
Programmable circuits 27 to 30 (output information expression conversion means) for converting the output patterns from the neural networks 1 to 4 in different conversion modes for each of the neural networks 1 to 4
However, the present embodiment is different from the first embodiment in that it has the same function and effect as the first embodiment.

Next, an expert system to which the neural network device according to the present invention is applied will be described.

The conventional expert system derives a certain conclusion or judgment result by using if A1 and A2 and... An then Y = ANS1 as a rule. However, Ai is true or false 2
It is a value and cannot use wide information or information containing noise. For this reason, in order to perform inference using information in the real world, Ai must be normalized to binary as preprocessing of the expert system. Due to this standardization, information in the real world is not transmitted, and the inference ability is low.

A fuzzy expert system is used as a technique for effectively using information in the real world for inference. However, there is no method for uniquely determining the "membership function", which is an important element of this system, from the inference to be realized. Membership functions are determined by trial and error based on human experience. For this reason, when the conditions of the if clause become complicated, it becomes difficult to know what membership function should be used, and it is difficult to realize complicated inference with a fuzzy expert system.

[0037] In contrast, an expert system that applies neural network apparatus according to the present invention, as shown in FIG. 5 as an application example, a plurality of the neural network device (4
Components) (hereinafter, referred to as “CSNN”) 62 arranged in parallel, an inference convergence determination unit 64, and an inference result output unit 6
6 to utilize the learning ability and the ambiguous information processing ability of the neural network. That is, complicated inference can be realized by "learning ability", and inference using real-world information can be realized by "ambiguous processing ability".

Each of the neural network devices U1 to U4 constituting the CSNN 62 is previously learned with rules 1 to 4 different from each other. The inference convergence determination unit 6
Reference numeral 4 indicates that the convergence is determined when all the reliability signals of the neural network devices U1 to U4 are output as “0”, and that the convergence is not determined otherwise. When the inference convergence determination unit 64 determines convergence, the inference result output unit 66 outputs an input at that time as an inference result, and otherwise outputs all zeros. The inference convergence determination unit 64, when it is determined that the non-convergence, so as to enter the output of the non-causes of convergence is a signal of signal outputs "1" neural network apparatus Ui to CSNN ing.

As a specific example of the expert system, an image inference apparatus that receives an image and outputs an image of an inference result based on a defined rule will be described. CSNN that trains the following rule Ri on the neural network device Ui
To build.

R1: A for B R2: B for C R3: C for D R4: D for E However, one neural network is composed of five perceptron-type neural networks having 64 inputs and 64 outputs. The device Ui is configured.
Error correction learning is used as a learning algorithm, and an 8 × 8 bitmap information expression of each image is used as an input.

The following steps show that the device can infer inputs E from A:

(1) Input A.

(2) Only U1 issues B, and other Ui
Outputs “0”.

(3) Since the CSNN output is B, the inference convergence determination unit issues a non-convergence determination and uses B as a new input.

(4) Only U2 issues C, and other Ui
Outputs “0”.

(5) Since the CSNN output is C, the inference convergence determination unit issues a non-convergence determination and uses C as a new input.

(6) Only U3 issues D, and other Ui
Outputs “0”.

(7) Since the CSNN output is D, the inference convergence determination unit issues a non-convergence determination and uses D as a new input.

(8) Only U4 issues E, and other Ui
Outputs “0”.

(9) Since the CSNN output is E, the inference convergence judging unit issues a judgment of non-convergence and uses E as a new input.

(10) All Ui output "0".

(11) Since the CSNN output is “0”, the inference convergence determination unit issues a convergence determination, and the inference result output unit outputs
The input at this time, that is, E is output as an inference result.

Even if noise is added to the input A and the input image is slightly different from A, the neural network device U1 outputs B. Therefore, correct inference is performed for an input including noise, and an inference result E is obtained.

That is, if the neural network device according to the present invention is applied, rules can be constructed by learning, and an expert system which is resistant to noise can be realized. Also, in order to add a new rule from the characteristics of CSNN, this can be easily realized by adding a neural network device that has learned the new rule.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram showing a first embodiment of a neural network device according to the present invention.

FIG. 2 is a configuration diagram showing each neural network in the embodiment.

FIG. 3 is a configuration diagram showing a programmable circuit in the embodiment.

FIG. 4 is a main part configuration diagram showing a second embodiment of the present invention.

FIG. 5 is an expert of the neural network device according to the present invention .
Configuration diagram showing an application example to a stem

[Explanation of symbols]

1, 2, 3, 4 Neural network 5 Matching degree detecting means 6 Reliability judging means 7, 8, 9, 10 Programmable circuit (input information expression conversion means) 27, 28, 29, 30 Programmable circuit (output information) Expression conversion means) U1, U2, U3, U4 Neural network device

Continuation of the front page (56) References Akimasa Shirosaka, "Column Structure Neural Network", IEICE Technical Report, Japan, IEICE, March 18, 1991, March 18, VOL. 90, NO. 483 (NC90-68 to 111), pp. 249-254, Patent Office CSDB literature number: C SNT199900646039 (58) Fields investigated (Int. Cl. ⁷ , DB name) G06N 1/00-7/00 G06G 7 / 60 CSDB (Japan Patent Office) JICST file (JOIS)

Claims (2)

(57) [Claims] 1. A coincidence detecting means for detecting a coincidence of a pattern between a plurality of neural networks and an output from each of the neural networks when the same pattern is input to each of the neural networks. And reliability determining means for determining the reliability of the output of each of the neural networks based on the degree of coincidence detected by the degree of coincidence detecting means. A neural network device comprising input information expression conversion means for converting an input pattern to a network in a different conversion mode for each of these neural networks. 2. A plurality of neural networks, and a coincidence detecting means for detecting a degree of coincidence of a pattern between outputs from the neural networks when the same pattern is input to each of the neural networks. And reliability determining means for determining the reliability of the output of each of the neural networks based on the degree of coincidence detected by the degree of coincidence detecting means. A neural network device comprising output information expression conversion means for converting a pattern of an output from the network in a different conversion mode for each of the neural networks.