JP2877413B2 - Surplus neuron decision processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention determines an inactive surplus neuron in a network-structured data processing device that executes data processing according to a network structure formed by internal connections of neurons. In the surplus neuron determination processing method.

In a conventional sequential processing computer (Neumann-type computer), the data processing function cannot be adjusted according to a change in the method of use or the environment. An adaptive data processing apparatus of a parallel distributed processing method using a data conversion function has been proposed. In this data processing device having a network configuration, an output signal (output pattern) from a network structure output in response to the presentation of an input signal (input pattern) prepared for learning without creating an explicit program. The weight value of the internal connection of the network structure is determined according to a predetermined learning algorithm so as to match the teacher signal (teacher pattern). When the weight value is determined by the learning process, a "soft" data processing function of outputting an appropriate output signal from the network structure even if an unexpected input signal is input is provided. Will be realized.

In the network configuration data processing device having such a structure, the arithmetic processing speed of the network structure can be increased,
In order to reduce the amount of hardware in the network structure, it is necessary to provide a means for identifying the extra inactive neurons in the network structure.

[Conventional technology]

The prior art will be described according to a network configuration data processing device having a hierarchical network configuration.

In a data processing device having a hierarchical network configuration, a hierarchical network is configured from a kind of node called a basic unit (which forms an aspect of a neuron) and an internal connection having a weight value corresponding to an internal state value. FIG. 12 shows the basic configuration of the basic unit 1. This basic unit 1
Is a multi-input, one-output system, a multiplication unit 2 for multiplying a plurality of inputs by respective weight values of the internal connection, an accumulation unit 3 for adding all the multiplication results thereof, A threshold processing unit 4 that performs a non-linear threshold process on the calculated value and outputs one final output.

Assuming that the h-th layer is the first layer and the i-th layer is the second layer, the accumulation processing unit 3 executes the operation of the following expression (1), and the threshold processing unit 4 executes the operation of the following expression (2).

Here, h: unit number of the h layer i: unit number of the i layer p: input signal pattern number θ _i : threshold value of the i unit of the i layer W _ih : weight value of the internal coupling between the _{hi and} i layers x _pi : y _ph : The output from the h-th unit of the h-th layer with respect to the input signal of the p-th pattern y _pi : The output of the i-th layer with respect to the input signal of the p-th pattern Output from the i-th unit Then, in the hierarchical network configuration data processing device,
A large number of basic units 1 having such a configuration are hierarchically connected as shown in FIG. 13 by using an input unit 1 'which distributes and outputs an input signal value as it is as an input layer, thereby forming a hierarchical network unit. When configured, a parallel data processing function of converting an input pattern into a corresponding output pattern is exhibited.

In a data processing device having a network configuration, it is necessary to obtain a weight value of a network structure that defines a data conversion function by a learning process. As a learning processing method of the weight value of the hierarchical network configuration data processing device, a learning processing method called a back propagation method has been particularly noted because of its high practicality.

In the back propagation method, learning is performed by adaptively and automatically adjusting the weight value W _ih and the threshold value θ _i of the hierarchical network by error feedback. (1)
As is apparent from the equation (2), the adjustment of the weight value W _ih and the threshold θ _i needs to be performed at the same time.
It is a difficult task to interfere with each other. Therefore, the applicant of the present application has filed a Japanese Patent Application No. 62-333484 (December 28, 1987)
As disclosed in Japanese Patent Application, “Network Configuration Data Processing Apparatus”), by providing a unit that always outputs “1” to the h layer on the input side and assigns a threshold value θ _i to the output as a weight value. It was suggested to treat the threshold theta _i as the weighting value incorporates threshold theta _i in the weight value W _ih. By doing so, the above (1)
Equation (2) is Will be represented by

Next, a description will be given of a weight value learning processing method by the back propagation method according to the description format of the expressions (3) and (4). Here, the description will be made on the assumption that the hierarchical network unit has a hierarchical network having a three-layer structure of the h layer-i layer-j layer shown in FIG.

By analogy with equations (3) and (4), the following equations (5) and (6)
An expression is obtained. That is, Where j: the unit number of the j-th layer W _ji : the weight value of the internal connection between the ij-layers x _pj : the product sum of the inputs from each unit of the i-th layer to the j-th unit of the j-th layer y _pj : the p-th pattern Output from the j-th unit of the j-th layer with respect to the input signal In the back propagation method, an output pattern y _pj from the output layer that is output when an input pattern for learning is presented, and an output pattern y _pi When the teacher pattern d _pj (teacher signal for the p-th pattern input signal to the j-th layer j-th unit) to be taken is determined, first, the difference value between the output pattern y _pj and the teacher pattern d _pj [ d _pj −y _pj ], then α _pj = y _pj (1−y _pj ) (d _pj −y _pj ), and then: Where ε: learning constant ζ: momentum t: learning frequency Δw _ij (t)
Is calculated. Here, for adding those pertaining to the update amount of the weight values determined in the previous update cycle of "ζΔW _ji (t-1)" is order to speed up the learning.

Subsequently, using the calculated α _pj , first, , Then , The update amount ΔW _ih (t) of the weight value between the h layer and the i layer
Is calculated.

Subsequently, the weight value W _ji for according to the update amount calculated as described above in the next update cycle _{(t) = W ji (t} -1) + ΔW ji (t) W ih (t) = W ih (t-1) + ΔW By repeating the method of determining _ih (t), an output pattern y _pj from the output layer that is output when an input pattern for learning is presented, and a signal to be taken by the output pattern y _pj Processing is performed to learn weight values W _ji , W _ih (including threshold values θ _i , θ _j ) at which a certain teacher pattern d _pj matches.

Then, the hierarchical network unit is composed of g layer-h layer-i layer-j
If you have a 4-layer hierarchical network called layers,
First of all, , Then The update amount ΔW _hg (t) of the weight value between the g layer and the h layer is calculated according to the following formula. By determining while using, all weight values and threshold values are processed to be learned.

[Problems to be solved by the invention]

However, conventionally, when the weight value of the internal connection is learned, processing is performed so as to directly enter the data processing using the weight value. Also, in the case where a method of performing weight value learning processing by a large computer and porting the obtained weight value to real hardware having a network structure is used,
The weight value obtained by the learning is transplanted as it is and the data processing is performed.

From this, in the related art, even if there is a neuron that does not work effectively for data processing, the data processing is executed without considering the fact at all. The presence of such extra neurons not only increases the amount of hardware but also increases the operation speed, and does not bring any advantage.

The present invention has been made in view of such circumstances,
By providing a means for identifying which extra neurons are not activated in the network structure of the network configuration data processing device, the calculation processing speed of the network configuration data processing device is increased and the amount of hardware is reduced. It is intended to provide a way to realize the above.

[Means for solving the problem]

 FIG. 1 is a diagram illustrating the principle of the present invention.

In the figure, reference numeral 10 denotes a network configuration data processing device that performs adaptive data processing according to the network configuration, and 11 denotes a network structure provided in the network configuration data processing device 10 and includes one or more input devices. And an internal state value to be multiplied with respect to the input to obtain a sum-of-products value. The product-sum value is converted by a predetermined function to obtain an output value. A device for calculating and outputting an output pattern corresponding to the pattern, 12 is an internal state value storage means provided in the network configuration data processing device 10, and an internal state value required by the network structure unit 11 at the time of executing data conversion processing. 20 is an internal state value learning device, which obtains an internal state value stored in the internal state value storage means 12 by a learning process. It is.

Reference numeral 30 denotes a surplus neuron determination device, which includes a network structure information management unit 31, an internal state value reading unit 32,
Surplus neuron identification means 33 and neuron group identification means
34, the same function neuron specifying means 35, the second surplus neuron specifying means 36, the output range specifying means 37, the third surplus neuron specifying means 38, and the surplus neuron output means
39.

The network structure information management means 31 manages the network structure information of the network structure part 11, and the internal state value reading means 32 processes to read the internal state value stored in the internal state value storage means 12, The surplus neuron identification means 33 is configured to determine whether one of the distance from the zero value of the learning value of the internal state value assigned to the input and the distance from the zero value of the learning value of the internal state value assigned to the output is zero. A neuron having a value close to the value is specified, and the specified neuron is processed to be determined as a surplus neuron. The neuron group specifying means 34 searches the network structure information management means 31 to The input / output destinations of the neurons are processed so as to specify a group of neurons having the same form. Out of the neuron groups specified by the neuron group specifying means 34, the neuron groups in which the learning values of the corresponding internal state values assigned to the inputs and outputs have substantially the same value are specified. Performs processing to determine a neuron other than one of the neuron groups specified by the same function neuron specifying means 35 as a surplus neuron,
Performs processing to specify the maximum value and the minimum value of the output value output by each neuron in response to the presentation of the input pattern group to be learned, and the third surplus neuron specifying means 38 A neuron whose difference value between the specified maximum value and the minimum value is close to zero is specified, and the specified neuron is processed as a surplus neuron. , The second and third surplus neuron identification means 33, 36, and 38 output the determined surplus neuron identifier.

[Action]

In the present invention, the internal state value learning device 20 presents the input pattern group to be learned to the network structure unit 11 and outputs the network structure unit 11 according to the presentation.
, And an error value between the output pattern group and the teacher pattern group indicating the pattern to be taken by the output pattern group, and the internal state value of the internal state value storage means 12 is sequentially updated based on the error value. By doing so, the internal state value of the network structure unit 11 for experimenting with the desired data conversion processing is learned and stored in the internal state value storage unit 12.

In this way, the internal state value of the network structure unit 11 that realizes the desired data conversion processing is stored in the internal state value storage unit.
When stored in the storage unit 12, the internal state value reading unit 32 sequentially refers to the management information of the network structure information management unit 31,
One neuron is selected, and all internal state values assigned to the internal connection on the input side of the selected neuron are read out from the internal state value storage means 12, and all internal state values assigned to the internal connection on the output side are stored in the internal state. Processing to read from the value storage means 12 is performed. Then, when the distance from the zero value of the read internal state value of the internal connection on the input side has a value close to zero value, the first surplus neuron specifying means 33 determines that the neuron which does not respond to the input is When it is determined that the neuron is an active surplus neuron, and when the distance from the zero value of the internal state value of the read internal connection on the output side has a value close to zero value, the neuron does not output any inactive. The surplus neuron output means 39 outputs an identifier of the determined surplus neuron.

Further, when the internal state value of the network structure unit 11 that realizes the desired data conversion processing is stored in the internal state value storage unit 12, the neuron group identification unit 34 refers to the management information of the network structure information management unit 31. This specifies a group of neurons whose input / output destinations in the network structure unit 11 take the same form. Subsequently, the internal state value reading means 32 reads from the internal state value storing means 12 all the internal state values assigned to the internal connection of the input and output of each neuron of the specified neuron group. Then, in accordance with the read internal state value, the same-function neuron identification means 35 outputs a neuron whose internal state value assigned to the input / output of the neuron group whose input / output destination takes the same form has substantially the same value. A second surplus neuron identifying means for identifying a group
36 determines that the group of neurons having substantially the same value can be originally replaced by one neuron, and the other neurons become inactive surplus neurons. , And outputs the determined surplus neuron identifier.

When the internal state value of the network structure unit 11 that realizes the desired data conversion processing is stored in the internal state value storage unit 12, the output range specifying unit 37 refers to the management information of the network structure information management unit 31. While selecting one neuron in sequence, and referring to the network output data of the network structure unit 11 that is output in response to the presentation of the input pattern group to be learned, the selected neuron presents the input pattern group to be learned. The maximum value and the minimum value of the output value to be output with respect to are specified. When the difference between the specified maximum value and the minimum value has a value close to zero, the third surplus neuron specifying means 38 determines that the neuron is an inactive surplus neuron that is not performing an effective operation. The surplus neuron output means 39 outputs the determined surplus neuron identifier.

As described above, according to the present invention, it is possible to specify whether or not extra inactive neurons in the network structure unit 11 of the network configuration data processing device 10 cannot be removed.

〔Example〕

Hereinafter, the present invention will be described in detail according to an embodiment applied to a hierarchical network configuration data processing device.

FIG. 2 shows an embodiment of the present invention. In the figure, the first
The same components as those described in FIG. 12, FIG. 12, and FIG. 13 are denoted by the same reference numerals. 10a is a hierarchical network configuration data processing device for executing adaptive data processing according to the hierarchical network configuration, 11a is a hierarchical network structure unit having the structure described in FIG. 13, and 12a is an internal state value storage means of FIG. Weight value storage means equivalent to 12, 20
a is a weight value learning device for learning the weight value of the hierarchical network structure unit 11a according to the back propagation method, and 30a is a surplus neuron determination device corresponding to the surplus neuron determination device 30 in FIG.

The weight value learning device 20a includes a learning signal storage unit 21, a learning signal presenting unit 22, a learning convergence determining unit 23, and a weight value learning unit 24.
And The learning signal storage means 21 stores a learning signal used for learning the weight value, and the learning signal presentation means 22 reads the learning signal from the learning signal storage means 21 and stores the learning presentation signal in the hierarchy. Network Structure Department 11
a, and processing to present the other pair of learning teacher signals to the learning convergence determining means 23 and the weight value learning means 24, and the learning convergence determining means 23 is output from the hierarchical network structure unit 11a. In response to the output signal and the learning teacher signal from the learning signal presenting means 22, it is determined whether or not an error in the data processing function of the hierarchical network structure unit 11a has entered an allowable range, and the determination result is represented by the learning signal presenting means 22. Weight value learning means 24 receives the learning teacher signal from the learning signal presentation means 22 and the network output data of the hierarchical network structure unit 11a, and
Calculate the update amount of the weight value according to the propagation method,
By updating the weight value in accordance with the update amount, processing is performed to learn to converge the weight value.

Then, the surplus unit determination device 30a characteristic of the present invention
When the weight value learning device 20a completes the learning of the weight value, the first redundant unit specifying means 301 which is constituted by the program means and processes to specify the inactive basic unit 1 in the hierarchical network structure section 11a. , A second surplus unit specifying unit 302, a third surplus unit specifying unit 303, and a fourth surplus unit specifying unit 304, and a judgment value γ and a distance function f () required by these surplus unit specifying units 301, 302, 303, 304. α _i ), a judgment data storage means 305 for managing these extra unit
And a result output means 306 for outputting the processing results of 1,302,303,304 to an external device. In FIG. 3, the first
4 is a flowchart executed by the surplus unit specifying means 301, FIG. 4 is a flowchart executed by the second surplus unit specifying means 302, FIG. 5 is a flowchart executed by the third surplus unit specifying means 303, and FIG. And the fourth
3 is a flowchart executed by the surplus unit identification means 304.

Next, in accordance with the flowcharts shown in FIGS. 3 to 6, the process of specifying the surplus basic unit 1 which is to be executed on condition that the learning of the weight value is completed by the weight value learning device 20a will be described in detail. Will be described.

As shown in the flowchart of FIG. 3, the first surplus unit identification means 301 selects one basic unit 1 for which the determination of active or inactive has not been made in step 30, and
In step 31, the learning value α _i (1) of the weight value assigned to all the inner connections output from the basic unit 1
≤ i ≤ n) is read from the weight value storage means 12a. Subsequently, in step 32, the judgment value γ assigned to the basic unit 1 and the distance function f (α _i ) (basic unit 1
Can be set every time or in units of the number of levels of the hierarchical structure) from the determination data storage means 305. Subsequently, in step 33, the weight value (n
The distance function f (α _i )
, The distance K from the origin of the read weight value is calculated. Here, the distance function f (α _i ) can be variously defined. And f (α _i ) = MAX | α _i | where 1 ≦ i ≦ n or Is calculated according to the following function formula.

In this way, when the distance K from the origin in the n-dimensional weight space of the weight value assigned to the internal connection on the output side of the selected basic unit 1 is obtained, the following step 34
Then, it is determined whether or not the distance K is smaller than the determination value γ read in step 32. When it is determined that the basic unit 1 is smaller than the determination value γ, the process proceeds to step 35 to determine that the selected basic unit 1 is a surplus unit. The determined basic unit 1 is determined not to be a surplus unit, and the result of the determination is notified to the result output means 306 in the next step 37. Then, when it is detected in step 38 that all the basic units 1 have not been determined to be active or inactive, the processing returns to step 30 so that all the basic units 1 of the hierarchical network structure unit 11a are processed. A determination is made as to whether unit 1 is active or inactive.

As described above, when the learning values of all the internal state values of the internal connection on the output side of the basic unit 1 have a value close to zero, the first surplus unit specifying unit 301 does not output the basic unit 1. Processing is performed to determine that the unit is an inactive surplus unit.

On the other hand, the second surplus unit identification means 302
By executing the flowchart of FIG. 4, when the learning values of all the internal state values of the internal connection on the input side of the basic unit 1 have a value close to zero, the basic unit 1 does not accept the input and is inactive. Processing is performed to determine that the unit is a surplus unit. The difference between the flowchart of FIG. 4 and the flowchart of FIG. 3 is that, in step 31 of the flowchart of FIG.
The learning values α _i of the weight values assigned to all the internal connections “output” from “ _i” are read, while in step 41 of the flowchart of FIG. At the point of reading the learning value α _i of the weight value assigned to the internal connection of.
The learning value of the threshold value of the basic unit 1 does not need to be included in the learning value of the weight value to be read.

As shown in the flowchart of FIG. 5, the third surplus unit specifying means 303 determines the group of the basic units 1 having the same weight space, such as the group of the basic units 1 of the second layer in the middle layer in step 50. One group is selected from (a group that is internally connected to the same basic unit group). Next, in step 51, the judgment value γ assigned to the group and the distance function f (α _i ,
α ′ _i ) (which can be set for each group) is read from the judgment data storage means 305. Subsequently, in step 52, one basic unit 1 is selected as a reference unit from the basic units 1 of the selected group, and in the following step 53, the selected reference unit is determined as an activated non-surplus unit. .

Next, in step 54, the undetermined and unselected basic unit 1
One basic unit 1 is selected from, and the following step 55
The distance at which the weight value α _{i of the} reference unit read from the weight value storage means 12a (which forms an n-dimensional space) and the weight value α ′ _{i of} this selected basic unit 1 which forms an n-dimensional space are read. By substituting into the function f (α _i , α ′ _i ), the distance K between the reference unit and the basic unit 1 selected in step 54 is calculated. Here, the threshold value is preferably included in the weight value, but may not be included because the degree of contribution is small. Also,
The distance function f (α _i , α ′ _i ) can be variously defined, for example, And f (α _i , α ′ _i ) = MAX | α _i , α ′ _i | where 1 ≦ i ≦ n or Is calculated according to the following function formula.

When the distance K between the reference unit and the basic unit 1 selected in step 54 is determined, it is determined in subsequent step 56 whether or not the distance K is smaller than the determination value γ read in step 51. When it is determined that the value is smaller than the determination value γ, the process proceeds to step 57, in which the selected basic unit 1 performs the same function as the reference unit (a function that can be omitted by increasing the weight value of the reference unit). Therefore, when it is determined that the unit is an inactive surplus unit, and when it is determined that the unit is larger than the determination value γ, nothing is determined and is left undetermined. Then, in a succeeding step 58, it is determined whether or not all of the undecided basic units 1 have been selected, and if there is any unselected basic unit 1, processing is returned to step 54.

When it is determined in step 58 that all of the basic units 1 have been selected, it is determined in subsequent step 59 whether or not undetermined basic units 1 are left in the basic units 1 of the group selected in step 50. If so, the process returns to step 52 to select a reference unit from the remaining ones, and a surplus basic unit that performs the same function as the selected reference unit among the remaining ones. Processing is performed to check whether or not 1 exists.
If it is determined in step 59 that no undetermined basic unit 1 is left, the process proceeds to step 60 to determine whether all groups of the basic unit 1 having the same weight space have been processed. When it is detected that the processing has not been performed, the process returns to step 50 to determine whether all the basic units 1 of the hierarchical network structure unit 11a are active or inactive. The result of the determination is notified to the result output means 306.

As described above, the third surplus unit specifying unit 303 groups the units that perform the same function as the basic unit 1 of the hierarchical network structure unit 11a, and only one of the grouped basic units 1 Is determined not to be a surplus unit, and the remaining basic unit 1 is determined to be a surplus unit. This third surplus unit specifying means 30
The basic unit 1 determined to be surplus according to 3 can be omitted by setting the weight value of the representatively selected non-surplus basic unit 1 large accordingly.

As shown in the flowchart of FIG. 6, the fourth surplus unit specifying means 304 selects one basic unit 1 which has not been determined to be active or inactive in step 70,
In step 71, the judgment value γ assigned to the basic unit 1 (which can be set for each basic unit 1 or in units of the number of levels of the hierarchical structure) is read from the judgment data storage unit 305. Subsequently, in step 72, the learning signal presenting unit 22 is instructed to present the learning presentation signal to the hierarchical network structure unit 11a, and the output value from the selected basic unit 1 outputted in response to the instruction is given. Is processed so as to obtain the maximum value and the minimum value of. Then, in a succeeding step 73, it is determined whether or not the difference value between the obtained maximum value and the minimum value is smaller than the judgment value γ read in the step 71. When it is determined that the selected basic unit 1 is a surplus unit, and when it is determined that the basic unit 1 is larger than the determination value γ, the process proceeds to step 75, where it is determined that the selected basic unit 1 is not a surplus unit. 76
Then, the result of the determination is notified to the result output means 306. When it is detected in step 77 that all the basic units 1 have not been determined to be active or inactive, the processing is returned to step 70, so that all of the hierarchical network structure units 11a are processed. A determination is made as to whether the basic unit 1 is active or inactive.

As described above, the fourth surplus unit specifying unit 304 determines that the basic unit 1 that hardly changes the output value with respect to the input pattern input to the hierarchical network structure unit 11a has the inactive surplus that does not contribute to the data processing. Processing is performed so as to be determined as a unit.

Next, the surplus unit identification processing will be specifically described according to a simulation. In this simulation, as shown in FIG. 7, the hierarchical network structure section 11a has a hierarchical network structure of two input layers, nine units, a one-stage intermediate layer, and one unit output layer. The device 20a uses the 20 learning signals shown in FIG. 8 and sets the learning constant ε to 5.0 and the momentum ζ
Set to 0.4, initialize the weight value and threshold with random numbers from -0.5 to 0.5, and present the learning signal 5490 times (at this time, the difference between the learning teacher signal and the output signal is 0.025 or less) Was.

FIG. 9 (a) illustrates the learning value of the weight value obtained by the simulation, and FIG. 9 (b) illustrates the learning value of the threshold value obtained by the simulation. here,
In FIG. 9A, “0.548750” indicates “i0 unit” and “h0
9B shows the weight value assigned to the internal connection between the “unit” and “0.049658” in FIG. 9B.
0 unit ”.

When such a weight value is obtained, the first surplus unit identifying means 301 determines that the weight value assigned to the inner connection on the output side of the “i4 unit” in the intermediate layer is “−0.014513”, which is close to zero. This basic unit 1 is specified as a surplus unit. Since the weight value assigned to the internal connection on the output side of the basic unit 1 of the other intermediate layer is relatively large, it is specified that the unit is a non-surplus activated unit.

FIG. 10 shows a plot of weight values (forming a two-dimensional weight space) assigned to the internal connection on the input side of the basic unit 1 of the intermediate layer in FIG. 9 (a).

When such a weight space is obtained, the second surplus unit specifying means 302 determines that the distance from the origin of the weight value assigned to the internal connection of the input side of “i4 unit” and “i8 unit” in the intermediate layer is small. Therefore, these basic units 1 are specified as surplus units. Then, since the distance from the origin of the weight value assigned to the internal connection on the input side of the basic unit 1 of the other intermediate layer is relatively large, the unit is specified as a non-surplus activated unit.

Further, the third surplus unit identification means 303
As can be seen from the figure, when the threshold is not taken into account, the “i2 unit” and “i6 unit” of the intermediate layer perform substantially the same function, and the “i4 unit” and “i8 unit”
Perform the same function as the i6 unit.
And “i8 unit” are specified as surplus units.

FIG. 11 illustrates the output values of the basic units 1 of the intermediate layer, which are output for each of the 20 learning signals when the learned weight value and the threshold value are set. As can be seen from FIG. 11, since the “i2 unit”, “i4 unit”, “i6 unit”, and “i8 unit” of the intermediate layer do not change the output value, the fourth surplus unit specifying means 304 Specifies these basic units 1 as surplus units.

Although the illustrated embodiment has been described above, the present invention is not limited to this. For example, the present invention is not limited to a hierarchical network configuration data processing device, but can be applied to other network configuration data processing devices as it is. Further, the neurons to which the present invention is applied are not limited to the basic units that perform the threshold value conversion processing as described in the embodiment, but may perform other function conversion processing.

〔The invention's effect〕

As described above, according to the present invention, in the network configuration data processing device, it is possible to specify which extra neurons are not activated in the network structure, and therefore, the hardware amount of the network structure is reduced. Need not be prepared unnecessarily large.
As a result, the arithmetic processing speed of the network configuration data processing device can be increased, and the amount of hardware can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram of the principle of the present invention, FIG. 2 is an embodiment of the present invention, FIG. 3 is a flowchart executed by the first surplus unit specifying means, and FIG. FIG. 5 is a flowchart executed by a third surplus unit specifying unit, FIG. 6 is a flowchart executed by a fourth surplus unit specifying unit, FIG. 7 is a description of a hierarchical network structure used in the simulation. Fig. 8, Fig. 8 is an explanatory diagram of a learning signal used in the simulation, Figs. 9, 10, and 11 are explanatory diagrams of simulation results, Fig. 12 is a basic configuration diagram of a basic unit, and Fig. 13 is a hierarchy. It is a basic block diagram of a network part. In the figure, 1 is a basic unit, 10 is a network configuration data processing device, 10a is a hierarchical network configuration data processing device, 11 is a network structure unit, 11a is a hierarchical network structure unit, 12 is an internal state value storage unit, and 12a is a weight value. Storage means, 20 is an internal state value learning device, 20a is a weight value learning device, 3
0 is a surplus neuron determination device, and 30a is a surplus unit determination device.

Continuing on the front page (72) Inventor Yuri Owada 1015 Uedanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture Inside Fujitsu Co., Ltd. Documents JP-A-2-231670 (JP, A) JP-A-3-83157 (JP, A) JP-A-3-83156 (JP, A) Masafumi Hagiwara, Masao Nakagawa, "Backpropagation with selection function- Evasion Algorithm from Local Minimum ", IEICE Spring National Convention (1989), P. 7-19 (58) Field surveyed (Int. Cl. ⁶ , DB name) G06F 15/18 G06G 7/00 JICST file (JOIS)

Claims (1)

(57) [Claims] 1. A method for receiving one or more inputs and an internal state value to be multiplied with respect to the input to obtain a product sum value, converting the product sum value by a predetermined function to obtain an output value With a network structure formed by internal connections of the obtained neurons as learning targets, an output pattern group output from the network structure in response to the presentation of an input pattern group to be learned, and a pattern to be taken by the output pattern group This is a surplus neuron determination processing method used in a learning processing system that learns an internal state value by calculating an error value between the instruction pattern group and a teacher pattern group that indicates the internal state value based on the error value. Output range specifying means for specifying the maximum value and the minimum value of the output value output by each neuron in response to the presentation of the input pattern group to be learned according to the learned internal state value And a surplus neuron specifying means for specifying a surplus neuron by specifying a neuron whose difference value between the maximum value and the minimum value specified by the output range specifying means is close to zero. Characteristic surplus neuron decision processing method.