JP7547956B2 - Method and program for determining edges to be corrected - Google Patents

Description

The present invention relates to a method for determining an edge to be corrected and a program for determining an edge to be corrected.

In a system that uses a machine learning model such as a Deep Neural Network (DNN), if an output that is undesirable to the user occurs, the machine learning model may be modified. Hereinafter, a DNN machine learning model may be referred to as a DNN model.

For example, in an autonomous driving system using a camera, if a stop road sign is recognized as a speed limit, there is a risk of an accident, so the DNN model needs to be modified to ensure correct recognition.
A DNN model is not created according to specifications, but according to input training data, so its modifications can also be done by inputting training data.

Japanese Patent Application Publication No. 9-128358

However, in such conventional methods of correcting DNN models using training data, it is necessary to collect the data necessary for correction, and the collected data does not necessarily allow errors to be corrected.

In other words, retraining can lead to incorrect inferences after retraining for data that was correctly inferred before retraining, resulting in degradation. Degradation occurs due to the principle of CACE (Changing Anything Changes Everything).

In one aspect, the present invention aims to correct only the weights of edge groups that have led to regression, while leaving the weights of edge groups that have led to improvement intact.

For this reason, the method for determining edges to be modified includes, when an information processing device determines edges to be modified among the edges included in a neural network model, a process of extracting, as a modified edge group, edges included in a second-generation trained machine learning model that have weights different from those of the edges included in a first-generation trained machine learning model prior to the second generation; a process of classifying the modified edge group into a first edge classification that influenced the output of regressed test data, which is test data in which regression has occurred, and a second edge classification that influenced the output of improved test data, which is test data in which improvement has occurred; and a process of identifying a third edge group obtained by subtracting the second edge group included in the second edge classification from the first edge group included in the first edge classification as a correction target.

According to one embodiment, the weights of the edge groups that led to improvement can be left as they are, while only the weights of the edge groups that led to regression can be modified.

FIG. 1 is a diagram illustrating a configuration of an information processing apparatus as an example of an embodiment. 1 is a diagram for explaining a process of a model training unit in an information processing device as an example of an embodiment; 1 is a diagram illustrating improvement test data and regression test data in an information processing device as an example of an embodiment; 10A and 10B are diagrams for explaining a process performed by a weight difference measurement unit in an information processing device as an example of an embodiment; 10 is a diagram illustrating a process performed by a weighting effect measuring unit in an information processing device as an example of an embodiment; 10 is a diagram illustrating a process performed by a weighting effect measuring unit in an information processing device as an example of an embodiment; 10 is a diagram for explaining a process performed by a selection weight number determination unit in an information processing device as an example of an embodiment; FIG. 11 is a diagram for explaining a process of a correction target weight specification unit in an information processing device as an example of an embodiment; FIG. 11 is a diagram for explaining a process of a correction target weight specification unit in an information processing device as an example of an embodiment; FIG. 10 is a diagram illustrating a method of selecting a correction target weight by a correction target weight specifying unit in an information processing device as an example of an embodiment; FIG. 1 is a flowchart illustrating a process in an information processing device as an example of an embodiment. 1 is a flowchart illustrating a process in an information processing device as an example of an embodiment. 11A and 11B are diagrams illustrating simulation results in an information processing device as an example of an embodiment; FIG. 1 is a diagram illustrating a hardware configuration of an information processing apparatus according to an embodiment;

Below, an embodiment of the present correction target edge determination method and correction target edge determination program will be described with reference to the drawings. However, the embodiments shown below are merely examples, and are not intended to exclude the application of various modified examples and techniques not explicitly stated in the embodiments. In other words, this embodiment can be implemented with various modifications without departing from its spirit. Furthermore, each figure is not intended to include only the components shown in the figure, but can include other functions, etc.

(A) Configuration FIG. 1 is a diagram illustrating a schematic configuration of an information processing device 1 as an example of an embodiment.

The information processing device 1 realizes a model correction function for correcting a trained DNN model (machine learning model). The trained DNN model is a trained machine learning model. As shown in FIG. 1, the information processing device 1 includes a model training unit 101, a test data classification unit 102, a weight difference measurement unit 103, a weight influence measurement unit 104, a selection weight number determination unit 105, and a correction target weight identification unit 106.
The model training unit 101 performs training (retraining) by inputting training data to the DNN model to be trained.
FIG. 2 is a diagram for explaining the processing of the model training unit 101 in the information processing device 1 as an example of an embodiment.
Training data and a pre-training DNN model are input to the model training unit 101. The training data includes, for example, input data and training labels (correct answer data).

A DNN inputs input data such as images and audio into an input layer, and sequentially executes predetermined calculations in hidden layers composed of convolutional layers and pooling layers, thereby executing forward processing (forward propagation processing) that sequentially transmits information obtained by the calculations from the input side to the output side. In each layer, nodes are connected by edges. Furthermore, edges have weights.

After performing forward processing, the model training unit 101 performs backward processing (backpropagation processing) to determine parameters to be used in the forward processing in order to reduce the value of the error function obtained from the output data output from the output layer and the correct answer data. Then, an update process is performed to update weights (variables) based on the results of the backpropagation processing. For example, gradient descent may be used as an algorithm to determine the update width of the weights used in the calculation of the backpropagation processing. Hereinafter, the DNN model may be simply referred to as a model.

The model training unit 101 stores information about the DNN model before training (pre-training model) and the DNN model after training (post-training model) in a storage area such as the storage device 13 (see FIG. 14). The model training unit 101 trains the same DNN model multiple times (n times), and thus n generations of DNN model information is stored in the storage device 13 or the like. The stored DNN model information also includes weights. Therefore, the model training unit 101 stores the weights of the model k generations ago during training in the storage device 13 or the like.

Hereinafter, the latest generation DNN model among multiple generations of DNN models may be referred to as the current generation model. Also, a DNN model of a generation (old generation) prior to the current generation model may be referred to as the previous generation model. The current generation model may be referred to as the nth generation model and may be represented by the symbol _Mn . Also, a previous generation model that is k generations prior to the nth generation model may be referred to as the nkth generation model and may be represented by the symbol _Mnk . Note that n is a natural number of 2 or more, and k is a natural number of 1 or more. Also, k<n.

The latest generation (current generation, nth generation) corresponds to the second generation, and the nth generation DNN model corresponds to the second generation trained machine learning model. Also, the previous generation (n-kth generation) corresponds to the first generation, and the n-kth generation DNN model corresponds to the first generation trained machine learning model.

The information processing device 1 also realizes a correction target edge determination function that determines edges to be corrected from among the edges included in a DNN (neural network model).
The test data classifier 102 collects improvement test data and regression test data.
FIG. 3 is a diagram for explaining improvement test data and regression test data in the information processing device 1 as an example of an embodiment.

Figure 3 shows an example in which the labels "cat" and "dog" are input as input data to the n-kth generation DNN model and the nth generation DNN model, respectively.

In the example shown in FIG. 3, when the label "dog" was input to the n-th generation DNN model, the incorrect estimation result "cat" was obtained (estimation failure), but when it was input to the n-th generation DNN model, the correct estimation result "dog" was obtained (estimation success). In other words, this indicates that the DNN model was correctly corrected (see symbol P1). Test data that is now correctly estimated by correcting the DNN model in this way is sometimes called improved test data. The test data "dog" is improved test data.
The test data classification unit 102 evaluates the test data with the DNN model M _n and collects a set of successful test data X _pos,n

The test data classification unit 102 generates a common set Xneg _,nk∩Xpos,n between a set Xneg,nk of test data that failed in the previous generation (here, the nk generation) DNN model _Mnk and the above-mentioned Xpos _,n . Hereinafter, this common set _{Xneg, nk∩Xpos ,n may} be referred to as improved test data _Ximp .

On the other hand, in the example shown in FIG. 3, when the label "cat" was input to the n-th generation DNN model, the correct estimated result "cat" was obtained (estimation successful), but when it was input to the n-th generation DNN model, the incorrect estimated result "tiger" was obtained (estimation failed). In other words, it is considered that degradation (performance reduction) occurred due to the correction of the DNN model (see symbol P2). Note that degradation is sometimes called "degradation." Test data in which degradation has occurred due to such correction of the DNN model is sometimes called regressed test data. The test data "cat" is regressed test data.
The test data classification unit 102 evaluates the test data with the n-th generation DNN model M _n and collects a set X _neg,n of failing test data.

The test data classification unit 102 generates a common set X _pos,nk ∩X _{neg,n between a set X pos,nk of test data that has been successful in the previous generation (here, the nk generation) of the DNN model M nk and the above-mentioned X neg,n . Hereinafter} , this common set X _pos,nk ∩X _{neg ,n} may be referred to as regression test data X _deg .

In this information processing device 1, DNN model correction is performed based on the first hypothesis that data for which inference was successful using the previous generation DNN model is easy to correct, and the second hypothesis that the difference in weights before and after correction affects degradation and correction.

The test data classification unit 102 inputs the same test data to both the n-kth generation model and the nth generation model. Then, if estimation is successful in the n-kth generation model and fails in the nth generation model, the test data is determined (classified) as regressed test data.

On the other hand, if the estimation fails in the nk-th generation model and succeeds in the n-th generation model, the test data classification unit 102 determines (classifies) the test data as improved test data.
The test data classification unit 102 stores the classified improvement test data and regression test data in a predetermined storage area such as the storage device 13 .

The weight difference measurement unit 103 calculates the weight difference between the weight of the current generation model and the weight of the previous generation model. The weight difference is the absolute value of the value obtained by subtracting the weight of the previous generation model from the weight of the current generation model.

The weight difference measurement unit 103 calculates the absolute value of the difference between the set of weights in the nth generation DNN model and the set of weights in the nkth generation DNN model in layer l, and generates a list of weights arranged in descending order of absolute value.
FIG. 4 is a diagram for explaining a process performed by the weight difference measurement unit 103 in the information processing device 1 as one example of an embodiment.

In Figure 4, the symbol A indicates the information of the previous generation (n-kth generation) DNN model, and the symbol B indicates the information of the current generation (nth generation) DNN model. In the example shown in Figure 4, the information of the DNN model includes layers, types, weights, and the number of weights.

Layer is identification information that identifies a layer, and natural numbers are used in the example shown in Figure 4. Type is the type of layer, and Conv2D and Dense are used in the example shown in Figure 4. Weight is the weight value used in each layer, and is flattened to one dimension in the example shown in Figure 4. The number of weights is the number of weights provided in each layer.

In addition, in each layer, an index is set for each weight, and for example, a natural number sequence such as 0, 1, 2, 3, ... is set as the index. In the example shown in Figure 4, for example, the weights [0.053, 0.124, ..., 0.122] of layer 1 are set to indexes [0, 1, ..., 895].

That is, the weights for each layer of the n-kth generation DNN model indicated by reference symbol A, and the weights for each layer of the nth generation DNN model indicated by reference symbol B, are arranged in index order.

In addition, in FIG. 4, the symbol C indicates weight difference information. This weight difference information indicated by the symbol C includes a layer, a weight difference, and the number of weights. Note that in the figure, items that are the same as those already described indicate similar content, and therefore their explanation is omitted. In addition, hereinafter, the weight difference information may be denoted by the symbol 201 and referred to as weight difference information 201.

The weight difference is calculated by subtracting the weight of each corresponding index in the same layer in the n-kth generation DNN model, indicated by symbol A, from the weight of each layer in the nth generation DNN model, indicated by symbol B.

For example, the weight difference for index 1 of layer 1 of the n-kth generation DNN model, "0.053", is obtained by subtracting the weight for index 1 of layer 1 of the nth generation DNN model, "0.054", from the weight for index 1 of layer 1 of the nth generation DNN model.

The weight difference information 201 represents a plurality of edges (modified edge group) that have weights different from those of the edges included in the n-th generation DNN model among the edges included in the n-th generation DNN model. Therefore, the weight difference measurement unit 103 executes a process of extracting, as a modified edge group, edges that have weights different from those of the edges included in the n-th generation DNN model among the edges included in the n-th generation DNN model.

4, reference symbol D indicates an index list in which the weight differences in the weight difference information indicated by reference symbol C are sorted in descending order of their values. Hereinafter, the index list in which the weight differences in the weight difference information are sorted in descending order of their values may be referred to as a weight difference ordered index list 210.
The weight difference order index list 210 includes a layer, a weight difference order sorted index, and a number of weights.

The weight difference sorted index sorts the weight differences in the weight difference information indicated by symbol C in descending order of value, and represents the weight difference values in the corresponding weight indexes instead.

As a result, by referring to the weight difference order index list 210, it is possible to easily know, for each layer, the weights that have a large weight change (weight difference) caused by modification of the DNN model.

The weighting influence measurement unit 104 measures the influence of the weighting on the output by differentiating the output of each of the regression test data and the improvement test data by the DNN model with the weighting.
FIG. 5 is a diagram for explaining the process performed by the weighting effect measuring unit 104 in the information processing device 1 as one example of an embodiment.
In the example shown in FIG. 5, regression test data is input to the n-th generation DNN model, and an output (estimation result) is obtained.
In FIG. 5, symbol A illustrates the weights for each layer of the n-th generation DNN model, and symbol B illustrates the output.

The weight influence measurement unit 104 calculates the influence (influence) of each weight value on the output by automatically differentiating the output of the DNN model using the weight. Hereinafter, the influence of the weight calculated based on the regression test data may be referred to as regression influence. The weight influence measurement unit 104 calculates the regression influence by automatically differentiating the output Y _deg when regression test data X _deg is input to the n-th generation model, using the weight W _n . The regression influence in layer l is expressed by the following formula.

図５において符号Ｃは退行影響度情報を示す。図５に例示する退行影響度情報は、レイヤ，出力に対する重みの影響度および重みの数を備える。

5, the reference character C indicates regression influence information. The regression influence information illustrated in FIG. 5 includes a layer, a weight influence on an output, and the number of weights.

The influence of the weight on the output is calculated for each layer in the DNN model, and is calculated for each value of the weight of the same layer. In the example shown in Fig. 5, for example, for the weight of layer 1 [0.053, 0.124, ..., 0.122], the influence of the weight on the output is calculated as [0.001, 0.002, ..., -0.004]. In addition, hereinafter, the regression influence information may be denoted by the symbol 202 and referred to as regression influence information 202.
Furthermore, the weight influence measuring unit 104 generates an index list in which the weights are sorted in descending order of the influence on the output for each layer.
In FIG. 5, the symbol D denotes an index list in which the indexes are sorted in order of the regression influence degree.
Hereinafter, the index list in which the regression influence of the weights is sorted in descending order of value may be referred to as a regression influence ordered index list 220 .

The regression influence order index list 220 indicates an index list sorted in descending order of the influence value of the weight on the output in the regression influence information indicated by the symbol C. This regression influence order index list 220 includes a layer, a regression influence order sorted index (index), and the number of weights.

The regression impact sorted index sorts the influence of the weights on the output of the regression impact information 202, indicated by the symbol C, in descending order of value, and represents the corresponding weight index instead of the influence value.

As a result, by referring to the regression influence order index list 220, it is easy to know the weights that have a high influence on the output in the DNN model for each layer.

図６は実施形態の一例としての情報処理装置１における重み影響測定部１０４による処理を説明するための図である。
図６に示す例において、n世代目のＤＮＮモデルに改善テストデータが入力され、出力（推定結果）が得られる。
図６において、符号Ａはn世代目のＤＮＮモデルのレイヤ毎の重みを例示し、符号Ｂはその出力を例示する。

FIG. 6 is a diagram for explaining the process performed by the weighting effect measuring unit 104 in the information processing device 1 as one example of an embodiment.
In the example shown in FIG. 6, improved test data is input to the n-th generation DNN model, and an output (estimated result) is obtained.
In FIG. 6, symbol A illustrates the weights for each layer of the n-th generation DNN model, and symbol B illustrates the output.

The weight influence measurement unit 104 calculates the influence (degree of influence) that each weight value has on the output by automatically differentiating the output of the DNN model using the weights. Hereinafter, the degree of influence of the weights calculated based on the improvement test data may be referred to as the improvement influence. The improvement influence in layer l is expressed by the following formula.

図６において符号Ｃは改善影響度情報を示す。図６に例示する改善影響度情報は、レイヤ，出力に対する重みの影響度および重みの数を備える。

6, the reference character C denotes improvement influence information. The improvement influence information exemplified in FIG. 6 includes a layer, the influence of weights on outputs, and the number of weights.

The influence of the weight on the output is calculated for each layer in the DNN model, and is calculated for each value of the weight of the same layer. In the example shown in Fig. 6, for example, for the weight of layer 1 [0.053, 0.124, ..., 0.122], the influence of the weight on the output is calculated as [0.001, 0.002, ..., -0.004]. In addition, hereinafter, the improvement influence information may be denoted by the symbol 203 and referred to as improvement influence information 203.
Furthermore, the weight influence measuring unit 104 generates an index list in which the weights are sorted in descending order of the influence on the output for each layer.
In FIG. 6, the symbol D denotes an index list in which the indexes are sorted in order of the degree of influence.
Hereinafter, the index list in which the improvement influence of the weights is sorted in descending order of value may be referred to as an improvement influence index list 230 .

The improvement impact index list 230 is an index list sorted in descending order of the impact value of the weights on the output in the improvement impact information indicated by the symbol C. This improvement impact index list 230 includes a layer, an improvement impact sorted index (index), and the number of weights.

The index sorted by improvement impact is obtained by sorting the weighting influence on the output of the improvement impact information indicated by symbol C in descending order of value, and expressing the weighting influence as an index instead of the influence value.

As a result, by referring to the improvement influence index list 230, it is easy to know the weights that have a high influence on the output in the DNN model for each layer.

選択重み個数決定部１０５は、重み差分情報２０１，退行影響度情報２０２および改善影響度情報２０３に基づいてレイヤ毎に選択重み個数を決定する。
図７は実施形態の一例としての情報処理装置１における選択重み個数決定部１０５による処理を説明するための図である。

The selection weight number determination unit 105 determines the number of selection weights for each layer based on the weight difference information 201, the regression influence information 202, and the improvement influence information 203.
FIG. 7 is a diagram for explaining the process performed by the selection weight number determination unit 105 in the information processing device 1 as one example of an embodiment.

In FIG. 7, the symbol A indicates weight difference information 201. The selection weight number determination unit 105 calculates the average value of the weight difference for each layer based on the weight difference information 201. The average weight difference for each layer is expressed as follows:

Furthermore, the selection weight number determination unit 105 normalizes the average value of the calculated weight differences for each layer so that the sum of all layers is 1. The average of the normalized weight differences can be expressed as follows:

図７においては、正規化した各レイヤの重み差分の平均の例を符号Ｂに示す。
選択重み個数決定部１０５は、算出したレイヤ毎の正規化した重み差分の平均を記憶装置１３等の所定の記憶領域に記憶させる。

In FIG. 7, the symbol B indicates an example of the average of the normalized weight differences of each layer.
The selection weight number determination unit 105 stores the calculated average of the normalized weight differences for each layer in a predetermined storage area such as the storage device 13 .

In addition, in FIG. 7, the symbol C indicates the regression influence information 202. The selection weight number determination unit 105 calculates the average value of the regression influence (the influence of the weight on the output) for each layer based on the regression influence information 202. The average regression influence for each layer is expressed as follows:

さらに、選択重み個数決定部１０５は、算出したレイヤ毎の退行影響度の平均値を正規化する。正規化した退行影響度の平均は以下のように表すことができる。

Furthermore, the selection weight number determination unit 105 normalizes the calculated average value of the regression influence degree for each layer. The normalized average value of the regression influence degree can be expressed as follows.

図７においては、正規化した各レイヤの退行影響度の平均の例を符号Ｄに示す。
選択重み個数決定部１０５は、算出したレイヤ毎の正規化した退行影響度の平均を記憶装置１３等の所定の記憶領域に記憶させる。

In FIG. 7, the symbol D indicates an example of the average normalized regression influence degree of each layer.
The selection weight number determination unit 105 stores the calculated average of the normalized regression influence degree for each layer in a predetermined storage area such as the storage device 13 .

In addition, in FIG. 7, the symbol E indicates the improvement impact information 203. The selection weight number determination unit 105 calculates the average value of the improvement impact (the impact of the weight on the output) for each layer based on the improvement impact information 203. The average improvement impact for each layer is expressed as follows:

さらに、選択重み個数決定部１０５は、算出した各レイヤの平均値を正規化する。正規化した改善影響度の平均は以下のように表すことができる。

Furthermore, the selection weight number determination unit 105 normalizes the calculated average value for each layer. The normalized average of the improvement influence degree can be expressed as follows.

図７においては、正規化した各レイヤの改善影響度の平均の例を符号Ｆに示す。
選択重み個数決定部１０５は、算出したレイヤ毎の正規化した改善影響度の平均を記憶装置１３等の所定の記憶領域に記憶させる。

In FIG. 7, the symbol F indicates an example of the average of the normalized improvement influence degrees of each layer.
The selection weight number determination unit 105 stores the calculated average of the normalized improvement influence degrees for each layer in a predetermined storage area such as the storage device 13 .

The selection weight number determination unit 105 calculates the suspicion value susp for each layer based on the normalized average weight difference, the average regression influence, and the average improvement influence using the following formula (1).

図７においては、正規化した各レイヤの疑惑値の例を符号Ｇに示す。
選択重み個数決定部１０５は、算出したレイヤ毎の疑惑値suspを記憶装置１３等の所定の記憶領域に記憶させる。

In FIG. 7, an example of the normalized suspicion value of each layer is indicated by the symbol G.
The selection weight number determination unit 105 stores the calculated suspicion value susp for each layer in a predetermined storage area such as the storage device 13.

The selection weight number determination unit 105 determines the number of weights to be selected as correction targets in each layer based on the suspicion value susp. The selection weight number determination unit 105 determines the number of selection weights for each layer using the following formula (2).

図７においては、算出した各レイヤの選択重み個数の例を符号Ｈに示す。
修正対象重み特定部１０６は、レイヤ毎に修正対象とする重みを決定（特定）する。
図８は実施形態の一例としての情報処理装置１における修正対象重み特定部１０６の処理を説明するための図である。

In FIG. 7, the symbol H indicates an example of the calculated number of selected weights for each layer.
The correction target weight specifying unit 106 determines (specifies) the weight to be corrected for each layer.
FIG. 8 is a diagram for explaining the process of the correction target weight identifying unit 106 in the information processing device 1 as one example of an embodiment.

The correction target weight identification unit 106 selects the number of selected weights' worth of indexes from the top of the weight difference sorted indexes for each layer based on the weight difference order index list 210. Hereinafter, the top indexes from the weight difference sorted indexes for the number of selected weights may be referred to as weight difference top indexes. The weight difference top indexes are a set of weights with a large weight difference from the previous generation (n-kth generation) DNN model. An example of a weight difference top index is shown in symbol A in Figure 8.

修正対象重み特定部１０６は、選択した重み差分上位インデックスを記憶装置１３等の所定の記憶領域に記憶させる。

The correction target weight specifying unit 106 stores the selected weight difference upper index in a predetermined storage area such as the storage device 13 .

The correction target weight identification unit 106 also selects, for each layer, indexes equal to the number of selected weights from the top of the regression influence sorted indexes for each layer based on the regression influence index list 220. Hereinafter, the top indexes equal to the number of selected weights selected from the regression influence sorted indexes may be referred to as top regression influence indexes. The top regression influence indexes are a set of weights that are heavily influenced by the regression test data. An example of a top regression influence index is shown in symbol B of Figure 8.

The upper regression influence index corresponds to the first edge classification that has influenced the output of the regression test data. Therefore, the correction target weight identification unit 106 executes a process of classifying the edges (weights) in each layer into the first edge classification that has influenced the output of the regression test data.
Then, the correction target weight specifying unit 106 prioritizes edges with a large degree of regression influence as the first edge classification.
The correction target weight specifying unit 106 stores the selected regression influence degree upper index in a predetermined storage area such as the storage device 13 .

Furthermore, the correction target weight identification unit 106 selects, for each layer, indexes equal to the number of selected weights from the top of the indexes sorted by improvement impact, based on the improvement impact index list 230. Hereinafter, the top indexes equal to the number of selected weights selected from the indexes sorted by improvement impact may be referred to as top improvement impact indexes. The top improvement impact indexes are a collection of weights that are greatly influenced by the improvement test data. An example of a top improvement impact index is shown in symbol C in Figure 8.

The upper improvement influence index corresponds to the second edge classification that has influenced the output of the improved test data. Therefore, the correction target weight identification unit 106 executes a process of classifying the edges (weights) in each layer into the second edge classification that has influenced the output of the improved test data.
Then, the correction target weight specifying unit 106 prioritizes edges with a large degree of improvement influence as the second edge classification.
The correction target weight specifying unit 106 stores the selected improvement influence degree upper index in a predetermined storage area such as the storage device 13 .

Then, the correction target weight identification unit 106 selects, from among the weights that have a large difference in weight from the previous generation DNN model in Fault Localization, those that are heavily influenced by the regression test data and less influenced by the improvement test data as the correction target weights.

In other words, the correction target weight identification unit 106 identifies the weights to be corrected by taking the difference (＼) between the intersection (∩) of the "set of weights with a large weight difference from the previous generation" and the "set of weights greatly influenced by regression test data" and the "set of weights greatly influenced by improvement test data."
FIG. 9 is a diagram for explaining the processing of the correction target weight identifying unit 106 in the information processing device 1 as one example of the embodiment.

In Figure 9, symbol A indicates a "set of weights with a large difference from the previous generation (n-k)," symbol B indicates a "set of weights that are heavily influenced by regression test data," and symbol C indicates a "set of weights that are heavily influenced by improvement test data."

For example, for a certain layer, the weight elements of the "set of weights with a large difference from the previous generation" are {w1, w2, w3, w4, w5}, the weight elements of the "set of weights with a large influence from regression test data" are {w1, w2, w3}, and the weight elements of the "set of weights with a large influence from improvement test data" are {w2, w4}. In such a case, the weight elements {w1, w3} are selected as the weights to be modified.

Note that weights with a large weight difference, weights with a large influence from regression test data, and weights with a large influence from improvement test data are the top-ranked weights (a predetermined number of selected weights) when sorted by weight difference or influence value.

図１０は実施形態の一例としての情報処理装置１における修正対象重み特定部１０６による修正対象重みの選択方法を例示する図である。
この図１０においては、レイヤ１における修正対象重みを修正対象重み特定部１０６が特定（選択）する例を示す。

FIG. 10 is a diagram illustrating a method of selecting a correction target weight by the correction target weight specification unit 106 in the information processing device 1 as one example of an embodiment.
FIG. 10 shows an example in which the correction weight specification unit 106 specifies (selects) the correction weights in layer 1 .

In FIG. 10, the symbol A indicates a weight difference top index extracted from the weight difference ordered index list 210, the symbol B indicates a regression impact top index extracted from the regression impact ordered index list 220, and the symbol C indicates an improvement impact top index extracted from the improvement impact ordered index list 230.

The correction target weight identification unit 106 takes the intersection of the weight difference upper index and the regression influence upper index. That is, the correction target weight identification unit 106 extracts the indexes "77" and "572" that are in both the weight difference upper index and the regression influence upper index (see symbol P1).

Next, the correction target weight specifying unit 106 obtains the difference set of the improvement influence upper index from the product set of the weight difference upper index and the regression influence upper index. In other words, the correction target weight specifying unit 106 does not specify the index "287" as a correction target if it is in the improvement influence upper rank even if it is in the weight difference upper rank and regression influence upper rank (see symbol P2).
As a result, the correction weight identifying unit 106 selects the weights with indexes "77" and "572" as correction weights for layer 1 (see symbol D).

In this way, the correction target weight identification unit 106 performs a process of identifying a third edge group, obtained by subtracting the second edge group included in the higher improvement impact index (second edge classification) from the first edge group included in the higher regression impact index (first edge classification), as a correction target.

The weighting of the correction target for each layer shown in FIG. 10 corresponds to the edge group (third edge group, correction target) obtained by subtracting the edge group (second edge group) included in the second edge classification from the edge group (first edge group) included in the first edge classification.

The weights are modified using, for example, PSO (Particle swarm optimization) for W _localize output in this way. Note that the fitness may be calculated using the following equation.

Note that N _patched indicates the number of failed test data that have been corrected, and N _intact indicates the number of successful test data that remain unchanged. Note that PSO is a known method, and a detailed description thereof will be omitted.
(B) Operation

The process of the information processing device 1 as one example of the embodiment configured as described above will be described with reference to the flowcharts (steps S1 to S34) shown in FIGS.
11 shows the process of steps S1 to S14, and FIG. 12 shows the process of steps S15 to S34.

In step S1 of FIG. 11, the model training unit 101 performs training by inputting training data into the DNN model. The model training unit 101 records information on the nth generation DNN model and the n-kth generation DNN model in the storage device 13, etc.

In step S2 in Fig. 11, a loop process is started in which the control up to step S7 in Fig. 11 is repeatedly performed for all test data. Note that x is a variable for identifying the test data.
In step S3 of FIG. 11, the test data classification unit 102 inputs the test data x to the n-th generation DNN model and the n-th generation DNN model.

In step S4 of FIG. 11, the test data classification unit 102 checks whether the test data x is one that has been successfully estimated by the n-kth generation DNN model and has failed to be estimated by the nth generation DNN model. If the result of this check is that the test data x is not one that has been successfully estimated by the n-kth generation DNN model and has not failed to be estimated by the nth generation DNN model (see NO route), the process proceeds to step S6 of FIG. 11.

Also, if the result of the check in step S4 shows that the test data x is successfully estimated by the n-kth generation DNN model and unsuccessfully estimated by the nth generation DNN model (see the YES route in step S4), the process proceeds to step S5 in FIG. 11. In step S5, the test data classification unit 102 adds the test data x to the regression test data. Then, the process proceeds to step S6.

In step S6, the test data classification unit 102 checks whether the test data x is one for which estimation has failed in the n-kth generation DNN model and estimation has been successful in the nth generation DNN model. If the result of this check is that the test data x is one for which estimation has failed in the n-kth generation DNN model and estimation has not been successful in the nth generation DNN model (see the NO route in step S6), the process proceeds to step S8.

Also, if the result of the check in step S6 shows that the estimation of the test data x has failed in the n-kth generation DNN model and has been successful in the nth generation DNN model (see the YES route in step S6), the process proceeds to step S7 in FIG. 11. In step S7, the test data classification unit 102 adds the test data x to the improved test data. After that, the process proceeds to step S8.

In step S8, loop end processing corresponding to step S2 is performed. When processing of all test data is completed, control proceeds to step S9 in FIG. 11.

In step S9, the weight difference measurement unit 103 records the weight difference calculated by taking the absolute value of the difference between the set of weights in the nth generation DNN model and the set of weights in the nkth generation DNN model in the storage device 13, etc.
In step S10 of FIG. 11, the weight difference measurement unit 103 records in the storage device 13 or the like a weight difference sorted index list in which the weight indices are sorted in order of the magnitude of the weight differences.

In step S11 of FIG. 11, the weighting influence measurement unit 104 calculates the regression influence ∂Y _deg /∂W n by automatically differentiating the output Y _deg when the regression test data X _deg is input to the n-th generation model with the weighting _{W n} , and records this calculated regression influence in the storage device 13 or the like.

In step S12 of FIG. 11, the weight influence measurement unit 104 records a regression influence sorted index list in which the weight indices are sorted in order of the magnitude of the regression influence in the storage device 13, etc.

In step S13 of FIG. 11, the weighting influence measurement unit 104 calculates the improvement influence degree ∂Y _imp /∂W n by automatically differentiating the output Y _imp when the improved test data X _imp is input to the n-th generation model with the weighting _{W n} , and records this calculated improvement influence degree in the storage device 13 or the like.

In step S14 of FIG. 11, the weighting influence measuring unit 104 records an index list sorted in order of improvement influence degree, in which the weighting indexes are sorted in order of the magnitude of the improvement influence degree, in the storage device 13 or the like.
In step S15 of FIG. 12, a loop process is started in which the control up to step S18 is repeatedly performed for all layers l in the DNN model.
In step S16 of FIG. 12, the selection weight number determination unit 105 calculates the average of the weight differences of the layer l, and records it in the storage device 13 or the like.
In step S17 of FIG. 12, the selection weight number determination unit 105 calculates the average of the regression influence degree of layer l, and records it in the storage device 13 or the like.
In step S18 of FIG. 12, the selection weight number determination unit 105 calculates the average improvement influence degree of layer l and records it in the storage device 13 or the like.

In step S19 in Fig. 12, a loop end process corresponding to step S15 is performed. When the process for all layers is completed, control proceeds to step S20 in Fig. 12.
In step S20, a loop process is started in which the control up to step S25 is repeatedly performed for all layers l in the DNN model.
In step S21 of FIG. 12, the selection weight number determination unit 105 calculates the normalized weight difference average of the layer l, and records it in the storage device 13 or the like.
In step S22 of FIG. 12, the selection weight number determination unit 105 calculates the normalized regression influence average of the layer l, and records it in the storage device 13 or the like.
In step S23 of FIG. 12, the selection weight number determination unit 105 calculates the normalized improvement influence average of the layer l, and records it in the storage device 13 or the like.
In step S24 of FIG. 12, the selection weight number determination unit 105 calculates the suspicion value of layer l based on the above formula (1).

In step S25 of FIG. 12, the selection weight number determination unit 105 determines the number of weights to be selected as correction targets in layer l based on the suspicion value, and records the number in the storage device 13, etc.

In step S26 of Fig. 12, a loop end process corresponding to step S20 is performed. When the process for all layers is completed, control proceeds to step S27 of Fig. 12.
In step S27, a loop process is started in which the control up to step S32 is repeatedly performed for all layers l in the DNN model.

In step S28 of FIG. 12, the correction target weight identification unit 106 generates a weight difference upper index for layer l, which is a set of weights selected from the top of the weight difference sorted index, and records the weight difference upper index in the storage device 13, etc.

In step S29 of FIG. 12, the correction target weight identification unit 106 generates a regression influence top index for layer l, which is a set of weights selected from the top of the regression influence sorted index up to the number of selected weights, and records the generated index in the storage device 13, etc.

In step S30 of FIG. 12, the correction target weight identification unit 106 generates a top improvement impact index for layer l, which is a set of weights selected from the top of the improvement impact sorted index up to the number of selected weights, and records the index in the storage device 13, etc.

In step S31 of FIG. 12, the correction target weight identification unit 106 generates an index set for layer l, which is weight difference upper index ∩ regression influence upper index ＼ improvement influence upper index, as an index of the correction target weight, and records it in the storage device 13, etc.

In step S32 of FIG. 12, for layer l, the correction target weight identification unit 106 records the weight identified by the correction target weight index as the correction target weight.

In step S33 of FIG. 12, loop end processing corresponding to step S27 is performed. When processing is completed for all layers l, control proceeds to step S34.

In step S34 of FIG. 12, the correction target weight identifying unit 106 outputs the correction target weights W _localize recorded in all layers l, and then ends the process.
Then, the weight localization is performed using, for example, PSO on the thus generated weight W _localize to be localized.
(C) Effects

In this way, according to the information processing device 1 as an example of an embodiment, the correction target weight identification unit 106 selects, in each layer, a weight that is included in the top ranks of the regression influence degree and the top ranks of the weight difference, but is not included in the top ranks of the improvement influence degree, as a correction target weight. This makes it possible to realize correction of weights that are less likely to cause regression.

That is, by excluding weights with a large improvement influence from the weights to be modified, the weights of the edge groups that led to the improvement are used as they are. On the other hand, by including weights with a large regression influence from the weights to be modified, the weights of the edge groups that led to the regression are included in the weights to be modified. Therefore, it is possible to use the weights of the edge groups that led to the improvement as they are, while only the weights of the edge groups that led to the regression are modified.
FIG. 13 is a diagram illustrating a simulation result in the information processing device 1 as an example of the embodiment.

In this Figure 13, the method of determining edges to be corrected by the information processing device 1 is compared with the conventional method (Arachne) using traffic sign images from the GTSRB (The German Traffic Sign Recognition Benchmark) as target data.

As shown in FIG. 13, it can be seen that the method for determining an edge to be repaired by the information processing device 1 is successful in suppressing the BreakRate while maintaining a sufficient RepairRate.
(D) Others FIG. 14 is a diagram illustrating a hardware configuration of the information processing device 1 as an example of an embodiment.

The information processing device 1 is a computer and has as its components, for example, a processor 11, a memory 12, a storage device 13, a graphics processing device 14, an input interface 15, an optical drive device 16, a device connection interface 17, and a network interface 18. These components 11 to 18 are configured to be able to communicate with each other via a bus 19.

The processor (control unit) 11 controls the entire information processing device 1. The processor 11 may be a multiprocessor. The processor 11 may be, for example, any one of a CPU, an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). The processor 11 may also be a combination of two or more types of elements from the CPU, MPU, DSP, ASIC, PLD, and FPGA.

Then, the processor 11 executes a control program for the information processing device 1 (a correction target edge determination program: not shown), thereby realizing the functions of the model training unit 101, the test data classification unit 102, the weight difference measurement unit 103, the weight influence measurement unit 104, the selected weight number determination unit 105, and the correction target weight identification unit 106 illustrated in FIG. 1.

The information processing device 1 realizes the functions of a model training unit 101, a test data classification unit 102, a weight difference measurement unit 103, a weight influence measurement unit 104, a selection weight number determination unit 105, and a correction target weight identification unit 106 by executing a program (a correction target edge determination program, an OS program) recorded on a computer-readable non-transitory recording medium, for example.

The program describing the processing contents to be executed by the information processing device 1 can be recorded on various recording media. For example, the program to be executed by the information processing device 1 can be stored in the storage device 13. The processor 11 loads at least a part of the program in the storage device 13 into the memory 12 and executes the loaded program.

The program to be executed by the information processing device 1 (processor 11) can also be recorded on a non-transitory portable recording medium such as an optical disk 16a, a memory device 17a, or a memory card 17c. The program stored on the portable recording medium becomes executable after being installed in the storage device 13, for example, under control of the processor 11. The processor 11 can also read and execute the program directly from the portable recording medium.

Memory 12 is a storage memory including ROM (Read Only Memory) and RAM (Random Access Memory). The RAM of memory 12 is used as the main storage device of information processing device 1. The RAM temporarily stores at least a portion of the program to be executed by processor 11. In addition, memory 12 stores various data necessary for processing by processor 11.

The storage device 13 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), a storage class memory (SCM), etc., and stores various data. The storage device 13 is used as an auxiliary storage device of the information processing device 1.
An OS program, a control program, and various data are stored in the storage device 13. The control program includes a correction target edge determination program.

As the auxiliary storage device, a semiconductor storage device such as an SCM or a flash memory can also be used. In addition, multiple storage devices 13 can be used to configure a RAID (Redundant Arrays of Inexpensive Disks).

The storage device 13 may store information on the previous generation model and the current generation model, and may also store regression test data and improvement test data. Furthermore, the storage device 13 may store at least a part of the weight difference information 201, regression influence information 202, improvement influence information 203, weight difference ordered index list 210, regression influence ordered index list 220, and improvement influence ordered index list 230. The storage device 13 may also store the number of selected weights for each layer determined by the selected weight number determination unit 105. Furthermore, the storage device 13 may record at least a part of the weight difference top index, regression influence top index, improvement influence top index, weight difference sorted index list, regression influence sorted index list, and improvement influence sorted index list.

The monitor 14a is connected to the graphics processing device 14. The graphics processing device 14 displays images on the screen of the monitor 14a in accordance with instructions from the processor 11. Examples of the monitor 14a include a display device using a CRT (Cathode Ray Tube) and a liquid crystal display device.

A keyboard 15a and a mouse 15b are connected to the input interface 15. The input interface 15 transmits signals sent from the keyboard 15a and the mouse 15b to the processor 11. Note that the mouse 15b is an example of a pointing device, and other pointing devices can also be used. Examples of other pointing devices include a touch panel, a tablet, a touch pad, and a trackball.

The optical drive device 16 uses laser light or the like to read data recorded on the optical disc 16a. The optical disc 16a is a portable, non-transient recording medium on which data is recorded so that it can be read by the reflection of light. Examples of optical discs 16a include DVDs (Digital Versatile Discs), DVD-RAMs, CD-ROMs (Compact Disc Read Only Memory), and CD-Rs (Recordable)/RWs (ReWritable).

The device connection interface 17 is a communication interface for connecting peripheral devices to the information processing device 1. For example, a memory device 17a or a memory reader/writer 17b can be connected to the device connection interface 17. The memory device 17a is a non-transient recording medium equipped with a communication function with the device connection interface 17, such as a USB (Universal Serial Bus) memory. The memory reader/writer 17b writes data to the memory card 17c or reads data from the memory card 17c. The memory card 17c is a card-type non-transient recording medium.

The network interface 18 is connected to a network. The network interface 18 transmits and receives data via the network. Other information processing devices, communication devices, etc. may be connected to the network.
Regardless of the embodiment described above, various modifications can be made without departing from the spirit of the present embodiment.
For example, in the above-described embodiment, an example is shown in which the invention is applied to a DNN, but the invention is not limited to this and may be applied to a NN.
Moreover, the above disclosure enables a person skilled in the art to implement and manufacture the present embodiment.
(E) Supplementary Note The following supplementary notes are further disclosed with respect to the above embodiment.

(Appendix 1)
When the information processing device determines an edge to be modified among edges included in the neural network model,
A process of extracting, from among the edges included in the second generation trained machine learning model, edges having weights different from those of the edges included in the first generation trained machine learning model prior to the second generation as a group of changed edges;
A process of classifying the changed edge group into a first edge classification that has influenced the output of regressed test data, which is test data in which regression has occurred, and a second edge classification that has influenced the output of improved test data, which is test data in which improvement has occurred;
and identifying a third edge group, obtained by subtracting a second edge group included in the second edge classification from a first edge group included in the first edge classification, as a correction target edge.

(Appendix 2)
2. The method for determining an edge to be corrected according to claim 1, wherein the extraction process, the classification process, and the identification process are performed for each layer of the neural network.

(Appendix 3)
In the extraction process,
The method for determining an edge to be corrected according to claim 1 or 2, characterized in that, among the edges included in the second generation trained machine learning model, edges having weights different from those of the edges included in the first generation trained machine learning model, edges having a large difference between the weight of the second generation trained machine learning model and the weight of the first generation trained machine learning model are preferentially extracted.

(Appendix 4)
The method for determining an edge to be corrected according to any one of appendices 1 to 3, characterized in that in the classification process, an edge having a large degree of regression influence, which is obtained by inputting the regression test data into the second generation trained machine learning model and differentiating the output obtained with a weight, is given priority as the first edge classification.

(Appendix 5)
The method for determining an edge to be modified according to any one of appendices 1 to 4, characterized in that in the classification process, an edge having a large improvement influence obtained by inputting the improved test data into the second generation trained machine learning model and differentiating the output obtained with a weight is given priority as the second edge classification.

(Appendix 6)
A processor of an information processing device that determines edges to be modified among edges included in a neural network model,
A process of extracting, from among the edges included in the second generation trained machine learning model, edges having weights different from those of the edges included in the first generation trained machine learning model prior to the second generation as a group of changed edges;
A process of classifying the changed edge group into a first edge classification that has influenced the output of regressed test data, which is test data in which regression has occurred, and a second edge classification that has influenced the output of improved test data, which is test data in which improvement has occurred;
and identifying a third edge group, which is obtained by subtracting a second edge group included in the second edge classification from a first edge group included in the first edge classification, as a correction target.

(Appendix 7)
The processor,
7. The correction target edge determination program according to claim 6, further comprising: executing the extraction process, the classification process, and the identification process for each layer of the neural network.

(Appendix 8)
The processor,
The correction target edge determination program described in Appendix 6 or 7, characterized in that it executes a process of preferentially extracting edges, among the edges included in the second generation trained machine learning model, that have weights different from those of the edges included in the first generation trained machine learning model, and that have a large difference between the weight of the second generation trained machine learning model and the weight of the first generation trained machine learning model.

(Appendix 9)
The processor,
The correction target edge determination program according to any one of appendices 6 to 8, characterized in that a process is executed in which edges with a large degree of regression influence, which is obtained by inputting the regression test data into the second generation trained machine learning model and differentiating the output obtained with a weight, are prioritized as the first edge classification.

(Appendix 10)
The processor,
The program for determining an edge to be modified according to any one of claims 6 to 9, characterized in that a process is executed in which edges with a large improvement influence, which is obtained by inputting the improvement test data into the second generation trained machine learning model and differentiating the output obtained with a weight, are prioritized and classified as the second edge.

1 Information processing device 11 Processor (processing unit)
12 Memory 13 Storage device 14 Graphics processing device 14a Monitor 15 Input interface 15a Keyboard 15b Mouse 16 Optical drive device 16a Optical disk 17 Device connection interface 17a Memory device 17b Memory reader/writer 17c Memory card 18 Network interface 18a Network 19 Bus 101 Model training unit 102 Test data classification unit 103 Weight difference measurement unit 104 Weight influence measurement unit 105 Selection weight number determination unit 106 Modification target weight identification unit 201 Weight difference information 202 Regression influence information 203 Improvement influence information 210 Weight difference ordered index list 220 Regression influence ordered index list 230 Improvement influence ordered index list

Claims (6)

When the information processing device determines an edge to be modified among edges included in a machine learning model of a neural network,
A process of extracting, from among the edges included in the second generation trained machine learning model, edges having weights different from those of the edges included in the first generation trained machine learning model prior to the second generation as a group of changed edges;
A process of classifying the changed edge group into a first edge classification that has influenced the output of regressed test data, which is test data in which regression has occurred, and a second edge classification that has influenced the output of improved test data, which is test data in which improvement has occurred;
and identifying a third edge group, obtained by subtracting a second edge group included in the second edge classification from a first edge group included in the first edge classification, as an edge to be corrected. 2. The method for determining an edge to be corrected according to claim 1, wherein the extraction process, the classification process, and the identification process are performed for each layer of the neural network. In the extraction process,
3. The method for determining an edge to be corrected according to claim 1 or 2, characterized in that, among the edges included in the second generation trained machine learning model, edges having weights different from those of the edges included in the first generation trained machine learning model, edges having a large difference between the weight of the second generation trained machine learning model and the weight of the first generation trained machine learning model are preferentially extracted. The method for determining an edge to be corrected according to any one of claims 1 to 3, characterized in that in the classification process, edges with a large degree of regression influence, which is obtained by inputting the regression test data into the second generation trained machine learning model and differentiating the output obtained with a weight, are given priority as the first edge classification. The method for determining an edge to be modified according to any one of claims 1 to 4, characterized in that in the classification process, edges having a large improvement influence obtained by inputting the improved test data into the second generation trained machine learning model and differentiating the output obtained with a weight are given priority as the second edge classification. A processor of an information processing device that determines edges to be modified among edges included in a neural network model,
A process of extracting, from among the edges included in the second generation trained machine learning model, edges having weights different from those of the edges included in the first generation trained machine learning model prior to the second generation as a group of changed edges;
A process of classifying the changed edge group into a first edge classification that has influenced the output of regressed test data, which is test data in which regression has occurred, and a second edge classification that has influenced the output of improved test data, which is test data in which improvement has occurred;
and identifying a third edge group, which is obtained by subtracting a second edge group included in the second edge classification from a first edge group included in the first edge classification, as a correction target.

Images