JP7842997B2 - Model parameter recovery device, method, and program - Google Patents

Description

The disclosed technology relates to a technique for recovering model parameters.

While various approaches to attacks on AI, or in other words, on model parameters, and countermeasures against them are being considered, many measures have been proposed to prevent attacks from occurring in the first place. However, there are not many technologies that allow for the continued use of model parameters that have been attacked.

Techniques for continuing to utilize model parameters that have been attacked are known from Non-Patent Documents 1 and 2.

The technology described in Non-Patent Document 1 is a technique for detecting backdoor attacks and, if a backdoor attack is detected, optimizing the retraining dataset to restore the model parameters to their pre-attack state.

The technology described in Non-Patent Document 2 is a technique that detects a backdoor attack and restores the system to its pre-attack state by changing parameters.

Fahmy, Hazem & Pastore, Fabrizio & Bagherzadeh, Mojtaba & Briand, Lionel. (2021). Supporting Deep Neural Network Safety Analysis and Retraining Through Heatmap-Based Unsupervised Learning. IEEE Transactions on Reliability. PP. 1-17. 10.1109/TR.2021.3074750. Yue Zhao, Hong Zhu, Kai Chen, and Shengzhi Zhang. 2021. AI-Lancet: Locating Error-inducing Neurons to Optimize Neural Networks. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (CCS '21). Association for Computing Machinery, New York, NY, USA, 141-158.

In the technologies described in Non-Patent Documents 1 and 2, it is necessary to retrain the model parameters to return them to their pre-attack state.

The disclosed technique aims to eliminate the need to retrain the model's parameters to return them to their pre-attack state.

A model parameter recovery device, one embodiment of the disclosed technology, comprises: a change determination unit that determines whether a predetermined change has occurred in the model parameter based on an error correction code assigned to the parameter; and a recovery unit that, if the change determination unit determines that a predetermined change has occurred in the parameter, recovers the parameter based on the error correction code.

According to the disclosed technology, retraining of the model parameters is unnecessary to return the model parameters to their pre-attack state.

Figure 1 shows an example of the functional configuration of a model parameter recovery device. Figure 2 shows an example of the processing procedure for model parameter recovery. Figure 3 shows an example of the functional configuration of a model learning device with a recovery function. Figure 4 shows an example of the processing procedure for a model learning method with recovery functionality. Figure 5 shows an example of a computer's functional configuration.

The embodiments of the disclosed technology will be described below with reference to the drawings. Note that components with the same function are numbered identically in the drawings, and redundant explanations will be omitted.

[Model parameter recovery device and method]
As shown in Figure 1, the model parameter recovery device includes, for example, a model storage unit 1, a change determination unit 2, and a recovery unit 3.

The model parameter recovery method is implemented, for example, by having each component of the model parameter recovery device perform the processing shown in steps S2 to S3 in Figure 2.

<Model Memory Unit 1>
The model memory unit 1 stores multiple parameters that constitute the model. The model is, for example, a deep neural network.

If the model is a deep neural network, each parameter is a weight vector corresponding to the edges connecting the nodes.

Error correction codes are assigned to all or some of the multiple parameters that constitute the model. The model storage unit 1 stores the parameters along with the error correction codes assigned to those parameters. An example of an error correction code is cyclic redundancy check.

For example, an example of a part of the multiple parameters that make up a model is at least one of the multiple parameters that make up the model.

When the model is a deep neural network, the parameters, which are weight vectors, may be elements of a matrix. In this case, some of the multiple parameters that make up the model may be parameters included in a given matrix among the multiple parameters that make up the model.

Furthermore, the generation of multiple parameters stored in the model storage unit 1 and the assignment of error correction codes are performed before the processing in step S1 is carried out by the model learning device with recovery function, which will be described later.

<Change Judgment Unit 2>
The change determination unit 2 reads a certain parameter and the error correction code assigned to that parameter from among the multiple parameters stored in the model storage unit 1.

The change detection unit 2 performs the following processing using the read parameters and the error correction codes assigned to those parameters.

The change determination unit 2 determines whether a predetermined change has occurred in the model parameters based on the error correction code assigned to the parameters (step S1). The determination result is output to the recovery unit 3.

For example, the change detection unit 2 first calculates the amount of change in the parameter based on the error correction code assigned to the model parameter. Of course, the change detection unit 2 may also calculate the amount of change using the parameter itself, not just the error correction code. Then, the change detection unit 2 compares the calculated amount of change with a predetermined threshold, and determines that a predetermined change has occurred in the parameter if the calculated amount of change is greater than the predetermined threshold.

An example of a predetermined threshold is a predetermined absolute value. Another example of a predetermined threshold is a representative value of the change in the parameter over the past n cycles, where n is a predetermined positive integer. Examples of representative values include the mean, maximum, minimum, or mode.

The change in parameters over the past n cycles can be obtained from the information about the parameter changes during each learning cycle, which is obtained by the learning unit 4 described later. The information about the parameter changes during each learning cycle, obtained by the learning unit 4, may also be input to the change determination unit 2.

Furthermore, the change determination unit 2 may determine that a predetermined change has occurred in the parameter if the absolute value of the calculated change is greater than a predetermined threshold, and the calculated change is either positive or negative.

Furthermore, the change determination unit 2 may determine that a predetermined change has occurred in the parameter if the past n changes in the parameter were positive and the currently calculated change is negative. Alternatively, the change determination unit 2 may determine that a predetermined change has occurred in the parameter if the past n changes in the parameter were negative and the currently calculated change is positive.

Furthermore, the change determination unit 2 may determine that a predetermined change has occurred in the parameter if the absolute value of the calculated change is greater than a predetermined threshold, and the past n changes in the parameter were positive, while the currently calculated change is negative. Alternatively, the change determination unit 2 may determine that a predetermined change has occurred in the parameter if the absolute value of the calculated change is greater than a predetermined threshold, and the past n changes in the parameter were negative, while the currently calculated change is positive.

Furthermore, the change detection unit 2 may determine whether a predetermined change has occurred in the parameter by combining it with existing parameter change detection methods based on vector distance or the like.

For example, the change determination unit 2 may make a final determination that a predetermined change has occurred in the parameter if it determines that a predetermined change has occurred in the parameter based on the calculated change amount described above, and also if it determines that a predetermined change has occurred in the parameter based on an existing parameter change detection method.

<Recovery section 3>
The recovery unit 3 receives the judgment result from the change judgment unit 2.

If the change determination unit 2 determines that a predetermined change has occurred in the parameter, the recovery unit 3 recovers the parameter based on the error correction code (step S3).

Recovering parameters based on error correction codes means restoring the parameters to their original values based on the error correction codes.

The recovery unit 3 may recover only the parameters that have been determined to have undergone a predetermined change, or it may recover not only the parameters that have been determined to have undergone a predetermined change but also other parameters of the model, or it may recover all parameters of the model.

Other parameters in the model include parameters adjacent to the parameter that was judged to have undergone a predetermined change, parameters at a distance L from the parameter that was judged to have undergone a predetermined change, and parameters belonging to the same matrix as the parameter that was judged to have undergone a predetermined change. L is a predetermined positive integer.

The parameters adjacent to the parameter that has been determined to have undergone a predetermined change are, in the case of a deep neural network, the parameters corresponding to each edge connected to the node to which the edge corresponding to that parameter is connected.

Here, if the model is a deep neural network, for example, the distance between parameter P1 and parameter P2 is the number of nodes between the edge corresponding to parameter P1 and the edge corresponding to parameter P2.

In this case, the parameter at a distance L from the parameter where a predetermined change has occurred is the parameter corresponding to the edge that has L nodes between it and the edge corresponding to that parameter.

When L=1, the parameter at a distance L from the parameter that was judged to have undergone a predetermined change is the parameter adjacent to the parameter that was judged to have undergone the predetermined change.

In this way, by incorporating recovery mechanisms such as error correction codes into the model parameters themselves, it becomes unnecessary to retrain the model parameters to return them to their pre-attack state.

After the learning unit 4 has learned the multiple parameters constituting the model, and the error correction code assignment unit 5 has assigned error correction codes to them, these parameters are stored in the model storage unit 1. In some cases, the learning unit 6, shown by the dashed line in Figure 1, may then perform retraining on these parameters.

In this case, if the training data used by the learning unit 6 is contaminated with noise or triggers, then changes may occur in multiple parameters that constitute the model stored in the model storage unit 1 as a result of the learning by the learning unit 6, changes unintended by the creator of the model stored in the model storage unit 1. Such unintended changes will be referred to as a contamination attack.

Even if multiple parameters constituting the model stored in the model storage unit 1 are altered due to such a contamination attack, the parameters can be restored to their original state using the model parameter recovery device and method described above.

[Model learning device and method with recovery function]
As illustrated in Figure 3, the model learning device with recovery function includes, for example, a model storage unit 1, a learning unit 4, and an error correction code assignment unit 5.

A model learning method with a recovery function can be implemented, for example, by performing the steps S4 to S5 shown in Figure 4.

<Learning Section 4>
Learning data is input into learning unit 4.

The learning unit 4 performs learning based on the input learning data and generates multiple parameters that constitute the model (step S4). The generated multiple parameters that constitute the model are stored in the model storage unit 1.

The learning unit 4 divides the training data into multiple groups and performs learning based on each of these groups. In other words, the learning unit 4 performs multiple learning iterations. These multiple learning iterations may include minor learning, known as fine-tuning.

When performing multiple learning cycles, the learning unit 4 may obtain information about the change in parameters during each learning cycle.

Information regarding the parameter changes during each learning session may be output to the change determination unit 2. In this case, the change determination unit 2 determines a predetermined threshold based on the information regarding the parameter changes during each learning session.

<Error correction code assignment unit 5>
The error correction code assignment unit 5 reads multiple parameters that constitute the model from the model storage unit 1.

The error correction code assignment unit 5 assigns error correction codes to all or some of the multiple parameters that constitute the loaded model (step S5). The error correction codes are stored in the model storage unit 1 along with the corresponding parameters.

In this way, by assigning error correction codes to all or some of the multiple parameters that make up the model, it becomes unnecessary to retrain the model parameters to return them to their pre-attack state.

Furthermore, before the error correction code is applied by the error correction code assignment unit 5, training may be performed using contaminated training data with noise or triggers. For example, if the training unit 4 performs multiple training cycles, at least one of these cycles may be performed using contaminated training data with noise or triggers. Even in this case, if the change determination unit 2 of the model parameter recovery device uses a representative value of the past n changes in the parameter as a predetermined threshold to determine whether a predetermined change has occurred, and n is sufficiently larger than the number of training cycles using contaminated training data with noise or triggers, the change determination unit 2 can determine whether a predetermined change has occurred.

[Differentiation]
The specific configuration of the embodiments of the disclosed technology is not limited to the configuration described above. The specific configuration of the embodiments of the disclosed technology can be modified as appropriate, as long as it does not deviate from the spirit of the embodiments of the disclosed technology.

The various processes described in the embodiments of the disclosed technology may be performed not only in chronological order according to the sequence described, but also in parallel or individually, depending on the processing capacity of the device performing the processes or as necessary.

For example, data exchange between components of the model parameter recovery device may occur directly or via a storage unit (not shown in the diagram).

[Programs, recording media]
The processing of each part of the above-mentioned devices may be implemented by a computer. In this case, the processing content of the functions that each device should have is described by a program. This program is then loaded into the storage unit 1020 of the computer 1000 shown in Figure 5, and the arithmetic processing unit 1010, input unit 1030, output unit 1040, display unit 1060, etc. are operated, thereby realizing the various processing functions of each of the above-mentioned devices on the computer.

The program describing this process can be recorded on a computer-readable recording medium. A computer-readable recording medium is, for example, a non-temporary recording medium, specifically a magnetic recording device, an optical disc, etc.

Furthermore, this program can be distributed, for example, by selling, transferring, or lending portable recording media such as DVDs or CD-ROMs containing the program. Alternatively, the program may be stored in the storage device of a server computer and distributed by transferring it from the server computer to other computers via a network.

A computer executing such a program may, for example, first store the program recorded on a portable recording medium or a program transferred from a server computer in its own non-temporary storage device, the auxiliary recording unit 1050. Then, when processing is to be executed, the computer reads the program stored in the auxiliary recording unit 1050 into the storage unit 1020 and executes the processing according to the loaded program. Alternatively, as another execution form of this program, the computer may directly read the program from the portable recording medium into the storage unit 1020 and execute the processing according to that program. Furthermore, each time a program is transferred to this computer from a server computer, it may sequentially execute the processing according to the received program. Alternatively, the above processing may be executed by a so-called ASP (Application Service Provider) type service, where the server computer does not transfer programs to this computer, but the processing function is realized only by execution instructions and result acquisition. Furthermore, the term "program" in this form includes information used for processing by an electronic computer that is equivalent to a program (data, etc., that is not a direct instruction to the computer but has the property of defining the computer's processing).

Furthermore, while this configuration involves executing a predetermined program on a computer, at least some of these processing steps may be implemented in hardware. For example, the change detection unit 2, recovery unit 3, learning unit 4, and error correction code assignment unit 5 may be configured using processing circuits.

Furthermore, it goes without saying that modifications may be made as appropriate without departing from the spirit of this invention.

All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually described as being incorporated by reference.

Claims (5)

A change determination unit that determines whether a predetermined change has occurred in the model parameters based on an error correction code assigned to the parameters,
If the change determination unit determines that a predetermined change has occurred in the parameter, the recovery unit restores the parameter based on the error correction code,
A model parameter recovery device that includes this. A model parameter recovery device according to claim 1,
The change determination unit determines that a predetermined change has occurred in the parameter when the amount of change in the parameter is greater than a predetermined threshold.
Model parameter recovery device. A model parameter recovery device according to claim 1,
The recovery unit recovers not only the parameter but also other parameters of the model.
Model parameter recovery device. A change determination step in which the change determination unit determines whether a predetermined change has occurred in the model parameters based on an error correction code assigned to the parameters,
If the recovery unit determines that a predetermined change has occurred in the parameter, the recovery unit performs a recovery step of recovering the parameter based on the error correction code.
A model parameter recovery method that includes this. A program for causing a computer to function as a component of any of the model parameter recovery devices according to claims 1 to 3.