JP6794593B2 - Methods and devices for optimizing and applying multi-layer neural network models, and storage media - Google Patents

Description

The present invention relates to the field of modeling multi-layer neural networks, and more particularly to methods and devices for optimizing and applying multi-layer neural network models to save processor resources, and storage media.

In the field of artificial intelligence (AI), network architectures based on deep learning are one of the most widely used architectures so far. A network architecture based on deep learning is actually a multi-layer neural network that generates a multi-layer neural network model using a prescribed learning set, such as a set of descriptive data corresponding to images or a set of descriptive data corresponding to voice. It is a structure. Each of the hierarchies in a multi-layer neural network model contains a neuron operation in which the result of the upper neuron operation is used as an input for the lower neuron operation in two adjacent layers, thereby learning from the top layer to the bottom layer. Therefore, we model a multi-layer neural network model. In the modeling procedure, each time one hierarchy is trained, the parameters are fine-tuned by using the method of back proper gaton to obtain a multi-layer neural network model with better performance. ..

The convolutional neural netork (CNN) model is currently the most commonly used multi-layer neural network model. Convolutional neural network models have been widely used in the technical fields of computer vision, computer audiory, and natural language processing. However, in the multi-layer neural network model modeled in recent years, the operation causes a large amount of memory overhead and occupies a large amount of processor resources. Better data processing results can be obtained when a multi-layer neural network model is calculated on a GPU-based workstation or server, but it still occupies less processor resources when the network model is calculated. , One of the purposes of improving the performance of the network model. On the other hand, since the processor resources of current embedded devices such as smartphones and tablets are limited, fixed-point numbers (integers) tend to be used in order to perform more efficient calculations. In this case, it is necessary to use fixed-point arithmetic to convert floating-point parameters in each layer of the network into fixed-point parameters and adapt them to the operation requirements of embedded devices. However, since fixed-point arithmetic is an operation accompanied by deterioration, executing fixed-point arithmetic in a neural network for each layer causes a cumulative error in the entire system, resulting in a significant deterioration in system performance. Therefore, the multi-layer neural network model is hardly calculated on the embedded device.

Considering the technical problems existing in the above-mentioned conventional multi-layer neural network model, the present invention provides the processor resources required for optimizing the conventional multi-layer neural network model and computing the multi-layer neural network model. The purpose is to provide a way to reduce.

According to the uniformity of the present invention, it is a substructure division module that divides at least one substructure from a multi-layer neural network model, and the last layer of the divided substructure is a quantization layer, which is a quantization layer. Having a data transfer module that transfers arithmetic parameters in layers other than the above to the quantization layer for each of the divided substructures and updates the quantization threshold parameters in the quantization layer based on the transferred arithmetic parameters. A device for optimizing a multi-layer neural network model is provided.

According to another aspect of the present invention, a storage module for storing a multi-layer neural network model and a data set corresponding to a task request that can be executed by the multi-layer neural network model for the stored multi-layer neural network model. The multi-layer neural network model has an input module for inputting data and an arithmetic module for calculating a data set in each of the layers from the top layer to the bottom layer in the multi-layer neural network model and outputting the result. The storage module stores the information of the quantization threshold parameter in the quantization layer, and the arithmetic module stores the basic coefficient, the shift coefficient, and the quantization with respect to the basic coefficient indicated by the information of the quantization threshold parameter . A device for applying a multi-layer neural network model characterized by performing quantization processing on a data set in a quantization layer based on a quantization threshold parameter determined based on a correlation between threshold parameters. Will be done.

According to another aspect of the present invention, at least one substructure is divided from the multi-layered neural network model, and here, the last layer of the divided substructure is the quantization layer, other than the quantization layer. The feature is that the arithmetic parameters in the hierarchy are transferred to the quantization layer for each of the divided substructures, and the quantization threshold parameters in the quantization layer are updated based on the transferred arithmetic parameters. A method of optimizing a multi-layered neural network model is provided.

According to another aspect of the present invention, a multi-layer neural network model is stored, and a data set corresponding to a task request that can be executed by the multi-layer neural network model is provided to the stored multi-layer neural network model. The multi-layer neural network model includes a quantization layer, which has input and output of a result by computing a data set in each of the layers from the top layer to the bottom layer in the multi-layer neural network model. , Information on the quantization threshold parameter in the quantization layer is stored, and further, in the calculation, the basic coefficient, the shift coefficient indicated by the information on the quantization threshold parameter , and the correlation between the quantization threshold parameter with respect to the basic coefficient. A method of applying a multi-layer neural network model characterized by performing quantization processing in a quantization layer for a data set based on a quantization threshold parameter determined based on a relationship is provided.

According to another aspect of the invention, when executed by a computer, it divides at least one substructure from a multi-layer neural network model, where the last layer of the divided substructures is quantized. It is a layer, and the arithmetic parameters in the layers other than the quantization layer are transferred to the quantization layer for each of the divided substructures, and the quantization threshold parameters in the quantization layer are updated based on the transferred arithmetic parameters. A non-temporary computer-readable storage medium is provided that stores instructions for causing a computer to execute a method of optimizing a multi-layer neural network model, which comprises.

Further features of the present invention will become apparent from the description of the exemplary embodiments below, with reference to the accompanying drawings.

The accompanying drawings are included in the specification and are used to form a part thereof, show an embodiment of the present invention, and explain the principle of the present invention together with the description thereof.

FIG. 1 shows an example of an optimized binary convolutional neural network model. FIG. 2A shows an example of a network model for dividing the substructure of the network model shown in FIG. FIG. 2B is an example of a part of the substructure in FIG. 2A. FIG. 2C is an example of a part of the substructure in FIG. 2A. FIG. 2D shows an example of a network model after simplifying FIG. 2A. FIG. 3 shows a block diagram of the hardware configuration of the device that optimizes the multi-layer neural network model. , 4 (a) and 4 (b) show flowcharts for optimizing the multi-layer neural network model, respectively. , , , 5A-5D show examples of four types of substructures. FIG. 6 shows a part of the calculation procedure of the multi-layer neural network model without using the optimization method of the embodiment of the present invention and the optimization procedure of the multi-layer neural network model using the optimization method of the embodiment of the present invention. And a part of. FIG. 7 shows a block diagram of a device that optimizes a multi-layer neural network model. FIG. 8 shows the structure of the arithmetic module in the application device.

Various embodiments of the present invention will be described in detail below with reference to the drawings. It should be understood that the invention is not limited to the various embodiments described below. Furthermore, it is not necessary to include a combination of features described in all of the various embodiments as a method for solving the problems of the present invention.

The structure of traditional multi-layer neural network models is usually complex. In order to simplify the structure, a binarization process is performed on the network model. FIG. 1 shows a convolutional neural network model that has been binarized.

With reference to FIG. 1, a binary convolutional neural network model trained and generated using well-known techniques is shown. When the binary convolutional neural network model shown in FIG. 1 is trained and generated, the training set is first input according to the task request. Here, the task request is referenced to model a binary convolutional neural network model, i.e., the network model is used to perform any kind of data processing. For example, if the task request is an image classification process, the binary convolutional neural network model is used to perform data processing to classify the image, and if the task request is a speech recognition process, for example, the binary convolutional neural network. The network model is used to perform data processing to recognize speech. After the task request is determined, the learning set that matches the task request is input. For example, when the task request is an image classification process, an image and a set of descriptive data (such as annotated true values) corresponding to the image are input as a learning set. Alternatively, for example, when the task request is a voice recognition process, the voice and a set of descriptive data corresponding to the voice are input as a learning set. Then, the binary convolutional neural network model shown in FIG. 1 is trained and generated by a prescribed learning method using the input learning set. The generated binary convolutional neural network model has the following two characteristic aspects.
Feature 1: The weight parameter (weight) is binarized In order to simplify the operation, the weight in the convolutional layer of the binary convolutional neural network model is binarized. That is, the weight is represented using one bit according to the sign (positive or negative) of the weight. Here, the weight having a positive value is represented by +1 after being binarized, and the weight having a negative value is represented by -1 after being binarized. Further, in consideration of the calculation accuracy, in other words, in order to obtain a calculation result based on the weight after the binarization process, which is close to the calculation result based on the full-precision weight (that is, the weight not subjected to the binarization process). In order to obtain the convolution operation result of the binarized weight and the input feature map input to the convolution layer, which is close to the convolution operation result of the full-precision weight and the input feature map input to the convolution layer, 1 Two scaling coefficients α are included in each of the convolution filters. This scaling factor α is used to improve the accuracy of the calculation result using the binarized weight. That is, the product of the binarized weight and α is used to approximate the full-precision weight. The scaling coefficient α is calculated according to the following equation (1).

Here, | W | _l1 indicates the norm L1 corresponding to the filter in the case of full-precision weights, and n indicates the number of weights of the filter.

Based on the above equation (1), the later convolution operation based on the binarized weight is expressed by the following equation (2).

Where W _b indicates the convolution filter when binary weights are used, and α indicates the scaling factor of the convolution filter when binary weights are used.

Indicates the convolution operation, b indicates the offset parameter of the convolution operation, X ₁ indicates the input feature map input to the convolution layer (in other words, the output of the upper layer of the convolution layer), and Y ₁ indicates the convolution layer. Shows the output of.

Feature 2: The input feature map (feature map) is quantized using multiple bits. In the quantization layer, the upper layer of the quantization layer is according to a certain quantization threshold method (for example, using a specific quantization threshold parameter). Each element in the input feature map is quantized (mapped) into a computational space represented by k bits. Where k is a positive integer and the choice of k depends on the computational accuracy requirements for the multi-layered neural network model to be modeled. If the computational accuracy requirement for the network model is higher, a larger k value is selected, such as a k value of 4, and if the computational accuracy requirement for the network model is lower, the k value is smaller, such as 2. The k value is selected.

Taking the k value of 2 as an example, some center point elements are selected from each of the elements in the input feature map from the upper layer of the quantized layer, for example, 0, 0.538, 1.076, and 1.614 is selected as the center point element, and based on these center point elements, the quantization threshold parameters used during the quantization process are determined. After that, each of the elements in the input feature map is compared with the quantization threshold parameter, and each of the elements is quantized into the calculation space represented by 2 bits according to the result of the comparison. In particular, the quantization process is executed by using the quantization threshold method shown in the following equation (3).

Here, X _i indicates the i-th element in the input feature map from the upper layer of the quantization layer, i is a positive integer, and Y _i is the output of the quantization layer for the i-th element in the input feature map. Is shown. As can be seen from the equation (3), the quantization threshold method used in the quantization layer sets the numerical value at which the i-th element in the input feature map is quantized as 0,0.807,1.345. It is determined based on the quantization threshold parameters of and ∞. If X _i is less than 0, the i-th element is quantized to 0, i.e. the output of the quantization layer is 0 and X _i is between the quantization threshold parameters 0 and 0.807. If the i-th element is quantized to 0.538, i.e. the output of the quantization layer is 0.538 and X _i is between the quantization threshold parameters 0.807 and 1.345. If the i-th element is quantized to 1.076, i.e. the output of the quantization layer is 1.076 and X _i is greater than the quantization threshold parameter 1.345, then the i-th element is 1. Quantized to 614, i.e. the output of the quantization layer is 1.614. By the quantization operation shown in the above equation (3), all the elements in the input feature map are selected in the quantization layer so that the result output from the quantization layer becomes the value of the selected center point element. It is quantized (mapped) to the center point element.

In the above equation (3), the quantization threshold parameter is calculated by a method such as averaging the selected center point elements.

0.807 = (0.538 + 1.076) / 2, 1.345 = (1.076 + 1.614) / 2
Of course, the calculation method for determining the quantization threshold parameter is not limited to the above method for averaging the selected center point elements, and other known calculation methods may be used.

If the minimum non-zero term is shown as β'in the above equation (3), the equation (3) can be further expanded into the equation (4).

Here, β'= 0.538.

Y _i is further expressed as Y _i = β'Y _q , where Y _q is a representation of the corresponding integer part of Y _i , and the value of that element takes an integer of 0, 1, 2, or 3. However, it can be seen from the above equation (4).

When a data processing task is executed using the binary convolutional neural network model shown in FIG. 1, the value output from the quantization layer and mapped via quantization becomes a floating point number. It can be seen from equation (4). Therefore, for the lower layer of the quantization layer, each element in the input feature map is a 32-bit floating point number, and the operation on the floating point number causes a large amount of memory overhead and occupies a large amount of processor resources. In general, hardware devices have tended to use more integers to perform operations in order to achieve more efficient operations. Based on this, the present invention proposes a method of optimizing a conventional multi-layer neural network model. Referring to FIG. 4A, first, the substructure is divided with respect to the floating point quantized multi-layer neural network. If the last layer of the divided substructure is a quantized layer (in other words, a substructure that can be processed using the method of the embodiment of the invention), then the method of the embodiment of the invention Fixed-point processing is performed using this to obtain a fixed-point multi-layer neural network, when the last layer of the divided substructure is not a quantization layer but another layer such as an activation layer (in other words, Substructures that cannot be processed using the methods of the embodiments of the invention), followed by processing using other known algorithms. In the fixed-point processing according to the embodiment of the present invention, the calculation procedure and the calculation parameter in each of the layers are transferred to the quantization layer to form a reconstruction substructure in the joint quantization form, and the network forward is performed. Based on algorithms such as calculation rules, the calculation procedure and calculation parameters during the calculation of neurons in each layer before the quantization layer are equally transferred downward, thereby in the form of integers from the quantization layer. Output the feature map. This effectively reduces the server resources used during the computation of the multi-layer neural network model and saves memory overhead.

The hardware configuration of the device for optimizing the multi-layer neural network model according to the embodiment of the present invention will be described below with reference to FIG.

<Hardware configuration of the device that optimizes the multi-layer neural network model>
FIG. 3 shows a schematic diagram of the hardware configuration of a device that optimizes a multi-layer neural network model, and the devices include a processor unit 101, an internal memory unit 102, a network interface unit 103, an input unit 104, and an external memory unit 105. And the bus unit 106.

The processor unit 101 is a CPU or GPU. The memory unit 102 includes a random access memory (RAM) and a read-only memory (ROM). The RAM is used as the main memory, work area, etc. of the processor unit 101. The ROM is used to store the control program of the processor unit 101, and is also used to store files or other data used when the control program is executed. The network interface unit 103 is connected to the network and performs network communication. The input unit 104 controls input from devices such as a keyboard and a mouse. The external memory unit 105 stores a startup program, various applications, and the like. The bus unit 106 is used to connect each of the units in the device that optimizes the multi-layer neural network model.

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

<First Embodiment>
As shown in FIG. 4B, it is a schematic flowchart of the optimization of the multi-layer neural network model in the first embodiment of the present invention. In the present embodiment, the processing flow for optimizing the multi-layer neural network model shown in FIG. 4B is a program (neural network forward prediction algorithm, etc.) stored in the ROM and / or the external memory unit 105. It is executed by causing the CPU 101 to execute the RAM using the RAM as the work memory.

Step S201: Preprocessing Step S201 is the preprocessing before the execution of the optimization of the multi-layer neural network model, and the details thereof are: Obtaining the optimized multi-layer neural network model.

It should be noted that step S201 is not a necessary step to achieve the object of the present invention. The multi-layer neural network model that needs to be optimized in step S201 is a multi-layer neural network model modeled by a known method. The binary convolutional neural network model shown in FIG. 1 is an example of a multi-layer neural network model that needs to be optimized. The multi-layer neural network model acquired in step S201 is a network model modeled by a known technique, and the embodiment of the present invention is not limited thereto.

Step S202: Substructure division step In step S202, the process of optimizing the multi-layer neural network model is started. The details are: Dividing at least one substructure from a multi-layered neural network model, where each trailing layer of the divided substructures is a quantization layer.

Preferably, when the multi-layer neural network model acquired in step S201 is a convolutional neural network model, the convolutional layer is further divided into each of the substructures in order to perform the convolutional processing performed in the substructure. included. More preferably, in order to improve the convolution operation, the head layer of each substructure is a convolution layer, and the bottom layer is a quantization layer. Between the convolutional layer and the quantization layer are other layers such as batch regularization layer, pooling layer, and scaling layer. Of course, there may be no other layers between the convolution layer and the quantization layer, that is, only the convolution layer and the quantization layer sequentially include substructures. 5A-5D show schematic views of four types of substructures. However, in the method of the embodiment of the present invention, the form of the substructure is not limited to the structure shown in FIGS. 5A to 5D, and the substructure satisfying the above conditions is according to the actual structure of the multi-layer neural network model. Divided from a multi-layer neural network model. A detailed example of substructure division in step S202 will be given later.

Step S203: Data transfer In step S203, the process of optimizing the multi-layer neural network model is continued. The details are: For each of the divided substructures, the quantization threshold parameters in the quantization layer are updated using the arithmetic parameters in the hierarchy other than the last hierarchy (quantization layer). That is, the calculation parameters and the calculation procedure in the layers other than the last layer are equally transferred to the quantization layer to update the quantization threshold parameters in the quantization layer. The "transfer" described in the embodiment of the present invention is essentially a merge operation. For example, for substructures containing N layers, the output of each layer is used as the input of the next layer starting from the first layer until transferred to the quantization layer, thereby leading and ending. Determine the mathematical relationship with the hierarchy.

In step S203 of the embodiment of the present invention, for each of the layers excluding the last layer of the substructure, until the operation parameters and operation procedures in each layer are equally transferred to the last layer (that is, the quantization layer). The calculation parameters and calculation procedures in the upper layer are similarly transferred from the top layer to the bottom layer toward the lower layer. This updates the quantization threshold parameters in the quantization layer. The updated quantization threshold parameter is quantized using the updated quantization threshold parameter, reflecting the calculation procedure and calculation parameters (that is, the calculation characteristics of each layer) in each layer performed during the data transfer procedure. When the (mapping) process is executed, as the quantization result, the quantization (mapping) process is performed without using the selected center point element in the floating point format in the input feature map from the upper layer as in the equation (3). An integer is used as the result of the quantization of. This reduces the processor resources required during the operation.

A detailed example of the data transfer procedure in step S203 will be given later.

For the multi-layer neural network model optimized through steps S201 to S203, the multi-layer operation for any substructure in the optimized network model is expanded to the quantization operation of the quantization layer in the substructure, and the quantum. The output from the optimization layer is no longer a floating-point arithmetic parameter, but a hierarchy after this substructure (which may be a hierarchy in another substructure or a hierarchy in a network model that does not belong to a substructure. The optimized multi-layer neural network model, which is an arithmetic parameter in integer format, can operate with less occupied processor resources so that the arithmetic (which may be) is simple.

In addition to the optimization processing procedure in steps S202 and S203, optimization processing such as substructure simplification processing and fixed-point processing is optimized through steps S202 to S203 in the embodiment of the present invention. Further execution may be performed on the model, the details of which are given in the description below.

Step S204: Substructure Simplification Continuing with reference to step 4 (b), the details of the process for simplifying the substructure in step S204 are as follows: Arithmetic parameters and arithmetic for hierarchies other than the last hierarchy in each substructure. The procedure is to remove the hierarchy equally transferred to the quantization layer from the substructure.

In the embodiment of the present invention, when there is a hierarchy in which the calculation parameters and the calculation procedure are the same, the layers are completely transferred to the quantization layer in a layer other than the last layer, and the multi-layer neural network model is calculated. Since the data is no longer calculated in such a hierarchy, deleting such a hierarchy does not adversely affect the calculation accuracy. Rather, the structure of the multi-layer neural network model becomes simpler. A detailed example for simplifying the substructure in step S204 will be given later.

Step S205: Fixed-Point Step Continuing with reference to FIG. 4B, the details of the fixed-point processing in step S205 are: Multi-layer neural network so that floating-point arithmetic parameters in each layer are converted to fixed-point parameters. It is to perform fixed-point processing for each layer in the model. Any technique for fixed-point processing is the Q-value technique, where the Q-value is a positive integer N, where the floating-point number is shifted left by N bits (equivalent to multiplying 2 to the Nth power). Amplification operation) means that it will be rounded later, which converts the floating point operation parameter to a fixed point parameter (integer). One N value is set for the entire multi-layer neural network model, and one N value is set for each of the different layers, depending on the accuracy requirement in the multi-layer neural network model. A detailed example of the fixed point in step S205 will be given later.

Note that step S205 is executed after the processes of steps S203 and S204 are completed. Of course, step S205 may be executed before step S203, for example, may be executed between step S201 and step S202, or may be executed between step S203 and step S204. The embodiment of the present invention does not limit the processing time point of step S205 as long as the floating-point arithmetic parameters in each layer of the multi-layer neural network model can be converted into fixed-point numbers. In addition, steps S204 and S205 are suitable steps in the embodiments of the present invention. In embodiments of the invention, the two steps may be present at the same time, one of them may be present, or neither of them may be present.

Step S206: Preservation Step The details of the preservation step in step S206 are: Preserving the multi-layer neural network model optimized through the previous steps S202 to S205. Preferably, when the multi-layer neural network model is a binary convolutional neural network model, the weights in the binary convolutional layer are stored in the form of a binary stream. On the other hand, since each of the hierarchies in the substructure equally transferred the arithmetic parameters and procedures in such hierarchies to the quantized layer in step S203, the arithmetic parameters in these hierarchies are included in the network model. I can't.

This preservation step is an arbitrary step according to the embodiment of the present invention. In the method of the embodiment of the invention, this preservation step does not have to be performed and the optimized multi-layer neural network model is output directly to the outside through a network interface or other input / output interface. ..

The optimization processing described in steps S202 to S205 in this embodiment may be executed a plurality of times, that is, even if all the optimization processing procedures are executed in steps S202 to S205, the optimization is performed. After the multi-layer neural network model is saved in step S206, the optimization processing in steps S202 to S205 may be executed again for the saved network model, thereby improving the calculation accuracy of the network model.

Steps S202-S205 will be described in detail below by a specific example.

<Example of dividing the substructure in step S202>
Taking the case where the binary convolutional neural network model shown in FIG. 1 is acquired in step S201 as an example, in step S202, the procedure for dividing the substructure from this network model is particularly described as follows.

Referring to FIG. 1, the binary convolutional neural network model contains eight convolutional layers. To ensure the accuracy of this network model, both the leading and trailing convolution layers are set as full-precision convolution layers (ie, non-binarized convolution layers), and the remaining convolution layers are binarized. It is a binary convolution layer. It is assumed that the substructure division method is that the first layer is a binary convolution layer and the last layer is a quantization layer. The five substructures are divided from the model shown in FIG. 1, and the substructures are divided. In FIG. 1, each wavy line block has the structure shown in FIG. 2A, which represents the substructure. Here, the first layer in the first substructure is the binary convolution layer 2, the intermediate layers are sequentially the pooling layer 2 and the batch normalization layer 2, the last layer is the quantization layer 2, and the second layer. The first layer in the substructure is the binary convolution layer 3, the middle layer is the batch normalization layer 3, the last layer is the quantization layer 3, and the first layer in the third substructure is the binary convolution layer 4. , The intermediate layer is the batch normalization layer 4, the last layer is the quantization layer 4, the first layer in the fourth substructure is the binary convolution layer 5, and the intermediate layers are sequentially the pooling layer 5 and the batch normalization. The chemical layer 5, the last layer is the quantization layer 5, the first layer in the fifth substructure is the binary convolution layer 6, the middle layer is the batch normalization layer 6, and the last layer is the quantization layer. Layer 6 2B and 2C show a part of the network model from FIG. 2A where the substructure is divided.

To simplify the description of the network model, different character identifiers are assigned to each layer of the multi-layer neural network model according to Table 1.

Here, IP represents a fully connected layer in which each neuron is connected to all input nodes. Based on the correspondence between the character identifiers shown in Table 1 and the hierarchy, the binary convolutional neural network model shown in FIG. 2A is represented as a character string, as shown below.

With reference to the above five divided substructures, it can be seen that the subcolumn CPBQ and the subcolumn CBQ represent the divided substructures in the character string. Therefore, using a method such as regular matching sub-string search, the five substructures divided are represented in the above character string in the following format.

In addition, to ensure the correct order of operation of the network model, the network model operates layer by layer from the top layer to the bottom layer, so there is no nesting between each of the divided substructures. For example, when there is a sub-column such as C-> C-> B-> Q-> B-> Q in the network model, the character sub-string C-> B-> Q in the middle is represented as a substructure and C and B-> Q at both ends are sub-columns. It is necessary to divide the substructure in a way that does not form the structure (without nesting).

When the substructure is divided according to the following method (nested), the problem that the operation is wrong arises.

<Example of data transfer in step S203>
Taking the substructure shown in FIG. 5A as an example, this substructure includes a convolutional layer, a batch regularization layer, and a quantization layer from the top layer to the bottom layer. After the operation shown in equation (2) is executed using the input feature map (that is, the output in the upper layer of the convolution layer) X ₁ input to the convolution layer, the operation result Y ₁ is batch-normalized in the convolution layer. Output for layers. In the batch normalization layer, the batch normalization process is performed on the received Y ₁ , and the output after the batch normalization process is equally transferred to the quantization layer. Then, in the quantization layer, the quantization threshold parameter in the equation (4) is updated using the input feature map from the batch regularization layer. The substructures shown in FIGS. 5B-5D are considered as modifications of the substructure of FIG. 5A, and their transfer procedures are similar to those described above. For example, in the substructure shown in FIG. 5B, the pooling layer is added between the convolutional layer and the batch regularization layer, as compared to the substructure in FIG. 5A. In the substructure shown in FIG. 5B, the method of the transfer operation result is consistent with the method based on the substructure of FIG. 5A, since the pooling operation itself does not require any parameters. Compared with the substructure of FIG. 5A, in the substructure shown in FIG. 5C, the batch regularization layer is canceled and the pooling layer is added, and the pooling operation itself does not require any parameters, so that the calculation result of the convolution layer Y ₁ is directly input to the quantization layer. Compared with the sub-structure of Figure 5A, the sub-structure of FIG. 5D, pooling layer and scaling layers sequentially are added between the convolution layer and batch normalization layer, operation output Y ₁ of the convolution layer, pooling Transferred to the scaling layer through the layer. The scaling layer, is performed scaling operations using the input Y _1, then the output result is transferred to the batch normalization layer. Finally, in the batch normalization layer, the batch normalization process is executed using the result output from the scaling layer, and the result after the batch normalization process is transferred to the quantization layer.

A specific algorithm for transferring and updating the quantization threshold parameters will be further described by taking the substructure shown in FIG. 5A as an example. Before the execution of step S203, the output of the convolution layer is output based on the equation (2).

Is calculated as. Here, β is a non-zero minimum term parameter in the input feature map of the quantized convolution layer, and X _q is a representation of the integer part of the feature map. The output of the convolutional layer is transferred to the batch normalization layer to execute the batch normalization process, and the procedure of the batch normalization process is expressed by the following equation (5).

Here, θ is the average parameter of the batch normalization layer, σ is the distribution parameter of the batch normalization layer, θ and σ are floating point numerical values, and X ₂ is transferred to the batch normalization layer. Is the output of the convolution layer to be (ie Y ₁ ), where Y ₂ is the output of the batch normalization layer to the quantized layer. The batch normalization process is a process for normalizing an input feature map into a standard normal distribution type form.

Furthermore, the quantization (mapping) procedure in the quantization layer is expressed by Eq. (4), and the feature map output to the quantization layer by the batch regularization layer is Y ₂ , and each of the elements in the feature map is X _i . .. Next, in step S203, the procedure for executing the update of the quantization threshold parameter in the quantization layer will be described in detail.

The quantization layer quantizes the input feature map with 2 bits, and the quantization (mapping) procedure in the quantization layer is expanded from the equation (4) to the equation (6).

Here, β'= 0.538, thr ₁ = 0, thr ₂ = 0.807, thr ₃ = 1.345, and X _i is an input feature output to the quantization layer by the batch regularization layer. It is the i-th element in the map, and Y _i is the output of the quantization layer for the i-th element in the input feature map.

From the combination of equations (2) and (5), Y ₁ in equation (2) is X ₂ in equation (5), and Y _{2 in} equation (5) is input to the quantization layer by the batch regularization layer. It turns out that it is the input feature map of equation (6), where the i-th element is

Is. Therefore, the above equation (6) is further expanded into the equation (7).

Equation (7) continues to be expanded into equation (8) through a mathematical transformation.

Equation (8) continues to be expanded into equation (9) again through mathematical transformation.

In the above formula (9)

After that, the above equation (9) is expanded into the equation (10).

Equation (10) shows the update result of the quantization threshold parameter after the operation parameters and the operation procedures in each of the layers above the quantization layer are equally transferred to the quantization layer. At this time, the updated quantization threshold parameters reflect the arithmetic parameters and procedures in the convolutional layer and the batch regularization layer. Furthermore, the result corresponding to the quantization (mapping) process is the product of the non-zero minimum term parameter and the integers 0, 1, 2, or 3, where

Shows the output of the convolution layer independent of the offset and scaling parameters of the convolution filter.

Compared to equation (6) and Equation (10), the X _i associated with convolution convolution layer and the batch normalization operation batch normalization layer, unrelated to the offset parameter and the scaling parameter of the convolution filter convolution Layer output

It can be seen that the input feature map input to the quantization layer has been changed. At the same time, the quantization threshold parameters in the quantization layer are changed from thr ₁ to thr ₁ ', thr ₂ to thr ₂ ', and thr ₃ to thr ₃ '. That is, the quantization threshold parameter of the equation (6) determined by the calculation of the center point element in the input feature map is the center point element (thr _{1 to} thr ₃ and β') and the calculation parameter (binarized weight) in the convolution layer. The scaling factor α of the convolution filter, the non-zero minimum parameter β quantized by the input feature map, and the offset parameter b of the convolution operation), and the operation parameters in the batch normalization layer (average parameter θ of the batch normalization layer, and It is changed to the quantization threshold parameter determined by the dispersion parameter σ) of the batch normalization layer. Equivalent transfer of the computational parameters and procedures of the convolutional and batch normalization layers to the quantization layer by reflecting the computational parameters and procedures of the convolutional layer and batch normalization layer in the quantization threshold parameters of the quantization layer. At this time, the quantization layer directly quantizes (maps) each of the elements in the input feature map to 0, 1, 2, or 3 and outputs it. At the same time, its non-zero minimum term parameter β'is also output for the subsequent fixed point.

In addition, referring to equation (10), W _b and X _q are integers, respectively, i.e.

There is an integer, if one or more of thr _{1 '~thr} 3' is the number of floating point, it rounded floating point portion, rounding rounding floating-point, i.e. before and after the rounding is lossless quantization it is the result, after being performed on floating point numbers in thr _{1 '~thr} 3', does not affect the threshold comparison result. Therefore, the equation (10) is converted into an integer quantization operation in the case of lossless conversion by the simple rounding process for rounding the floating point number.

Parameters such as the scaling factor α in the convolutional layer, which are used when the corresponding operations are performed in each layer of the network model by the method of the first embodiment, especially by the expression of equation (10), are weighted. Rather than being stored directly in form, it is stored in the form of quantization threshold parameters after the computational parameters and procedures of each layer above the quantization layer are similarly transferred to the quantization layer, thereby. Reduce storage space when these parameters are stored.

Taking the scaling factor α as an example, α is generally a floating point number, and the parameter W _{b of the} convolution filter represented by the binary weight = {1, -1, 1, 1. .. .. 1, -1} is an integer. When the conventional method is used to store the original parameter W (W = W _b * α) of the convolution filter, it is necessary to store each of W _b and α. However, after the method of the first embodiment of the present invention is applied, especially after the expansion of equation (10), α is transferred to the quantization threshold parameter of the quantization layer. In this way, it is not necessary to store α, and W _b = {1, -1, 1, 1. .. .. It is necessary to store only 1, -1}. Therefore, the space for storing the arithmetic parameters of the convolution layer of the network model can be efficiently reduced, and the storage resources can be saved.

In the conventional method, when it is desired to store the original parameter W in a low bit (for example, 1 bit), it is difficult to include α in the original parameter W in a low bit, and α is used during the operation of the network model. It's difficult to do. However, after the method of the first embodiment of the present invention is applied, α is transferred to the quantization threshold parameter of the quantization layer, especially after the expansion of equation (10). Even if not included in the original parameter W stored in the form of lower bits, the transferred α is used in the form of a quantized threshold parameter during the operation of the network model, which improves the accuracy of the network model. To do.

The calculation procedure of equations (6) to (10) will be described by taking the substructure of FIG. 5A as an example, and the embodiment of the present invention is not limited to other substructures. Quantization thresholds when the arithmetic parameters and procedures in each of the layers are similarly transferred to the quantization layer from the top layer to the bottom layer, such as updating the quantization threshold parameters for different substructures. The algorithms and results for updating the parameters are also different. In the embodiment of the present invention, the substructure described in the embodiment of the present invention is satisfied, and the calculation parameters and the calculation procedure in each of the layers above the quantization layer are sequentially transferred to the quantization layer. The algorithm for updating the quantization threshold parameter for a specific substructure is not limited as long as it is transferred according to the method described in step S203.

Preferably, the equation (4) is expanded into the equation (11) in consideration of the relationship between the quantization threshold parameters in the equation (4) when the transfer step is not executed.

After the equation (4) is expanded into the equation (11) and then the equation (11) is expanded into the equation (9) according to the same algorithm as the algorithm of the equation (7), the equation (12) is obtained.

Further, equation (12) is expanded into equation (13).

In equation (13),

The basic coefficient γ,

Is a shift coefficient δ, the equation (13) is simplified to the equation (14).

With reference to equation (14), the quantization threshold parameter updated after the transfer step is represented by the correlation between different quantization threshold parameters for one fundamental coefficient γ, one shift coefficient δ, and basic coefficient γ. It turns out that Therefore, when the updated quantization threshold parameters are saved, only the fundamental coefficient γ, the shift coefficient δ, and the correlation between the quantization threshold parameters with respect to the fundamental coefficient γ are saved, and all the quantization threshold parameters are sequentially saved. This saves storage space.

Since the expression form of the above equation (14) is changed by a simple mathematical transformation, the embodiment of the present invention particularly describes the correlation between the quantization threshold parameters with respect to the basic coefficient γ in the above equation (14), δ. , Γ is not limited. For example, 1.5γ in the formula (14) may be represented as γ + 0.5γ, 2.5γ may be represented as 2γ + 0.5γ, and then 1.5γ + δ in the formula (14) may be represented as γ + 0.5γ + δ = γ + δ'. , 2.5γ + δ may be expressed as 2γ + 0.5γ + δ = 2γ + δ', where δ'= 0.5γ + δ. In this way, the above equation (14) is converted into a new equation represented by γ and δ'.

In addition, four numerical values (0,0.538, 1.076, 1.614) are used as the selected center point elements, and the quantization threshold parameter is determined by a method of averaging the selected center point elements. Moreover, since the equation (4) is explained by taking as an example the case where each element in the input feature map from the upper layer of the quantization layer is quantized (mapped) into the numerical space represented by k = 2 bits. Four kinds of results (that is, 0, 1, 2, or 3) are output in the equation (14), and a scaling relationship (that is, 1.5 times) between the quantization threshold parameter and the non-zero minimum term β'. And 2.5 times). However, when the quantization threshold parameter is determined by another method in the equation (4) and the value of k is a numerical value other than 2, between the quantization threshold parameters in the expanded equations (11) to (14). The correlation changes accordingly. For example, an equation (15) that covers a wider range than the equation (14) can be obtained.

Here, k indicates that the numerical space for quantization (mapping) is represented by k bits, and thr1 ”~.

Represents a simplified representation that embodies the correlation between thresholds when using a particular central element selection method and a particular quantization threshold parameter determination method.

However, even if the method is used in equation (4) to determine the quantization threshold parameters, as long as there is a correlation between the quantization threshold parameters, the equation (11) is used to simplify the quantization threshold parameters. ) ~ It should be understood that the method of equation (14) is used, which saves storage space.

Equation (14) is substantially the same as Equation (10), which are described here in two different forms of expression. Embodiments of the present invention are not limited to other modifications in formula (14) and formula (10).

<Example of simplifying the substructure in step S204>
Further, taking the substructure shown in FIG. 5A as an example, the arithmetic parameters and procedures in the convolution layer and the batch normalization layer were similarly transferred to the quantization layer as represented by Eq. (10). Later, on the one hand, the operation parameters and procedures in the batch normalization layer were completely transferred to the quantization layer as well, so that the batch normalization layer was removed from the substructure, and on the other hand, the scaling of the convolution filter of the convolution layer. Parameters and offset parameters are also transferred to the quantization layer, but operations independent of the convolution filter scaling and offset parameters.

The convolution layer cannot be deleted because the convolution layer does more. After the process of simplifying the substructure in step S204, the structure of the multi-layer neural network model is greatly simplified. Based on the above simplified principle, the binary convolutional neural network model shown in FIG. 2A is transformed into the simplified structure shown in FIG. 2D, where the quantization layer in the substructure becomes the joint quantization layer. ..

Taking the substructure shown in FIG. 5A as an example for explanation, the embodiment of the present invention does not limit a specific substructure. Such hierarchies may be removed from the substructure as long as the arithmetic parameters and procedures in the substructure are completely transferred to the quantization layer.

<Example of fixed point in step S205>
An example of fixed-point processing in the embodiment of the present invention will be described by taking Q value technology as an example. Assuming that Q values are set for each layer of the simplified binary convolutional neural network model shown in FIG. 2D, the specific fixed-point procedure is as follows.

The five substructures in the network model shown in FIG. 2D are subjected to joint quantization processing based on the data transferred in step S203 and substructure simplification processing in step S204, and the binary convolution layer 2 in the network model is performed. Since the arithmetic parameters in the binary convolution layer 7 are integers, the Q value for the binary convolution layer 2 to the binary convolution layer 7 is 1, and the arithmetic parameters in the other layers are still floating point numbers, so the full precision convolution layer. The Q value for 1 and the quantization layer 1 is 9, and the Q value for the full precision convolution layer 8 is 13, so that all the arithmetic parameters in each of the hierarchies in the model shown in FIG. 2D are fixed point parameters. Will be converted. Table 2 is a table of Q values specified for the network model shown in FIG. 2D.

<Effect of the first embodiment>
The left side of FIG. 6 shows a part of the calculation procedure of a partial multi-layer neural network model that does not use the optimization method of the embodiment of the present invention, and the right side shows the optimization method of the embodiment of the present invention. A part of the calculation procedure of the multi-layer neural network model used is shown. For ease of understanding, the operations on the left convolution layer have the same meaning as equation (2), Convolve (W _b , X _q ) and

Represented by, where Convolve means a convolution operation, where β and X _q are similar to the representations of the integer part quantized by X ₁ and the non-zero minimum parameter in equation (2), respectively, in batches. The operation in the normalization layer has the same meaning as in Eq. (5).

Here, BN means a batch normalization operation, and the operation in the quantization layer has the same meaning as in Eq. (6).

Here, thr ₀ , thr ₁ ... Are the quantization threshold parameters in the equation (6), and Quant means the quantization operation. After the optimization method of the embodiment of the present invention, the operation in the convolution layer on the right side is independent of the scaling parameter and offset parameter of the convolution filter and is represented by Convolve (W _b , X _q ), and is a batch normalization operation. Is omitted, and the operations in the quantization layer are

Represented by.

In the method based on the first embodiment, the substructure is divided from the multi-layer neural network model, and the arithmetic parameters and the arithmetic parameters in each of the layers are divided from the top layer to the bottom layer in each of the substructures, with respect to the quantized layer at the end. By transferring the arithmetic procedure, the quantization threshold parameter in the quantization layer is updated, and the quantization layer quantizes (maps) the elements in the input feature map to integers. When an optimized network model is applied, the arithmetic procedure in the optimized network model efficiently reduces the required server resources and saves memory space compared to the arithmetic procedure in the conventional network model. It saves, and at the same time, saves the storage space of the network model, allowing the multi-layer neural network model to work in embedded devices. Further, according to the method of the embodiment, if simplification processing and fixed point processing are further performed on the network model and a further optimized network model is applied, the optimized multi-layer neural network model is saved. The storage space for this is further reduced, and the server resources required to operate the network model are further reduced as compared with the calculation procedure in the conventional network model.

After optimizing the multi-layer neural network model using the optimization method of the first embodiment, the optimized network model is applied. For example, an image classification application is performed by entering a dataset that corresponds to a task request that can be performed by a multi-layer neural network model, such as entering a photo and the corresponding annotated truth value. Applicants are optimized using conventional multi-layer neural network models (network models that are not optimized using the methods of the embodiments of the invention) and the methods of the embodiments of the invention when applied. By comparing the accuracy performance of the multi-layer neural network model, it was found that the overall accuracy performance of the two network models is substantially close to each other. Taking the case where the photo classification service is executed using the binary convolutional neural network model as an example, the accuracy performance of the binary convolutional neural network model and the embodiment of the present invention without using the method of the embodiment of the present invention. The accuracy performance of the network model using the method is close to each other. Referring to Table 3, their photo classification accuracy is 52.22% and 52.21%, respectively, and the accuracy difference is 0.01%, which is very small. Therefore, after the multi-layer neural network model is optimized using the method of the embodiment of the present invention, the overall calculation cost of the network model is maintained while maintaining the accuracy of task processing as compared with the conventional multi-layer neural network model. Significantly reduce (required processor and content resources) and storage requirements.

<Second embodiment>
A second embodiment of the present invention describes an apparatus for optimizing a multi-layer neural network model, which includes a substructure division module 301 and a data transfer module 302, as shown in FIG. The substructure division module 301 executes the process in step S202 of the first embodiment, and the data transfer module 302 executes the process in step S203 of the first embodiment. Specifically, the substructure division module 301 is configured to divide at least one substructure from the multi-layer neural network model, and here, the last layer of the divided substructure is a quantization layer. Yes, the data transfer module 302 transfers the arithmetic parameters and procedures in the layers other than the quantization layer to the quantization layer for each of the divided substructures to update the quantization threshold parameters in the quantization layer. It is configured to generate an optimized multi-layer neural network model.

Preferably, the convolution layer is further included in the divided substructure. More preferably, the first layer of the substructure is the convolution layer and the last layer is the quantization layer. There may be other layers such as a batch regularization layer, a pooling layer, a scaling layer, etc. between the first layer and the last layer, or there may be no other layers. If the number of substructures divided is at least 2, there is no nesting between the substructures.

The data transfer module 302 further increases from the top layer to the bottom layer for each layer other than the quantization layer in one substructure until all the operation parameters and procedures in each layer are transferred to the quantization layer. It is configured to transfer the calculation parameters and the calculation procedure in the lower layer to update the quantization threshold parameters in the quantization layer.

In addition to the substructure division module 301 and the data transfer module 302 described above, the device that optimizes the multi-layer neural network model is configured to acquire the optimized multi-layer neural network model Preprocessing module 303. Is further included. The pre-processing module 303 executes processing in step S201 in the first embodiment.

Preferably, the device that optimizes the multi-layer neural network model removes from the substructure the layers in which the arithmetic parameters and procedures have been transferred to the quantization layer for the layers other than the quantization layer in each substructure. It further includes a substructure simplification module 304 configured as such. The substructure simplification module 304 performs the process in step S204 in the first embodiment.

Preferably, the device that optimizes the multi-layer neural network model performs fixed-point processing for each layer in the multi-layer neural network model and converts the floating-point arithmetic parameters in each layer to fixed-point parameters. It further includes a configured fixed-point module 305. The fixed-point module 305 performs processing in step S205 in the first embodiment.

Preferably, the device for optimizing the multi-layer neural network model further includes a storage module 306 configured to store the optimized multi-layer neural network model.

After the multi-layer neural network model has been optimized by the optimizer of the second embodiment, the optimized network model is operated using an device that applies the multi-layer neural network model. The applicable device is a known device that operates the network model, and is optimized with an input module that inputs a data set corresponding to a task request that can be executed by the network model into an optimized multi-layer neural network model. It includes a calculation module that calculates a data set and outputs a result in each of the layers from the top layer to the bottom layer in a multi-layer neural network model.

Further, in order to save storage space for storing the quantization threshold parameters, in the optimization method for the multi-layered neural network model of the first embodiment, the quantization threshold parameters updated after the transfer step are the basic coefficients. It is expressed based on the above equation (14) by the correlation between different quantization threshold parameters for γ, shift coefficient δ, and basic coefficient γ, that is, the correlation between different quantization threshold parameters for γ, δ, and basic coefficient γ. Relationships are stored. Then, when the network model optimized through the first embodiment is operated by the applicable apparatus, the arithmetic module is computed on the input data set for the top-to-bottom quantization layer in the network model. In this case, the quantization threshold parameter is determined using the correlation between the quantization threshold parameters for γ, δ, and γ, and then the quantization process is performed on the data set based on the determined quantization threshold parameter. Do.

FIG. 8 shows the configuration of the arithmetic module in the application device, and when the data set is input via the input module, the data set is processed via the configuration shown in FIG. Referring to FIG. 8, the arithmetic module includes a network model storage unit 401, a feature map storage unit 402, a convolution unit 403, a pooling / activation unit 404, a quantization unit 405, and a control unit 406. Each unit is described below.

The network model storage unit 401 stores information about the multi-layer neural network model acquired by using the optimization method of the first embodiment. However, the information about the multi-layer neural network model is not limited to the network structure information, but includes the information of the quantization threshold parameter transferred to the quantization layer and the information required for the calculation performed in each of the other layers. Here, the information of the quantization threshold parameter transferred to the quantization layer may be the correlation between the quantization threshold parameters for γ, δ, and γ, and the quantization threshold parameter can be characterized. The information and the quantization threshold parameters themselves may be. The feature map storage unit 402 stores the feature map information required during the calculation in the network model.

The convolution unit 403 relates to the data set according to the feature map information of the i-th hierarchy (that is, X _q ) input by the feature map storage unit 402 and the information input by the network model storage unit 401 (that is, W _b ). It is used to perform the convolution process. The pooling / activation unit 404 is the output of the convolution unit 403.

And according to the information input by the network model storage unit 401 (eg, the information required when the pooling / activation process is performed, where the network model storage unit 401 does not have to input any information). Used to perform pooling / activation processing on a dataset. The quantization unit 405 is used to perform the quantization process on the data set according to the quantization threshold parameter input by the network model storage unit 401 and the result output from the pooling / activation unit 404. The result after the quantization process is stored in the feature map storage unit 402. The control unit 406 controls the operation of the network model storage units 401 to the quantization unit 405 by outputting a control signal to the other units in FIG.

It should be noted that the pooling / activation unit 404 capable of performing pooling / activation processing is shown as an example in the configuration shown in FIG. 8 and performs normalization processing and scaling processing which may be included. Other possible units are omitted. Not repeated here. In addition, in the substructure indicated by the network structure information stored in the network model storage unit 401, one or more of the batch regularization layer, the pooling layer, the scaling layer, etc. are located between the convolution layer and the quantization layer. It may be, or none of these hierarchies may be present. For example, if only the convolution layer and the quantization layer are present in the substructure, the output result of the convolution unit 403 is directly accessible to the quantization unit 405.

Other Examples Of the above-described embodiment, a computer-executable instruction (for example, one or more programs) recorded on a storage medium (which may also be a “non-temporary computer-readable storage medium”) is read and executed. A computer of a system or device that performs one or more functions, and / or a system that includes one or more circuits that perform one or more functions of the aforementioned embodiments (eg, an application-specific integrated circuit (ASIC)). Alternatively, the computer of the device and, for example, the computer of the system or device may read and execute a computer-executable instruction from the storage medium to perform one or more functions of the above-described embodiment, and / or one. An embodiment of the present invention can also be realized by a method performed by controlling the above circuit and executing one or more functions of the above-described embodiment. The computer constitutes one or more processors (for example, a central processing unit (CPU)), a microprocessing unit (MPU), and has a network of different computers or a separate processor to read computer-executable instructions. You may want to do it. The computer-executable instructions may be provided to the computer, for example, from a network or storage medium. The storage medium includes, for example, a hard disk, a random access memory (RAM), a read-only memory (ROM), a storage of a distributed computing system, an optical disk (compact disk (CD), a digital versatile disk (DVD), a Blu-ray Disc). (BD) (Recording trademark), etc.), a flash memory device, a memory card, or the like.

Software (program) that executes the functions of the above-described embodiment is supplied to the system or device via a network or various storage media, and the computer of the system or device, or the central processing unit (CPU), microprocessing unit (MPU). The embodiment of the present invention can also be realized by a method of reading and executing the method of the program by means of) or the like.

Although the present invention has been described with reference to embodiments, it should be understood that the invention is not limited to the described embodiments. The scope of the following claims should be construed to the maximum extent, including all modifications and equivalent configurations and functions.

Claims (15)

A device that optimizes multi-layer neural network models
A substructure division module that divides at least one substructure from the multi-layer neural network model, wherein the last layer of the divided substructure is a quantization layer.
For each of the divided substructures, the arithmetic parameters in the layers other than the quantization layer are transferred to the quantization layer, and the quantization threshold parameters in the quantization layer are updated based on the transferred arithmetic parameters. A device characterized by being provided with a data transfer module. The device according to claim 1, wherein the convolution layer is included in the divided substructure. The apparatus according to claim 2, wherein the leading layer in the divided substructure is a convolution layer. The data transfer module has the arithmetic parameters of the upper layer from the top layer to the bottom layer for each of the layers other than the quantization layer in the substructure until the arithmetic parameters in each of the layers are transferred to the quantization layer. The apparatus according to claim 1, wherein the device is transferred to a lower layer. When the convolution layer is included in the divided substructure, the data transfer module has the quantization threshold in the quantization layer based on the parameter scaling factor transferred from the convolution layer to the quantization layer. The apparatus according to claim 1, wherein the parameters are updated. When the convolution layer is included in the divided substructure, the data transfer module has the quantization threshold parameter in the quantization layer based on the offset parameter transferred from the convolution layer to the quantization layer. The apparatus according to claim 1, wherein the device is updated. The apparatus according to claim 1, wherein the updated quantization threshold parameter is represented by a fundamental coefficient, a shift coefficient, and a correlation between the quantization threshold parameter with respect to the fundamental coefficient. In addition
The apparatus according to claim 1, further comprising a substructure simplification module for deleting the layer in which the arithmetic parameters are completely transferred to the quantization layer from the substructure. In addition
The apparatus according to claim 1, further comprising a fixed-point module that performs fixed-point processing for each of the layers in the multi-layer neural network model so that the floating-point arithmetic parameters are converted into fixed-point parameters. A device that applies a multi-layer neural network model
A storage module that stores the multi-layer neural network model,
An input module that inputs a data set corresponding to a task request that can be executed by the multi-layer neural network model into the stored multi-layer neural network model.
It is provided with a calculation module that calculates the data set and outputs the result in each of the layers from the top layer to the bottom layer in the multi-layer neural network model.
The multi-layer neural network model includes a quantization layer, and the storage module stores information on quantization threshold parameters in the quantization layer.
The calculation module is based on the quantization threshold parameter determined based on the fundamental coefficient, the shift coefficient, and the correlation between the quantization threshold parameter with respect to the basic coefficient indicated by the information of the quantization threshold parameter. An apparatus characterized in that a set is subjected to a quantization process in the quantization layer. The device according to claim 10, wherein the storage module stores the multi-layer neural network model optimized by the device according to claim 1 as the multi-layer neural network model. A method of optimizing a multi-layer neural network model
Dividing at least one substructure from the multi-layered neural network model, where the last layer of the divided substructure is the quantization layer.
For each of the divided substructures, the arithmetic parameters in the layers other than the quantization layer are transferred to the quantization layer, and the quantization threshold parameters in the quantization layer are updated based on the transferred arithmetic parameters. A method characterized by having to do and have. A method of applying a multi-layer neural network model,
Saving the multi-layer neural network model and
Inputting a data set corresponding to a task request that can be executed by the multi-layer neural network model into the stored multi-layer neural network model, and
It has the function of calculating the data set in each of the layers from the top layer to the bottom layer in the multi-layer neural network model and outputting the result.
The multi-layer neural network model includes a quantization layer and stores information on quantization threshold parameters in the quantization layer.
Further, in the calculation, based on the quantization threshold parameter determined based on the basic coefficient, the shift coefficient, and the correlation between the quantization threshold parameters with respect to the basic coefficient indicated by the information of the quantization threshold parameter. A method characterized in that a data set is subjected to a quantization process in the quantization layer. A non-temporary computer-readable storage medium that stores instructions for causing the computer to execute a method of optimizing a multi-layer neural network model when executed by a computer.
Dividing at least one substructure from the optimized multi-layer neural network model, where the last layer of the divided substructure is the quantization layer.
For each of the divided substructures, the arithmetic parameters in the layers other than the quantization layer are transferred to the quantization layer, and the quantization threshold parameters in the quantization layer are updated based on the transferred arithmetic parameters. A non-temporary computer-readable storage medium characterized by having to do. A non-temporary computer-readable storage medium that stores instructions for causing the computer to perform a method of applying a multi-layer neural network model when executed by a computer.
Saving the multi-layer neural network model and
Inputting a data set corresponding to a task request that can be executed by the multi-layer neural network model into the stored multi-layer neural network model, and
It has the function of calculating the data set in each of the layers from the top layer to the bottom layer in the multi-layer neural network model and outputting the result.
The multi-layer neural network model includes a quantization layer and stores information on quantization threshold parameters in the quantization layer.
Further, in the calculation, based on the quantization threshold parameter determined based on the basic coefficient, the shift coefficient, and the correlation between the quantization threshold parameters with respect to the basic coefficient indicated by the information of the quantization threshold parameter. A non-temporary computer-readable storage medium characterized in that a data set is quantized in the quantization layer.

Images