JP7614475B2 - Calculation device and calculation method - Google Patents

Description

Embodiments of the present invention relate to a computing device and a computing method.

Edge devices and AI (artificial intelligence) chips have fewer resources than computers equipped with GPUs (Graphics Processing Units). Therefore, it is difficult to run a trained deep neural network model (DNN (Deep Neural Network)) on an edge device. Therefore, quantization is performed to reduce the weight of the model.
Normally, floating-point numbers with 32-bit arithmetic precision are used inside DNNs. In order to detect overflow during multiplication and accumulation operations and to verify operation on AI chips, the precision is converted to a smaller bit width, such as 8 bits, during quantization, and the weights are reduced by performing the calculations.

Regarding the technology for quantizing the weights of a neural network, a technology is known that allows parameters to be changed in the pre-stage, during learning, and during inference when quantizing the parameters (see, for example, Patent Document 1). In this technology, overflow and the like are checked each time, and the parameters are updated.
Furthermore, a technique is known in which weights are quantized for a convolutional neural network (CNN) and then executed (see, for example, Patent Document 2).
There is also known a technique for converting the weights of a model trained in floating point to fixed point (see, for example, Patent Document 3). This technique makes it possible to determine parameters such as bit width and shift amount based on statistics, and to set parameters for each CNN channel.

JP 2020-64635 A JP 2020-060967 A JP 2019-032833 A

The present invention has been made to solve the above-mentioned problems, and aims to provide a calculation device and a calculation method that can improve the accuracy of the output value of a learning model for input values in fixed-point format.

One embodiment of the present invention is a calculation device comprising: an acquisition unit that acquires information specifying the amount of decimal point shift of parameters of a neural network format learning model; a calculation unit that calculates output values of the learning model for multiple input values converted to fixed-point format with different bit widths based on the information specifying the amount of decimal point shift of the parameters acquired by the acquisition unit, and shifts the decimal point of each of the calculated multiple output values based on each of the calculated multiple output values and the amount of decimal point shift of the output value; and a derivation unit that derives the accuracy of the result of shifting the decimal point of each of the calculated multiple output values , wherein the input values are in fixed-point format.
In one embodiment of the present invention, in the aforementioned computing device, the parameters include parameters for each of a plurality of layers.
In one embodiment of the present invention, in the aforementioned computing device, the parameters include weights and outputs.
In one embodiment of the present invention, in the aforementioned computing device, the parameters further include a bias.
In one embodiment of the present invention, in the aforementioned arithmetic device, the parameters are in a fixed-point format.
One embodiment of the present invention is in the aforementioned computing device, wherein the learning model is created by converting parameters into fixed-point format based on a trained model in deep neural network format.
One embodiment of the present invention is the aforementioned arithmetic device, wherein the arithmetic unit performs a MAC (multiply-accumulate) operation on an input value based on the parameter .

One embodiment of the present invention is a computer-executed calculation method comprising the steps of: acquiring information specifying the amount of decimal point shift of parameters of a neural network-type learning model; determining output values of the learning model for multiple input values converted to fixed-point format with different bit widths based on the information specifying the amount of decimal point shift of the parameters acquired in the acquiring step; shifting the decimal point of each of the multiple output values based on each of the multiple output values and the amount of decimal point shift of the output value; and deriving the precision of the result of shifting the decimal point of each of the multiple output values , wherein the input values are in fixed-point format.

According to an embodiment of the present invention, it is possible to provide a calculation device and a calculation method that can improve the accuracy of the output value of a learning model for an input value in fixed-point format.

FIG. 2 is a configuration diagram of a calculation device according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of a learning model possessed by the arithmetic device according to the present embodiment. 1 is a diagram for explaining an example of an input value in a floating-point format processed by the arithmetic device according to the present embodiment. FIG. 1 is a diagram for explaining an example of an input value in a fixed-point format processed by the arithmetic device according to the present embodiment. FIG. FIG. 2 is a diagram for explaining an example of a calculation executed by the calculation device according to the present embodiment. 11 is a diagram for explaining derivation of the decimal point shift amount of the parameter of the terminal device according to the present embodiment. FIG. FIG. 2 is a diagram showing an example of a learning model possessed by a terminal device according to the present embodiment. 5 is a flowchart showing an example of the operation of the arithmetic device according to the present embodiment. 6 is a flowchart showing an example of an operation of the terminal device according to the embodiment. 1 is a diagram for explaining the accuracy of a calculation device according to the present embodiment. FIG. FIG. 13 is a configuration diagram of a calculation device according to a modified example of the embodiment. 13 is a flowchart illustrating an example of an operation of a calculation device according to a modified example of the embodiment. FIG. 11 is a diagram showing an example of a derived result of the accuracy of the output of a calculation device according to a modified example of the present embodiment.

Next, a calculation device and a calculation method according to an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.
In addition, in all the drawings for explaining the embodiments, the same reference numerals are used for the parts having the same functions, and the repeated explanation is omitted.
In addition, in this application, "based on XX" means "based on at least XX" and includes cases where it is based on other elements in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is directly used, but also includes cases where it is based on XX that has been calculated or processed. "XX" is any element (for example, any information).

(Embodiment)
[Overall configuration]
FIG. 1 is a configuration diagram of a computing device according to an embodiment of the present invention.
An input value is input to the arithmetic device 100. An example of the input value is a floating-point format. The following will continue with the explanation of the case where the input value is in the floating-point format.
The arithmetic unit 100 receives an input value. The arithmetic unit 100 converts the received input value from a floating-point format to a fixed-point format. The number of digits (bit width) n when converting to the fixed-point format is set in advance.

The arithmetic device 100 acquires information for specifying the amount of shift of the decimal point of a parameter included in the learning model based on the learning model. An example of the learning model is a neural network type learning model such as a deep neural network. An example of the learning model is written in Python.
The arithmetic device 100 calculates an output value of the learning model for an input value based on the input value converted into fixed-point format and information specifying the amount of shift of the decimal point of the parameter.
When deriving the calculation result of the output value of the learning model, the calculation device 100 derives the final output value by shifting the decimal point of the result of the product-sum calculation based on the shift amount osf of the decimal point of the output value. The calculation device 100 outputs the derived final output value.
The arithmetic device 100 will now be described.

[Calculation device 100]
The arithmetic device 100 is realized as a smartphone, a mobile terminal, a personal computer, a tablet terminal device, or other information processing devices. The arithmetic device 100 includes, for example, an input unit 110, a reception unit 120, an acquisition unit 125, a arithmetic unit 130, an output unit 140, and a storage unit 150.

The input unit 110 includes an input device. Information that specifies an input value is input to the input unit 110. The input unit 110 acquires information that specifies the input value that has been input. An example of an input value is a number in a 32-bit floating-point format. Below, as an example, the explanation will continue for the case where the input value is a number in a 32-bit floating-point format.

The storage unit 150 is realized by a hard disk drive (HDD), a flash memory, a random access memory (RAM), a read only memory (ROM), etc. The storage unit 150 stores a learning model 152 and shift amount information 154.
An example of the learning model 152 is a learning model in the form of a deep neural network. This learning model 152 is quantized. A method for quantizing a learning model will be described later. Below, the explanation will be continued for the case where the learning model 152 is a quantized learning model in the form of a deep neural network.

2 is a diagram showing an example of a learning model included in the arithmetic device according to the present embodiment. As an example of the learning model 152, FIG. 2 shows a quantized learning model 152 in a deep neural network format.
The learning model 152 includes an input layer 151-1, an intermediate layer 151-2, and an output layer 151-3. The input layer 151-1 has one or more units, the intermediate layer 151-2 has one or more units, and the output layer 151-3 has one or more units. Each of the one or more units included in the input layer 151-1 is connected to each of the one or more units included in the intermediate layer 151-2 by one or more links. Each of the one or more units included in the intermediate layer 151-2 is connected to each of the one or more units included in the output layer 151-3 by one or more links.

Each of the one or more units acquires information output by one or more other units and processes the acquired information. Each of the one or more units outputs a result of processing the information. Specifically, each of the one or more units acquires one or more input values. Each of the one or more units derives an output based on the acquired one or more input values and parameters. Here, the parameters include weights.
Furthermore, the parameters may include a bias. As an example, the description will be continued assuming that the parameters include a weight and a bias. Each of the one or more units multiplies one or more input values by one or more weights, respectively, and adds the sum of the results and the bias.

The shift amount information 154 stores information specifying the shift amount of the decimal point of each input value of the input layer 151-1, the intermediate layer 151-2, and the output layer 151-3 included in the learning model 152, information specifying the shift amount of the decimal point of the parameter, and information specifying the shift amount of the decimal point of each output of the input layer 151-1, the intermediate layer 151-2, and the output layer 151-3. The parameters include weights and biases.
That is, the information specifying the amount of shift of the decimal point of the parameter includes information specifying the amount of shift of the decimal point of the weight wsf and information specifying the amount of shift of the decimal point of the bias. A method for deriving information specifying the amount of shift of the decimal point of the input value of each of the input layer 151-1, the intermediate layer 151-2, and the output layer 151-3, the amount of shift of the decimal point of the parameter, and the amount of shift of the decimal point of the output of each of the input layer 151-1, the intermediate layer 151-2, and the output layer 151-3 will be described later.

The receiving unit 120 obtains information specifying an input value input to the input unit 110, and receives the obtained information specifying the input value. The information specifying the input value received by the receiving unit 120 is output to the calculation unit 130.
When the receiving unit 120 receives information specifying an input value, the acquiring unit 125 acquires information specifying the shift amount of the decimal point of each of the input values of the input layer, the intermediate layer, and the output layer, information specifying the shift amount of the decimal point of the parameter, and information specifying the shift amount of the decimal point of each of the outputs of the input layer, the intermediate layer, and the output layer, which are stored in the shift amount information 154 of the storage unit 150. The information specifying the shift amount of the decimal point of each of the input values of the input layer, the intermediate layer, and the output layer, information specifying the shift amount of the decimal point of the parameter, and information specifying the shift amount of the decimal point of each of the outputs of the input layer, the intermediate layer, and the output layer acquired by the acquiring unit 125 are output to the calculation unit 130.

式（１）において、「ｘｉｎｔ」は浮動小数点の形式の数値を固定小数点の形式の数値へ変換したものである。「ｘ」は浮動小数点の形式の数値である。「ｓｆ」は小数点のシフト量である。「ｘｑ」は、浮動小数点の形式の数値を固定小数点の形式の数値へ変換したものを浮動小数点の形式の数値へ変換したものである。「ｃｌａｍｐ（－２^ｎ－１，２^ｎ－１－１，ｘｉｎｔ）」は、ｘｉｎｔが－２^ｎ－１以下である場合には－２^ｎ－１とし、ｘｉｎｔが２^ｎ－１－１以上である場合には２^ｎ－１－１とすることを表す。
例えば、演算部１３０は、ビット幅ｎを８とした場合には、３２ビットの浮動小数点の形式の入力値を、８ビットの固定小数点の形式に変換する。 The calculation unit 130 acquires information that identifies the input value accepted by the acceptance unit 120. The calculation unit 130 converts the input value into a fixed-point format by calculating formula (1) based on the acquired information that identifies the input value and information that identifies the amount of shift of the decimal point of the input value.

In formula (1), "xint" is a floating-point number converted to a fixed-point number. "x" is a floating-point number. "sf" is the shift amount of the decimal point. "xq" is a floating-point number converted to a fixed-point number, which is then converted back to a floating-point number. "clamp(-2 ^n-1 , 2 ^n-1 -1, xint)" indicates that if xint is -2 ^n-1 or less, it is set to -2 ^n-1 , and if xint is 2 ^n-1 -1 or more, it is set to 2 ^n-1 -1.
For example, when the bit width n is 8, the calculation unit 130 converts an input value in a 32-bit floating-point format into an 8-bit fixed-point format.

FIG. 3 is a diagram for explaining an example of an input value in a floating-point format processed by the arithmetic device according to the present embodiment.
3, an input value 200 in floating-point format includes a sign bit 210, an exponent part 220, and a mantissa part 230. Floating-point is a notation method that expresses a number by dividing it into a part indicating a decimal point and a part indicating the decimal point position. The mantissa part 230 is the part indicating the decimal point, and the exponent part 220 is the part indicating the decimal point position. The sign bit 210 is the part that determines the sign of the input value 200.

FIG. 4 is a diagram for explaining an example of an input value in a fixed-point format processed by the arithmetic device according to the present embodiment.
4, the input value 205 in the fixed-point format includes a sign bit 215, an integer part 225, a fractional part 235, and a decimal point 245. The fixed-point format refers to a notation method that uses a decimal point to indicate a decimal number with a fixed number of digits. The sign bit 215 determines the sign of the input value 205, the integer part 225 corresponds to the part of the input value 205 that indicates an integer, and the fractional part 235 corresponds to the part of the input value 205 that indicates a decimal. The decimal point 245 indicates a reference point that divides the integer part 225 and the fractional part 235 of the input value 205.

The calculation unit 130 acquires information specifying the amount of decimal point shift of each of the input values of the input layer, intermediate layer, and output layer acquired by the acquisition unit 125, information specifying the amount of decimal point shift of the parameters, and information specifying the amount of decimal point shift of each of the outputs of the input layer, intermediate layer, and output layer.
The calculation unit 130 calculates an output value of the learning model 152 for an input value based on information specifying the amount of decimal point shift of each of the acquired input values of the input layer, intermediate layer, and output layer, information specifying the amount of decimal point shift of the parameters, and information specifying the amount of decimal point shift of each of the outputs of the input layer, intermediate layer, and output layer.

5 is a diagram for explaining an example of a calculation executed by the calculation device according to the present embodiment. FIG. 5 shows a MAC (multiply-and-accumulate) calculation executed by the calculation unit 130.
The calculation unit 130 includes a multiplier 10, an adder 20, and an adder 30, and performs a product-sum calculation. The calculation unit 130 acquires an input value in and information specifying the amount of shift of the decimal point of the parameter. Here, the input value in is converted into a fixed-point format based on the information specifying the amount of shift isf of the decimal point of the input value.
The calculation section 130 shifts the decimal point of the weight WE based on information specifying the amount of shift of the decimal point of the parameter.
In the calculation unit 130, the multiplier 10 multiplies the input value in by a weight WE with a shifted decimal point. The adder 20 adds the result of multiplying the input value in by the weight WE with a shifted decimal point. The adder 30 derives the final output by shifting the decimal point of the output value OU based on the output value OU of the adder 20 and the shift amount osf of the decimal point of the output. Specifically, the adder 30 adds 0.5 to the output value OU. By shifting the decimal point of the output value OU, the accuracy of the output value OU can be improved.

Here, information for specifying the amount of shift of the decimal point of the input value and a process for deriving the amount of shift of the decimal point of the parameter will be described.
The derivation of the information specifying the amount of shift of the decimal point of the input value and the amount of shift of the decimal point of the parameter is performed by the terminal device.
The terminal device acquires an input layer from the stored learning model, and derives the amount of shift of the decimal point of the weights of the input layer and the amount of shift of the decimal point of the bias based on the weights and biases included in the acquired input layer. The terminal device stores information specifying the amount of shift of the decimal point of the weights of the derived input layer and information specifying the amount of shift of the decimal point of the bias.
The terminal device converts the weights and biases into a fixed-point format. The terminal device creates a new input layer including the weights and biases converted into the fixed-point format. The terminal device replaces the input layer of the stored learning model with the newly created input layer.

The terminal device acquires an intermediate layer from the stored learning model, and derives the amount of shift of the decimal point of the weights of the intermediate layer and the amount of shift of the decimal point of the bias based on the weights and biases included in the acquired intermediate layer. The terminal device stores information specifying the amount of shift of the decimal point of the weights of the intermediate layer and information specifying the amount of shift of the decimal point of the bias.
The terminal device converts the weights and biases into a fixed-point format. The terminal device creates a new intermediate layer including the weights and biases converted into the fixed-point format. The terminal device replaces the intermediate layer of the stored learning model with the newly created intermediate layer.

The terminal device acquires an output layer from the stored learning model, and derives the amount of shift of the decimal point of the weights of the output layer and the amount of shift of the decimal point of the bias based on the weights and biases included in the acquired output layer. The terminal device stores information specifying the amount of shift of the decimal point of the weights of the output layer and information specifying the amount of shift of the decimal point of the bias derived.
The terminal device converts the weights and biases into a fixed-point format. The terminal device creates a new output layer including the weights and biases converted into the fixed-point format. The terminal device replaces the output layer of the stored learning model with the newly created output layer. This completes the quantized learning model.

The weights and biases included in the quantized learning model are converted from floating-point format to fixed-point format. By using the quantized learning model, multiplication and accumulation operations can be performed in fixed-point format. This allows them to be executed directly on edge devices and AI chips.
However, if this continues, the accuracy of the calculation results of the learning model will decrease. The decrease in accuracy of the calculation results of this learning model is not only caused by the decrease in calculation accuracy. In the learning model in the floating-point format, the calculation of (Equation 1) is performed in the output part.
However, the output part of the quantized learning model does not perform the same processing as (Equation 1). Therefore, in all layers, the input value, weight, and output shift amount are made executable even with integers, and the output decimal point shift amount is applied to the output OU in each layer of the quantized learning model. By configuring in this way, it is possible to prevent a decrease in the accuracy of the calculation result.

The terminal device acquires information for identifying an input value that has been input, and accepts the information for identifying the acquired input value. The terminal device acquires information for identifying the amount of shift of the decimal point of each parameter of the stored input layer, intermediate layer, and output layer. The terminal device converts the input value into a fixed-point format based on the information for identifying the acquired input value.
The terminal device derives the output of the input layer of the quantized learning model for the input value converted to fixed-point format based on the acquired information specifying the shift amount of the decimal point of the weight of the input layer and the information specifying the shift amount of the decimal point of the bias. The terminal device derives the shift amount of the decimal point of the output of the input layer based on the derived output of the input layer. By configuring in this way, quantization can be performed even if the output of the input layer is not quantized. The terminal device stores information specifying the shift amount of the decimal point of the derived output of the input layer, in other words, the shift amount of the decimal point of the input value of the intermediate layer.

The terminal device derives an output of the intermediate layer of the quantized learning model for the output value of the input layer of the quantized learning model based on the acquired information specifying the shift amount of the decimal point of the intermediate layer weight and the information specifying the shift amount of the decimal point of the bias. The terminal device derives the shift amount of the decimal point of the intermediate layer output based on the derived output of the intermediate layer.
By configuring in this way, even if the output of the intermediate layer is not quantized, quantization can be performed. The terminal device stores information specifying the amount of shift of the decimal point of the derived output of the intermediate layer, in other words, the amount of shift of the decimal point of the input value of the output layer.

The terminal device derives the output of the output layer of the quantized learning model for the output value of the intermediate layer of the quantized learning model based on the information specifying the amount of shift of the decimal point of the weight of the output layer and the information specifying the amount of shift of the decimal point of the bias acquired.
The terminal device derives the amount of shift of the decimal point of the output of the output layer by performing a calculation such as variance based on the derived output of the output layer. By configuring in this way, even if the output of the output layer is not quantized, quantization can be performed. The terminal device stores information that specifies the amount of shift of the decimal point of the derived output of the output layer.
The terminal device 300 will now be described.
FIG. 6 is a diagram for explaining derivation of the decimal point shift amount of the parameter of the terminal device according to this embodiment.

[Terminal Device 300]
The terminal device 300 is realized by a device such as a personal computer, a server, a tablet computer, an industrial computer, etc. The terminal device 300 includes, for example, an input unit 310, a reception unit 320, an acquisition unit 325, a calculation unit 330, an output unit 340, and a storage unit 350.

The storage unit 350 is realized by a HDD, a flash memory, a RAM, a ROM, etc. The storage unit 350 stores a learning model 352 and shift amount information 354.
An example of the learning model 352 is a trained learning model in the form of a deep neural network. Hereinafter, the case where the learning model 352 is a trained learning model in the form of a deep neural network will be described.

FIG. 7 is a diagram showing an example of a learning model possessed by a terminal device according to this embodiment.
The learning model 352 includes an input layer 351-1, an intermediate layer 351-2, and an output layer 351-3. The input layer 351-1 has one or more units, the intermediate layer 351-2 has one or more units, and the output layer 351-3 has one or more units.
Each of one or more units included in the input layer 351-1 is connected to each of one or more units included in the hidden layer 351-2 by one or more links. Each of one or more units included in the hidden layer 351-2 is connected to each of one or more units included in the output layer 351-3 by one or more links.

Each of the one or more units acquires information output by one or more other units and processes the acquired information. Each of the one or more units outputs the result of processing the information. Specifically, each of the one or more units acquires one or more input values. Each of the one or more units derives an output based on the acquired one or more input values and parameters. Here, the parameters include weights and biases. Each of the one or more units adds the sum of the results of multiplying the one or more input values by the one or more weights and the bias.

The calculation unit 330 acquires the input layer 351-1 from the learning model 352 in the storage unit 350 (1). The calculation unit 330 derives the amount of shift of the decimal point of the weights and the amount of shift of the decimal point of the bias of the input layer 351-1 by performing calculations such as variance based on the weights and biases included in the acquired input layer 351-1. The calculation unit 330 stores information specifying the amount of shift of the decimal point of the weights and the amount of shift of the decimal point of the bias of the derived input layer 351-1 in the shift amount information 354 of the storage unit 350.
The calculation unit 330 converts the weights and biases into a fixed-point format. The calculation unit 330 creates an input layer 351a-1 including the weights and biases converted into the fixed-point format (2). The calculation unit 330 replaces the input layer 351-1 of the learning model 352 in the storage unit 350 with the created input layer 351a-1 (3).

The calculation unit 330 acquires the intermediate layer 351-2. The calculation unit 330 derives the amount of shift of the decimal point of the weights and the amount of shift of the decimal point of the bias of the intermediate layer 351-2 by performing calculations such as variance based on the weights and biases included in the acquired intermediate layer 351-2. The calculation unit 330 stores information specifying the amount of shift of the decimal point of the weights and the amount of shift of the decimal point of the bias of the derived intermediate layer 351-2 in the shift amount information 354 of the storage unit 350.
The calculation unit 330 converts the weights and biases into a fixed-point format. The calculation unit 130 creates an intermediate layer 351a-2 that includes the weights and biases converted into the fixed-point format. The calculation unit 330 replaces the intermediate layer 351-2 of the learning model 352 with the created intermediate layer 351a-2.

The calculation unit 330 acquires the output layer 351-3. The calculation unit 330 performs calculations such as variance based on the weights and biases included in the acquired output layer 351-3 to derive the amount of shift of the decimal point of the weights and the amount of shift of the decimal point of the biases of the output layer 351-3.
The calculation unit 330 stores, in the shift amount information 354 of the storage unit 350, information specifying the shift amount of the decimal point of the weights of the derived output layer 351-3 and information specifying the shift amount of the decimal point of the biases.
The calculation unit 330 converts the weights and biases into a fixed-point format. The calculation unit 330 creates an output layer 351a-3 that includes the weights and biases converted into the fixed-point format. The calculation unit 330 replaces the output layer 351-3 of the learning model 352 with the created output layer 351a-3 (4).
By performing the above-described processing, the calculation unit 330 creates a quantized learning model 352 a from the learning model 352 .
The calculation unit 330 can derive the weight decimal point shift amount and bias decimal point shift amount of the input layer 351-1, the weight decimal point shift amount and bias decimal point shift amount of the intermediate layer 351-2, and the weight decimal point shift amount wsf and bias decimal point shift amount of the output layer 351-3.

The input unit 310 includes an input device. For example, the input device includes a device for inputting character information such as a keyboard, a pointing device such as a mouse or a touch panel, a button, a dial, a joystick, a touch sensor, a touch pad, etc. Information specifying an input value is input to the input unit 310. The input unit 310 acquires information specifying the input value that has been input. An example of an input value is a number in a 32-bit floating-point format. Below, as an example, the explanation will continue for a case where the input value is a number in a 32-bit floating-point format.

The reception unit 320 acquires information that identifies the input value input to the input unit 310, and accepts the information that identifies the acquired input value. The information that identifies the input value accepted by the reception unit 320 is output to the calculation unit 330.

When the receiving unit 320 receives information specifying an input value, the acquiring unit 325 acquires information specifying the shift amount of the decimal point of each parameter of the input layer, intermediate layer, and output layer stored in the shift amount information 354 of the storage unit 350. The information specifying the shift amount of the decimal point of each parameter of the input layer, intermediate layer, and output layer acquired by the acquiring unit 325 is output to the calculation unit 330.

The calculation unit 330 acquires information specifying the input value accepted by the acceptance unit 320. The calculation unit 330 converts the input value into a fixed-point format based on the acquired information specifying the input value.
The calculation unit 330 acquires information specifying the amount of decimal point shift of each parameter of the input layer, the intermediate layer, and the output layer from the acquisition unit 325. The calculation unit 330 derives an output of the input layer 351a-1 of the quantized learning model 352a for the input value converted to the fixed-point format, based on the acquired information specifying the amount of decimal point shift of the weight of the input layer 351-1 and the information specifying the amount of decimal point shift of the bias.
The calculation unit 330 derives the amount of shift of the decimal point of the output of the input layer 351a-1 by performing calculations such as variance based on the derived output of the input layer 351a-1. With this configuration, even if the output of the input layer 351a-1 is not quantized, quantization can be performed. The calculation unit 330 stores information specifying the amount of shift of the decimal point of the derived output of the input layer 351a-1, in other words, the amount of shift of the decimal point of the input of the intermediate layer 351a-2, in the shift amount information 354 of the storage unit 350.

The calculation unit 330 derives the output of the quantized intermediate layer 351a-2 of the learning model 352a corresponding to the output value of the input layer 351a-1 of the learning model 352a, based on the acquired information specifying the amount of decimal point shift of the weight of the intermediate layer 351a-2 and information specifying the amount of decimal point shift of the bias.
The calculation unit 330 derives the amount of shift of the decimal point of the output of the intermediate layer 351a-2 by performing calculations such as variance based on the derived output of the intermediate layer 351a-2. With this configuration, even if the output of the intermediate layer 351a-2 is not quantized, quantization can be performed. The calculation unit 330 stores information specifying the amount of shift of the decimal point of the derived output of the intermediate layer 351a-2, in other words, the amount of shift of the decimal point of the input of the output layer 351a-3, in the shift amount information 354 of the storage unit 350.
The calculation unit 330 derives the output of the quantized output layer 351a-3 of the learning model 352a for the output value of the intermediate layer 351a-2 of the learning model 352a based on the information specifying the amount of decimal point shift of the weight of the acquired output layer 351a-3 and the information specifying the amount of decimal point shift of the bias.

The calculation unit 330 derives the amount of shift of the decimal point of the output of the output layer 351a-3 by performing calculations such as variance based on the derived output of the output layer 351a-3. By configuring in this way, even if the output of the output layer 351a-3 is not quantized, quantization can be performed. The calculation unit 330 stores information specifying the amount of shift of the decimal point of the derived output of the output layer 351a-3 in the shift amount information 354 of the storage unit 350.
By performing the above-mentioned processing, the calculation unit 330 derives the amount of shift of the decimal point of the output of the input layer 351a-1, in other words, the amount of shift of the decimal point of the input of the intermediate layer 351a-2, and the amount of shift of the decimal point of the output of the intermediate layer 351a-2, in other words, the amount of shift of the decimal point of the input of the output layer 351a-3, and the amount of shift of the decimal point of the output of the output layer 351a-3.
The quantized learning model 352a created by the terminal device 300 is used as the learning model 152 in the arithmetic device 100. The shift amount information 354 created by the terminal device 300 is used as the shift amount information 154 in the arithmetic device 100.

(Operation of the arithmetic device 100)
FIG. 8 is a flowchart showing an example of the operation of the arithmetic device according to the present embodiment.
(Step S1-1)
In the arithmetic device 100, the receiving unit 120 obtains information specifying an input value input to the input unit 110, and receives the obtained information specifying the input value.
(Step S2-1)
In the arithmetic device 100, the acquiring unit 125 acquires information specifying the amount of shift of the decimal point of each parameter of the input layer, the intermediate layer, and the output layer, and information specifying the amount of shift of the decimal point of each output of the input layer, the intermediate layer, and the output layer, which are stored in the shift amount information 154 of the storage unit 150. The information specifying the amount of shift of the decimal point of each output of the input layer, the intermediate layer, and the output layer includes information specifying the amount of shift of the decimal point of an input value of the input layer, information specifying the amount of shift of the decimal point of an input value of the intermediate layer, and information specifying the amount of shift of the decimal point of an input value of the output layer.

(Step S3-1)
In the arithmetic device 100, the arithmetic unit 130 acquires information that specifies an input value accepted by the acceptance unit 120. The arithmetic unit 130 converts the input value into a fixed-point format based on the acquired information that specifies the input value and information that specifies the amount of shift of the decimal point of the input value.
(Step S4-1)
In the calculation device 100, the calculation unit 130 acquires information specifying the amount of decimal point shift of each of the parameters of the input layer, the intermediate layer, and the output layer, and information specifying the amount of decimal point shift of each of the outputs of the input layer, the intermediate layer, and the output layer acquired by the acquisition unit 125. The calculation unit 130 calculates an output value OU for an input value, based on the acquired information specifying the amount of decimal point shift of each of the parameters of the input layer, the intermediate layer, and the output layer, and information specifying the amount of decimal point shift of each of the outputs of the input layer, the intermediate layer, and the output layer.
(Step S5-1)
In the calculation device 100, the calculation section 130 derives the final output by shifting the decimal point of the output value OU based on the output value OU and the shift amount osf of the decimal point of the output.

(Operation of Terminal Device 300)
FIG. 9 is a flowchart showing an example of the operation of the terminal device according to this embodiment.
(Step S1-2)
In the terminal device 300, the calculation unit 330 acquires the input layer 351-1 from the learning model 352 in the storage unit 350. The calculation unit 330 derives the amount of decimal point shift of the weights and the amount of decimal point shift of the biases of the input layer 351-1, based on the weights and biases included in the acquired input layer 351-1.
The calculation unit 330 stores, in the shift amount information 354 of the storage unit 350, information specifying the shift amount of the decimal point of the weights of the derived input layer 351-1 and information specifying the shift amount of the decimal point of the biases.
(Step S2-2)
In the terminal device 300, the calculation unit 330 converts the weights and biases of the input layer 351-1 into a fixed-point format.

(Step S3-2)
In the terminal device 300, the calculation unit 330 creates an input layer 351a-1 including weights and biases converted into a fixed-point format. The calculation unit 330 replaces the input layer 351-1 of the learning model 352 in the storage unit 350 with the created input layer 351a-1.
(Step S4-2)
In the terminal device 300, the calculation unit 330 acquires the intermediate layer 351-2 from the learning model 352 in the storage unit 350. The calculation unit 330 derives the amount of shift of the decimal point of the weights and the amount of shift of the decimal point of the bias of the intermediate layer 351-2 based on the weights and biases included in the acquired intermediate layer 351-2. The calculation unit 330 stores information specifying the amount of shift of the decimal point of the weights and the amount of shift of the decimal point of the bias of the derived intermediate layer 351-2 in the shift amount information 354 of the storage unit 350.
(Step S5-2)
In the terminal device 300, the calculation unit 330 converts the weights and biases of the hidden layer 351-2 into a fixed-point format.

(Step S6-2)
In the terminal device 300, the calculation unit 130 creates an intermediate layer 351a-2 including weights and biases converted into a fixed-point format. The calculation unit 330 replaces the intermediate layer 351-2 of the learning model 352 with the created intermediate layer 351a-2.
(Step S7-2)
In the terminal device 300, the calculation unit 330 acquires the output layer 351-3 from the learning model 352 in the storage unit 350. The calculation unit 330 derives the amount of shift of the decimal point of the weights of the output layer 351-3 based on the weights and biases included in the acquired output layer 351-3. The calculation unit 330 stores information specifying the amount of shift of the decimal point of the weights of the derived output layer 351-3 and information specifying the amount of shift of the decimal point of the bias in the shift amount information 354 of the storage unit 350.

(Step S8-2)
In the terminal device 300, the calculation unit 330 converts the weights and biases of the output layer 351-3 into a fixed-point format.
(Step S9-2)
In the terminal device 300, the calculation unit 330 creates an output layer 351a-3 including weights and biases converted into a fixed-point format. The calculation unit 330 replaces the output layer 351-3 of the learning model 352 with the created output layer 351a-3.
(Step S10-2)
In the terminal device 300, the receiving unit 320 obtains information specifying an input value input to the input unit 310, and receives the obtained information specifying the input value.

(Step S11-2)
In the terminal device 300, the acquisition unit 325 acquires information that specifies the amount of shift of the decimal point of each parameter of the input layer, the intermediate layer, and the output layer, which is stored in the shift amount information 354 of the storage unit 350.
(Step S12-2)
In the terminal device 300, the calculation unit 330 acquires information specifying the input value accepted by the acceptance unit 320. The calculation unit 330 converts the input value into a fixed-point format based on the acquired information specifying the input value and information specifying the amount of shift of the decimal point of the input value.
(Step S13-2)
In the terminal device 300, the calculation unit 330 acquires information specifying the amount of decimal point shift of each parameter of the input layer, the intermediate layer, and the output layer from the acquisition unit 325. The calculation unit 330 derives an output of the input layer 351a-1 of the quantized learning model 352a for the input value converted to the fixed-point format, based on the acquired information specifying the amount of decimal point shift of the weight of the input layer 351-1 and the information specifying the amount of decimal point shift of the bias.
The calculation unit 330 derives the amount of shift of the decimal point of the output of the input layer 351a-1 based on the derived output of the input layer 351a-1. The calculation unit 330 stores information specifying the amount of shift of the decimal point of the derived output of the input layer 351a-1, in other words, the amount of shift of the decimal point of the input value of the intermediate layer 351a-2, in shift amount information 354 of the storage unit 350.

(Step S14-2)
In the terminal device 300, the calculation unit 330 derives the output of the quantized intermediate layer 351a-2 of the learning model 352a corresponding to the output value of the input layer 351a-1 of the learning model 352a, based on the acquired information specifying the amount of decimal point shift of the weight of the intermediate layer 351a-2 and information specifying the amount of decimal point shift of the bias.
The calculation unit 330 derives the amount of shift of the decimal point of the output of the intermediate layer 351a-2 based on the derived output of the intermediate layer 351a-2. The calculation unit 330 stores information specifying the amount of shift of the decimal point of the derived output of the intermediate layer 351a-2, in other words, the amount of shift of the decimal point of the input value of the output layer 351a-3, in the shift amount information 354 of the storage unit 350.
(Step S15-2)
In the terminal device 300, the calculation unit 330 derives the output of the quantized output layer 351a-3 of the learning model 352a for the output value of the intermediate layer 351a-2 of the learning model 352a based on the information specifying the amount of decimal point shift of the weight of the acquired output layer 351a-3 and the information specifying the amount of decimal point shift of the bias.
The calculation unit 330 derives the amount of shift of the decimal point of the output of the output layer 351a-3 by performing calculations such as variance based on the derived output of the output layer 351a-3. The calculation unit 330 stores information specifying the amount of shift of the decimal point of the derived output of the output layer 351a-3 in the shift amount information 354 of the storage unit 350.

The accuracy of the final output value of the arithmetic device 100 according to this embodiment will be described.
Fig. 10 is a diagram for explaining an example of the accuracy of the arithmetic device 100 according to this embodiment. Fig. 10 shows an example of the accuracy of the final output value when the number of digits (bit width) n when converting an input value into a fixed-point format is set to 2 to 8, and when the final output value is the output value OU with the decimal point shifted, and when the output value OU is set to the final output value.
10, it can be seen that, for bit widths up to six digits, the same accuracy is obtained up to the first decimal place regardless of whether the decimal point of the output value OU is shifted or not. For bit widths of five digits, there is a difference in the first decimal place between when the decimal point of the output value OU is not shifted and when it is shifted. For bit widths of four digits or less, when the decimal point is not shifted, the accuracy is significantly reduced compared to when the decimal point of the output value OU is shifted.

In the above embodiment, a deep neural network type learning model has been described as an example of the learning model, but the present invention is not limited to this example. For example, a convolutional neural network type learning model may be used as the learning model. A learning model having a layer including a product-sum operation may be used.
In the above embodiment, an example of the learning model is described in Python, but this is not limited to this example. For example, an example of the learning model may be described in a library such as Pytorch or Tensorflow.
In the above embodiment, the input value is, for example, a 32-bit floating-point number, but the input value is not limited to this example. For example, the input value may be a floating-point number other than 32 bits, such as a 16-bit floating-point number.
In the above-described embodiment, each of the learning model 152 and the learning model 352 may include multiple hidden layers. When multiple hidden layers are included, the weight decimal point shift amount, bias decimal point shift amount, and output decimal point shift amount (input decimal point shift amount) are derived for each of the multiple hidden layers.
In the above-mentioned embodiment, the arithmetic device 100 may select whether or not to shift the decimal point of the output value OU based on the output value OU and the shift amount osf of the decimal point of the output. For example, as shown in Fig. 10, when the bit width n is 6 to 8, the difference in precision between the case where the decimal point of the output value OU is shifted and the case where the output value OU is the final output value is small, so there is no need to shift the decimal point of the output value OU. By configuring in this way, the hardware for shifting the decimal point of the output value OU can be reduced.

According to the computing device of the embodiment, the computing device 100 includes an acquisition unit 125 that acquires information specifying the amount of decimal point shift of the parameters of a neural network-type learning model 152, a computation unit 130 that calculates an output value of the learning model 152 for an input value based on the information specifying the amount of decimal point shift of the parameters acquired by the acquisition unit 125, and an output unit 140 that shifts the decimal point of the output value based on the output value and the amount of decimal point shift of the output value, and outputs the output value with the shifted decimal point, and the input value is in a fixed-point format.
By configuring in this manner, the calculation device 100 calculates the output value of the learning model 152 for an input value in fixed-point format based on information specifying the amount of decimal point shift of the parameters of the learning model 152, shifts the decimal point of the output value based on the output value and the amount of decimal point shift of the output value, and outputs the output value with the shifted decimal point.
Since the decimal point of the output value can be shifted based on the output value and the amount of shift of the decimal point of the output value, the calculation accuracy can be improved when the input value is converted to a fixed-point format. Therefore, accurate accuracy can be obtained before implementing it in hardware. Since it is possible to perform emulation equivalent to hardware, it can be used for debugging hardware. It is possible to check for overflow at the emulation stage. It can be used in conjunction with commonly used deep neural network frameworks.

Moreover, the parameters include parameters for each of the multiple layers. With this configuration, the arithmetic device 100 can calculate an output value of the learning model 152 for an input value based on the parameters for each of the multiple layers of the learning model 152, and therefore, when the input value is converted into a fixed-point format, the arithmetic accuracy can be improved.
In addition, the parameters include weights and outputs. With this configuration, the calculation device 100 can calculate the output value of the learning model 152 for the input value based on information specifying the amount of decimal point shift of each of the multiple weights of the learning model 152 and the amount of decimal point shift of each of the multiple layers of the output, and therefore can improve the calculation accuracy when the input value is converted into a fixed-point format.
In addition, the parameters further include a bias. With this configuration, the calculation device 100 can calculate the output value of the learning model 152 for the input value based on information specifying the amount of shift of the decimal point of each of the multiple biases of the learning model 152, and therefore can improve the calculation accuracy when the input value is converted into a fixed-point format.

Moreover, the parameters are in a fixed-point format. With this configuration, the arithmetic device 100 calculates the output value of the learning model 152 for the input value in the fixed-point format based on information specifying the shift amount of the decimal point of the quantized learning model 152 parameters, shifts the decimal point of the output value based on the output value and the shift amount of the decimal point of the output value, and outputs the output value with the shifted decimal point. Therefore, accurate accuracy can be obtained before being implemented in hardware. Since it is possible to perform emulation equivalent to hardware, it can be used for debugging hardware. It can be used in conjunction with a commonly used deep neural network framework.
In addition, the learning model is created by converting parameters into a fixed-point format based on a trained model in the form of a deep neural network. By configuring in this way, it is possible to prepare a learning model from a trained model.
Furthermore, the calculation unit 130 performs a MAC (multiply-accumulate) calculation on the input value based on the parameters. With this configuration, it is possible to calculate the output value of the learning model 152 for the input value by performing a MAC calculation on the input value.

Because of their wide range of applications, systems using neural networks are used for a variety of purposes, particularly in edge devices. However, because neural networks require a large amount of calculation, quantization to reduce the amount of calculation is essential to run them on edge devices. For this reason, applications of the arithmetic device according to the embodiment include quantization of neural networks used in surveillance cameras, embedded cameras, and the like. In particular, the arithmetic method is capable of emulating actual hardware, allowing for quick verification. In addition, it is compatible with any neural network, making it possible to try quantization on new models as well.

(Modification of the embodiment)
[Overall configuration]
FIG. 11 is a configuration diagram of a calculation device according to a modified example of the embodiment.
The user U performs an operation to input an input value to the arithmetic device 100a. An example of the input value is a floating-point format. The following will continue with the explanation of the case where the input value is in the floating-point format.
The arithmetic device 100a accepts an input value input by a user U. The arithmetic device 100a converts the accepted input value from floating-point format to fixed-point format. The number of digits (bit width) n when converting to the fixed-point format is set in advance. Here, in the arithmetic device 100a according to the modified example of this embodiment, a plurality of bit widths n are set. Specifically, n=6, 7, 8, 9, 10, 16, etc.

The arithmetic device 100a acquires parameters included in a learning model based on the learning model in the form of a deep neural network. An example of the parameters is a floating-point format. Below, the explanation will continue for the case where the input value is in floating-point format. The arithmetic device 100a converts the acquired parameters to a fixed-point format. The number of digits (bit width) n when converting to the fixed-point format is set in advance.

The arithmetic device 100a derives an output value ou for each of a plurality of input values by calculating the output value ou for each of the plurality of input values based on the plurality of input values and parameters after conversion to a fixed-point format. The arithmetic device 100a derives a final output value by shifting the decimal point of each of the plurality of output values ou based on the derived plurality of output values ou and the shift amount osf of the decimal point of the output value.
The arithmetic device 100a derives the precision of each of a plurality of final output values. The arithmetic device 100a outputs the bit width and the precision in association with each other for each of a plurality of bit widths n when converting the input value into the fixed-point format.
The arithmetic device 100a will now be described.

[Calculation device 100a]
The arithmetic device 100a is realized as a smartphone, a mobile terminal, a personal computer, a tablet terminal device, or other information processing equipment. The arithmetic device 100a includes, for example, an input unit 110, a reception unit 120, an acquisition unit 125, a arithmetic unit 130a, a derivation unit 135, an output unit 140a, and a storage unit 150.

The calculation unit 130a acquires information specifying the amount of decimal point shift for each parameter of the input layer, intermediate layer, and output layer acquired by the acquisition unit 125, and information specifying the amount of decimal point shift for each output of the input layer, intermediate layer, and output layer. The calculation unit 130a calculates the output value of the learning model 152 for the input value based on the acquired information specifying the amount of decimal point shift for each parameter of the input layer, intermediate layer, and output layer, and information specifying the amount of decimal point shift for each output of the input layer, intermediate layer, and output layer.

The MAC calculation performed by the calculation unit 130a can be applied to Fig. 5. The calculation unit 130a includes a multiplier 10, an adder 20, and an adder 30, and performs a multiply-and-accumulate calculation and an addition calculation. The calculation unit 130a acquires an input value in and information specifying the amount of shift of the decimal point of the parameter. Here, the input value in is converted to a fixed-point format based on the information specifying the amount of shift of the decimal point of the input value.
The calculation unit 130a shifts the decimal point of the weight WE based on information specifying the amount of shift of the decimal point of the parameter. In the calculation unit 130a, the multiplier 10 multiplies the input value in by the weight WE with the decimal point shifted. The adder 20 adds the result of multiplying the input value in by the weight WE with the decimal point shifted.

The adder 30 derives the final output by shifting the decimal point of the output value OU based on the output value OU of the adder 20 and the shift amount osf of the decimal point of the output. Specifically, the adder 30 adds 0.5 to the output value OU.
The derivation unit 135 acquires the final output for each of the multiple bit widths n from the calculation unit 130a. The derivation unit 135 derives the accuracy of the final output based on the multiple acquired final outputs.
The output unit 140a acquires information associating each of a plurality of bit widths obtained when the input value is converted into a fixed-point format with the precision of the final output from the derivation unit 135. The output unit 140a outputs the information associating each of the acquired plurality of bit widths with the precision of the final output.

(Operation of arithmetic device 100a)
FIG. 12 is a flowchart illustrating an example of the operation of a calculation device according to a modified example of the embodiment.
(Step S1-3)
In the arithmetic device 100a, the receiving unit 120 obtains information specifying an input value input to the input unit 110, and receives the obtained information specifying the input value.
(Step S2-3)
In the calculation device 100a, the acquisition unit 125 acquires information specifying the shift amount of the decimal point of the input value stored in the shift amount information 154 of the storage unit 150, information specifying the shift amount of the decimal point of each parameter of the input layer, the intermediate layer, and the output layer, and information specifying the shift amount of the decimal point of each output of the input layer, the intermediate layer, and the output layer.
(Step S3-3)
In the calculation device 100a, the calculation unit 130a sets the bit width.

(Step S4-3)
In the arithmetic device 100a, the arithmetic unit 130a acquires information specifying an input value accepted by the accepting unit 120. The arithmetic unit 130a converts the input value into a fixed-point format based on the acquired information specifying the input value and the set bit width.
(Step S5-3)
In the calculation device 100a, the calculation unit 130a acquires information specifying the amount of decimal point shift of each parameter of the input layer, the intermediate layer, and the output layer, and information specifying the amount of decimal point shift of each output of the input layer, the intermediate layer, and the output layer acquired by the acquisition unit 125. The calculation unit 130a calculates an output value of the learning model 152 for an input value, based on the acquired information specifying the amount of decimal point shift of each parameter of the input layer, the intermediate layer, and the output layer, and information specifying the amount of decimal point shift of each output of the input layer, the intermediate layer, and the output layer.

(Step S6-3)
In the calculation device 100a, the calculation section 130a derives the final output by shifting the decimal point of the output value OU based on the output value and the shift amount osf of the decimal point of the output.
(Step S7-3)
In the calculation device 100a, the calculation unit 130a judges whether or not the final output has been derived for all bit widths. If there is a bit width for which the final output has not been derived, the process returns to step S3-3.
(Step S8-3)
In the computing device 100a, when it is determined that the computing unit 130a has derived the final output for all bit widths, the derivation unit 135 acquires the final output for each of the multiple bit widths n from the computing unit 130a. The derivation unit 135 derives the accuracy of the final output based on the multiple acquired final outputs.

(Step S9-3)
In the arithmetic device 100a, the output unit 140a acquires information associating each of a plurality of bit widths obtained when an input value is converted into a fixed-point format with the precision of the final output from the derivation unit 135. The output unit 140a outputs the information associating each of the acquired plurality of bit widths with the precision of the final output.
12, the case where the accuracy of the final output is derived after the final output is derived for all bit widths has been described, but this is not limiting. For example, the accuracy of the final output may be derived every time the final output is derived.

According to the arithmetic device 100a according to the modified embodiment, the arithmetic device 100a is the arithmetic device 100 described above, in which the arithmetic unit 130a calculates output values of a learning model for a plurality of input values converted into fixed-point format with different bit widths, and shifts the decimal point of each of the calculated output values based on each of the calculated output values and the shift amount of the decimal point of the output value. The arithmetic device 100a includes a derivation unit that derives the precision of the result of shifting the decimal point of each of the plurality of output values.
By configuring in this way, the arithmetic device 100a can derive the accuracy of the result of shifting the decimal point of the output value of the learning model for each of a plurality of bit widths. Therefore, if the accuracy required for the learning model is set, it is possible to obtain a bit width that satisfies the required accuracy. In other words, if a plurality of learning models are prepared, it is possible to obtain an allowable bit width. In addition, it is possible to know the degree to which accuracy is reduced by quantization at the time of emulation.

13 is a diagram showing an example of a result of deriving the accuracy of the output of a calculation device according to a modified example of this embodiment, in which the horizontal axis indicates the format of the input value, and the vertical axis indicates the accuracy.
"FP32" is 32-bit floating point format, "FP16" is 16-bit floating point format, "INT16" is 16-bit fixed point format, "INT10" is 10-bit fixed point format, "INT9" is 9-bit fixed point format, "INT8" is 8-bit fixed point format, "INT7" is 7-bit fixed point format, and "INT6" is 6-bit fixed point format.
Accuracy was evaluated using a learning model trained on ImageNet and CIFAR 100. Here, AlexNet, VGG16, and MobileNet-v2 are shown as examples of the learning model.

According to FIG. 13, in the case of AlexNet (Innet_AlexNet) trained with ImageNet, the accuracy is lower for INT8, INT7, and INT6 compared to FP32, FP16, INT16, INT10, and INT9.
In the case of VGG16 (Innet_VGG16) trained with ImageNet, it can be seen that the accuracy is lower for INT7 and INT6 compared to FP32, FP16, INT16, INT10, INT9, and INT8.
In the case of MobileNet-v2 (Innet_MobileNet-v2) trained with ImageNet, it can be seen that the accuracy is lower for INT9, INT8, INT7, and INT6 compared to FP32, FP16, INT16, and INT10.

Furthermore, according to FIG. 13, in the case of AlexNet trained with CIFAR100 (C100_AlexNet), the accuracy is lower for INT8, INT7, and INT6 compared to FP32, FP16, INT16, INT10, and INT9.
In the case of VGG16 (C100_VGG16) trained with CIFAR100, it can be seen that the accuracy is lower for INT7 and INT6 compared to FP32, FP16, INT16, INT10, INT9, and INT8.
In the case of MobileNet-v2 (C100_MobileNet-v2) trained with CIFAR100, it can be seen that the accuracy is lower for INT8, INT7, and INT6 compared to FP32, FP16, INT16, INT10, and INT9.
Furthermore, according to FIG. 13, when GAP (Global average Pooling) is applied to AlexNet trained with CIFAR100 (C100_AlexNet_GAP), the accuracy is lower for INT8, INT7, and INT6 compared to FP32, FP16, INT16, INT10, and INT9.
When GAP is applied to VGG16 trained with CIFAR100 (C100_VGG16_GAP), it can be seen that the accuracy is lower for INT7 and INT6 compared to FP32, FP16, INT16, INT10, INT9, and INT8.
According to FIG. 13, when a learning model is prepared, it is possible to find how many bits are permissible based on the permissible accuracy.

Although the embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents described in the claims.
The above-mentioned arithmetic device 100, terminal device 300, and arithmetic device 100a each have a computer inside. The process of each of the above-mentioned devices is stored in a computer-readable recording medium in the form of a program, and the above-mentioned processes are performed by the computer reading and executing this program. Here, the computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Also, the computer program may be distributed to a computer via a communication line, and the computer that receives the program may execute the program.
The program may be for realizing part of the functions described above.
Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program).

10...multiplier, 20, 30...adder, 100, 100a...arithmetic unit, 110...input unit, 120...reception unit, 125...acquisition unit, 130, 130a...arithmetic unit, 135...derivation unit, 140, 140a...output unit, 150...storage unit, 151-1...input layer, 151-2...intermediate layer, 151-3...output layer, 152...learning model, 154...shift amount information, 200, 205...input value, 210, 215...sign bit, 220...exponent part, 225...integer part, 230...mantissa part, 235...fractional part, 245...decimal point, 300...terminal device, 310...input unit, 320...reception unit, 325...acquisition unit, 330: Calculation unit; 340: Output unit; 350: Storage unit; 352, 352a: Learning model; 354: Shift amount information; 351-1, 351a-1: Input layer; 351-2, 351a-2: Intermediate layer; 351-3, 351a-3: Output layer

Claims (8)

An acquisition unit that acquires information that specifies a decimal point shift amount of a parameter of a neural network type learning model;
a calculation unit that calculates output values of the learning model for a plurality of input values converted to a fixed-point format with different bit widths based on information specifying the amount of shift of a decimal point of the parameter acquired by the acquisition unit, and shifts the decimal point of each of the calculated output values based on each of the calculated output values and the amount of shift of the decimal point of the output value ;
a derivation unit that derives a precision of a result of shifting a decimal point of each of the plurality of output values;
Equipped with
The input values are in fixed-point format. The computing device of claim 1, wherein the parameters include parameters for each of a plurality of layers. The computing device of claim 2, wherein the parameters include weights and outputs. The computing device of claim 3, wherein the parameters further include a bias. The arithmetic device according to any one of claims 1 to 4, wherein the parameters are in fixed-point format. The arithmetic device according to any one of claims 1 to 5, wherein the learning model is created by converting parameters into a fixed-point format based on a learned model in a deep neural network format. The arithmetic device according to any one of claims 1 to 6, wherein the arithmetic unit performs a MAC (multiply-accumulate) operation on the input value based on the parameters. Obtaining information specifying a decimal point shift amount of a parameter of a neural network type learning model;
A step of obtaining output values of the learning model for a plurality of input values converted to a fixed-point format with different bit widths based on information specifying the amount of shift of the decimal point of the parameter obtained in the obtaining step, and shifting the decimal point of each of the obtained plurality of output values based on each of the obtained plurality of output values and the amount of shift of the decimal point of the output value;
deriving a precision resulting from shifting the decimal point of each of the plurality of output values;
having
A computer implemented method of arithmetic, wherein the input values are in fixed point format.

Images