JP7747585B2 - arithmetic device - Google Patents

Description

This disclosure relates to a computing device.

It is known that the power consumption of a computing device that performs arithmetic processing to execute a neural network model increases when the computational load is increased. DVFS (Dynamic Voltage and Frequency Scaling) control is known, which reduces power consumption by controlling voltage and clock frequency. Patent Document 1 also discloses that the power consumption when executing a neural network model that performs calculations using a low-bit weight parameter (1 bit) is lower than the power consumption when executing a neural network model that performs calculations using a high-bit weight parameter (16 to 32 bits) represented by floating point.

Japanese Patent Application Laid-Open No. 2021-103441 Japanese Patent Application Laid-Open No. 2020-74099 Japanese Patent Application Laid-Open No. 2022-507704

When using DVFS control to reduce power consumption, the time required for neural network model calculations can increase significantly. Neural network models using low bit-weight parameters can reduce power consumption while minimizing increases in calculation time, but the structure of the neural network model dictates the hardware configuration, making it difficult to easily switch neural network models.

According to one embodiment of the present disclosure, there is provided a computing device (100, 100B, 100C, 100D). The computing device includes: a computing unit (10) that executes a neural network model to perform computation; an acquisition unit (20) that acquires, as a determination value, one or more of the power consumption of the computing device, the temperature of the computing device, and the availability rate of the computing unit; a switching unit (30, 30B) that can switch the weight of the neural network model used in the computation between a high bit weight and a low bit weight having a lower number of bits than the high bit weight; and a storage unit (40) that stores the high bit weight and the low bit weight in advance . The switching unit switches the weight from the high bit weight to the low bit weight when the determination value is equal to or greater than a predetermined first threshold.
According to another aspect of the present disclosure, there is provided a computing device including: a computing unit that executes a neural network model to perform a computation process; an acquisition unit that acquires, as a determination value, one or more of the power consumption of the computing unit, the temperature of the computing unit, and an availability rate of the computing unit; a switching unit that can switch a weight of the neural network model used in the computation process between a high bit weight and a low bit weight having a bit number lower than the high bit weight; and a storage unit that stores the high bit weight. When the determination value is equal to or greater than a predetermined first threshold, the switching unit switches the weight from the high bit weight to the low bit weight and quantizes the high bit weight to switch it to the low bit weight.
According to another aspect of the present disclosure, there is provided a computing device including: a computing unit that executes a neural network model to perform a computation; an acquisition unit that acquires, as a determination value, one or more of the power consumption of the computing unit, the temperature of the computing unit, and an availability rate of the computing unit; and a switching unit that can switch a weight of the neural network model used in the computation between a high bit weight and a low bit weight having a bit number lower than the high bit weight. The computing unit performs, in parallel, a first computation process that executes a first neural network model with the high bit weight and a second computation process that executes a second neural network model with the low bit weight, and the switching unit switches the weight from the high bit weight to the low bit weight and switches the weight of the first neural network model from the high bit weight to the low bit weight when the determination value is equal to or greater than a predetermined first threshold.
According to another aspect of the present disclosure, there is provided a computing device including: a computing unit that executes a neural network model to perform computational processing; an acquisition unit that acquires, as a determination value, one or more of the power consumption of the computing unit, the temperature of the computing unit, and an availability rate of the computing unit; and a switching unit that can switch a weight of the neural network model used in the computational processing between a high bit weight and a low bit weight having a bit number lower than the high bit weight. The switching unit switches the weight from the high bit weight to the low bit weight when the determination value is equal to or greater than a predetermined first threshold, and switches the weight from the low bit weight to the high bit weight when the determination value is equal to or less than a predetermined third threshold that is lower than the first threshold.

With this type of computing device, the switching unit can switch the weights of the neural network model between high-bit weights and low-bit weights depending on power consumption, etc. This allows calculations to be performed with the desired accuracy, and when power consumption, etc. increases, power consumption can be reduced while preventing an increase in the time required for calculations.

FIG. 2 is an explanatory diagram showing an outline of the configuration of a computing device. FIG. 10 is an explanatory diagram of a calculation process using a neural network model. 10 is a flowchart showing an example of a calculation process. FIG. 10 is an explanatory diagram illustrating an outline of the configuration of a calculation device according to a second embodiment. 10 is a flowchart illustrating an example of a calculation process in the second embodiment. FIG. 10 is an explanatory diagram illustrating an outline of the configuration of a calculation device according to a third embodiment. FIG. 10 is an explanatory diagram illustrating an outline of the configuration of a calculation device according to a fourth embodiment.

A. First embodiment:
As shown in FIG. 1 , the arithmetic device 100 includes a calculation unit 10, an acquisition unit 20, a switching unit 30, and a storage unit 40. The arithmetic device 100 is a device that executes a neural network model to perform arithmetic processing. The arithmetic device 100 includes a microcomputer configured with a central processing unit (CPU), RAM, and ROM, and realizes the functions of each of these units by the microcomputer executing a pre-installed program. In this embodiment, the arithmetic device 100 is a SoC (System on a Chip). The arithmetic device 100 is mounted on, for example, a vehicle and performs arithmetic processing for image analysis. The arithmetic device 100 may also be implemented on a cloud.

The calculation unit 10 executes a neural network model to perform calculations. As shown in FIG. 2, the neural network comprises an input layer L1, multiple intermediate layers L2, and an output layer L3. The number of intermediate layers L2 can be determined arbitrarily. The input layer L1 is a layer to which information is input. The intermediate layer L2 is a layer that calculates feature values based on information transmitted from the input layer L1. The output layer L3 is a layer that outputs results based on information transmitted from the intermediate layer L2. Each layer includes multiple nodes. Edges connecting nodes in adjacent layers are weighted. The calculation unit 10 performs calculations including a product-sum operation, for example, to multiply node N1 by weight W11, multiply node N2 by weight W12, and multiply node N3 by weight W13, and then add these results together to find the value of node N4.

The storage unit 40 (see Figure 1) stores weights by which values input to nodes in the neural network model are multiplied. In this embodiment, the storage unit 40 stores a predetermined high bit weight Wh and a predetermined low bit weight Wl, which has a lower number of bits than the high bit weight Wh. In this embodiment, the storage unit 40 stores optimized 32-bit floating-point data as the high bit weight Wh, and 4-bit integer data as the low bit weight Wl. The high bit weight Wh and the low bit weight Wl are each optimized.

The acquisition unit 20 acquires a judgment value related to the power consumption of the calculation device 100. In this embodiment, the acquisition unit 20 acquires the temperature value of the calculation device 100 from the sensor 200 as the judgment value. The acquisition unit 20 can estimate that the higher the temperature of the calculation device 100, the higher the power consumption of the calculation device 100.

The switching unit 30 switches the weights of the neural network model used in the calculation process. The switching unit 30 can switch the weights of the neural network model between high-bit weight Wh and low-bit weight Wl.

The switching process shown in Figure 3 is a process in which the switching unit 30 switches the weights of the neural network model. This process is repeatedly executed when the calculation unit 10 executes calculation processing. In step S100, the acquisition unit 20 acquires a judgment value.

In step S110, the switching unit 30 determines whether the judgment value acquired in step S100 is equal to or greater than the first threshold. If the judgment value is equal to or greater than the first threshold, the switching unit 30 proceeds to step S120 and switches the weight from the high bit weight Wh to the low bit weight Wl. On the other hand, if the judgment value is less than the first threshold, the switching unit 30 ends the calculation process without switching the weight.

According to the computing device 100 of this embodiment described above, the switching unit 30 can switch the weights of the neural network model between high bit weight Wh and low bit weight Wl depending on power consumption, etc. As a result, computations can normally be performed with the desired accuracy, and when power consumption, etc. increases, power consumption can be reduced while preventing an increase in the time required for computation.

In addition, because the switching unit 30 switches to an already optimized low bit weight Wl, the accuracy of calculations using the low bit weight Wl can be improved compared to when switching to an unoptimized low bit weight Wl.

B. Second embodiment:
The arithmetic device 100B of the second embodiment shown in Fig. 4 differs from the first embodiment in that the storage unit 40 stores only the high bit weight Wh, but the other configurations are the same. The switching process in the second embodiment shown in Fig. 5 differs from the first embodiment in that the switching unit 30B performs post-quantization on the high bit weight Wh to switch it to a low bit weight in step S125, but the processes in steps S100 and S110 are the same.

In step S125, the switching unit 30B performs post-quantization on the high bit weight Wh to switch it to a low bit weight. More specifically, the switching unit 30B reduces the number of bits of the mantissa represented by 23 bits of the 32-bit high bit weight Wh to 7 bits, switching it to a low bit weight of 16-bit floating-point data.

According to the arithmetic device 100 of the second embodiment described above, the switching unit 30B performs post-quantization on the high bit weight Wh and switches it to a low bit weight, so the number of bits for the low bit weight can be determined arbitrarily.

C. Third embodiment:
6 is different from the first embodiment in that it includes a control unit 50. The third embodiment differs from the first embodiment in that the control unit 50 controls the voltage and clock frequency of the calculation device 100.

When the judgment value rises above a predetermined second threshold, the control unit 50 controls the voltage and clock frequency of the calculation device 100 to be higher than when the judgment value is below the second threshold. The second threshold is a value smaller than the first threshold. In other words, the control unit 50 performs DVFS control.

According to the arithmetic device 100C of the third embodiment described above, when the judgment value rises above the second threshold, the control unit 50 controls the voltage and clock frequency of the arithmetic device 100 to be higher than when the judgment value is below the second threshold, thereby reducing power consumption.

D. Fourth embodiment:
7 is different from the first embodiment in that it includes an estimation unit 60. The fourth embodiment is different from the first embodiment in that the switching unit 30 switches the weight from the high bit weight Wh to the low bit weight Wl even when the amount of calculation estimated by the estimation unit 60 is equal to or greater than a predetermined threshold.

The estimation unit 60 estimates the amount of calculation required for the requested calculation process.

In this embodiment, the switching unit 30 switches the weight of the neural network model from a high bit weight Wh to a low bit weight Wl when the amount of calculation is equal to or greater than a predetermined threshold. Note that if the judgment value is equal to or greater than a first threshold and the weight of the neural network model has already been switched from a high bit weight Wh to a low bit weight Wl, the switching unit 30 may switch to a weight with an even lower number of bits.

The computing device 100D of the fourth embodiment described above can reduce power consumption when performing computational processing that requires a large amount of computation.

E. Fifth embodiment:
The fifth embodiment differs from the first embodiment in that the calculation unit 10 executes a plurality of neural network models to perform calculation processing in parallel. The configuration of the calculation device 100 of the fifth embodiment is the same as the configuration of the calculation device 100 of the second embodiment, so a description of the configuration of the calculation device 100 will be omitted. The fifth embodiment also has substantially the same effects as the second embodiment.

The calculation unit 10 performs a first calculation process in parallel, which executes a first neural network model with a high bit weight Wh, and a second calculation process in parallel, which executes a second neural network model with a low bit weight Wl. In other words, the calculation unit 10 has a redundant configuration. The calculation device 100 typically uses the calculation results obtained by the calculation unit 10 executing the first neural network model as the calculation results of the calculation unit 10.

In this embodiment, if the judgment value is equal to or greater than the first threshold, the switching unit 30 switches the weight of the first neural network model, which is the main model, from the high-bit weight Wh to the low-bit weight Wl in step S120 (see Figure 3).

F. Other Embodiments:
(F1) In the above-described embodiment, the acquisition unit 20 acquires the temperature of the arithmetic device 100 as the judgment value. However, the acquisition unit 20 may acquire one or more of the power consumption of the arithmetic device 100, the temperature of the arithmetic device 100, and the operation rate of the arithmetic device 10 as the judgment value. The operation rate of the arithmetic device 10 is, for example, the memory usage rate due to multiply-accumulate (MAC) operations of the arithmetic device 10. When multiple types of values are acquired as the judgment value, the first threshold is set for each type of judgment value.

(F2) In the above-described embodiment, the switching unit 30 may switch the weight from the low bit weight Wl to the high bit weight Wh when the judgment value falls below the first threshold. Furthermore, the switching unit 30 may switch the weight from the low bit weight Wl to the high bit weight Wh when the judgment value is below a predetermined third threshold that is lower than the first threshold. According to this embodiment, when the judgment value rises and falls near the first threshold, the switching unit 30 can avoid switching the weight more frequently than when the switching unit 30 switches the weight from the low bit weight Wl to the high bit weight Wh when the judgment value falls to the first threshold.

(F3) In the above-described embodiment, the memory unit 40 is provided in the computing device 100. However, this is not limiting, and the memory unit 40 may also be an external storage area.

(F4) In the second embodiment described above, the switching unit 30B may determine the number of low-bit-weight bits in accordance with the judgment value. More specifically, the switching unit 30B determines the number of low-bit-weight bits using a table or function prepared in advance. For example, the switching unit 30B reduces the number of low-bit-weight bits as the judgment value increases.

(F5) In the second embodiment described above, the switching unit 30B reduces the number of bits in the mantissa part of the 32-bit high bit weight Wh and switches it to a 16-bit low bit weight. This is not a limitation, and the switching unit 30B may perform post-quantization, for example, using a histogram of the high bit weight Wh. In this case, the storage unit 40 stores the histogram of the high bit weight Wh. More specifically, the switching unit 30B performs post-quantization using the width of the histogram interval calculated by the following equation (1). Note that the time required for post-quantization by the switching unit 30B is approximately a few milliseconds, which does not significantly affect the time required for the calculation processing by the calculation unit 10.

S=A/2 ^(n-1) ...(1)
Here, S is the width of the histogram interval, A is the difference between the minimum and maximum values of the histogram, and n is the number of low-order bits, which can be determined arbitrarily.

(F6) In the third embodiment described above, the estimation unit 60 controls the voltage and clock frequency of the calculation device 100. However, this is not limiting, and the estimation unit 60 may control at least one of the voltage and clock frequency of the calculation device 100.

(F7) In the fourth embodiment described above, the switching unit 30 may switch the weight of the neural network model from high bit weight Wh to low bit weight Wl when the amount of calculation is equal to or greater than a threshold and the control unit 50 cannot perform DVFS control. In this case, the calculation device 100D includes a control unit 50, and the control unit 50 performs DVFS control when the amount of calculation is equal to or greater than a threshold.

This disclosure is not limited to the above-described embodiments, and can be realized in various configurations without departing from its spirit. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention section can be replaced or combined as appropriate to solve the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate.

10... Calculation unit, 20... Acquisition unit, 30, 30B... Switching unit, 40... Memory unit, 50... Control unit, 60... Estimation unit, 100, 100B, 100C, 100D... Calculation device, 200... Sensor

Claims (7)

A computing device (100, 100B, 100C, 100D),
A calculation unit (10) that executes a neural network model and performs calculation processing;
an acquisition unit (20) that acquires one or more of the power consumption of the arithmetic unit, the temperature of the arithmetic unit, and the operating rate of the arithmetic unit as a judgment value;
a switching unit (30, 30B) capable of switching the weight of the neural network model used in the arithmetic processing between a high bit weight and a low bit weight having a bit number lower than the high bit weight;
a storage unit (40) for storing the high bit weight and the low bit weight in advance ,
The switching unit switches the weight from the high bit weight to the low bit weight when the determination value is equal to or greater than a predetermined first threshold. A computing device,
a calculation unit that executes a neural network model and performs calculations;
an acquisition unit that acquires, as a determination value, one or more of the power consumption of the arithmetic unit, the temperature of the arithmetic unit, and the availability rate of the arithmetic unit;
a switching unit that can switch the weight of the neural network model used in the calculation process between a high bit weight and a low bit weight having a bit number lower than the high bit weight;
a storage unit that stores the high bit weight ;
The switching unit is
When the determination value is equal to or greater than a predetermined first threshold, the weight is switched from the high bit weight to the low bit weight ;
A computing device that performs quantization on the high bit weights and switches to the low bit weights . 3. The computing device according to claim 2 ,
The switching unit determines the number of low-bit-weight bits in accordance with the determination value. The arithmetic device according to claim 1 or 2 , further comprising:
A computing device comprising a control unit (50) that controls at least one of the voltage and clock frequency of the computing device to be higher when the judgment value rises to or above a second threshold value that is smaller than the first threshold value than when the value is less than the second threshold value. 2. The computing device according to claim 1 , further comprising:
An estimation unit (60) for estimating the amount of calculation in the calculation processing,
The switching unit switches the weight from the high bit weight to the low bit weight when the amount of calculation is equal to or greater than a predetermined threshold. A computing device,
a calculation unit that executes a neural network model and performs calculations;
an acquisition unit that acquires, as a determination value, one or more of the power consumption of the arithmetic unit, the temperature of the arithmetic unit, and the availability rate of the arithmetic unit;
a switching unit that can switch the weight of the neural network model used in the calculation process between a high bit weight and a low bit weight having a number of bits lower than the high bit weight,
the calculation unit performs a first calculation process for executing the first neural network model with a high bit weight and a second calculation process for executing the second neural network model with a low bit weight in parallel;
The switching unit is
When the determination value is equal to or greater than a predetermined first threshold, the weight is switched from the high bit weight to the low bit weight ;
A computing device that switches the weights of the first neural network model from the high-bit weights to the low-bit weights . A computing device,
a calculation unit that executes a neural network model and performs calculations;
an acquisition unit that acquires, as a determination value, one or more of the power consumption of the arithmetic unit, the temperature of the arithmetic unit, and the availability rate of the arithmetic unit;
a switching unit that can switch the weight of the neural network model used in the calculation process between a high bit weight and a low bit weight having a number of bits lower than the high bit weight,
The switching unit is
When the determination value is equal to or greater than a predetermined first threshold, the weight is switched from the high bit weight to the low bit weight ;
When the determination value is equal to or less than a predetermined third threshold value that is lower than the first threshold value, the weight is switched from the low bit weight to the high bit weight .