JP7666333B2 - Processing Unit - Google Patents

Description

The present disclosure relates to a computing device.

In recent years, there has been progress in the development of deep neural networks, which are mathematical models that mimic the mechanisms of the nervous system. These deep neural networks have achieved high performance by increasing the amount of calculations and coefficient capacity.

JP 2018-26027 A

Squeeze-and-Excitation Network J. Hu, L. Shen, and G. Sun, arXiv:1709.01507, 2017 (https://arxiv.org/pdf/1709.01507.pdf)

For a processing device that performs arithmetic processing in a deep neural network, there is a demand for suppressing an increase in power consumption even when the amount of calculation increases. Therefore, various methods have been proposed for reducing the processing load of calculations in a deep neural network and suppressing an increase in power consumption, and one of such proposals is the method disclosed in the above-mentioned Patent Document 1. However, even when the technology disclosed in the above-mentioned Patent Document 1 is applied, the effect of suppressing an increase in power consumption is still insufficient, and further improvements are strongly required.

Therefore, this disclosure proposes a processing device that can further reduce the increase in power consumption of a processing device that performs calculations in a deep neural network.

According to the present disclosure, there is provided an arithmetic processing device comprising: an arithmetic circuit unit that reads out first and second data and repeatedly performs an arithmetic operation using the read out first and second data; a first memory unit that stores the first data; a second memory unit that stores the second data; a register unit that reads out from the first memory unit or the second memory unit and temporarily holds data that is repeatedly used in the arithmetic circuit unit among the first and second data; a first selector unit that selects data to be held in the register unit among the first and second data; and a control unit that controls the first selector unit based on at least one of the type of arithmetic operation, the amount of arithmetic operation, the type of the first and second memory units, the capacity of the first and second memory units, and the type of the arithmetic circuit unit.

FIG. 1 is a conceptual diagram (part 1) for explaining an overview of an example of a deep neural network according to an embodiment of the present disclosure. FIG. 2 is a conceptual diagram (part 2) for explaining an overview of an example of a deep neural network according to an embodiment of the present disclosure. FIG. 13 is a conceptual diagram (part 3) for explaining an overview of an example of a deep neural network according to an embodiment of the present disclosure. 1 is an example of a functional block diagram of a processing device 10 according to a first embodiment of the present disclosure. 2 is an example of a circuit block diagram of a calculation unit 120 according to the first embodiment of the present disclosure. FIG. 2 is an example of a circuit block diagram of a product adder 500 according to the first embodiment of the present disclosure. FIG. 4 is a flowchart illustrating an example of a calculation processing method according to the first embodiment of the present disclosure. FIG. 11 is an example of a circuit block diagram of a calculation unit 120a according to a second embodiment of the present disclosure. 1 is a hardware configuration diagram illustrating an example of a computer 1000 that realizes the functions of a processor 10 according to an embodiment of the present disclosure.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration will be given the same reference numerals to avoid duplicated explanations. In this specification and drawings, similar components in different embodiments may be distinguished by adding different alphabets after the same reference numerals. However, if there is no particular need to distinguish between similar components, only the same reference numerals will be used.

The explanation will be given in the following order.
1. History leading to the creation of the embodiment of the present disclosure 1.1 Background 1.2 History 1.3 Outline of the embodiment of the present disclosure 2. First embodiment 2.1. Example of functional configuration of the arithmetic processing device 10 2.2. Example of configuration of the arithmetic unit 120 2.3. Example of configuration of the multiply-accumulator 500 2.4. Arithmetic processing method 3. Second embodiment 4. Summary 5. Regarding hardware configuration 6. Supplementary information

<<1. How the embodiment of the present disclosure was created>>
1.1 Background
First, before describing the details of the embodiment of the present disclosure, the background and process by which the present inventor created the embodiment of the present disclosure will be described. First, an overview of a fully connected layer type deep neural network, which is an example of a general deep neural network, will be described with reference to Fig. 1. Fig. 1 is a conceptual diagram for explaining the overview of an example of a deep neural network according to an embodiment of the present disclosure.

In detail, in the image of processing in the fully connected layer type deep neural network shown in FIG. 1, the fully connected layer type deep neural network has at least an input layer, a fully connected layer (middle layer), and an output layer, i.e., performs hierarchical calculation processing. Note that in the fully connected layer type deep neural network, the fully connected layer is not limited to one, and multiple fully connected layers may be included in stages. In other words, the fully connected layer type deep neural network can have an input layer, multiple fully connected layers (middle layers), and an output layer.

In the fully connected layer type deep neural network, a plurality of input data X ₁ , X ₂ , ..., X _m are input to the input layer. Next, each input data X ₁ , X ₂ , ..., X _m is weighted by a different predetermined weighting coefficient and integrated to obtain the values of each node Y ₁ , Y ₂ , ..., Y _n-1 , Y _n of the fully connected layer (note that a weight matrix can be obtained by combining these multiple predetermined weighting coefficients) (Affine operation). Similarly, the values of each node Y ₁ , Y ₂ , ..., Y _n-1 , Y _n of the fully connected layer are weighted by a different predetermined weighting coefficient and integrated to obtain the output data Z of the output layer.

That is, in the example of the fully connected layer type deep neural network shown in FIG. 1, the calculations are repeatedly executed hierarchically so that data propagates (unidirectionally) from the input layer through the fully connected layer to the output layer. Furthermore, in the fully connected layer type deep neural network, since all the nodes between the layers are connected as shown in FIG. 1, the calculations are repeatedly executed using the same data (same input data). In other words, in the fully connected layer type deep neural network (Affine calculation), the calculations are executed so that the data (input data) is reused.

Next, an overview of a convolutional deep neural network, which is an example of a general deep neural network, will be described with reference to Figures 2 and 3. Figures 2 and 3 are conceptual diagrams for explaining the overview of an example of a deep neural network according to an embodiment of the present disclosure.

In the above-mentioned fully connected deep neural network, the data of each node in the same layer is treated independently of each other. In other words, the relationship between each node in the same layer is not taken into consideration when performing the calculation. In such a case, the accuracy of the output data obtained by the calculation may be poor depending on the characteristics of the input data being handled (for example, when the input data is image data). Therefore, in order to perform calculations while taking into consideration the relationship between each node in the same layer, a convolutional deep neural network was devised that performs convolution using a filter. Below, the convolutional deep neural network is explained using an example in which the convolutional deep neural network is applied to image processing (image recognition).

In detail, in the image of processing in the convolutional deep neural network shown in Figure 2, features of the image data are extracted by applying a filter (shown on the left side of Figure 2) to the input image data (Input in Figure 2).

Therefore, referring to FIG. 3, a method of extracting features from image data using a filter will be described. Specifically, as shown in the left end of FIG. 3, the filter is divided into a matrix, and a weighting factor is assigned to each divided area in advance. First, as shown in the center of FIG. 3, the filter is applied to a predetermined area of the image data (Input in FIG. 3) (each feature amount in the predetermined area of the image data is multiplied by the weighting factor of the filter at the corresponding position and integrated), to extract features in the predetermined area, and as shown on the right side of FIG. 3, the extracted features are assigned to a predetermined area of the output data (Output in FIG. 3). Furthermore, as shown in the center of FIG. 3, the filter is applied by sliding it sequentially on the image data, to extract features in each area of the image data, and the extracted features are assigned to a predetermined area of the output data. In this way, in a convolutional deep neural network, by using a filter, the feature amount of the image data can be convoluted (aggregated) into the output data while maintaining the positional relationship on the image data (Convolution operation).

Returning to Figure 2, as shown in the top part of Figure 2, a filter is applied to the upper left region of the input image data to extract features of the upper left region, and the extracted features are generated as output data (Output in Figure 2). As shown in Figure 2, the convolutional deep neural network repeatedly applies a filter to each region of the image data while sliding it, extracting features of each region of the image data. Furthermore, in the convolutional deep neural network, such extraction is repeatedly performed hierarchically.

That is, in the example of the convolutional deep neural network shown in Figures 2 and 3, a filter is used to perform calculations taking into account the vertical and horizontal positional relationships on the image data. Furthermore, in this example, calculations are performed using a repetitive filter (weighting coefficient). In other words, in the convolutional deep neural network (convolution calculation), calculations are performed such that coefficients are reused.

Furthermore, this convolutional deep neural network hierarchically combines extracted features while maintaining the vertical and horizontal positional relationships on the image data, thereby enabling it to demonstrate excellent performance in processes such as image recognition.

In addition, in an embodiment of the present disclosure, the processing in the deep neural network may be performed in combination with the above-mentioned Affine operation (fully connected deep neural network) and Convolution operation (convolutional deep neural network), and is not particularly limited.

1.2 Background
In recent years, deep neural networks as described above have achieved high performance by increasing the amount of calculation and the coefficient capacity. However, for a processing device that performs such processing, even if the amount of calculation increases, it is required to suppress the increase in power consumption.

In detail, examples of deep neural networks, such as fully connected deep neural networks and convolutional deep neural networks, are realized by a combination of product-sum operations called affine operations and convolution operations, as explained above. To execute such a large number of product-sum operations, many memory accesses are executed to read data used in the operations from a storage unit (memory), but in order to suppress an increase in power consumption of the arithmetic processing device, it is necessary to suppress an increase in the number of such memory accesses.

As explained above, in the Affine operation (fully connected deep neural network) and the Convolution operation (convolutional deep neural network), the same input data or coefficients are repeatedly used to perform the operation. Therefore, if the input data or coefficients once read from the storage unit (memory) can be reused when performing the Affine operation and the Convolution operation, the increase in the number of memory accesses can be suppressed.

Therefore, in the technology disclosed in the above Patent Document 1, input data and coefficients are read out from the storage unit (memory) one row at a time, each of which is temporarily stored in each register unit, and the stored input data and coefficients are output to the calculation unit. Specifically, in the technology disclosed in the above Patent Document 1, when performing a convolution calculation, the coefficients stored in the register unit are slid to perform multiplication and accumulation on the first row of input data read out from the storage unit (memory) to the register unit, and the intermediate results are stored in yet another register unit. Furthermore, in this technology, the multiplication and accumulation of the second row of input data and coefficients, the multiplication and accumulation of the third row of input data and coefficients, and so on are sequentially read out from the register unit and similar calculations are repeatedly performed.

In addition, in the technology disclosed in the above-mentioned Patent Document 1, a crossbar switch is provided between the register unit and the arithmetic unit, so that the input data and coefficients stored in the register unit can be shared among multiple arithmetic units. In this way, the number of times the register unit needs to be rewritten is reduced.

In this way, in the technology disclosed in Patent Document 1, the input data and coefficients read from the storage unit (memory) are temporarily stored in the register unit and reused, thereby suppressing an increase in the number of memory accesses. However, in the technology disclosed in Patent Document 1, three register units are provided for each calculation unit, so there is a concern that the circuit size will increase when there is a large amount of input data, coefficients, or output data for one row, or when the number of calculation units is increased to increase the number of parallel operations. In other words, even if the technology disclosed in Patent Document 1 can suppress an increase in the number of memory accesses, it is expected that it will be difficult to suppress an increase in power consumption due to the increase in circuit size.

Therefore, in consideration of the above-mentioned situation, the inventor has created a processing device according to an embodiment of the present disclosure that can further reduce the increase in power consumption of a processing device that performs calculations in a deep neural network.

1.3 Overview of the embodiment of the present disclosure
Next, an overview of a processing device according to an embodiment of the present disclosure created by the present inventor will be described.

The arithmetic processing device according to the embodiment of the present disclosure has two storage units (memories) (first storage unit, second storage unit) that store input data, coefficients, and other data (first data, second data), and a multiplier/accumulator (arithmetic circuit unit) that reads data from these storage units and repeatedly performs calculations using the read data. Furthermore, the arithmetic processing device has a register unit that reads data from the storage units and temporarily holds the data that is repeatedly used in the multiplier/accumulator, and a selector unit (first selector unit) that selects the data to be held in the register unit.

In an embodiment of the present disclosure, a selector unit installed before the register unit is controlled to select data (input data or coefficients) to be temporarily stored in the register unit so as to reduce the number of reads from the storage unit (number of memory accesses). In an embodiment of the present disclosure, the input data or coefficients temporarily stored in the register unit are repeatedly reused by the multiply-accumulator.

In addition, the processing device according to an embodiment of the present disclosure may further have a memory unit (third memory unit) for storing intermediate results (partial sums) calculated by the multiplier-accumulator.

In addition, in an embodiment of the present disclosure, the multiplier-accumulator may have a plurality of multipliers, an adder, and an accumulator. Furthermore, in an embodiment of the present disclosure, the multiplier-accumulator may have a selector unit (second selector unit) that selects whether to perform the next addition using the accumulated result temporarily held in the accumulator or the intermediate result (partial sum) stored in the memory unit (third memory unit).

In an embodiment of the present disclosure, the selection operation of each of the selector units described above is controlled based on at least one of the type of calculation (e.g., convolution calculation, affine calculation, etc.), the number of memory access bits determined from the amount of calculation (e.g., kernel size, output size, address, number of maps, number of bits), the type of memory unit (e.g., DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), etc.), the capacity of the memory unit, and the type of multiplier/accumulator.

Specifically, when the type of operation to be executed is an affine operation, a multiply-and-accumulate operation is executed between the input data and the coefficient, but in this multiply-and-accumulate operation, the same input data is used repeatedly as described above. Therefore, in the embodiment of the present disclosure, in the case of an affine operation, the repeatedly used input data is temporarily stored in a register unit, thereby suppressing an increase in the number of memory accesses, and as a result, an increase in power consumption can be further suppressed.

In addition, when the type of operation to be performed is a convolution operation, a multiplication and accumulation operation between the input data and the coefficients is performed, but in this multiplication and accumulation operation, as described above, the same input data and coefficients are repeatedly used. Therefore, in the embodiment of the present disclosure, in the case of a convolution operation, data that can be temporarily stored in the register unit and repeatedly used to minimize the number of reads (memory access counts) from the storage unit (memory) is selected and temporarily stored in the register unit. As a result, in the embodiment of the present disclosure, it is possible to suppress an increase in the number of memory accesses, and thus to further suppress an increase in power consumption.

More specifically, when the type of calculation to be performed is a convolution calculation, in an embodiment of the present disclosure, data is selected based on the amount of calculation (e.g., kernel size, output size, address, number of maps, number of bits), etc., so that the number of reads (number of memory accesses) can be minimized by temporarily storing the data in a register and repeatedly using the data. According to an embodiment of the present disclosure, by doing so, it is possible to suppress an increase in the number of memory accesses, and as a result, it is possible to further suppress an increase in power consumption.

In addition, when the type of calculation to be performed is a convolution calculation, in an embodiment of the present disclosure, data to be temporarily stored in the register unit is selected so that the capacity of the memory unit, which is determined based on the amount of calculation (e.g., kernel size, output size, address, number of maps, number of bits), is minimized. In this embodiment, for example, when the capacity of the memory unit is reduced by storing coefficients in the register unit, the coefficients are temporarily stored in the register unit. In addition, in this embodiment, for example, when the capacity of the memory unit is reduced by storing input data in the register unit, the input data is temporarily stored in the register unit. According to an embodiment of the present disclosure, by doing so, it is possible to suppress an increase in the number of memory accesses and an increase in power consumption associated with memory access, and as a result, it is possible to further suppress an increase in power consumption.

Furthermore, in the case where the type of calculation to be performed is a convolution calculation, in the embodiment of the present disclosure, data to be temporarily stored in the register unit is selected so as to minimize the power consumed by accessing the memory unit. In this embodiment, for example, if the memory unit that stores the coefficients is a DRAM and the memory unit that stores the input data is an SRAM, the coefficients are temporarily stored in the register unit because the DRAM consumes more power when accessed. According to the embodiment of the present disclosure, by doing so, it is possible to further suppress the increase in power consumption.

In other words, according to the embodiments of the present disclosure, it is possible to further suppress the increase in power consumption of a processing device that performs arithmetic processing in a deep neural network. Below, the details of the embodiments of the present disclosure created by the present inventor will be explained in order.

<<2. First embodiment>>
<2.1. Example of functional configuration of the arithmetic processing device 10>
First, an example of the functional configuration of a processor 10 that realizes the processor method according to the present embodiment will be described with reference to Fig. 4. Fig. 4 is an example of a functional block diagram of the processor 10 according to the present embodiment.

In detail, as shown in Fig. 4, the arithmetic processing device 10 according to this embodiment mainly has an input unit 110, a calculation unit 120, a memory unit 130, and an output unit 140. Below, each functional block of the arithmetic processing device 10 will be described in order.

(Input unit 110)
The input unit 110 has a function of detecting various input operations by an operator and receiving input data related to the input operations. For example, the input unit 110 can be realized by various devices for detecting input operations by an operator, such as various buttons, a keyboard, a touch panel, a mouse, and a switch.

(Calculation unit 120)
The calculation unit 120 has a function of performing calculations based on a plurality of input data and a plurality of coefficients corresponding to the input data, and calculating output data. Specifically, the calculation unit 120 according to this embodiment can perform calculations related to the deep neural network described above. The details of the calculation unit 120 will be described later.

(Memory unit 130)
The storage unit 130 has a function of storing programs and the like used in each block of the arithmetic processing device 10 .

(Output unit 140)
The output unit 140 has a function of outputting various information to the operator. For example, the output unit 140 outputs output data which is a result of the calculation by the calculation unit 120. In detail, the output unit 140 can be configured to include a display device which outputs visual information. Here, the above-mentioned display device can be, for example, a CRT (Cathode Ray Tube) display device, a Liquid Crystal Display (LCD) device, an OLED (Organic Light Emitting Diode) device, or the like.

The above describes an example of the functional configuration of the arithmetic processing device 10 according to this embodiment. Note that the functional configuration example described above is merely an example, and the functional configuration example of the arithmetic processing device 10 according to this embodiment is not limited to the above example. The arithmetic processing device 10 according to this embodiment may further have a configuration other than that shown in FIG. 4. For example, the arithmetic processing device 10 may further have a communication unit that communicates information with an information processing terminal. In other words, the functional configuration of the arithmetic processing device 10 according to this embodiment can be flexibly redesigned.

2.2. Example of configuration of calculation unit 120
Next, the arithmetic unit 120 according to the present embodiment will be described with reference to Fig. 5. Fig. 5 is an example of a circuit block diagram of the arithmetic unit 120 according to the present embodiment.

In detail, as shown in Fig. 5, the calculation unit 120 according to this embodiment mainly includes memory units 200, 202, and 204, selector units 300 and 302, a register unit 400 (REG in Fig. 5), a product-sum calculator 500, and a control unit 600. Each circuit block of the calculation unit 120 will be described below in order.

(Storage units 200, 202)
The storage unit (first storage unit) 200 can store, for example, coefficients (first data), and the storage unit (second storage unit) 202 can store, for example, input data (second data). In this embodiment, these storage units 200 and 202 may be volatile memories such as DRAM and SRAM, or non-volatile memories such as NAND. However, from the viewpoint of access speed, these storage units 200 and 202 are preferably volatile memories such as DRAM and SRAM. In this embodiment, the input data and coefficients stored in the storage units 200 and 202 are temporarily held in a register unit 400 described later, or directly read out by a product-sum unit 500 without being held in the register unit 400.

(Memory unit 204)
The storage unit 204 can store intermediate results (partial sums) calculated by the multiply-adder 500. In this embodiment, the storage unit 204 may be a volatile memory such as DRAM or SRAM, or a non-volatile memory such as NAND, but from the viewpoint of access speed, a volatile memory such as DRAM or SRAM is preferable. In this embodiment, the intermediate results stored in the storage unit 204 are read out by the multiply-adder 500 and reused in the calculation.

(Selector Units 300, 302)
The selector units 300 and 302 are configured with logic circuits and the like, and can select data to be temporarily held in the register unit 400 (described later) from, for example, the coefficients stored in the storage unit 200 or the input data stored in the storage unit 202, under the control of the control unit 600 (described later). For example, when a selection signal "0" is input from the control unit 600, the selector unit 300 outputs the coefficients stored in the storage unit 200 to the register unit 400, and the selector unit 302 outputs the input data stored in the storage unit 202 to the product-accumulator 500. On the other hand, when a selection signal "1" is input from the control unit 600, for example, the selector unit 300 outputs the input data stored in the storage unit 202 to the register unit 400, and the selector unit 302 outputs the coefficients stored in the storage unit 200 to the product-accumulator 500.

(Register unit 400)
The register unit 400 can temporarily hold either the coefficients stored in the memory unit 200 or the input data stored in the memory unit 202, selected by the selector units 300 and 302. In this embodiment, the register unit 400 is configured by logic circuits (flip-flops, etc.) built into the CPU (Central Processing Unit) 1100 (see FIG. 9 ), and is a dedicated storage device that has a smaller capacity but a faster operating speed than the above-mentioned storage units 200 and 202. Since the register unit 400 is a dedicated storage device, the circuit configuration is simplified and the power consumption is low. Therefore, in this embodiment, data to be repeatedly used in the product-adder 500 described later is temporarily held in the register unit 400 and repeatedly reused in the calculations in the product-adder 500 described later, thereby suppressing an increase in power consumption.

(Product Accumulator 500)
The product-accumulator 500 can read input data, coefficients, and other data from the storage units 200, 202 and the register unit 400, and repeatedly perform calculations using the read data. Furthermore, the product-accumulator 500 can output calculation results (intermediate results) to the storage units 202, 204, etc. In this embodiment, the product-accumulator 500 may be a digital circuit or an analog circuit depending on the calculation process, etc., and is not particularly limited. Details of the product-accumulator 500 will be described later.

(Control unit 600)
The control unit 600 is composed of a CPU 1100 (see FIG. 9 ) and the like, and can control the selector units 300 and 302 based on the type of calculation, the amount of calculation (e.g., kernel size, output size, address, number of maps, number of bits), the type and capacity of the storage units 200 and 202, and the type of the product adder 500. The control unit 600 can also control the storage units 200 and 202.

Specifically, in this embodiment, for example, when the type of operation to be performed is an affine operation, the control unit 600 performs control so that the repeatedly used input data is temporarily stored in the register unit 400.

In addition, in this embodiment, the control unit 600, for example, in the case of a convolution calculation, selects data that can be temporarily stored in the register unit 400 and repeatedly used to minimize the number of reads (memory accesses) from the storage units 200 and 202, and controls to store the selected data in the register unit 400. Alternatively, in this embodiment, for example, the control unit 600 selects data that can be temporarily stored in the register unit 400 and repeatedly used to minimize the number of reads (memory accesses) based on the amount of calculation (for example, kernel size, output size, address, number of maps, number of bits), etc., and controls to store the selected data in the register unit 400. Alternatively, the control unit 600 selects data to be temporarily stored in the register unit 400 so that the capacity used by the storage units 200 and 202, which is determined based on the amount of calculation (for example, kernel size, output size, address, number of maps, number of bits), etc., is minimized, and controls to store the selected data in the register unit 400. Furthermore, the control unit 600 selects data to be temporarily stored in the register unit 400 based on the type of the memory units 200 and 202 (e.g., DRAM, SRAM, etc.) so as to minimize the power consumed by accessing the memory units 200 and 202, and performs control so as to store the selected data in the register unit 400.

In this embodiment, by controlling the selector units 300 and 302 by the control unit 600 in this manner, it is possible to suppress an increase in the number of memory accesses, and as a result, it is possible to further suppress an increase in power consumption. Furthermore, according to this embodiment, since there is no need to provide many register units, the circuit size does not increase, and as a result, it is possible to further suppress an increase in power consumption.

The above describes an example of the functional configuration of the calculation unit 120 according to this embodiment. Note that the above-described example of the functional configuration is merely an example, and the example of the functional configuration of the calculation unit 120 according to this embodiment is not limited to the example. For example, the calculation unit 120 according to this embodiment may further have a configuration other than that shown in FIG. 5.

<2.3. Example of the configuration of the product-accumulator 500>
Next, an example of the product-accumulator 500 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is an example of a circuit block diagram of the product-accumulator 500 according to this embodiment.

In detail, as shown in FIG. 6, the product-sum calculator 500 according to this embodiment mainly includes a plurality of multipliers 502, a selector unit 504, an adder 506, and an accumulator 508. In this embodiment, a plurality of product-sum calculators 500 are provided in the arithmetic unit 120 according to the number of processing layers of the arithmetic processing in the deep neural network, for example. Each circuit block of the product-sum calculator 500 will be described below in order.

(Multiplier 502)
Each multiplier 502 is configured with a logic circuit or the like, and can read input data or coefficients stored in the storage units 200 and 202 and input data or coefficients temporarily held in the register unit 400, and multiply them sequentially. Furthermore, each multiplier 502 can output the obtained calculation result to an adder 506, which will be described later.

(Selector Unit 504)
The selector unit 504 is composed of a logic circuit or the like, and can select data (third data) to be output to the adder 506 described later from among the intermediate result (partial sum) stored in the memory unit 204 and the accumulated result temporarily held in the accumulator 508 described later, according to control (selection signal) from the control unit 600. For example, when a selection signal "0" is input from the control unit 600, the selector unit 504 outputs the intermediate result (partial sum) stored in the memory unit 204 to the adder 506. On the other hand, when a selection signal "1" is input from the control unit 600, the selector unit 504 outputs the accumulated result temporarily held in the accumulator 508 to the adder 506. In this embodiment, the selector unit 504 may be controlled by the control unit 600 depending on the type of calculation, or may be controlled by the control unit 600 to further suppress an increase in power consumption, similar to the above-mentioned selector units 300 and 302.

(Adder 506)
The adder 506 is configured by a logic circuit or the like, and can add the multiplication results from each multiplier 502 described above to either the intermediate result (partial sum) stored in the memory unit 204 and the accumulated result temporarily held in the accumulator 508, selected by the selector unit 504 described above. Furthermore, the adder 506 can output the addition result to the accumulator 508 and the memory units 202 and 204 described below.

(Accumulator 508)
The accumulator 508 temporarily holds the addition result of the adder 506 described above, and can output the held accumulation result to the storage units 202, 204, or output it to the adder 506 according to the selection of the selector unit 504. According to this embodiment, the adder 506 can perform an arithmetic operation using the accumulation result held in the accumulator 508, so that it is possible to reduce the number of times data is read out to the storage units 200, 202, etc., and perform a closed arithmetic operation within the product-sum unit 500. In addition, a reset signal for resetting the accumulated value to 0 may be input to the accumulator 508 as appropriate.

The accumulator 508 is configured, for example, like the register unit 400, by logic circuits (such as flip-flops) built into the CPU 1100 (see FIG. 9), and is a dedicated memory device with a smaller capacity but faster operating speed than the above-mentioned memory units 200, 202, etc. Such an accumulator 508 is a dedicated memory device, so that the circuit configuration is simplified and power consumption is low. Therefore, by using such an accumulator 508, it is possible to suppress an increase in power consumption.

The above describes an example of the functional configuration of the product-accumulator 500 according to this embodiment. Note that the above-described example of the functional configuration is merely an example, and the example of the functional configuration of the product-accumulator 500 according to this embodiment is not limited to the example. For example, the product-accumulator 500 according to this embodiment may further have a configuration other than that shown in FIG. 6.

2.4. Calculation processing method
The above is a detailed description of exemplary configurations of the arithmetic processing device 10, the arithmetic unit 120, and the product adder 500 according to this embodiment. Next, the arithmetic processing method according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the arithmetic processing method according to this embodiment.

In detail, as shown in Fig. 7, the calculation processing method according to this embodiment includes a number of steps from step S101 to step S116. Each step included in the calculation processing method according to this embodiment will be described in detail below.

First, the control unit 600 sets (resets) a counter for the processing layer of the arithmetic processing in the deep neural network to 0 (step S101). Next, the control unit 600 initializes counters for various processes (step S102).

Next, the control unit 600 generates a control signal and a selection signal according to the type of calculation to be performed, and outputs them to each block of the calculation unit 120 (step S103). In detail, the control unit 600 generates a selection signal "0" or a selection signal "1" based on the type of calculation, the amount of calculation (e.g., kernel size, output size, address, number of maps, number of bits), the type and capacity of the storage units 200 and 202, and the type of the multiplier-accumulator 500, and outputs them to the selector units 300 and 302. In this embodiment, the control unit 600 selects data that can be temporarily stored in the register unit 400 and used repeatedly to minimize the number of reads (memory access count) from the storage units 200 and 202, and generates a selection signal to store the selected data in the register unit 400. In this embodiment, by generating a selection signal that controls the selector units 300 and 302 in this way, it is possible to suppress an increase in the number of memory accesses, and as a result, it is possible to further suppress an increase in power consumption.

When the type of calculation to be performed is a convolution calculation, as described above, the control unit 600 may select data to be temporarily stored in the register unit 400 so as to minimize the capacity used by the memory unit, which is determined based on the amount of calculation (e.g., kernel size, output size, address, number of maps, number of bits), etc. The control unit 600 may also select data to be temporarily stored in the register unit 400 so as to minimize the power consumed by accessing the memory units 200 and 202.

Furthermore, under the control of the control unit 600, coefficients and input data are read out from the memory units 200, 202 (step S104).

Here, the selector units 300 and 302 determine whether the selection signal from the control unit 600 is "0" (step S105). If the selector units 300 and 302 determine that the selection signal is "0" (step S105: Yes), they proceed to processing in step S106, and if they determine that the selection signal is not "0" (step S105: No), they proceed to processing in step S110.

Next, based on the selection signal "0" input from the control unit 600, the selector unit 300 outputs the coefficients stored in the memory unit 200 to the register unit 400, and the selector unit 302 outputs the input data stored in the memory unit 202 to the product-accumulator 500. Therefore, the register unit 400 temporarily holds the coefficients (step S106).

Then, the multiplier 500 performs a multiply-and-accumulate operation using the coefficients temporarily stored in the register unit 400 and the input data read from the memory unit 202 (step S107). Next, the multiplier 500 reads the partial sum from the memory unit 204 and multiplies it by the multiply-and-accumulate result of step S107 described above (step S108). Furthermore, the multiplier 500 outputs the multiplication result of step S108 described above to the memory unit 204 (step S109).

Next, based on the selection signal "1" input from the control unit 600, the selector unit 300 outputs the input data stored in the memory unit 202 to the register unit 400, and the selector unit 302 outputs the coefficients stored in the memory unit 200 to the product-accumulator 500. Therefore, the register unit 400 temporarily holds the input data (step S110).

Then, the multiplier-accumulator 500 performs a multiply-and-accumulate operation using the input data temporarily stored in the register unit 400 and the coefficients read from the memory unit 200 (step S111). Next, the multiplier-accumulator 500 reads the accumulation result (partial sum) from the accumulator 508 and adds it to the multiply-and-accumulate result in step S111 described above (step S112). According to this embodiment, since the adder 506 can perform an operation using the accumulation result stored in the accumulator 508, it is possible to reduce the number of times reads are made from the memory units 200, 202, etc. and perform a closed operation within the multiplier-accumulator 500.

Furthermore, the multiplier-accumulator 500 outputs the accumulation result in step S112 described above to the accumulator 508 (step S113).

Next, the control unit 600 determines whether all output data for the target processing layer (target layer) has been generated (step S114). If the control unit 600 determines that all output data for the target layer has been generated (step S114: Yes), it proceeds to processing of step S115, and if the control unit 600 determines that all output data for the target layer has not been generated (step S114: No), it returns to processing of step S103.

Next, the control unit 600 determines whether the target layer on which the processing has been performed is the final layer (step S115). If the control unit 600 determines that the target layer on which the processing has been performed is the final layer (step S115: Yes), it ends the calculation process, and if the control unit 600 determines that the target layer on which the processing has been performed is not the final layer (step S115: No), it proceeds to the process of step S116.

Next, the control unit 600 refers to the counter of the processing layer and determines whether the processed target layer is the final layer (step S115). If the control unit 600 determines that the processed target layer is the final layer (step S115: Yes), it ends the calculation process, and if the control unit 600 determines that the processed target layer is not the final layer (step S115: No), it proceeds to the process of step S116.

Furthermore, the control unit 600 increments the processing layer counter by 1 (step S116) and returns to processing of step S103.

As described above, according to this embodiment, it is possible to suppress an increase in power consumption of the calculation processing device 10 that performs calculation processing in the deep neural network.

In detail, in this embodiment, the selector units 300, 302 installed in front of the register unit 400 are controlled to select data (input data or coefficients) to be temporarily stored in the register unit 400 so as to reduce the number of reads (number of memory accesses) from the storage units 200, 202. Therefore, according to this embodiment, it is possible to suppress an increase in the number of memory accesses, and as a result, it is possible to further suppress an increase in power consumption.

In addition, in this embodiment, the selector units 300 and 302 can share and use the data stored in the register unit 400, so there is no need to provide many register units. Therefore, according to this embodiment, the circuit size does not increase, and as a result, the increase in power consumption can be further suppressed.

<<3. Second embodiment>>
Incidentally, in the above-mentioned first embodiment, the arithmetic processing device 10 has been described as performing convolution calculations, affine calculations, and the like, but the arithmetic processing device 10 can also be applied to other calculations.

For example, as explained above, in a convolutional deep neural network (convolution operation), the operation is performed by taking into account the relationship between each node in the same layer. However, in a convolutional deep neural network (convolution operation), a filter is used to extract features by taking into account local relationships, and the operation is a convolution operation, and the operation does not take into account the relationship in the entire input data (for example, the entire image data). Therefore, in recent years, a method called a squeeze-and-excitation network, which performs an operation by taking into account the relationship in the entire input data, has been considered (see Non-Patent Document 1). In the squeeze-and-excitation network, there is a phase (SE Block) in which the sum of products of input data is calculated. Specifically, in the squeeze-and-excitation network, a scale factor that determines the degree of emphasis for each map of input data is calculated from the input data, and the sum of products of the input data and the scale factor is further calculated.

Therefore, in the second embodiment, input data (scale data) is stored in place of coefficients in the memory unit 200 of the calculation unit 120 according to the first embodiment described above, and the calculation unit 120 performs calculations, thereby suppressing an increase in the number of reads (memory access count) from the memory units 200 and 202, and as a result, an increase in power consumption can be further suppressed. The second embodiment is described below.

Note that the configuration of the arithmetic processing device 10 of this embodiment is similar to that of the arithmetic processing device 10 of the first embodiment described above, so description thereof will be omitted here.

The calculation unit 120a according to this embodiment will be described with reference to FIG. 8. FIG. 8 is an example of a circuit block diagram of the calculation unit 120a according to this embodiment. As shown in FIG. 8, the calculation unit 120a according to this embodiment includes memory units 200a, 202, 204, selector units 300, 302, a register unit 400 (REG in FIG. 8), a product adder 500, and a control unit 600, similar to the first embodiment. However, unlike the first embodiment, the memory unit 200a stores at least a portion of the input data (scale data).

Note that the configuration of the multiplier-accumulator 500 in this embodiment is similar to that of the multiplier-accumulator 500 in the first embodiment described above, so its explanation will be omitted here.

In this embodiment, for example, when a selection signal "0" is input from the control unit 600, the selector unit 300 outputs the input data (data for scaling) stored in the memory unit 200 to the register unit 400, and the selector unit 302 outputs the input data stored in the memory unit 202 to the product-accumulator 500. The product-accumulator 500 can then calculate the product-accumulator of the input data (data for scaling) and the input data.

In this embodiment, too, input data (scaling data) is stored in the memory unit 200a of the calculation unit 120 instead of coefficients, and the stored input data (scaling data) is temporarily held in the register unit 400 for calculation, thereby suppressing an increase in the number of reads (memory access count) from the memory units 200 and 202. As a result, according to this embodiment, an increase in power consumption can be further suppressed.

Furthermore, as can be seen from the above, the processing device 10 according to an embodiment of the present disclosure is capable of handling various deep neural network calculations by controlling the selector units 300, 302 according to the calculation.

<<4. Summary>>
As described above, according to each of the above-described embodiments of the present disclosure, it is possible to suppress an increase in power consumption of the arithmetic processing device 10 that performs arithmetic processing in a deep neural network.

In detail, in each embodiment of the present disclosure, the selector units 300, 302 installed in front of the register unit 400 are controlled to select data (input data or coefficients) to be temporarily stored in the register unit 400 so as to reduce the number of reads (number of memory accesses) from the storage units 200, 202. Therefore, according to each embodiment of the present disclosure, it is possible to suppress an increase in the number of memory accesses, and as a result, it is possible to further suppress an increase in power consumption.

In addition, in each embodiment of the present disclosure, the selector units 300 and 302 can share and use the data held in the register unit 400, so there is no need to provide many register units. Therefore, according to each embodiment of the present disclosure, the circuit size does not increase, and as a result, the increase in power consumption can be further suppressed.

<<5. Hardware configuration>>
The arithmetic processing device 10 according to each of the above-described embodiments is realized by a computer 1000 having a configuration as shown in FIG. 9, for example. The arithmetic processing device 10 of the present disclosure will be described below as an example. FIG. 9 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the arithmetic processing device 10. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input/output interface 1600. Each unit of the computer 1000 is connected by a bus 1050.

The CPU 1100 operates based on the programs stored in the ROM 1300 or the HDD 1400 and controls each part. For example, the CPU 1100 expands the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

ROM 1300 stores boot programs such as BIOS (Basic Input Output System) executed by CPU 1100 when computer 1000 is started, as well as programs that depend on the hardware of computer 1000.

HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records the computational processing program related to the present disclosure, which is an example of program data 1450.

The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

The input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input/output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input/output interface 1600. The input/output interface 1600 may also function as a media interface that reads programs and the like recorded on a specified recording medium. The media may be, for example, optical recording media such as DVD (Digital Versatile Disc) and PD (Phase change rewritable Disk), magneto-optical recording media such as MO (Magneto-Optical disk), tape media, magnetic recording media, or semiconductor memory.

For example, when computer 1000 functions as arithmetic processing device 10 according to an embodiment of the present disclosure, CPU 1100 of computer 1000 executes an arithmetic processing program loaded onto RAM 1200 to realize functions of arithmetic unit 120, etc. Also, HDD 1400 stores arithmetic processing programs, etc. according to an embodiment of the present disclosure. Note that CPU 1100 reads and executes program data 1450 from HDD 1400, but as another example, these programs may be obtained from other devices via external network 1550.

The arithmetic processing device 10 according to the present embodiment may also be applied to a system consisting of multiple devices, which is premised on connection to a network (or communication between devices), such as cloud computing. In other words, the arithmetic processing device 10 according to the present embodiment described above can also be realized as an arithmetic processing system that performs the arithmetic processing according to the present embodiment using multiple devices, for example.

＜＜6. Supplementary Information＞＞
The above-described embodiment of the present disclosure may include, for example, a program for causing a computer to function as a processing device according to the present embodiment, and a non-transitory tangible medium on which the program is recorded. The program may also be distributed via a communication line (including wireless communication) such as the Internet.

In addition, each step in the computational processing of each of the above-mentioned embodiments does not necessarily have to be processed in the order described. For example, each step may be processed in a different order as appropriate. Also, each step may be processed partially in parallel or individually instead of being processed in chronological order. Furthermore, the processing method of each step does not necessarily have to be processed in the manner described, and for example, each step may be processed in another manner by another functional unit.

Although the preferred embodiment of the present disclosure has been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person with ordinary knowledge in the technical field of the present disclosure can conceive of various modified or revised examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

In addition, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology disclosed herein may provide other effects that are apparent to a person skilled in the art from the description of this specification, in addition to or in place of the above effects.

The present technology can also be configured as follows.
(1)
an arithmetic circuit unit that reads out first and second data and repeatedly performs an arithmetic operation using the first and second data that have been read out;
a first storage unit that stores the first data;
a second storage unit that stores the second data;
a register unit that reads out from the first storage unit or the second storage unit data that is repeatedly used in the arithmetic circuit unit and temporarily holds the data;
a first selector unit that selects data to be held in the register unit from the first and second data;
a control unit that controls the first selector unit based on at least one of a type of operation, an amount of operation, a type of the first and second storage units, a capacity of the first and second storage units, and a type of the arithmetic circuit unit;
Equipped with
Processing unit.
(2)
The processing device according to (1) above, wherein the amount of calculation includes at least one of a kernel size, an output size, an address, a map number, and a number of bits.
(3)
An input unit that accepts input data;
an output unit that outputs a calculation result by the calculation circuit unit;
The processing device according to claim 1 or 2, further comprising:
(4)
The arithmetic processing device according to (3) above, wherein the first data includes coefficient information and the second data includes the input data.
(5)
The processor according to claim 3, wherein the first and second data include at least a portion of the input data.
(6)
a third storage unit for storing the partial sums calculated by the arithmetic circuit unit;
The processor according to any one of (1) to (5) above.
(7)
The arithmetic circuit unit includes:
a plurality of multipliers for multiplying the first and second data;
an adder that adds multiplication results of the plurality of multipliers and third data;
an accumulator for temporarily storing the sum of the adder;
a second selector unit that selects either the accumulator or the third storage unit as an acquisition source of the third data;
Has
The second selector unit is controlled by the control unit.
The processing device according to (6) above.
(8)
The arithmetic processing device according to any one of (1) to (7) above, wherein the first and second storage units are made of volatile memory.
(9)
The arithmetic processing device according to any one of (1) to (7) above, wherein the first and second storage units are made of non-volatile memory.
(10)
The arithmetic processing device according to any one of (1) to (9) above, wherein the arithmetic circuit unit is made of a digital circuit.
(11)
The arithmetic processing device according to any one of (1) to (9) above, wherein the arithmetic circuit unit is made of an analog circuit.

REFERENCE SIGNS LIST 10 Arithmetic processing device 110 Input section 120, 120a Arithmetic section 130 Memory section 140 Output section 200, 200a, 202, 204 Memory section 300, 302, 504 Selector section 400 Register section 500 Product-sum unit 502 Multiplier 506 Adder 508 Accumulator 600 Control section

Claims (8)

an arithmetic circuit unit that reads out first and second data and repeatedly performs an arithmetic operation using the first and second data that have been read out;
a first storage unit that stores the first data;
a second storage unit that stores the second data;
a register unit that reads out from the first storage unit or the second storage unit data that is repeatedly used in the arithmetic circuit unit and temporarily holds the data;
a first selector unit that selects data to be held in the register unit from the first and second data;
a control unit that controls the first selector unit based on at least one of a type of operation and an amount of operation;
a third storage unit for storing the partial sums calculated by the arithmetic circuit unit;
Equipped with
The arithmetic circuit unit includes:
a plurality of multipliers for multiplying the first and second data;
an adder that adds multiplication results of the plurality of multipliers and third data;
an accumulator for temporarily storing the sum of the adder;
a second selector unit that selects either the accumulator or the third storage unit as an acquisition source of the third data;
Has
The second selector unit is controlled by the control unit.
Processing unit. An input unit that accepts input data;
an output unit that outputs a calculation result by the calculation circuit unit;
The processor of claim 1 further comprising: The processor according to claim 2 , wherein the first data includes coefficient information and the second data includes the input data. The processor according to claim 2 , wherein the first and second data include at least a portion of the input data. The processor according to claim 1 , wherein the first and second storage units are made of volatile memories. The processor according to claim 1 , wherein the first and second storage units are made of non-volatile memories. The arithmetic processing device according to claim 1 , wherein the arithmetic circuit section is made up of a digital circuit. The arithmetic processing device according to claim 1 , wherein the arithmetic circuit section is an analog circuit.

Images