JP7795766B2 - Neural network circuit device - Google Patents

Description

The present invention relates to a neural network circuit device.

Conventionally, neural networks such as convolutional neural networks (CNNs) are trained by updating (adjusting) the weights of the connections (edges) between nodes (neurons) during the training phase. In contrast, Non-Patent Document 1 discloses an algorithm for a neural network (HNN: Hidden Network) that does not update weights even during the training phase. The algorithm in Non-Patent Document 1 extracts subnetworks hidden in the neural network with high inference accuracy during the training phase. In doing so, the algorithm does not update the randomly set weights. In other words, the technology in Non-Patent Document 1 fixes the randomly set weights. Furthermore, in the technology in Non-Patent Document 1, a supermask (sometimes simply referred to as a "mask") that defines whether connections between nodes are valid or invalid is determined during the training phase. In the technology in Non-Patent Document 1, a model (subnetwork) used in inference is constructed using the supermask and fixed weights.

Vivek Ramanujan et al., "What's Hidden in a Randomly Weighted Neural Network?", CVPR 2020, https://arxiv.org/abs/1911.13299

Non-Patent Document 1 does not disclose a method for implementing the HNN model used in inference in hardware. Therefore, if one were to simply try to implement the algorithm described in Non-Patent Document 1 in hardware, there is a risk that if the amount of model information data becomes enormous, the resulting hardware would have poor processing efficiency, requiring enormous amounts of memory and memory access.

The objective of the present invention is to provide a neural network circuit device that can perform processing efficiently.

The neural network circuit device of the present invention is a neural network circuit device that realizes a neural network model in which, during the learning stage, the weights of each connection between nodes are set by fixed random numbers, a mask is generated by determining whether each of the connections is valid or invalid, and the weights and the mask are used. The neural network circuit device includes: a weight generation circuit that generates the weights using a random number generator configured to generate the same random numbers as those generated during the learning stage using the same seed as used during the learning stage when performing inference; and an arithmetic circuit that performs a product-sum operation using input data that is the data to be inferred or activation value data corresponding to the data to be inferred, the weights, and the mask.

The objective of the present invention is to provide a neural network circuit device that can perform processing efficiently.

FIG. 1 is a diagram illustrating an HNN algorithm. FIG. 1 is a diagram illustrating an HNN algorithm. 10 is a graph showing the relationship between the value of k of the supermask in the HNN algorithm and the accuracy of inference. FIG. 10 is a diagram for explaining selection of a supermask according to the present embodiment. FIG. 1 is a diagram illustrating a configuration of a neural network circuit device according to a first embodiment. 10A and 10B are diagrams for explaining the processing of a weight generation circuit and a mask development circuit according to the first embodiment; FIG. 2 is a diagram illustrating a configuration of a weight generation circuit according to the first embodiment; FIG. 2 is a diagram illustrating a configuration of a mask developing circuit according to the first embodiment; FIG. 2 is a diagram for explaining input data according to the first embodiment; FIG. 2 is a diagram for explaining the operation of the arithmetic circuit according to the first embodiment; FIG. 10 is a diagram for explaining processing of a PE vector according to the first embodiment; FIG. 2 is a timing chart illustrating each component of the neural network circuit device according to the first embodiment. FIG. 2 is a diagram illustrating a configuration of a post-processing circuit according to the first embodiment. 10A and 10B are diagrams illustrating results of calculations performed by the neural network circuit device according to the first embodiment. 10A and 10B are diagrams illustrating results of calculations performed by the neural network circuit device according to the first embodiment. 10A and 10B are diagrams illustrating results of calculations performed by the neural network circuit device according to the first embodiment. 10A and 10B are diagrams illustrating results of calculations performed by the neural network circuit device according to the first embodiment.

(Outline of the embodiment)
Before describing the embodiments, an overview of the embodiments of the present invention will be described. Note that, although the embodiments of the present invention will be described below, the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

<About HNN>
1 and 2 are diagrams for explaining the HNN algorithm. The HNN algorithm corresponds to the technology described in Non-Patent Document 1. On the left side of Fig. 1, a diagram for explaining the learning of a model (neural network) according to the HNN algorithm is shown.

In the HNN algorithm, an initial model 10 and a supermask 20 are created during the learning phase. The initial model 10 is composed of nodes 12 (neurons) and connections 13 between the nodes 12. Here, a weight 14 is assigned to each connection 13. The weight 14 for each connection is set, for example, using a random number. As mentioned above, in the HNN algorithm, the weights 14 are not updated during learning and are fixed.

The supermask 20 defines whether each connection 13 is valid or invalid. The supermask 20 is generated using a score 30 that indicates the importance of each connection 13. The score 30 is determined during the training phase. In other words, the HNN algorithm learns a score 30 for each of the multiple connections 13. The score 30 is trained, for example, by backpropagation.

Figure 2 is a diagram illustrating a method for determining a supermask 20. Figure 2 is a diagram illustrating the distribution of scores 30. The horizontal axis of Figure 2 represents the score 30 value, and the vertical axis represents the number of connections 13 having the corresponding score 30 value. Here, the supermask 20 is generated by determining that connections 13 with high score 30 values are valid and connections 13 with low score 30 values are invalid. Specifically, the supermask 20 is generated by determining that connections 13 corresponding to the top k% of scores 30 (connections corresponding to the hatched portions in Figure 2) are valid and the remaining connections 13 are invalid. Here, the value of k is set in advance for each supermask 20 to be generated. A supermask 20 can be created for each of multiple k values.

The thick solid lines in the supermask 20 shown in Figure 1 correspond to connections 13 corresponding to the top k% of scores 30. Therefore, these thick solid lines correspond to connections 13 determined to be valid. On the other hand, the dashed lines in the supermask 20 shown in Figure 1 correspond to connections 13 that do not correspond to the top k% of scores 30 (i.e., connections 13 that correspond to the bottom (100-k)% of scores 30). Therefore, these dashed lines correspond to connections 13 determined to be invalid. Here, in the supermask 20 of this embodiment, a "1" is assigned to connections 13 determined to be valid (i.e., connections 13 that correspond to the top k% of scores 30). On the other hand, a "0" is assigned to connections 13 determined to be invalid (i.e., connections 13 that do not correspond to the top k% of scores 30). Note that the score 30 (and the supermask 20) is learned based on the value of k corresponding to the supermask 20. Specifically, the score 30 is learned for each value of k corresponding to the supermask 20 to be generated. For example, when generating a supermask 20 with k=20%, the top 20% of scores 30 are used to learn (update) all scores 30. Note that, as mentioned above, the weights 14 are not updated at this time.

Then, by combining the initial model 10 and the supermask 20, the subnetwork 40 shown on the right side of Figure 1 is constructed. In other words, a neural network (subnetwork 40) model is constructed using each connection 13 (weight 14) of the initial model 10 and the supermask 20. Specifically, the subnetwork 40 is constructed by the logical product (AND) of each connection 13 (weight 14) of the initial model 10 and the valid or invalid information (e.g., "1" or "0") of each connection 13 shown in the supermask 20. The HNN algorithm uses this subnetwork 40 (neural network model) to perform inference.

Figure 3 is a graph showing the relationship between the k value of SuperMask 20 in the HNN algorithm and inference accuracy. The top graph in Figure 3 shows the results when the HNN algorithm is applied to ResNet50 and ImageNet. The bottom graph in Figure 3 shows the results when the HNN algorithm is applied to VGG16 and CIFAR-100. The horizontal axis of each graph corresponds to the k value, and the vertical axis corresponds to inference accuracy. As shown in Figure 3, when the k value is between 0 and 30%, the larger the k value, the higher the inference accuracy can be.

As described above, the supermask 20 in this embodiment is binary information of "0" or "1." Therefore, the supermask 20 generated in the learning stage can be compressed and saved, and the compressed supermask 20 can be read and expanded in the inference stage. By compressing the supermask 20, the size of the supermask 20 corresponding to the model (trained model) read in the inference stage can be reduced. In other words, the amount of data transmitted when reading the supermask 20 can be reduced. This allows for power consumption savings. Therefore, inference processing can be performed more efficiently.

The smaller the value of k, the fewer the number of "1"s in the supermask 20 (i.e., the number of connections 13 determined to be "valid") and the greater the number of "0"s (i.e., the number of connections 13 determined to be "invalid"). Therefore, by compressing the supermask 20 using the Zero-Run-Length (ZRL) compression method, the smaller the value of k for the supermask 20, the higher the compression rate in the compressed state. Note that a "high compression rate" means that the ratio of the data size after compression to the data size before compression is small, i.e., good compression. Here, a supermask 20 with a small value of k has fewer "1"s, and can therefore be said to be sparse. Therefore, the sparser the supermask 20, the higher the compression rate. In other words, the sparser the supermask 20 in its compressed state, the higher the compression rate.

As mentioned above, in the example of Figure 3, when the value of k is approximately 0 to 30%, the larger the value of k, the higher the inference accuracy, and the smaller the value of k, the lower the inference accuracy. As mentioned above, the smaller the value of k, the sparser the supermask 20, and therefore the higher the compression rate. Therefore, when the value of k is approximately 0 to 30%, if you want to increase the inference accuracy, you can select a supermask 20 with a large k value. On the other hand, if you want to perform inference processing efficiently (reduce communication volume and power consumption), you can select a supermask 20 with a small k value (i.e., a high compression rate). This easily achieves a trade-off between inference accuracy and inference processing efficiency. Therefore, it is possible to separately generate a supermask 20 that prioritizes inference accuracy and a supermask 20 that prioritizes inference processing efficiency (reduce communication volume and power consumption).

Furthermore, a supermask 20 may be generated for each inference target. For example, in inference to detect animals from image data, different supermasks 20 may be generated when the inference target is a "dog" and when the inference target is a "cat." In this case, for example, a supermask 20 with a relatively large k value (e.g., approximately 20-30%) and a supermask 20 with a relatively small k value (e.g., approximately 1-10%) can be generated, where the inference target is trained as a "dog." The same applies when the inference target is trained as a "cat." Therefore, a supermask 20 may be generated for at least one of the inference accuracy and the inference target. This allows inference to be easily performed for different inference targets, and with different accuracy, simply by switching the supermasks 20 without changing the weights 14.

Figure 4 is a diagram illustrating the selection of a supermask 20 according to this embodiment. As described above, in the initial model 10, a weight 14 is assigned to each connection 13. These weights 14 are fixed. Inference can be performed by superimposing a supermask 20 on the initial model 10. Assume that supermasks 20A with k = 30%, 20B with k = 20%, and 20C with k = 10% have been generated. Selecting supermask 20A results in relatively high power consumption during inference, but relatively good inference accuracy. Selecting supermask 20B, on the other hand, can reduce power consumption during inference compared to supermask 20A, but can result in poorer inference accuracy. Selecting supermask 20C can further reduce power consumption during inference compared to supermask 20B, but can result in poorer inference accuracy. In this way, the supermask 20 to be used can be selected depending on whether emphasis is placed on inference accuracy or on inference processing efficiency (power consumption reduction).

Also, as mentioned above, in the HNN algorithm, the weights 14 are set by random numbers in the initial model 10 and are fixed. Here, the weights used in the inference stage must be the same as the weights set in the learning stage. In this case, one method is to store the weights set by random numbers in the learning stage and use these stored weights in the inference stage. However, in this case, a storage area is required to store the weights.

On the other hand, as described above, weights are generated using random numbers, so weights 14 can be generated using a random number generator. Here, if the same random number generator is used and the same seed is input to the random number generator, the same random numbers will be output. In this embodiment, the seed and random number generator used in the learning phase are used in the inference phase. As a result, the same random numbers generated in the learning phase are also generated in the inference phase. Therefore, the same weights set in the learning phase can be used in the inference phase. In this case, there is no need to store weights 14 in the device, so the size of the device's memory area can be reduced. This makes it possible to reduce memory capacity. Furthermore, there is no need to access memory when setting weights in the inference phase, so memory access can be suppressed. This makes it possible to achieve good processing efficiency.

The HNN algorithm maintains good inference accuracy even when the weight 14 is set to "+1" or "-1." Therefore, the weight 14 can be treated as binary information. For example, the weight 14 can be treated as binary information by interpreting the binary information "1" as "+1" and "0" as "-1." Furthermore, as mentioned above, the supermask 20 is also binary information. Therefore, in this embodiment, when the HNN algorithm is implemented in hardware, it is possible to reduce the size of the device.

(Embodiment 1)
Hereinafter, embodiments will be described with reference to the drawings. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In addition, the same elements in each drawing are designated by the same reference numerals, and duplicate explanations have been omitted as necessary.

<Neural network circuit device>
FIG. 5 is a diagram showing a configuration of a neural network circuit device 100 according to the first embodiment. The neural network circuit device 100 according to the first embodiment includes a control circuit 110, a weight generation circuit 200, a mask expansion circuit 300, an arithmetic circuit 400, an activation value data memory 500, a barrel shifter 130, and a post-processing circuit 140. The neural network circuit device 100 is hardware configured to perform inference using the above-described HNN algorithm during inference. The neural network circuit device 100 may include the above circuits on a single chip. The neural network circuit device 100 realizes a neural network model configured using weights set in the learning phase and a supermask generated in the learning phase.

When performing inference, the weight generation circuit 200 (WGU: Weight Generation Unit) generates weights 14 corresponding to each connection 13 using a random number generator configured to generate the same random numbers as those generated in the learning phase using the same seed as used in the learning phase. Note that the weight generation circuit 200 generates weights 14 for each kernel (filter, output channel). Details will be described later.

The mask expansion circuit 300 (SEU: Supermask Expansion Unit) expands the compressed supermask 20 when performing inference. The mask expansion circuit 300 expands the supermask 20 for each kernel (filter, output channel). Details will be provided later.

Figure 6 is a diagram illustrating the processing of the weight generation circuit 200 and mask expansion circuit 300 according to the first embodiment. The weight generation circuit 200 generates weights 14 corresponding to each connection 13, thereby constructing the initial model 10 set in the learning stage. The mask expansion circuit 300 expands the supermask 20, thereby obtaining the original, uncompressed supermask 20. Then, inference can be performed by superimposing the supermask 20 on the initial model 10.

When performing inference, the control circuit 110 (MCC: Model Construction Controller) controls the construction of the inference model (combination of the initial model 10 and the supermask 20) constructed during the learning phase. In particular, the control circuit 110 controls the weight generation circuit 200 and the mask expansion circuit 300. The control circuit 110 generates a seed to be input to the random number generator used in the weight generation circuit 200. The control circuit 110 also outputs (provides) information on the compression type (code length) of the compressed supermask 20 to the mask expansion circuit 300. The control circuit 110 also controls the operation of the weight generation circuit 200 so that it is synchronized with the operation of the mask expansion circuit 300. Details will be described later. The control circuit 110 may include a processor and a storage device. The functions of the control circuit 110 may be implemented by a program stored in the storage device and executed by the processor.

The activation data memory 500 (AMEM: Activation Memory) stores input data (iAct). Here, "input data" refers to inference target data such as image data, or activation data corresponding to the inference target data. The activation data corresponds to data generated by processing in the first convolutional layer using the inference target data, or data generated by processing in the second or subsequent convolutional layers. In other words, the activation data corresponds to output data from processing in the previous convolutional layer. When the inference target data is image data, the input data can be configured three-dimensionally by the horizontal and vertical directions of the image and input channels. This will be described in more detail below. Note that when the input data is inference target data, the input channel corresponds to the channel of that image (e.g., each RGB value). When the input data is activation data, the input channel corresponds to the kernel type from processing in the previous convolutional layer (i.e., the output channel of the output data from processing in the previous convolutional layer).

The barrel shifter 130 adjusts the position of the input data. Specifically, when the input data is processed by the arithmetic circuit 400, the barrel shifter 130 adjusts the position of the input data so that each piece of input data is placed in the appropriate position for processing. The barrel shifter 130 shifts the three-dimensionally configured input data vertically or horizontally.

The arithmetic circuit 400 performs a product-sum operation using the input data, the weight 14 generated by the weight generation circuit 200, and the supermask 20 expanded by the mask expansion circuit 300. The arithmetic circuit 400 performs an operation so that the product of the input data and the weight 14 is validated or invalidated by the supermask 20. The arithmetic circuit 400 performs a product-sum operation for each input channel and each kernel (output channel). The arithmetic circuit 400 outputs the partial sums (PSUM) obtained by each product-sum operation to the post-processing circuit 140. Details will be described later.

The post-processing circuit 140 (PPU: Post-Processing Unit) performs the necessary processing for the convolutional layer of the CNN. The post-processing circuit 140 performs processing subsequent to the processing performed by the arithmetic circuit 400. The post-processing circuit 140 stores the obtained output data (oAct) in the activity value data memory 500. This will be described in more detail below.

Figure 7 is a diagram showing the configuration of the weight generation circuit 200 according to the first embodiment. The weight generation circuit 200 is configured to generate weights 14 when performing inference using a random number generator configured to generate the same random numbers as those generated in the learning stage using the same seed as those used in the learning stage. The weight generation circuit 200 can perform processing in parallel for each output channel (i.e., for each kernel). In other words, the weight generation circuit 200 can generate weights 14 for multiple kernels (i.e., output channels) in parallel.

The weight generation circuit 200 has weight generation units 210 that perform processing for each output channel (i.e., for each kernel) equal to the number of output channels that can be processed in parallel. In the example of Figure 7, the weight generation circuit 200 performs processing for each of 16 output channels (16oChs). In other words, the weight generation circuit 200 has 16 weight generation units 210. The weight generation circuit 200 has a random number generator 202 and a selector 204 for each output channel (i.e., for each weight generation unit 210). The random number generator 202 can be implemented using a pseudo-random number generation algorithm such as Xorshift16, for example.

The number of output channels of the neural network model may be greater than the number of output channels to be processed (i.e., the number of weight generation units 210). In other words, the number of output channels does not have to be 16, and may be, for example, 256 or 512. If there are 256 output channels, the weight generation circuit 200 may perform processing on each of the 16 output channels 16 times. This also applies to the mask expansion circuit 300 and arithmetic circuit 400 described below.

As described above, the weight generation unit 210 generates a weight for each kernel. That is, one weight generation unit 210 generates a kernel weight group 220, which is a collection of weights corresponding to one kernel. Here, the kernel weight group 220 is configured three-dimensionally to correspond to the three-dimensional structure of the input data. The kernel weight group 220 has a width kw corresponding to the horizontal direction of the input data, a height kh corresponding to the vertical direction of the input data, and a depth of 16 channels (16iChs) corresponding to the input channels to be processed of the input data. One weight generation unit 210 generates a kernel weight portion 222 of size 1 x 1 x 16 in one cycle. Note that the kernel weight portion 222 is a collection of 1-bit weights 14, equal to the number of input channels to be processed (16). Therefore, by the weight generation unit 210 repeating the process in kw x kh cycles, a kernel weight group 220 consisting of kw x kh kernel weight portions 222 is generated.

The seed input to the random number generator 202 can be generated by the control circuit 110 using a method described below. In the example of FIG. 7, the control circuit 110 generates a different 16-bit seed for each of the 16 output channels. Therefore, the weight generation circuit 200 (random number generator 202) generates random numbers using a different seed for each output channel. This allows the weight generation circuit 200 to generate different weights (different kernel weight portions 222 and kernel weight groups 220) for each output channel.

In the first cycle, the random number generator 202 generates a random number using the seed output from the control circuit 110. This generates the first kernel weight portion 222. Then, in the nth cycle (n is an integer greater than or equal to 2 and less than or equal to (kw × kh)), the random number generator 202 generates a random number using the random number (weight) generated in the (n-1)th cycle. This generates the nth kernel weight portion 222. Therefore, in the first cycle, the selector 204 selects the random number seed (initial value) obtained from the control circuit 110 and outputs it to the random number generator 202, and in the second cycle and thereafter, it selects the previous output (generated random number) of the random number generator 202. Each weight generation unit 210 outputs the generated kernel weight portion 222 to the calculation circuit 400. Therefore, in one cycle, the weight generation circuit 200 outputs 16 kernel weight portions 222 to the calculation circuit 400, corresponding to the number of output channels to be processed. As a result, the weight generation circuit 200 outputs 16 (iCh) x 16 (oCh) = 256 (i.e., 256 bits) weights 14 to the calculation circuit 400 in one cycle.

In this way, the weight generation circuit 200 is configured to generate weights in the inference stage that are the same as the weights set in the learning stage. This eliminates the need to store weights, making it possible to reduce memory capacity. Furthermore, since there is no need to access memory when setting weights in the inference stage, it is possible to reduce memory accesses. This makes it possible to reduce power consumption, resulting in improved processing efficiency.

The control circuit 110 generates a seed using an output channel number (#oCh) and a layer number (#Layer). Here, the output channel number corresponds to the identification information (index number) of the output channel to be processed. The layer number corresponds to the identification information (index number) of a layer (convolutional layer) that constitutes the neural network model. For example, in processing the first layer, the control circuit 110 may use "#oCh=0" and "#Layer=0" when generating a seed to be supplied to the front-most weight generation unit 210 shown in FIG. 7.

In this way, by generating a seed using the output channel number (#oCh) and layer number (#Layer), a different seed can be generated for each process and output channel for each layer. This breaks the periodicity of random number generation for each process and output channel for each layer. In other words, random numbers can be generated with different periods for each process and output channel for each layer. Therefore, even if random numbers generated using a simple pseudo-random number generation algorithm such as Xorshift16 are used as weights, deterioration in inference accuracy can be suppressed. This makes it possible to use a pseudo-random number generation algorithm with a short pseudo-random number period. This allows the random number generator 202 to be made smaller.

The control circuit 110 may also generate a seed using any hash function that takes the output channel number (#oCh) and layer number (#Layer) as input. By generating a seed in this manner, it is possible to further reduce the amount of data stored for generating weights while suppressing degradation in inference accuracy.

Figure 8 is a diagram showing the configuration of the mask expansion circuit 300 according to the first embodiment. The mask expansion circuit 300 is configured to expand the supermask 20 in a compressed state when performing inference. As described above, prior to the inference process, the supermask 20 is compressed using the zero-run length compression method. The supermask 20 is compressed to a predetermined code length. In this embodiment, the code length may be determined according to the value of k corresponding to the supermask 20 to be processed.

Specifically, as described above, the smaller the value of k, the sparser the supermask 20. In other words, the smaller the value of k, the fewer the number of "1s" (i.e., the number of connections 13 determined to be "valid") in the supermask 20, and the more "0s" (i.e., the number of connections 13 determined to be "invalid"). Therefore, in this case, performing zero run length compression with a long code length can achieve a higher compression rate. On the other hand, the larger the value of k, the denser the supermask 20. In other words, the larger the value of k, the more "1s" there are in the supermask 20, and the fewer "0s" there are. Therefore, in this case, performing zero run length compression with a long code length can result in a longer data length after compression than before compression, so performing zero run length compression with a short code length can achieve a higher compression rate.

For example, the code length for zero run length compression is {2, 3, 4} bits. In this case, a supermask 20 with a k value of approximately 1 to 15% may be compressed using a compression method with a code length of "4 bits." A supermask 20 with a k value of approximately 15 to 25% may be compressed using a compression method with a code length of "3 bits." A supermask 20 with a k value of approximately 25 to 35% may be compressed using a compression method with a code length of "2 bits."

The mask expansion circuit 300 has a compressed data memory 302, a FIFO buffer 304, a decoder 306, and a mask memory 310. A FIFO buffer 304 and a decoder 306 are provided for each output channel to be processed (i.e., for each kernel to be processed). In the example of Figure 8, a FIFO buffer 304 and a decoder 306 are provided for each of the 16 output channels. As a result, the FIFO buffer 304 and the decoder 306 perform processing in parallel for each output channel (i.e., for each kernel).

The compressed data memory 302 (ZMEM: ZRL Memory) stores compressed supermask 20 (compressed supermask) data. The compressed data memory 302 acquires compressed supermask (ZRL Encoded Supermask) data. The compressed data memory 302 may acquire compressed supermask data from the control circuit 110. Here, as shown in FIG. 8, the compressed supermask data may be acquired in a state where it is divided into a predetermined data amount for each output channel and stored in the compressed data memory 302. For example, in FIG. 8, each piece of "oCh0" data stored in the compressed data memory 302 corresponds to the divided compressed supermask data corresponding to the kernel of output channel #0. Note that the supermask 20 acquired by the mask expansion circuit 300 does not have to be compressed. In this case, the acquired supermask 20 may be input directly to the mask memory 310 (described later) rather than to the compressed data memory 302 (pass-through mode).

The FIFO buffer 304 (FIFO: First-In First-Out) is configured to combine the divided compressed supermasks. Specifically, the FIFO buffer 304 combines the divided compressed supermasks in parallel for each output channel (i.e., for each kernel). For example, the frontmost FIFO buffer 304 shown in FIG. 8 combines multiple "oCh0" data stored in the compressed data memory 302. Note that the FIFO buffer 304 for each output channel may combine the divided compressed supermasks so that a compressed supermask corresponding to one kernel to be processed is generated. The FIFO buffer 304 for each output channel receives 32 bits of data from the compressed data memory 302 at a time. The FIFO buffer 304 may then output up to 4 bits (corresponding to the maximum code length) of data to the decoder 306 at a time. Here, the amount of data output corresponds to the code length of the compressed supermask. Code length information for the obtained compressed supermask may be received from the control circuit 110.

The decoder 306 (ZRL Decoder) is configured to expand (decode) the compressed supermask. The decoder 306 expands the compressed supermask in parallel for each output channel (i.e., for each kernel). The decoder 306 may receive code length information from the control circuit 110. The decoder 306 uses the received code length information to expand the compressed data output from the FIFO buffer 304. The decoder 306 may output the expanded supermask data to the mask memory 310, for example, in 8-bit increments for each output channel.

The mask memory 310 (SMEM: Supermask Memory) temporarily stores the expanded supermask data. The mask memory 310 stores the kernel mask group 320, which is a collection of supermask data corresponding to one kernel to be processed, in the same number as the number of kernels to be processed (number of output channels: 16oChs). Here, the size of the kernel mask group 320 is substantially the same as the size of the kernel weight group 220 described above. In other words, the kernel mask group 320 has a width kw corresponding to the horizontal direction of the input data, a height kh corresponding to the vertical direction of the input data, and a depth of 16 bits (16iChs) corresponding to the input channel of the input data to be processed.

The mask memory 310 outputs a kernel mask portion 322 of size 1 x 1 x 16 to the arithmetic circuit 400 in one cycle in parallel for each output channel. The size of the kernel mask portion 322 is substantially the same as the size of the kernel weight portion 222 described above. Therefore, the kernel mask portion 322 is a collection of 1-bit mask data, the number of which is equal to the number of input channels to be processed (16). The kernel mask group 320 is then composed of kw x kh kernel mask portions 322. The kernel mask portion 322 is then output to the arithmetic circuit 400 in kw x kh cycles for each output channel (kernel).

Mask memory 310 has two banks 312 (bank 312A (Bank 1) and bank 312B (Bank 2)). Banks 312A and 312B implement double buffering. Specifically, for example, data output from decoder 306 is first stored in bank 312A. Then, after data from kernel mask group 320 corresponding to the kernel to be processed is stored in bank 312A, the data stored in bank 312A is read out and output to arithmetic circuit 400, where it is used in the calculation. While the data stored in bank 312A is being read out and output to arithmetic circuit 400, data from kernel mask group 320 corresponding to the next kernel to be processed is stored in bank 312B.

When all the data stored in bank 312A has been read out and output to the arithmetic circuit 400, the data stored in bank 312B is read out and output to the arithmetic circuit 400, where it is used in the calculation. While the data stored in bank 312B is being read out and output to the arithmetic circuit 400, data from the kernel mask group 320 corresponding to the next kernel to be processed is stored in bank 312A. This process is repeated thereafter. This allows the processing corresponding to the arithmetic circuit 400 to be performed efficiently. In other words, because the mask memory 310 reads and writes (stores) data in parallel, the arithmetic circuit 400 can perform processing efficiently.

Here, the control circuit 110 synchronizes the timing at which the kernel weight portion 222 is output from the weight generation circuit 200 (random number generator 202) with the timing at which the kernel mask portion 322 is output from the mask expansion circuit 300 (mask memory 310). Specifically, the control circuit 110 outputs a read signal (read_addr) to the mask memory 310 at the timing corresponding to the timing at which the seed is output to the weight generation circuit 200. This allows the kernel weight portion 222 and kernel mask portion 322 corresponding to the same position (position on kw × kh) in the same kernel to be output to the calculation circuit 400 at substantially the same timing.

Figure 9 is a diagram for explaining input data according to the first embodiment. As described above, the input data 510 has a three-dimensional structure consisting of the horizontal direction (width) of the image, the vertical direction (height) of the image, and the input channel (depth: iChs). In the first embodiment, chunks 520 are considered as units of calculation of the input data 510 in the calculation circuit 400. The chunks 520 correspond to a portion of the data in the three-dimensionally structured input data 510. Data corresponding to one chunk 520 can be input to the calculation circuit 400 at one time.

For example, chunk 520 has a width of cw = 4, height ch = 4, and depth (input channel) iCh = 16. In this case, if the size of input data 510 is Width = 12, Height = 12, and iChs = 48, then input data 510 has 3 x 3 x 3 = 27 chunks 520. Here, input data 510 is, for example, a collection of 8-bit data. In this case, the data size of input data 510 is 12 x 12 x 48 x 8 [bits]. Furthermore, the data size of one chunk 520 is 4 x 4 x 16 x 8 [bits].

Figure 10 is a diagram for explaining the operation of the arithmetic circuit 400 according to the first embodiment. As described above, the kernel weight portions 222 corresponding to each output channel (kernel) are input from the weight generation circuit 200 to the arithmetic circuit 400. In the example of Figure 10, as indicated by the upward arrows, the kernel weight portions 222 corresponding to each of the 16 output channels (16 oChs) are input to the arithmetic circuit 400. Similarly, the mask expansion circuit 300 inputs the kernel mask portions 322 corresponding to each output channel (kernel) to the arithmetic circuit 400. In the example of Figure 10, as indicated by the downward arrows, the kernel mask portions 322 corresponding to each of the 16 output channels (16 oChs) are input to the arithmetic circuit 400.

The arithmetic circuit 400 also receives input of chunk 520 data. Here, chunk 520 is composed of cw x ch chunk portions 522. In the above example, chunk 520 has 4 x 4 = 16 chunk portions 522. The size of chunk portion 522 is 1 x 1 x 16. In other words, the size of chunk portion 522 corresponds to the size of kernel weight portion 222 and kernel mask portion 322. The 16 (= 4 x 4) chunk portions 522 are then input in parallel to the arithmetic circuit 400.

The arithmetic circuit 400 has a large number of processing elements (PEs). The arithmetic circuit 400 also has multiple PE matrices 410. The number of PE matrices 410 corresponds to the number of output channels to be processed (i.e., the number of kernels to be processed). In the example of FIG. 10, the arithmetic circuit 400 has 16 PE matrices 410. Each PE matrix 410 has multiple PE vectors 420. Each PE matrix 410 has a number of PE vectors 420 corresponding to the number of input chunk portions 522 (cw × ch). In the example of FIG. 10, each PE matrix 410 has 16 (= 4 × 4) PE vectors 420. In other words, the PE vectors 420 in the PE matrix 410 correspond to the positions (positional deviations) of the chunk portions 522 in the chunks 520. Furthermore, as described below, the PE vector 420 has a number of processing elements corresponding to the number of input channels to be processed.

Here, the kernel weight portion 222 and kernel mask portion 322 corresponding to the same position (position on kw × kh) in the same kernel are input to the same PE matrix 410. That is, assume that the kernel weight group 220 and kernel mask group 320 shown in FIG. 10 correspond to the same kernel. In this case, the kernel weight portion 222 at the position (kw, kh) = (x1, y1) in this kernel weight group 220 and the kernel mask portion 322 at the position (kw, kh) = (x1, y1) in this kernel mask group 320 are input to the same PE matrix 410. In this case, in parallel with the above, the kernel weight portion 222 and kernel mask portion 322 at the position (kw, kh) = (x1, y1) for a different kernel are input to a different PE matrix 410.

These kernel weight portions 222 and kernel mask portions 322 are then input to multiple PE vectors 420 of the same PE matrix 410. In other words, the same kernel weight portion 222 and the same kernel mask portion 322 are input to multiple PE vectors 420 included in a certain PE matrix 410. In other words, each PE matrix 410 is associated with a respective kernel (output channel).

Furthermore, chunk portions 522 at the same position are input to PE vectors 420 at the same position in each PE matrix 410, as indicated by the right-pointing arrows. In other words, each of the multiple chunk portions 522 is input to the PE vector 420 in each PE matrix 410 that corresponds to its position in that chunk 520. For example, in Figure 10, the topmost chunk portion 522 is input to the topmost PE vector 420 in each PE matrix 410. The same applies to chunk portions 522 at other positions. In this way, the PE vectors 420 in each PE matrix 410 are associated with positions (positional offsets) within the chunk 520.

Therefore, each PE vector 420 receives the chunk portion 522 at a position within the chunk 520 that corresponds to that PE vector 420. Furthermore, each PE vector 420 receives the kernel weight portion 222 and kernel mask portion 322 for the kernel that corresponds to that PE vector 420 (PE matrix 410). Then, as described below, each PE vector 420 performs a multiply-and-accumulate operation using these chunk portion 522, kernel weight portion 222, and kernel mask portion 322.

Note that by inputting another chunk 520 of the input data 510 to the arithmetic circuit 400, similar product-sum operations can be performed on the data of the other chunk 520. Furthermore, by inputting the kernel weight portion 222 and kernel mask portion 322 at other positions (other positions in kw × kh) in the kernel weight group 220 and kernel mask group 320, respectively, to the arithmetic circuit 400, similar product-sum operations can be performed on positions on other kernels. For example, assume that the kernel weight portion 222 and kernel mask portion 322 corresponding to a certain position of each kernel (e.g., (kw, kh) = (x1, y1)) are input to each PE vector 420 in the corresponding PE matrix 410. Meanwhile, by changing the input chunk 520, the arithmetic circuit 400 can perform product-sum operations on the above positions for all data in the input data 510. Then, the kernel weight portion 222 and kernel mask portion 322 corresponding to a different position of each kernel (for example, (kw, kh) = (x1, y2)) are input to each PE vector 420 in the corresponding PE matrix 410. Meanwhile, by changing the input chunk 520, the arithmetic circuit 400 can similarly perform a multiply-and-accumulate operation for the above positions for all data in the input data 510. In this way, a multiply-and-accumulate operation can be performed for all positions in the input data 510 for all positions in each kernel. Note that the timing of inputting this data may be adjusted by, for example, the control circuit 110 using a clock input to the neural network circuit device 100.

Figure 11 is a diagram illustrating the processing of the PE vector 420 according to the first embodiment. The PE vector 420 has a number of processing elements 430 (PEs) corresponding to the number of input channels to be processed. In the above example, the number of input channels to be processed is 16, so the PE vector 420 has 16 processing elements 430 (16 PEs). Note that due to space limitations, only eight processing elements 430 are shown in Figure 11.

As described above, one PE vector 420 receives one kernel weight portion 222, one kernel mask portion 322, and one chunk portion 522 as input. Then, data corresponding to each input channel of the kernel weight portion 222, kernel mask portion 322, and chunk portion 522 is input to one arithmetic element 430. That is, the PE vector 420 has an arithmetic element 430 corresponding to each input channel. For example, data for input channel #0 of the kernel weight portion 222, kernel mask portion 322, and chunk portion 522 is input to the arithmetic element 430 corresponding to input channel #0 (the arithmetic element 430 at the bottom right and forefront in FIG. 11). Furthermore, data for input channel #15 of the kernel weight portion 222, kernel mask portion 322, and chunk portion 522 is input to the arithmetic element 430 corresponding to input channel #15 (the arithmetic element 430 at the top left and furthest back in FIG. 11).

The arithmetic element 430 performs a logical product operation using the data from each input channel of the input kernel weight portion 222, kernel mask portion 322, and chunk portion 522. The PE vector 420 then calculates the sum of the logical products obtained by each of the 16 arithmetic elements 430 as a partial sum (PSUM). The arithmetic circuit 400 then outputs the partial sums obtained by each PE vector 420 (a total of 16 x 16 partial sums) to the post-processing circuit 140.

Here, the partial sums are
PSUM=Σif(SMask) {iAct×Weight}
...(1)
Here, Equation 1 means that for each input channel, when the value of the supermask (SMask) is "1" (i.e., not "0"), the product of the input data value (iAct) and the weight value (Weight) is calculated, and the sum of the products obtained for each input channel is calculated. In other words, Equation 1 means that an operation is performed so that the product of the input data and the weight value is enabled or disabled depending on the value of the supermask (1 or 0).

Here, Equation 1 is equivalent to Equation 2 below.
PSUM=Σ{(iAct and SMask)×Weight}
... (2)
Equation 2 means that for each input channel, the product of the logical product of the input data value (iAct) and the supermask value (SMask) and the weight value (Weight) is calculated, and the sum of the products obtained for each input channel is calculated.

The arithmetic element 430 according to the first embodiment performs the calculation corresponding to Equation 2. That is, the arithmetic element 430 (arithmetic circuit 400) multiplies the logical product of the input data input to a certain connection and the value (1 or 0) in the supermask indicating whether that connection is valid or invalid by the weight corresponding to that connection. The PE vector 420 (arithmetic circuit 400) then calculates the sum of the multiplied values obtained for each input channel as the partial sum (PSUM). Note that since the weight value is either "+1" or "-1," multiplying the logical product of the input data value (iAct) and the supermask value (SMask) by the weight value (Weight) corresponds to simple sign inversion.

For simplicity of explanation, a specific example of the above calculation will be described assuming that the number of input channels is four. It is assumed that the data input to the PE vector 420 is as follows:
"iAct" = {5, 3, 7, 1}
"SMask" = {1, 0, 1, 1}
"Weight" = {+1, -1, -1, +1}

In this case, for the first input channel, "SMask" is "1", so "iAct" is valid. The calculation element 430 corresponding to the first input channel calculates the product of "iAct" = "5" and "Weight" = "+1" as "+5". For the second input channel, "SMask" is "0", so the input data is invalid. Therefore, the calculation element 430 corresponding to the second input channel does not perform a product calculation.

For the third input channel, "SMask" is "1", so "iAct" is valid. The arithmetic element 430 corresponding to the third input channel calculates the product of "iAct" = "7" and "Weight" = "-1" as "-7". For the fourth input channel, "SMask" is "1", so "iAct" is valid. The arithmetic element 430 corresponding to the fourth input channel calculates the product of "iAct" = "1" and "Weight" = "+1" as "+1". The PE vector 420 then calculates the sum (PSUM) of the obtained products "+5", "-7", and "+1" as "-1".

Figure 12 is a timing chart illustrating the components of the neural network circuit device 100 according to the first embodiment. The numbers in the timing chart illustrated in Figure 12 indicate the index of the output channel (kernel). At timing T0, the control circuit 110 outputs information on the code lengths of the compressed supermasks for the 16 kernels #0 to #15 (output channels #0 to #15) to the decoder 306 of the mask expansion circuit 300. Between timings T0 and T1, the decoder 306 expands the compressed supermasks for kernels #0 to #15 and stores the expanded supermasks 20 in bank 312A of the mask memory 310.

At timing T1, the control circuit 110 outputs a seed for generating weights 14 for each of kernels #0 to #15 to the random number generator 202 of the weight generation circuit 200. As a result, the random number generator 202 generates weights 14 for kernels #0 to #15 and outputs the generated weights 14 to the arithmetic circuit 400. Meanwhile, at timing T1, the control circuit 110 outputs a read signal to bank 312A for reading out supermasks 20 for each of kernels #0 to #15. As a result, the supermasks 20 for each of kernels #0 to #15 are output to the arithmetic circuit 400. Therefore, the arithmetic circuit 400 performs calculations for kernels #0 to #15 (output channels #0 to #15) between timings T1 and T2.

Furthermore, at timing T1, the control circuit 110 outputs information on the code lengths of the compressed supermasks for the next 16 kernels #16 to #31 (output channels #16 to #31) to the decoder 306 of the mask expansion circuit 300. Between timings T1 and T2, the decoder 306 expands the compressed supermasks for kernels #0 to #15 and stores the expanded supermasks 20 in bank 312B of the mask memory 310. In this way, the calculations for kernels #0 to #15 (output channels #0 to #15) and the expansion of the compressed supermasks for the next kernels #16 to #31 (output channels #16 to #31) are performed in parallel.

At timing T2, the control circuit 110 outputs a seed for generating weights 14 for each of kernels #16 to #31 to the random number generator 202 of the weight generation circuit 200. As a result, the random number generator 202 generates weights 14 for kernels #16 to #31 and outputs the generated weights 14 to the arithmetic circuit 400. Meanwhile, at timing T2, the control circuit 110 outputs a read signal to bank 312B for reading out supermasks 20 for each of kernels #16 to #31. As a result, the supermasks 20 for each of kernels #16 to #31 are output to the arithmetic circuit 400. Therefore, the arithmetic circuit 400 performs calculations for kernels #16 to #31 (output channels #16 to #31) between timings T2 and T3.

Furthermore, at timing T2, the control circuit 110 outputs information about the code lengths of the compressed supermasks for the next 16 kernels #32 to #47 (output channels #32 to #47) to the decoder 306 of the mask expansion circuit 300. Between timings T2 and T3, the decoder 306 expands the compressed supermasks for kernels #16 to #31 and stores the expanded supermasks 20 in bank 312A of the mask memory 310. In this way, the calculations for kernels #16 to #31 (output channels #16 to #31) and the expansion of the compressed supermask for the next kernels #32 to #47 (output channels #32 to #47) are performed in parallel. Processing for the next 16 kernels #32 to #47 (output channels #32 to #47) and the following 16 kernels #48 to #63 (output channels #48 to #63) is then performed in a similar manner.

Figure 13 is a diagram showing the configuration of the post-processing circuit 140 according to the first embodiment. The post-processing circuit 140 includes a floating-point type converter 141, a batch norm unit 142, a block floating-point type converter 143, an activation function processor 144, and a pooling unit 145. The floating-point type converter 141 (INT2FP) converts PSUM data, which is of integer type, to floating-point type. This enables the post-processing circuit 140 to perform floating-point operations.

The batch normalization unit 142 (BN) performs batch normalization on the data converted to floating-point format. The block floating-point conversion unit 143 (FP2BFP) converts the normalized floating-point data to block floating-point format, thereby reducing the amount of data. The activation function processing unit 144 (ReLU) applies an activation function such as ReLU (Rectified Linear Unit) to the data converted to block floating-point format to activate the data. This obtains the output result of the activation function. The pooling unit 145 (MaxPool) performs max pooling on the output result.

The post-processing circuit 140 stores the obtained data for each output channel (kernel) in the activation value data memory 500. Then, using the data stored in the activation value data memory 500 as input data, the neural network circuit device 100 repeats the above-mentioned process for the next neural network layer. In this way, the neural network circuit device 100 processes multiple layers of the neural network. In other words, the neural network circuit device 100 processes each layer of the neural network by repeating the loop process shown in FIG. 5, which starts from the activation value data memory 500, passes through the barrel shifter 130, the arithmetic circuit 400, and the post-processing circuit 140, and returns to the activation value data memory 500.

<Computation result>
14 to 17 are diagrams showing the results of calculations performed by the neural network circuit device 100 according to the first embodiment. FIG. 14 shows that a reduction in external memory access can be achieved when the technique according to the first embodiment is applied. By generating weights 14 inside the neural network circuit device 100 (chip) using the weight generation circuit 200, external memory access can be reduced by 48.0%. Furthermore, by expanding the compressed supermask using the mask expansion circuit 300, external memory access can be reduced by 45.9%. In this way, the neural network circuit device 100 (on-chip model construction) according to the first embodiment can reduce external memory access.

Figure 15 shows that when the technique according to the first embodiment is applied to ResNet 50, the amount of data related to weights 14 stored within the device (within the chip) can be reduced. By generating weights 14 by generating random numbers using a different seed for each output channel (Random Seed), the amount of data stored for setting weights 14 can be reduced by 98.2% compared to not using this technique (Binaried Weight). Furthermore, by generating weights 14 by generating seeds using a hash function that takes the output channel number and layer number as input (Hashed Seed), the amount of data stored for setting weights 14 can be reduced by 100%. In this way, the neural network circuit device 100 according to the first embodiment is able to reduce the amount of data stored for setting weights 14.

Figure 16 shows that accuracy degradation can be suppressed even when the method according to the first embodiment is applied. Compared to the existing case (PyTorch Default), accuracy degradation can be suppressed even when weights 14 are generated by generating random numbers using different seeds for each output channel (Random Seed). Furthermore, accuracy degradation can be suppressed even when weights 14 are generated by generating seeds using a hash function that takes the output channel number and layer number as input (Hashed Seed), compared to the existing case (PyTorch Default).

Figure 17 shows the effect of compressing the supermask 20 using the method according to the first embodiment. Figure 17 shows the compression ratio when compressing the supermask 20 with k values of 10%, 20%, and 30%. The supermask 20 with k value of 10% has a large number of "0"s and a small number of "1"s. Therefore, in this case, the compression ratio can be increased by compressing using a compression method with a code length of "4 bits" (4-bit encoding). The compression ratio in this case is approximately 0.5, which is a 49.6% improvement (data volume reduction) compared to when the supermask 20 is not compressed.

Furthermore, a supermask 20 with k set to 20% has fewer "0s" and more "1s" than a supermask 20 with k set to 10%. Therefore, in this case, the compression ratio can be increased by compressing using a compression method with a code length of "3 bits" (3-bit encoding). The compression ratio is approximately 0.75. Furthermore, a supermask 20 with k set to 30% has fewer "0s" and more "1s" than a supermask 20 with k set to 20%. Therefore, in this case, the compression ratio can be increased by compressing using a compression method with a code length of "2 bits" (2-bit encoding). The compression ratio is approximately 0.9. Thus, the sparser the corresponding supermask 20 (the fewer "1s"), the higher the compression ratio of the compressed supermask.

(Modification)
The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention. For example, the neural network circuit device 100 according to the first embodiment described above does not need to include the mask expansion circuit 300. In other words, the supermask 20 does not need to be provided in a compressed state. On the other hand, if the supermask 20 is provided in a compressed state and the mask expansion circuit 300 expands the compressed supermask, memory accesses can be reduced, as described above.

Furthermore, in the above-described embodiment, the number of input channels to be processed is 16, but this is not limited to this configuration. The number of input channels to be processed does not have to be 16. Further, in the above-described embodiment, the number of output channels (kernels) to be processed is 16, but this is not limited to this configuration. The number of output channels (kernels) to be processed does not have to be 16. Furthermore, in the above-described embodiment, the horizontal and vertical sizes of chunk 520 are 4x4, but this is not limited to this configuration. The horizontal and vertical sizes of chunk 520 do not have to be 4x4.

In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disk (DVD), Blu-ray® disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

10 Initial model 12 Node 13 Connection 14 Weight 20 Supermask 30 Score 40 Sub-network 100 Neural network circuit device 110 Control circuit 130 Barrel shifter 140 Post-processing circuit 141 Floating point type conversion unit 142 Batch norm unit 143 Block floating point type conversion unit 144 Activation function processing unit 145 Pooling unit 200 Weight generation circuit 202 Random number generator 204 Selector 210 Weight generation unit 220 Kernel weight group 222 Kernel weight part 300 Mask expansion circuit 302 Compressed data memory 304 FIFO buffer 306 Decoder 310 Mask memory 312 Bank 320 Kernel mask group 322 Kernel mask part 400 Arithmetic circuit 410 PE matrix 420 PE vector 430 Arithmetic element 500 Activity value data memory 510 Input data 520 Chunk 522 Chunk part

Claims (9)

A neural network circuit device that realizes a neural network model in which, in a learning stage, weights of each connection between nodes are set by fixed random numbers, a mask is generated by determining whether each of the connections is valid or invalid, and the weights and the mask are used to generate a neural network model,
a weight generation circuit that generates the weights using a random number generator configured to generate the same random numbers as those generated in the learning stage using the same seed as those used in the learning stage when performing inference;
an arithmetic circuit that performs a product-sum operation using input data that is inference target data or activation value data corresponding to the inference target data, the weight, and the mask;
and
the weight generation circuit generates the random numbers as the weights, the random numbers being generated using a different seed for each output channel;
Neural network circuit device. a control circuit that generates the seed using an output channel number and a layer number of a layer that constitutes the neural network model;
2. The neural network circuit device of claim 1 , further comprising: the control circuit generates the seed by a hash function that receives the output channel number and the layer number as input.
3. The neural network circuit device according to claim 2 . a mask expansion circuit for expanding the compressed mask when making an inference;
and
the arithmetic circuit performs a product-sum operation using the mask expanded by the mask expansion circuit;
4. The neural network circuit device according to claim 1. The mask in a compressed state is compressed using a zero run length compression method.
5. The neural network circuit device according to claim 4 . The mask in the compressed state is compressed at a higher compression rate as the mask becomes sparser.
6. The neural network circuit device according to claim 5 . the calculation circuit performs a calculation such that a product of the input data and the weight is enabled or disabled by the mask.
7. The neural network circuit device according to claim 1. the arithmetic circuit multiplies a logical product of the input data input to a certain connection and a value in the mask indicating validity or invalidity corresponding to the connection by the weight corresponding to the connection;
8. The neural network circuit device according to claim 7 . The mask is generated for at least one of the accuracy of inference and the inference target.
9. The neural network circuit device according to claim 1.