JP7282070B2 - compressed data - Google Patents

Description

The present invention relates to compressed data in which information about the configuration of a neural network is encoded and compressed.

Machine learning is a method for solving input data classification and regression problems.
In machine learning, there is a method called a neural network that simulates the neural circuits (neurons) of the brain. In a neural network, classification (discrimination) or regression of input data is performed by a probabilistic model (discrimination model, generative model) represented by a network in which neurons are interconnected.
In addition, the convolutional neural network, which is a neural network with not only a fully-connected layer but also a convolution layer and a pooling layer, realizes data filtering. It is possible to generate networks that perform data processing other than classification and regression, such as networks that For example, using an image or sound as an input, image or sound filter processing that realizes noise reduction or high quality of the input signal, high-frequency restoration processing of compressed sound that has lost high-frequency, and partial area Restoration processing (inpainting) of an image in which a pixel is missing, super-resolution processing of an image, and the like can be realized with a convolutional neural network.
In addition, the data generated by the generative model is input to the identification model that determines whether it is genuine data (is not the data generated by the generative model), and the authenticity of the data is determined. A network is constructed by combining a model, and the generative model is generated data so that the discriminative model cannot detect that the generated data is generated data. In recent years, a new neural network called a generative adversarial network that achieves generation of a highly accurate generative model through adversarial learning has also been announced.

These neural networks can achieve high performance by optimizing network parameters through learning using a large amount of data.
However, the data size of neural networks tends to increase, and the computational load on computers using neural networks is also increasing.

On the other hand, Non-Patent Document 1 describes a technique for encoding by scalar quantizing edge weights, which are parameters of a neural network. By scalar quantizing and encoding edge weights, the data size of edge-related data is compressed.

Vincent Vanhoucke, Andrew Senior, Mark Z.; Mao, "Improving the speed of neural networks on CPUs," Proc. Deep Learning and Unsupervised Feature Learning NIPS Workshop, 2011.

However, the optimum value of the weight given to each of the plurality of edges in the neural network differs depending on the learning result of the network and is not constant.
For this reason, variations occur in the compression size of edge weights, and the technique described in Non-Patent Document 1 has a problem that high compression of parameter data relating to neural network edges cannot be achieved.

SUMMARY OF THE INVENTION An object of the present invention is to obtain compressed data capable of highly compressing parameter data of a neural network.

Compressed data according to the present invention includes quantization information defining a quantization step, which is a quantization width when quantizing parameter data of a neural network, and parameter data of the neural network quantized by the quantization step. and network configuration information, wherein the quantization information is encoded by the first data processing device either per edge, per node, per kernel, or per layer of the neural network. The quantization step determined as the minimum switching unit is defined, and the network configuration information is calculated by the first data processing device quantizing the parameter data at the quantization step defined in the quantization information. the quantization information and the network configuration information are encoded with a constant compression size between the first data processing device and the second data processing device; An apparatus decodes the encoded quantization information and network configuration information , and the decoded quantization information indicates the minimum switching unit for either each edge, each node, each kernel, or each layer of the neural network. Inverse quantization of the quantization parameter data included in the decoded network configuration information at the quantization step of the minimum switching unit, and using the inverse quantization parameter data to generate a neural network having the configuration indicated by the network configuration information. It is used .

According to the present invention, the compressed data includes quantization information defining a quantization step for quantizing the parameter data of the neural network and parameter data quantized by the quantization step in the quantization information. It is generated by encoding information. Thereby, the parameter data of the neural network can be highly compressed. By using quantization information and network configuration information decoded from compressed data, a neural network optimized on the encoding side can be configured on the decoding side.

1 is a block diagram showing the configuration of a data processing device (encoder) according to Embodiment 1 of the present invention; FIG. 1 is a block diagram showing a configuration of a data processing device (decoder) according to Embodiment 1; FIG. 3A is a block diagram showing a hardware configuration that implements the functions of the data processing apparatus according to Embodiment 1. FIG. 3B is a block diagram showing a hardware configuration for executing software realizing the functions of the data processing apparatus according to Embodiment 1. FIG. 4 is a flowchart showing the operation of the data processing device (encoder) according to Embodiment 1; 4 is a flowchart showing the operation of the data processing device (decoder) according to Embodiment 1; 1 is a diagram showing a configuration example of a neural network according to Embodiment 1; FIG. FIG. 4 is a diagram showing an example of convolution processing of one-dimensional data according to Embodiment 1; FIG. 4 is a diagram showing an example of convolution processing of two-dimensional data according to Embodiment 1; FIG. 4 is a diagram showing a matrix of edge weight information for each node in the first layer of the neural network; FIG. 4 is a diagram showing a matrix of quantization steps of edge weight information for each node in the first layer of the neural network; FIG. 4 is a diagram showing a matrix of edge weight information in a convolutional layer; FIG. 10 is a diagram showing a matrix of quantization steps of edge weight information in a convolutional layer; 4 is a diagram showing syntax of information forming quantization information in Embodiment 1. FIG. FIG. 3 is a diagram showing syntax for each matrix of information constituting quantized information according to Embodiment 1; FIG. 2 is a diagram showing a syntax for each layer of information forming quantized information according to Embodiment 1; FIG.

Hereinafter, in order to describe the present invention in more detail, embodiments for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1.
FIG. 1 is a block diagram showing the configuration of a data processing device 100 according to Embodiment 1 of the present invention. In FIG. 1, a data processing device 100 processes input data using a trained neural network and outputs a processing result.
The data processing apparatus 100 also functions as an encoder that encodes quantization information and network configuration information, and includes a data processing section 101 , a compression control section 102 and an encoding section 103 .

The data processing unit 101 processes input data using the above neural network.
The data processing unit 101 also receives the quantization information generated by the compression control unit 102, and quantizes the parameter data of the neural network at the quantization step defined in the quantization information. Then, data processing section 101 outputs network configuration information including the quantized parameter data to encoding section 103 .

A predetermined neural network may be used as the neural network used in the data processing unit 101, or parameter data may be optimized by learning.
When learning the parameter data of the neural network, the neural network in a predetermined initial state (initial value of the parameter data) is trained using the input data to be learned, and then the compression control unit 102 Input the quantization information generated by and quantize the parameter data of the neural network at the quantization step defined in the quantization information.
Then, using this quantized neural network as an initial state for the next learning, the above learning and quantization are performed. A neural network obtained as a result of repeating this learning and quantization processing L times (L is an integer equal to or greater than 1) is output to encoding section 103 as part of network configuration information.
In the case of L=1, since the neural network after quantization is not re-learned, it can be said that the processing is the same as using an externally trained neural network without learning in the data processing unit 101 . That is, the only difference is whether learning is performed in the data processing unit 101 or outside.

The network configuration information is information indicating the configuration of the neural network, and includes, for example, the number of layers in the network, the number of nodes in each layer, edges connecting nodes, weight information given to each edge, and activation representing the output of a node. Type information for each function and layer (eg, convolutional layer, pooling layer, fully connected layer), etc. are included.
Parameter data of a neural network includes, for example, weight information given to edges connecting nodes in the neural network.

The compression control unit 102 determines a quantization step when quantizing the parameter data of the neural network, and generates quantization information defining the quantization step.
For example, the compression control unit 102 determines a quantization step that switches for each edge, each node, each kernel, or each layer in the neural network.

The quantization information is information defining a quantization step for quantizing the parameter data of the neural network. A quantization step is a width (quantization width) when parameter data is quantized, and the larger the quantization step is, the coarser the parameter data is decomposed, resulting in a higher compression rate. The smaller the quantization step, the finer the parameter data is decomposed, resulting in a lower compression rate.
Specifically, the quantization value k is represented by the following formula (1).
k=floor((x/Q)+ _d0 )+ _d1 (1)
In the above formula (1), x is the value of the quantization target parameter, Q is the quantization step, and d ₀ (0≦d ₀ < 1) is the adjustment of the range of the quantization target value corresponding to each quantization value. d ₁ (0≦d 1 <1) is an offset, d 1 (0≦d ₁ <1) is an offset for adjusting the quantization value, and floor( ) is a decimal point rounding function. Furthermore, there is also a method of providing a dead zone in which the quantization value k of the quantization target value x within the range set for the above equation (1) is set to 0.
Also, the value y of the quantized parameter is given by the following equation (2).
y=kQ (2)
When the minimum switching unit of the quantization steps described above is changed from the edge unit to the layer unit, the compression rate of the parameter data by quantization increases, so the parameter data before encoding can be reduced.

The encoding unit 103 encodes the network configuration information including the parameter data quantized by the data processing unit 101 and the quantization information generated by the compression control unit 102 to generate compressed data.
The network configuration information input from the data processing unit 101 to the encoding unit 103 is network configuration information including parameter data quantized by the data processing unit 101 at the quantization step determined by the compression control unit 102. be.

FIG. 2 is a block diagram showing the configuration of the data processing device 200 according to Embodiment 1. As shown in FIG. In FIG. 2, a data processing device 200 processes input data using a neural network obtained by decoding compressed data, and outputs a processing result. Similar to the data processing apparatus 100, the processing result includes the classification result of the input data or the regression analysis result.
Data processing device 200 functions as a decoder that decodes quantization information and network configuration information from compressed data, and includes decoding section 201 and data processing section 202 .

Decoding section 201 decodes quantization information and network configuration information from the compressed data encoded by encoding section 103 as described above.
The network configuration information decoded by the decoding unit 201 includes the result (quantization value k) of quantizing parameter data such as edge weight information optimized by the learning result of the data processing device 100, which is an encoder. is
The quantization information decoded by the decoding unit 201 defines the quantization step Q when quantizing the parameter data. From the quantization result k of the parameter data and the quantization step Q, the quantized parameter y is decoded according to the above equation (2). These decoding results are output from the decoding section 201 to the data processing section 202 .

The data processing unit 202 processes input data using a neural network.
The data processing unit 202 also uses the quantization information and network configuration information decoded from the compressed data by the decoding unit 201 to inversely quantize edge weight information, which is parameter data. Furthermore, the data processing unit 202 configures a neural network using network configuration information including dequantized parameter data.

In this way, the data processing unit 202 uses information decoded from the compressed data to configure a neural network including parameter data such as edge weight information optimized by the learning results on the data processing device 100 side. , processes the input data using this neural network. As a result, the compression size of the optimized parameter data can be made constant between the data processor 100 and the data processor 200, and high compression of the parameter data can be realized.

FIG. 3A is a block diagram showing a hardware configuration that implements the functions of the data processing device 100. As shown in FIG. In FIG. 3A, processing circuitry 300 is a dedicated circuitry that functions as data processing device 100 . FIG. 3B is a block diagram showing a hardware configuration for executing software realizing the functions of the data processing device 100. As shown in FIG. In FIG. 3B, processor 301 and memory 302 are connected together by a signal bus.

The functions of the data processing unit 101, the compression control unit 102, and the encoding unit 103 in the data processing device 100 are implemented by processing circuits.
That is, the data processing device 100 has a processing circuit for executing the processing from step ST1 to step ST3, which will be described later with reference to FIG.
The processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in memory.

If the processing circuit is the dedicated hardware shown in FIG. 3A, the processing circuit 300 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA. (Field-Programmable Gate Array) or a combination thereof.
The functions of the data processing unit 101, the compression control unit 102, and the encoding unit 103 may be realized by separate processing circuits, or these functions may be collectively realized by one processing circuit.

When the processing circuit is the processor shown in FIG. 3B, the functions of the data processing unit 101, the compression control unit 102, and the encoding unit 103 are implemented by software, firmware, or a combination of software and firmware.
Software or firmware is written as a program and stored in memory 302 .
The processor 301 implements the functions of the data processing unit 101 , the compression control unit 102 and the encoding unit 103 by reading and executing programs stored in the memory 302 . That is, data processing apparatus 100 includes memory 302 for storing a program that, when executed by processor 301, results in the processing from steps ST1 to ST3 shown in FIG.
These programs cause the computer to execute the procedures or methods of the data processing unit 101 , compression control unit 102 and encoding unit 103 .
The memory 302 may be a computer-readable storage medium storing a program for causing the computer to function as the data processing unit 101, the compression control unit 102, and the encoding unit 103.

The memory 302 includes, for example, non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically-EPROM), Magnetic discs, flexible discs, optical discs, compact discs, mini discs, DVDs, and the like are applicable.

The functions of the data processing unit 101, the compression control unit 102, and the encoding unit 103 may be partly realized by dedicated hardware and partly by software or firmware.
For example, the function of the data processing unit 101 is realized by a processing circuit as dedicated hardware, and the compression control unit 102 and the encoding unit 103 are executed by the processor 301 reading out a program stored in the memory 302. The function may be realized by
As such, the processing circuitry may implement each of the above functions through hardware, software, firmware, or a combination thereof.

Although the data processing device 100 has been described, the same applies to the data processing device 200 as well. For example, the data processing device 200 includes a processing circuit for executing the processing from step ST1a to step ST4a, which will be described later with reference to FIG. This processing circuit may be dedicated hardware, or may be a CPU that executes a program stored in memory.

If the processing circuitry is dedicated hardware as shown in FIG. 3A, processing circuitry 300 may be, for example, a single circuit, multiple circuits, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. is applicable.
The functions of the decoding unit 201 and the data processing unit 202 may be realized by separate processing circuits, or these functions may be collectively realized by one processing circuit.

If the processing circuit is the processor shown in FIG. 3B, the functions of the decoding unit 201 and the data processing unit 202 are implemented by software, firmware, or a combination of software and firmware.
Software or firmware is written as a program and stored in memory 302 .
The processor 301 implements the functions of the decoding unit 201 and the data processing unit 202 by reading and executing programs stored in the memory 302 .
That is, data processing apparatus 200 includes memory 302 for storing a program that, when executed by processor 301, results in the processing from steps ST1a to ST4a shown in FIG.
These programs cause the computer to execute the procedures or methods of the decoding unit 201 and the data processing unit 202 .
The memory 302 may be a computer-readable storage medium storing a program for causing the computer to function as the decoding unit 201 and the data processing unit 202 .

Note that the functions of the decoding unit 201 and the data processing unit 202 may be partly realized by dedicated hardware and partly by software or firmware.
For example, the function of the decoding unit 201 is realized by a processing circuit as dedicated hardware, and the function of the data processing unit 202 is realized by the processor 301 reading and executing a program stored in the memory 302. may be realized.

Next, the operation will be explained.
FIG. 4 is a flow chart showing the operation of the data processing device 100. As shown in FIG.
A case where the parameter data of the neural network is edge weight information will be described below.
The compression control unit 102 determines a quantization step when quantizing weight information of each of a plurality of edges forming a trained neural network, and generates quantization information in which the quantization step is defined ( step ST1). The quantization information is output from compression control section 102 to data processing section 101 and encoding section 103 .

When the data processing unit 101 receives the quantization information from the compression control unit 102, the data processing unit 101 quantizes the edge weight information of the neural network at the quantization step in the quantization information (step ST2). Data processing section 101 generates network configuration information including quantized edge weight information, and outputs the network configuration information to encoding section 103 .

Encoding section 103 encodes the network configuration information input from data processing section 101 and the quantization information input from compression control section 102 (step ST3).
The compressed data of the network configuration information and the quantization information encoded by the encoding section 103 are output to the data processing device 200 .

FIG. 5 is a flow chart showing the operation of the data processing device 200. As shown in FIG.
Decoding section 201 decodes quantization information and network configuration information from the compressed data encoded by encoding section 103 (step ST1a). The quantization information and network configuration information are output from decoding section 201 to data processing section 202 .

Next, the data processing unit 202 calculates inverse-quantized edge weight information using the quantization information decoded from the compressed data by the decoding unit 201 and the network configuration information (step ST2a).

Subsequently, the data processing unit 202 configures a neural network using network configuration information including weight information of the inversely quantized edges (step ST3a).
Thereby, the data processing device 200 can configure a neural network that has been trained by the data processing device 100 .

The data processing unit 202 processes the input data using the neural network constructed in step ST3a (step ST4a).

6 is a diagram illustrating a configuration example of a neural network according to Embodiment 1. FIG.
In the _neural network shown in FIG. 6, input data (x ₁ , x ₂ , . . . , x _N1 ) are processed in each layer, and processing results (y ₁ , . .
In FIG. 6, N _l (l=1, 2, . . . , L) indicates the number of nodes in the l-th layer, and L indicates the number of layers of the neural network.
A neural network, as shown in FIG. 6, has an input layer, a hidden layer and an output layer, and each of these layers has a structure in which a plurality of nodes are connected by edges.
The output value of each of the plurality of nodes can be calculated from the output value of the node in the previous layer connected by the edge, the weight information of the edge, and the activation function set for each layer.

An example of a neural network is a CNN (Convolutional Neural Network). In the hidden layer of CNN, a convolutional layer and a pooling layer are alternately connected, and a fully-connected neural network layer (fully-connected layer) is added according to the final output. ) is provided. A ReLU function, for example, is used as the activation function of the convolutional layer.
Note that a network called a DNN (Deep Neural Network) (also called deep learning, DCNN (Deep CNN), etc.) is a multi-layered CNN.

FIG. 7 is a diagram showing an example of one-dimensional data convolution processing according to Embodiment 1, and shows a convolution layer that performs one-dimensional data convolution processing. One-dimensional data includes, for example, audio data and time-series data.
The convolutional layer shown in FIG. 7 has nine nodes 10-1 to 10-9 in the previous layer and three nodes 11-1 to 11-3 in the next layer.
The same weight is given to the edges 12-1, 12-6 and 12-11, the same weight is given to the edges 12-2, 12-7 and 12-12, and the edges 12-3 and 12-12 are given the same weight. -8, 12-13 are given the same weight, edges 12-4, 12-9, 12-14 are given the same weight, edges 12-5, 12-10, 12-15 are given the same weight. Further, the weights of edges 12-1 to 12-5 may all have different values, or a plurality of weights may have the same value.

Of the nine nodes 10-1 to 10-9 in the previous layer, five nodes are connected to one node in the next layer with the above weights. The kernel size K is 5 and the kernel is defined by the combination of these weights.
For example, as shown in FIG. 7, node 10-1 connects to node 11-1 via edge 12-1, node 10-2 connects to node 11-1 via edge 12-2, and node 10-3 connects to node 11-1 via edge 12-3, node 10-4 connects to node 11-1 via edge 12-4, and node 10-5 connects to edge 12-5. It is connected to the node 11-1 through the A kernel is defined by a combination of weights of edges 12-1 to 12-5.

Node 10-3 connects to node 11-2 via edge 12-6, node 10-4 connects to node 11-2 via edge 12-7, node 10-5 connects to edge 12-8. , the node 10-6 is connected to the node 11-2 via the edge 12-9, and the node 10-7 is connected to the node 11-2 via the edge 12-10. there is A kernel is defined by a combination of weights of edges 12-6 through 12-10.
Node 10-5 connects to node 11-3 via edge 12-11, node 10-6 connects to node 11-3 via edge 12-12, node 10-7 connects to edge 12-13. , the node 10-8 is connected to the node 11-3 via an edge 12-14, and the node 10-9 is connected to the node 11-3 via an edge 12-15. there is A kernel is defined by a combination of weights of edges 12-11 to 12-15.

In the processing of input data using CNN, the data processing unit 101 or the data processing unit 202 uses a combination of edge weights of the convolutional layers to perform an interval of steps S (S=2 in FIG. 7) for each kernel. performs a convolution operation on . Edge weight combinations are determined by learning for each kernel.
Note that in CNNs for image recognition, networks are often configured with convolution layers having a plurality of kernels.

FIG. 8 is a diagram showing an example of convolution processing of two-dimensional data according to Embodiment 1, and shows convolution processing of two-dimensional data such as image data.
Among the two-dimensional data shown in FIG. 8, the kernel 20 is a block area having a size of K _x in the x direction and a size of K _y in the y direction. The kernel size K is K=K _x ×K _y .
The data processing unit 101 or the data processing unit 202 performs a convolution operation on the data for each kernel 20 in the two-dimensional data at intervals of the x-direction step number _Sx and the y-direction step number _Sy . Here, steps S _x and S _y are integers of 1 or more.

FIG. 9 is a diagram showing a matrix of edge weight information for each node in the l-th layer (l=1, 2, . . . , L), which is a fully connected layer of the neural network.
FIG. 10 is a diagram showing a matrix of quantization steps of edge weight information for each node in the l-th layer (l=1, 2, . . . , L) which is a fully connected layer of the neural network.

In a neural network, a combination of weights w _ij for each layer shown in FIG. 9 constitutes data constituting the network. Therefore, in a multilayer neural network such as a DNN, the amount of data is generally several hundred Mbytes or more, and a large memory size is also required. Note that i is a node index, i=1, 2, . . . , _Nl . j is the edge index, j=1, 2, . . . , N _l-1 .

Therefore, in the data processing apparatus 100 according to Embodiment 1, the weight information is quantized in order to reduce the data amount of the edge weight information. As shown in FIG. 10, the quantization step q _ij is set for each edge weight w _ij .
Furthermore, the quantization step may be common to multiple node indexes or multiple edge indexes, or multiple node indexes and edge indexes. By doing so, the quantization information to be encoded can be reduced.

FIG. 11 is a diagram showing a matrix of edge weight information in a convolutional layer.
FIG. 12 is a diagram showing a matrix of quantization steps of edge weight information in a convolutional layer. In the convolution layer, edge weights for one kernel are common to all nodes, and the kernel can be made into a small region by reducing the number of edges connected per node, that is, the kernel size K.

FIG. 11 shows data in which the edge weight w _i′j′ is set for each kernel, and FIG. 12 shows data in which the quantization step q _i′j′ is set for each kernel.
Note that i' is a kernel index and i'=1, 2, ..., _Ml (l = 1, 2, ..., L). j' is an edge index, j'=1, 2, . . . , _Kl .
Furthermore, the quantization step may be common to multiple kernel indices or multiple edge indices, or multiple kernel indices and edge indices. By doing so, the quantization information to be encoded can be reduced.

In step ST1 in FIG. 4, compression control section 102 determines a quantization step to be used in weight quantization processing by data processing section 101, and outputs the quantization step to data processing section 101 as quantization information. The quantization steps are the quantization step q _ij shown in FIG. 10 and the quantization step q _i′j′ shown in FIG.

In step ST2 in FIG. 4, data processing section 101 quantizes edge _{weights wij shown} in _FIG . 9 at quantization steps qij shown in FIG. The information is output to encoding section 103 .
Similarly, in step ST2 of FIG. 4, data processing section 101 quantizes edge weight w _{i′j′ shown} in FIG. 11 at quantization step q i′j′ shown in FIG. network configuration information including the weights w _i′j′ that have been converted to encoding section 103 .
In addition to the quantized weights, the network configuration information includes the number of network layers, the number of nodes in each layer, edges connecting nodes, weight information assigned to each edge, and an activation function representing the output of a node. , type information for each layer (convolutional layer, pooling layer, fully connected layer), etc. are included. However, the information fixed (defined) in advance between the data processors 100 and 200 is not included in the network configuration information to be encoded.

13 is a diagram showing syntax of information forming quantization information according to Embodiment 1. FIG.
FIG. 14 is a diagram showing a syntax for each matrix of information constituting quantized information according to Embodiment 1. FIG.
FIG. 15 is a diagram showing a layer-by-layer syntax of information constituting quantization information according to the first embodiment.

In FIG. 13 , a flag quant_enable_flag, a flag layer_adaptive_quant_flag, a flag matrix_adaptive_quant_flag, and a quantization step fixed_quant_step are encoding parameters of quantization information encoded by the encoding section 103 .
Also, L is the number of layers.

14, quantization step base_quant_step[j], flag prev_quant_copy_flag[i−1], and difference value diff_quant_value[i−1][j] are encoding parameters of quantization information encoded by encoding section 103. is.
Also, C is the number of nodes N _{layer_id} or the number of kernels M _{layer_id} . Furthermore, E is the number of edges N _{layer_id−1} or the kernel size K _{layer_id} .

In FIG. 15 , quantization step base_layer_quant_step, flag layer_quant_copy_flag[i−2], and quantization step layer_quant_step[i−2] are encoding parameters of quantization information encoded by encoding section 103 . Also, L is the number of layers.

The information shown in FIG. 13 includes a flag quant_enable_flag for setting whether or not to quantize edge weight information in the network.
If the flag quant_enable_flag is 0 (false), weight information of all edges in the network is not quantized. That is, the quantization step is not set in the quantization information.
On the other hand, when the flag quant_enable_flag is 1 (true), the compression control unit 102 refers to the flag layer_adaptive_quant_flag.

When the flag layer_adaptive_quant_flag is 0 (false), the compression control unit 102 sets the quantization step fixed_quant_step common to all edges in the network in the quantization information.
When the flag layer_adaptive_quant_flag is 1 (true), the compression control unit 102 refers to the flag matrix_adaptive_quant_flag.

When the flag matrix_adaptive_quant_flag is 0 (false), the compression control unit 102 determines a common quantization step for each layer as a quantization step for weight information of each of multiple edges in the network.
However, since the input layer (first layer) does not have edges, no quantization step is set.
Note that FIG. 15 shows the syntax regarding the quantization steps common to each layer.

When the flag matrix_adaptive_quant_flag is 1 (true), the compression control unit 102 uses the quantization step shown in FIG. 10 or the quantization step shown in FIG. to decide. FIG. 14 shows the syntax of the quantization steps shown in FIG. 10 or FIG.

The syntax shown in FIG. 14 will be described.
As mentioned above, the input layer (first layer) does not have edges.
Therefore, the layer whose ID information layer_id is from 1 to L−1 (layer_id+1) becomes the target of setting the quantization step.
First, compression control section 102 performs base_quant_step[j] (j=0, 1 , . . . , E-1) are set.
Note that E is the number of edges N _{layer_id} or the kernel size K _{layer_id+1} .

Next, the compression control unit 102, for each node (or kernel) of the second and subsequent (i≧1) nodes (or kernels), has the same quantization step as the node (or kernel) of the previous index. Refer to the flag prev_quant_copy_flag[i-1] indicating whether or not.
If the flag prev_quant_copy_flag[i−1] is 1 (true), the i+1 th node (or kernel) has the same quantization step as the i th node (or kernel).
On the other hand, when the flag prev_quant_copy_flag[i−1] is 0 (false), the compression control unit 102 uses the difference value diff_quant_value[i−1] as information for generating the quantization step of the i+1-th node (or kernel). Set [j] (i = 1, 2, ..., C-1, j = 0, 1, ..., E-1).

The quantization step can be generated by adding the quantization step set in the previous node (or kernel) to the difference value diff_quant_value[i−1][j].
That is, at the second (i=1) node (or kernel), the quantization step is base_quant_step[j]+diff_quant_value[0][j]. For the third and subsequent (i≧2) nodes (or kernels), the quantization step is diff_quant_value[i−2][j]+diff_quant_value[i−1][j].

Although the difference value diff_quant_value[i−1][j] of quantization steps between nodes (or kernels) is shown as an encoding parameter, an independent quantization step is set for each node (or kernel). good too.
When the correlation of quantization steps between nodes (or kernels) tends to be low, this configuration increases the encoding efficiency in encoding section 103 .

The syntax shown in FIG. 15 will be described.
As mentioned above, the input layer (first layer) does not have edges.
Therefore, the compression control unit 102 sets base_layer_quant_step as a quantization step common to weight information of all edges in the second layer.
Next, for the third and subsequent layers (i≧2), compression control section 102 sets the quantization step common to all edge weight information in the i+1-th layer to all edge weight information in the i-th layer. A flag layer_quant_copy_flag[i−2] (i=2, 3, .

If the flag layer_quant_copy_flag[i−2] is 1 (true), the compression control unit 102 sets the quantization step common to the weight information of all edges in the i+1-th layer to that of all edges in the i-th layer. It is assumed to be the same as the quantization step common to weight information. On the other hand, if the flag layer_quant_copy_flag[i−2] is 0 (false), the compression control unit 102 sets layer_quant_step[i−2] as a quantization step common to all edge weight information in the i+1 layer. do.

Although compression control section 102 defines layer_quant_step[i−2] as an independent quantization step for each layer, the quantization step of the previous layer (i-th layer) is A layer_quant_step[i-2] may be defined as a difference value. Since many difference values near 0 are generated by using the difference value, the encoding efficiency in the encoding unit 103 can be improved.

The encoding unit 103 encodes the encoding parameters in FIGS. 13 to 15 as quantization information to generate compressed data.
Although the edge unit is used as the minimum switching unit of the quantization step, the node unit (the kernel unit in FIG. 12) may be used as the minimum switching unit of the quantization step as shown in FIG. This is the same as setting E=1 in FIG. In this case, the quantization step may be encoded independently in node units (kernel units in FIG. 12).

Also, the minimum switching unit of the quantization step may be a layer unit.
This is equivalent to always doing only layer_quant_coding( ) without the flag matrix_adaptive_quant_flag when the flag layer_adaptive_quant_flag=1 (true) in FIG. By making the minimum switching unit of the quantization step larger than the edge unit in this way, the data size of the quantization information before encoding can be reduced.

As described above, in the data processing apparatus 100 according to the first embodiment, the data processing unit 101 processes input data using a neural network. Compression control section 102 determines the quantization step and generates quantization information in which the quantization step is defined. The encoding unit 103 encodes the network configuration information including the parameter data quantized at the quantization step determined by the compression control unit 102 and the quantization information to generate compressed data.
In particular, in the above description, an example of handling weight information given to edges connecting nodes in a neural network as parameter data of the neural network has been described. With these configurations, quantization information defining a quantization step and network configuration information including parameter data quantized by the quantization step in the quantization information are encoded into compressed data. Thereby, the parameter data of the neural network can be highly compressed.
Also, by using quantization information and network configuration information decoded from compressed data, a neural network optimized on the encoding side can be configured on the decoding side.

In the data processing device 200 according to Embodiment 1, the data processing unit 202 processes input data using a neural network. A decoding unit 201 decodes the compressed data.
In this configuration, the data processing unit 202 inversely quantizes the parameter data using the quantization information and the network configuration information decoded from the compressed data by the decoding unit 201, and uses the network configuration information including the inversely quantized parameter data. to construct a neural network.
As a result, a neural network optimized on the encoding side can be constructed using the quantization information and network configuration information decoded from compressed data.

In data processing apparatus 100 according to Embodiment 1, compression control section 102 switches the quantization step for each edge. The encoding unit 103 encodes quantization information in which a quantization step for each edge is defined. By configuring in this way, the parameter data can be quantized with high precision.

In data processing apparatus 100 according to Embodiment 1, compression control section 102 switches the quantization step for each node or each kernel. The encoding unit 103 encodes quantization information in which a quantization step is defined for each node or each kernel.
Even with this configuration, the parameter data can be quantized with high precision.

In data processing apparatus 100 according to Embodiment 1, compression control section 102 switches the quantization step for each layer of the neural network. The encoding unit 103 encodes quantization information defining a quantization step for each layer of the neural network.
By configuring in this way, the compression rate of the parameter data by quantization is increased, so the data amount of the weight information before encoding can be reduced.

Embodiment 2.
In the first embodiment, an example in which the output result of the neural network is directly used as the result of data processing has been described. is used as a feature amount for data processing of image data and audio data, and a final data processing result is obtained by another data processing method as in the following references.
For example, when using the output of an intermediate layer of a neural network as an image feature amount for image processing such as image retrieval, matching, and object tracking, HOG (Histogram of Oriented Gradients), which is an image feature amount used in the conventional image processing, is used. ), SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), etc., by replacing or adding the image feature amount, image processing can be performed in the same processing flow as the image processing using the conventional image feature amount. realizable.
In this case, in the data processing apparatus 100, the network configuration information and the quantization information are encoded by the neural network up to the intermediate layer from which the output as the characteristic amount of the data processing is obtained.
Further, the data processing device 100 performs data processing such as image retrieval using the feature amount of the data processing. The data processing device 200 decodes the neural network from the compressed data to the intermediate layer, and performs data processing such as image retrieval using the output obtained by inputting the input data as a feature quantity for data processing.
(Reference) ISO/IEC JTC1/SC29/WG11/m39219, "Improved retrieval and matching with CNN feature for CDVA", Chengdu, China, Oct. 2016.

Therefore, in the data processing apparatus 100 according to the second embodiment, the compression ratio of the parameter data by quantization is increased, so that the data amount of weight information before encoding can be reduced. The data processing device 200 according to the second embodiment can perform data processing by decoding the compressed data output from the data processing device 100 and generating a neural network.

It should be noted that the present invention is not limited to the above-described embodiments, and any constituent elements of the embodiments may be modified or omitted within the scope of the present invention.

INDUSTRIAL APPLICABILITY A data processing apparatus according to the present invention can highly compress parameter data of a neural network, so that it can be used for image recognition technology, for example.

10-1 to 10-9, 11-1 to 11-3 nodes, 12-1 to 12-15 edges, 20 kernels, 100, 200 data processing units, 101, 202 data processing units, 102 compression control units, 103 codes conversion unit, 201 decoding unit, 300 processing circuit, 301 processor, 302 memory.

Claims (1)

quantization information defining a quantization step, which is a quantization width when quantizing the parameter data of the neural network;
Compressed data encoded with network configuration information including parameter data of the neural network quantized in the quantization step,
The quantization information is
wherein the quantization step is defined by the first data processing device, which is determined as a minimum switching unit of each edge, each node, each kernel, or each layer of the neural network;
The network configuration information is
The first data processing device includes quantized parameter data calculated by quantizing the parameter data at the quantization step defined in the quantization information,
The quantization information and the network configuration information are
encoded with a constant compression size between the first data processing device and the second data processing device;
The second data processing device is
decoding the encoded quantization information and the network configuration information ;
The network configuration decoded in the quantization step of the minimum switching unit indicated by the decoded quantization information among the minimum switching units for each edge , each node, each kernel, or each layer of the neural network. inversely quantizing the quantization parameter data included in the information;
Compressed data using the inversely quantized quantized parameter data to generate the neural network having the configuration indicated by the network configuration information.

Images