JP7014393B2 - Data processing device and data processing method in this - Google Patents

Description

The present invention relates to a data processing apparatus and a data processing method thereof, and more particularly to a data processing apparatus suitable for a convolutional operation in a convolutional neural network, and a data processing method thereof.

A convolutional neural network (hereinafter referred to as CNN as appropriate), which is a neural network with convolution (convolution) added, is widely recognized as machine learning that is particularly effective for image recognition.

FIG. 1 is a diagram showing an outline of a system configuration of CNN. The input data is processed in a plurality of layers (layers L1-L5)) structure.

In FIG. 1, each of the layers L1 and L2 includes a convolutional layer and a pooling layer, and this is repeated.

The convolution layer is a layer that multiplies the input data by the filter (kernel) features (convolutions the features). When the input data is image data, the input data (image) is multiplied by different filter features to obtain an image corresponding to the number of filters. Various features of the input image can be captured by using multiple filters, and patterns in the image can be detected by convolving the features.

The pooling layer is placed immediately after the convolutional layer, reducing the layer to make it easier to handle and reducing the position sensitivity of the extracted features.

The CNN then arranges a multi-layer perceptron fully coupled by layers L3-L5 to recognize the input data (image).

Here, the convolution calculation performed in the early layer requires a huge number of calculations. For this reason, power saving in such a part has become a very important issue.

However, if a plurality of arithmetic elements (PE: processing elements) are arranged in an array-type parallel arithmetic processing element, a large amount of wiring resources and transfer time are required to calculate the similarity with peripheral functional blocks. Will be.

In view of this point, the present inventors have previously provided a highly expandable data processing device capable of avoiding data collisions in communication between PEs and increasing PEs without being biased in a specific direction. It has been proposed (Patent Document 1).

In the above-mentioned invention technique, all PEs arranged in the n-dimensional direction constituting the n-dimensional network input / output data in synchronization with the transfer clock. Then, the first data is received from the first adjacent PE adjacent to the shift direction which is the direction of input / output of data, and the second data is output to the second adjacent PE adjacent to the opposite side to be adjacent to each other. It is an arithmetic unit having a feature that the data transfer rate between PEs is equal regardless of the shift direction.

Japanese Patent No. 5939572

The technique for CNN calculation so far is such that the processing amount of each PE becomes large in the parallel calculation even when the invention proposed in Patent Document 1 is followed, that is, it is technically the fastest but energy. The number of poor PEs or cores is much smaller than the number of data stored in memory.

In other words, the number of active processors and memory operations in each arithmetic step was basically smaller than possible in the CNN algorithm.

As a result, the time to resolve the CNN is much greater than the minimum and the energy to be resolved is high.

In view of this point, an object of the present invention is to provide a data processing device capable of performing high-speed calculation with low power consumption while maintaining parallelism by arranging a large number of arithmetic elements PE three-dimensionally, and a data processing method thereof. To do.

A first aspect according to the present invention that achieves the above object is a data processing apparatus having a plurality of processing elements having multiplication and addition functions in a three-dimensional direction, in which the plurality of processing elements are arranged in the two-dimensional direction. , Has a plurality of two-dimensional surfaces stacked in the Z-axis direction, has a filter memory in which feature weights are arranged corresponding to each of the plurality of two-dimensional surfaces, and input data is an upper surface in the Z-axis direction. The processing element arranged from the two-dimensional surface of the above, sequentially calculates the product of the input data and the feature weight by the multiplication function, calculates the two-dimensional convolution data, and is further transferred from the lower surface. It is characterized in that an operation of adding data and its own data is performed, and the operation result is transferred to a processing element on an adjacent upper surface.

In the first aspect according to the present invention to achieve the above object, as the first aspect, the processing element arranged in the two-dimensional direction is connected to the torus network, and the PE adjacent to the upper and lower surfaces is connected in the Z-axis direction. , It is characterized by being connected in both directions by a network.

In the first aspect according to the present invention to achieve the above object, as a second aspect, the two-dimensional convolution data is added to the transfer data from the adjacent processing element and its own data, and is further adjacent in the shift direction. It is characterized in that the addition operation result is transferred to a processing element.

In the first aspect according to the present invention to achieve the above object, as a third aspect, the processing element on the uppermost surface in the Z-axis direction is the two-dimensional convolution data transferred from the plurality of processing elements on the lower surface and itself. It is characterized in that 2.5-dimensional convolution data is calculated by adding the data of.

In the third aspect of the first aspect according to the present invention, which achieves the above object, as a fourth aspect, the 2.5-dimensional convolution data is characterized in that the processing element on the lower surface is sequentially shifted.

In any of the above aspects of the first aspect according to the invention that achieves the above object, the feature weight is divided by the number of two-dimensional planes in which the input data is arranged, and for each plane in which the input data is arranged. It is characterized in that it is arranged in the filter memory corresponding to the above, and at the time of the convolution operation, the feature weight arranged in the filter memory is broadcast to all the processing elements of the corresponding surfaces.

A second aspect according to the present invention that achieves the above object is a data processing method in a data processing apparatus having a plurality of processing elements having multiplication and addition functions in a three-dimensional direction, and the data processing apparatus has a plurality of each. The processing element is arranged in the two-dimensional direction, has a plurality of two-dimensional surfaces stacked in the Z-axis direction, and has a filter memory in which feature weights are arranged corresponding to each of the plurality of two-dimensional surfaces. , The process of arranging the input data from the two-dimensional surface of the upper surface in the Z-axis direction and the processing element arranged on one surface sequentially calculate the product of the input data and the feature weight by the multiplication function. It is characterized by having a step of calculating dimensional convolution data and a step of further performing a calculation of adding the data transferred from the lower surface and its own data and transferring the calculation result to a processing element on the adjacent upper surface.

In the second aspect according to the present invention to achieve the above object, as the first aspect, the two-dimensional convolution data is added to the transfer data from the adjacent processing element and its own data, and is further adjacent in the shift direction. It is characterized by having a step of transferring the addition operation result to a processing element.

In the second aspect according to the present invention to achieve the above object, as a second aspect, the processing element on the uppermost surface in the Z-axis direction is the two-dimensional convolution data transferred from the plurality of processing elements on the lower surface and itself. It is characterized by having a step of adding the data of the above and calculating the 2.5-dimensional convolution data.

A second aspect according to the present invention that achieves the above object is characterized in that, as a third aspect, there is a step of sequentially shifting the 2.5-dimensional convolution data to a processing element on the lower surface.

According to the feature configuration according to the present invention, the calculation of each layer of CNN can be executed with the minimum number of execution steps by performing parallel computation with the same number of processors as the data to be processed. Therefore, it can be executed at the minimum operating clock frequency that can achieve the execution time constraint required for real-time processing, and real-time and low power consumption calculation can be performed.

It is a figure which shows the outline of the system configuration of CNN. It is a figure which shows the PME (Processor in Memory) in the form of a three-dimensional array in a TAP (Tensor Array Processor). It is a block diagram of a functional configuration example of each PME. It is a figure which shows the 3D array of TAP expanded in each XY plane. It is a figure which shows the PME belonging to one plane in FIG. 4 in an enlarged manner. It is a figure which shows the change of the operation result by the convolution operation. It is a figure which shows the initial state of a certain layer in the data processing method of this invention. It is a figure which shows the state which perform 2D convolution about the 1st filter. It is a figure which shows the process which the 1st 2D convolution operation data is transferred from the lower surface, and the result of 2.5-dimensional convolution operation is obtained by adding with the own convolution operation result. It is a figure which shows the state which the result (white circle) of the first 2.5-dimensional convolution is data-shifted to the lower surface. It is a figure which shows the state which perform 2D convolution about the 2nd filter. It is a figure which shows the process which obtains the result of the 2.5-dimensional convolution operation by the addition of the 2D convolution operation data for the 2nd filter transferred from the lower surface, and the own convolution operation result. It is a figure which shows the state which the result (white circle) of the 1st and 2nd 2.5-dimensional convolution is data-shifted to the lower surface. It is a figure which shows the state which perform 2D convolution about the 3rd filter. It is a figure which shows the process which obtains the result of the 2.5-dimensional convolution operation by the 2D convolution operation data for the 3rd filter transferred from the lower surface, the own convolution operation result, and addition. It is a figure which shows the state which the result (white circle) of the 1st to 3rd 2.5-dimensional convolution is data-shifted to the lower surface. It is a figure which shows the state which perform 2D convolution about the 4th filter. It is a figure which shows the process which obtains the result of the 2.5-dimensional convolution operation by the 2D convolution operation data for the 4th filter transferred from the lower surface, the own convolution operation result, and addition. It is a figure which shows the calculation result of the convolution layer using four filters. It is a figure explaining the example of a convolution operation with concrete numerical values. It is a figure which considers the shift direction on the data when the vertical and horizontal of a feature weight are different. It is a time chart diagram which shows the state of the 2.5-dimensional convolution operation according to this invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. These examples are for facilitating the understanding of the present invention, and the application of the present invention is not limited to these examples. In addition, the scope of protection of the present invention extends to the same or similar scope as the claims.

The data processing device according to the present invention is a system composed of a PME (Processor in Memory), which is an arithmetic element having a memory function and is arranged in a three-dimensional array, and a network, and is hereinafter referred to as a TAP (Tensor Array Processor). ..

FIG. 2 is a diagram showing three-dimensional array-shaped PMEs in such a TAP, in which a plurality of PMEs each having a calculation module are stacked and arranged in a three-dimensional (X, Y, Z) direction.

Such a structure can be created as a three-dimensional processor by semiconductor technology. That is, it is possible to stack two-dimensional semiconductor surfaces in which a plurality of PMEs are arranged in the X and Y directions in the Z direction to form a three-dimensional structure with one chip.

In FIG. 2, a plurality of PMEs are arranged in a three-dimensional array of Ws × Hs × Cs (Ws in the X-axis direction, Hs in the Y-axis direction, and Cz in the Z-axis direction). On each XY plane, each PME is connected to a torus network (Lx, Ly), and the data possessed by each PME is shifted in four directions: X-axis positive direction, X-axis negative direction, Y-axis positive direction, and Y-axis negative direction. Has the function of

In the Z-axis direction, PMEs adjacent to the upper and lower surfaces are connected in both directions via the network Lz, and data can be transferred simultaneously in the Z-axis positive direction (upper surface direction) and in the Z-axis negative direction (lower surface direction). It is configured in. The direction control of the data shift and the common shift clock for that purpose are later supplied by the control processor whose data processing device is shown in the developed view.

FIG. 3 is a block diagram of a functional configuration example of each PME, and has a multiplication and addition function block 10 required for the calculation of the convolution layer and a MAX (maximum value) calculation function block 11 necessary for the calculation of the pooling layer. .. In addition, it is possible to add additional functional blocks as needed.

FIG. 4 is a diagram showing the three-dimensional array of the TAP expanded on each XY plane. Such a TAP has a control processor 20 common to the system and an instruction memory 21.

Each of the three-dimensional array has a filter memory FM1-FMS corresponding to each XY plane (Z = 1, Z = 2, ... Z = S), and each of the filter memory FM1-FMS has a corresponding identical plane. It is connected to all PMEs that exist in. During the calculation, the filter memory FM1-FMS can broadcast the feature weights of the filter to all PMEs of the corresponding faces.

Here, it is assumed that the data processing method according to the present invention is used for the convolution operation processing of CNN.

The filter size and the number of filters of each layer obtained by prior learning are input to the instruction memory 21 as a convolutional neural network structure. Based on this, the control processor 20 supplies weights corresponding to the filter memories FM1-FMS on each surface, a common clock signal, and the like.

In the convolution layer, the PME multiplies its own data with the weight broadcast from the corresponding filter memory, then adds the data transferred from the adjacent PME, and the result is the adjacent PME on the opposite side. Transfer to. In the pooling layer, the maximum value of the data transferred from its own data and the adjacent PME is obtained, and the data is sequentially transferred to the adjacent PME.

As described above, based on the instruction data obtained by pre-learning from the instruction memory 21, the timing clock for the direction and common shift of the calculation and transfer by the above PME is transmitted from the control processor 20 to all PMEs. Sent.

FIG. 5 is an enlarged view showing a PME belonging to one surface in FIG. 4. In this example, the plane of Z = k is shown in the case of Ws = 4 and Hs = 3. A filter (feature weight) is commonly supplied from the filter memory FMk to all PMEs on the Z = k plane.

Three-dimensional tensor data of the number of columns W _o × the number of rows H _o × the number of channels _Co is input to the TAP having such a configuration as the first layer of the CNN.

By repeatedly applying the calculation of the convolutional layer of CNN to this input data, the data size of each layer changes.

Here, the data size of the kth layer is W _k × H _k × C _k . In the system of the present invention

It is assumed that.

If the number of input channels is smaller than this value, the input data is arranged in order from the upper XY plane of the TAP. The PME in which the input data is placed is the active surface (active), otherwise it is the inactive surface (inactive). However, data transfer is performed even if the PME is an inactive surface.

For example, when considering one image data as input data, the following can be assumed. One image data is divided into areas of the same size, cut out as multiple area data (channels), and each area data is placed in the PME on the corresponding surface in order from the top surface of the TAP.

At this time, the filter (feature weight) is processed as follows. One feature weight is divided according to the number of faces on which the image data is arranged, and each split feature weight is stored in the file memory FM of the corresponding face. Then, the feature weights arranged in the corresponding filter memory FM at the time of calculation are broadcast to all PMEs belonging to the surface.

Each PME computes the product of the broadcast feature weights and its own data. Further, the PME adds the data transferred from the adjacent memory elements in one direction to the calculation result of the product, and transfers the data to the adjacent PMEs in the opposite direction. This process is repeated to perform two-dimensional convolution. The transfer control in such a case is executed according to the invention of Patent Document 1 described above.

Further, as a feature of the present invention, the two-dimensional convolution data is transferred upward from the lowermost surface to the uppermost surface of the TAP in which the data obtained by performing the two-dimensional convolution with respect to the input data is arranged. At this time, the PME of each surface adds its own data and the convolution calculation result from the next lower surface, and transfers the result to the PME of the next upper surface.

Finally, the top-level two-dimensional array processor can obtain a 2.5-dimensional convolution operation result by adding the two-dimensional convolution results of all the two-dimensional arrays.

Further, as will be described in detail later, the 2.5-dimensional convolution calculation result obtained on the uppermost surface is sequentially shifted to the lower surface.

FIG. 6 shows a change in the calculation result due to the convolution calculation. As shown in FIG. 6 (1), the input data arranged in N × M × Cin PMEs is the result of the convolution operation, and the convolution operation result is obtained in N × M × Cout PMEs. At this time, the size of Cout depends on the number of feature weights (kernels).

This result is the data of the convolution calculation processing result of one layer of CNN, and then, as shown in FIG. 6 (2), the calculation processing of the pooling layer is performed to reduce the layer and make it easier to handle. At the same time, the result of this operation becomes the input of the next layer.

Here, for the sake of easy understanding of the characteristics of the convolution operation according to the data processing executed in the data processing apparatus of the present invention, changes in each surface are schematically shown in FIGS. 7A to 7M.

In FIGS. A to 7M, the active surface is the surface on which the input data is arranged. FIG. 7A shows the initial state. The rectangular parallelepiped of the solid line represents the PME of the active surface, and the rectangular parallelepiped of the broken line represents the PME of the inactive surface. Hereinafter, the same applies to FIGS. 7B to 7M. Furthermore, the display occupied by the gray circle represents the input data Din. FIG. 7B shows a state in which two-dimensional convolution is performed on each surface of the first filter. The arrows in the figure indicate the data transfer direction to the PME on each surface, and the black circles indicate the calculation results. Hereinafter, the same applies to FIGS. 7C to 7M.

FIG. 7C shows a process in which the two-dimensional convolution operation data is transferred from the lower surface and added to the own two-dimensional convolution operation result to obtain the result of the 2.5-dimensional convolution operation.

FIG. 7D shows a state in which the 2.5-dimensional convolution calculation result is shifted to the lower surface in preparation for performing the convolution calculation for the second filter. This shift is performed including the inactive surface.

FIG. 7E shows a state in which two-dimensional convolution is performed on each surface of the second filter. FIG. 7F shows a process of obtaining the result of the 2.5-dimensional convolution operation by transferring the two-dimensional convolution operation data to the second filter from the lower surface and adding it to the own convolution operation result.

Similarly, FIGS. 7G to 7L show the state of the convolution calculation for the third and fourth filters. That is, FIG. 7G shows a state in which the results (white circles) of the first and second 2.5-dimensional convolutions are data-shifted from the upper surface to the lower surface as shown by arrows.

FIG. 7H shows a state in which two-dimensional convolution is performed for the third filter. FIG. 7I shows a process of transferring the two-dimensional convolution operation data to the third filter from the bottom and adding it to the own convolution operation result to obtain the result of the 2.5-dimensional convolution operation.

FIG. 7J shows a state in which the first to third 2.5-dimensional convolution results (white circles) are data-shifted from the upper surface to the lower surface. At this time, the data is also shifted to the inactive surface.

FIG. 7K shows a state in which a two-dimensional convolution operation is performed on the fourth filter. FIG. 7L is the same as the process of FIG. 7I, but the two-dimensional convolution operation data for the fourth filter from the bottom is transferred, and the 2.5-dimensional convolution operation result is obtained by adding it to the own convolution operation result. Shows the desired process.

FIG. 7M finally shows the calculation result of the convolution layer using the four filters, and this is the input of the next layer.

Here, the convolution operation can be expressed by an equation as shown in equation (1) below. b ₀ is the bias constant term. Bias b ₀ is used to increase or decrease the result of the convolution operation by a certain value. Both the bias b ₀ and the filter weight ω are automatically determined during CNN learning.

However, s is a natural number and represents the stride when performing the convolution calculation. Further, for the sake of simplicity, the description of the present invention describes only when the stride is 1, but the applicability of the present invention cannot be denied even if the stride is 2 or more.

Is the weight of the oth filter of the lth layer,

Is the input data of the first layer.

The PME on each side is the latter half of equation (1).

Do the calculation. At this time, the PME on the top surface of the TAP calculates the two-dimensional convolution operation of C = C ^l , and similarly, the two-dimensional convolution operation of C = C ^l -1 is performed on the surface below it. Calculated by PME. By adding the calculation results of the latter half of the above calculated on each surface, the calculation of the whole equation (1) is performed.

FIG. 8 is a diagram illustrating an example of the above-mentioned convolution operation with specific numerical values, and the case of two upper and lower surfaces is taken as an example.

Multiply the input (Input) x60 and the weight (kernel) w61 on one surface (Ch1) while moving in the arrow direction, and at the same time, input (Input) x62 and the weight (kernel) w63 on the lower surface (Ch2) in the arrow direction. Multiply while moving to. As a result, the two-dimensional convolution calculation result 64 is obtained, respectively.

The multiplication result of the input 60 and the weight 61 at the initial time of one surface (Ch1) is as follows.
(-3 * 1) + (-2 * 2) + (1 * 2) + (3 * 2) = 1
Next, the multiplication result of the input 60 and the weight 61 when shifted in the direction of the arrow by one digit is as follows.

(1 * 1) + (-3 * 2) + (-2 * 3) + (2 * 2) + (1 * 2) + (3 * 1) = -2
These are as shown in the two-dimensional convolution operation result 64.

On the other hand, the multiplication result of the input 62 and the weight 63 at the initial time of the lower surface (Ch2) is as follows.

(-2 * 2) + (3 * 3) + (-3 * 1) + (1 * 3) = 5
Next, the multiplication result of the input 60 and the weight 61 when shifted in the direction of the arrow by one digit is as follows.

(2 * 2) + (-2 * 3) + (3 * 3) + ( 1 * 1) + ( -3 * 3) + (1 * 2) = 1.

These are as shown in the two-dimensional convolution calculation result 65.

Then, on the one surface (Ch1), its own two-dimensional convolution calculation result 64 is obtained, and the two-dimensional convolution operation result 65 transferred from the lower surface (Ch2) is transferred. Therefore, the two-dimensional convolution calculation results 64 and 65 and the bias b ₀ = 1 are added and obtained as shown in the 2.5-dimensional convolution calculation result 66.

As described above, the weight is sequentially shifted by a predetermined number of digits on the input data, and multiplication and addition are repeated to obtain a two-dimensional convolution operation result.

At this time, based on the instruction stored in the instruction memory 21, the direction of data transfer from the PME transfers the data in one stroke, eliminates unnecessary transfer, and prevents multiple data transfers to the same PME. It is controlled and can avoid data collisions.

Here, in the example shown in FIG. 8, the feature weight is set to 3 × 3 in length and width, that is, w × w having the same length and width, but it can be generalized so that the length and width are different, and this is w_1. Expressed as × w_2.

FIG. 9 is a diagram for considering the shift direction on the data when the vertical and horizontal features weights are different. FIG. 9 (1) has an odd number with the same vertical and horizontal lengths, and can be shifted with a single stroke so as to move toward the center. In FIG. 9 (2), either the length of the 縱 or the horizontal length is an even number, and as in FIG. 9 (1), there is a Hamilton path that passes through all the points exactly once. On the other hand, FIG. 9 (3) shows a case where both the length and the horizontal length of the 縱 are odd numbers, and one PME passes twice, so that unnecessary transfer occurs.

Here, as described above, the result of the 2.5-dimensional convolution operation is obtained, and this data is held by the PME at the top of the TAP. In the algorithm according to the present invention, assuming that the lowermost surface of the TAP is Z = 1 and the uppermost surface is Z = C _S , the result of the 2.5-dimensional convolution operation when the 0th filter is used in the lth layer.

を

of

Place on a planar PME and prepare for the calculation of the next layer convolution. Therefore, each time the calculation result is obtained on the uppermost surface, the 2.5-dimensional convolution calculation result held by each PME is shifted once in the lower surface direction (see FIG. 7B).

FIG. 10 is a time chart diagram showing a state of the 2.5-dimensional convolution operation according to the present invention. This time chart shows the operation of PME (i, j, 1), PME (i, j, 2),… PME (i, j, Cs). Also, in this figure, the Cin planes are active.

In the time chart, a two-dimensional convolution operation is performed on the blackened portion A of each surface. In the next step after this calculation result is completed, the calculation result and bias B are added and the data is transferred to the PME on the upper surface (up arrow).

The PME on the next surface adds the data transferred from the PME on the lower surface and the result of its own two-dimensional convolution operation, and as a result, transfers C to the PME on the next upper surface repeatedly. Finally, the calculation result of PME on the top surface is the calculation result D of 2.5-dimensional convolution.

Next, the calculation result D of the 2.5-dimensional convolution is shifted to the lower surface in the direction of the down arrow each time the 2.5-dimensional convolution operation is completed for one weight. At this time, the data shifted from the upper surface is only saved in PME, and no other processing is performed. Shift only once to obtain the 2.5-dimensional convolution operation.

The above operation is repeated Cout times, and the 2.5-dimensional convolution operation result is held from the top surface to Cout.

Here, in each layer of the CNN, after the 2.5-dimensional convolution process is performed as described above, a pooling operation is performed to obtain input data for the next layer.

In the pooling operation, the data held by the surrounding PMEs is received by the same method as in the two-dimensional convolution operation, and the maximum value is calculated by the following equation (2).

However, s'is a natural number of 2 or more and represents the stride when performing the convolution calculation. Furthermore, act is an activation function, for example

Is used.

In pooling, data is transferred to the upper surface side in the same way as two-dimensional convolution, but each PME uses y _in received from the PME on the lower surface adjacent to its own data x.

To calculate.

While performing this calculation, the maximum value of the data possessed by the surrounding PMEs is obtained by first transferring the data as described with reference to FIG.

As described above, when the data processing apparatus according to the present invention is used as a data processing apparatus in CNN, the structure of CNN (filter size of each layer obtained by learning, number of filters) is given as an input. The weight of each filter is given on the filter memory of each side of the TAP. In addition, the CNN main input data is placed in the TAP's PME.

The PME of each surface multiplies its own data by the weight, adds the two-dimensional convolution data to the data transferred from the periphery, and sends it to the PME on the upper surface. Therefore, by adding the two-dimensional convolution calculation results of all the lower surfaces with the PME on the uppermost surface, it is possible to obtain a high-speed 2.5-dimensional convolution calculation result without labor while maintaining parallelism.

PME memory element 10 Multiplication and addition function block 11 MAX (maximum value) arithmetic function block 20 control processor 21 instruction memory
FM1-FMS filter memory

Claims (7)

A data processing device having a plurality of processing elements having multiplication and addition functions in a three-dimensional direction.
The plurality of processing elements are arranged on each of the plurality of two-dimensional planes laminated in the Z-axis direction.
The processing elements arranged on the two-dimensional plane are connected to the torus network, and the processing elements adjacent to the upper and lower surfaces in the Z-axis direction are connected in both directions by the network.
A processing element arranged on the two-dimensional plane corresponding to each of the plurality of two-dimensional planes has a filter memory for broadcasting feature weights.
The input data is sequentially arranged from the two-dimensional surface of the upper surface in the Z-axis direction to the lower surface.
Each of the processing elements arranged on one two-dimensional plane multiplies the input data and the feature weight by the multiplication function, and further obtains the transfer data from the adjacent processing elements on the two-dimensional plane and the result of the multiplication. Addition is performed, and then the addition operation result is transferred to a processing element adjacent to the opposite side of the two-dimensional surface .
The multiplication, addition, and transfer are repeated to perform two-dimensional convolution of the input data, and further.
The processing element of each two-dimensional surface adds its own data and the convolution data from the surface one below in the Z-axis direction, and transfers the data to the surface one above in the Z-axis direction . The data obtained by two-dimensional convolution of the data is transferred upward from the lowermost surface to the uppermost surface in the Z-axis direction.
A data processing device characterized by that. In claim 1,
The processing element on the uppermost surface in the Z-axis direction calculates the 2.5-dimensional convolution data by adding the two-dimensional convolution data transferred from the plurality of processing elements on the lower surface and its own data.
A data processing device characterized by that. In claim 2,
The 2.5-dimensional convolution data is sequentially shifted from the processing element on the uppermost surface to the processing element on the lower surface.
A data processing device characterized by that. In any one of claims 1 to 3,
The feature weight is divided by the number of two-dimensional surfaces on which the input data is arranged, and is arranged in the filter memory corresponding to each surface on which the input data is arranged. Broadcast the placed feature weights to all processing elements on the corresponding faces,
A data processing device characterized by that. A data processing method in a data processing apparatus having a plurality of processing elements having multiplication and addition functions in a three-dimensional direction.
In the data processing device, a plurality of processing elements are arranged on each of a plurality of two-dimensional planes stacked in the Z-axis direction.
The processing elements arranged on the two-dimensional plane are connected to the torus network, and the processing elements adjacent to the upper and lower surfaces in the Z-axis direction are connected in both directions by the network.
A processing element arranged on the two-dimensional plane corresponding to each of the plurality of two-dimensional planes has a filter memory for broadcasting feature weights.
The process of sequentially arranging the input data from the two-dimensional surface of the upper surface in the Z-axis direction to the lower surface, and
With each of the processing elements placed on one two-dimensional plane,
A step of multiplying the input data and the feature weight by the multiplication function,
The step of adding the transfer data from the processing elements adjacent to each other on the two-dimensional plane and the result of the multiplication, and
Next, it has a step of transferring the addition operation result to a processing element adjacent to the opposite side of the two-dimensional surface .
The step of multiplying, the step of adding, and the step of transferring are repeated to perform two-dimensional convolution of the input data, and further.
The processing element of each two-dimensional surface adds its own data and the convolution data from the surface one below in the Z-axis direction, and transfers the data to the surface one above in the Z-axis direction . The data obtained by two-dimensional convolution of the data is transferred upward from the lowermost surface to the uppermost surface in the Z-axis direction.
A data processing method characterized by that. In claim 5,
The processing element on the uppermost surface in the Z-axis direction is a step of calculating 2.5-dimensional convolution data by adding two-dimensional convolution data transferred from a plurality of processing elements on the lower surface and its own data.
A data processing method characterized by having. In claim 6,
The step of sequentially shifting the 2.5-dimensional convolution data from the processing element on the uppermost surface to the processing element on the lower surface.
A data processing method characterized by having.

Images