JP6832307B2 - Information processing equipment, information processing methods, and programs - Google Patents

Description

The present invention relates to an information processing apparatus, an information processing method, and a program, and more particularly to a technique for concealing input data of a convolutional neural network.

In recent years, the speed of CPU (Central Processing Unit) and GPU (Graphics Processing Unit) has been increased, the capacity of memory has been increased, and machine learning technology has been rapidly advanced. For this reason, machine learning using learning data on the order of hundreds of thousands to millions has become possible, and highly accurate identification technology and classification technology are being established (see Non-Patent Document 1).

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia (pp. 675-678) ). ACM.

A large amount of computational cost is required to perform machine learning based on a large amount of learning data. In addition, a huge amount of labor is required for preparing a large amount of learning data and preprocessing for processing the prepared learning data for use in machine learning. On the other hand, the learning model generated by machine learning is digital data, and its duplication is easy. Therefore, if the creator of the learning model provides the learning model to the user for the purpose of making the user use it, there is a risk that the provided learning model will be used illegally thereafter.

Therefore, for example, the creator of the learning model acquires the processing target data from the user via the communication network without providing the learning model itself to the user of the learning model, and provides only the processing result of the learning model to the user. Aspects are also conceivable. However, in this aspect, in the case of data in which leakage becomes a problem due to the inclusion of personal information in the data input to the learning model, the user may be reluctant to use the learning model.

The present invention has been made in view of these points, and an object of the present invention is to provide a technique for concealing data input to a learning model of a convolutional neural network.

The first aspect of the present invention is an information processing device. This device includes a model acquisition unit that acquires a learning model of a convolutional neural network that uses a matrix as input data, a convolution filter acquisition unit that acquires a convolution filter in the convolution layer in the training model, and the convolution filter. Generates a conversion filter acquisition unit that acquires a conversion filter of the same size as the input data by Fourier conversion, and a key matrix that is a matrix of the same size as the input data and is composed of elements other than 0. A concealment filter generation unit that generates a concealment filter that is an Adamal product of the conversion filter and the key matrix, and an inverse key matrix that is the inverse element of the Adamal product of the key matrix are generated. The inverse key matrix generation unit, the data acquisition unit that acquires the concealment data that is the Adamal product of the convolutional neural network processing target data and the inverse key matrix, the concealment data, and the concealment filter. A filter execution unit that inversely transforms the Adamal product of the above, and a model execution unit that acquires the output of the learning model related to the processing target data by using the execution result of the filter execution unit as the input of the layers after the convolutional layer. To be equipped with.

The information processing device may further include a reverse key matrix transmission unit that transmits the reverse key matrix to a user terminal via a communication network, and the data acquisition unit may be provided from the user terminal via the communication network. , The concealment data may be acquired, and the model execution unit may transmit the output of the learning model to the user terminal via the communication network.

The key matrix generation unit includes a random number generation unit that generates a random number in which both the absolute value and the inverse number of the absolute value are equal to or greater than a predetermined value, a matrix parameter acquisition unit that acquires the size of the input data, and a random number generation unit. It may include a matrix generation unit that uses the generated random number as an element and generates a matrix having the same size as the input data as the key matrix.

The convolution filter acquisition unit may acquire a convolution filter in at least the first convolution layer among the plurality of convolution layers in the learning model.

The inverse key matrix generator may generate the inverse element of the Hadamard product by substituting each of the plurality of elements constituting the key matrix with the reciprocal of each element.

A second aspect of the present invention is an information processing method. In this method, the processor obtains a learning model of a convolutional neural network using a matrix as input data, a step of acquiring a convolutional filter in a convolutional layer in the training model, and a Fourier transform of the convolutional filter. A step of acquiring a conversion filter having the same size as the input data, and a step of generating a key matrix having the same size as the input data and each of which is a matrix composed of elements other than 0. A step of generating a concealment filter which is an Adamal product of the conversion filter and the key matrix, a step of generating an inverse key matrix which is an inverse element of the Adamal product of the key matrix, and data to be processed by the convolutional neural network. And the step of acquiring the concealment data which is the Adamal product of the inverse key matrix, the step of acquiring the Adamal product of the concealment data and the concealment filter, and the concealment data and the concealment. A step of inversely Fourier transforming the Adamar product with the conversion filter, and a step of acquiring the output of the learning model regarding the data to be processed by using the matrix obtained by the inverse Fourier transform as the input of the layers after the convolutional layer. , Execute.

A third aspect of the present invention is a program. This program has a function of acquiring a learning model of a convolutional neural network using a matrix as input data, a function of acquiring a convolutional filter in the convolutional layer in the training model, and a Fourier transform of the convolutional filter. A function to acquire a conversion filter having the same size as the input data, a function to generate a key matrix which is a matrix having the same size as the input data and is composed of elements other than 0, and each conversion. A function to generate a concealment filter which is an Adamal product of the filter and the assigned key matrix, a function to generate an inverse key matrix which is an inverse element of the Adamal product of the key matrix, and data to be processed by the convolutional neural network. Fourier transformation, a function to acquire concealment data which is the Adamal product of the inverse key matrix, a function to acquire the Adamal product of the concealment data and the concealment filter, and the concealment data and the concealment. A function of inversely Fourier transforming the Adamar product with a filter and a function of acquiring the output of the learning model related to the data to be processed by using the matrix obtained by the inverse Fourier transform as the input of the layers after the convolutional layer. make it happen.

According to the present invention, the data input to the learning model of the convolutional neural network can be concealed.

It is a figure which shows typically the general functional structure of a convolutional neural network. It is a figure for demonstrating the relationship of the convolution of the input data and a feature map. It is a figure for demonstrating the generation of a feature map using a convolution filter. It is a figure for demonstrating the relationship between N convolution filters and N-stage feature map. It is a figure for demonstrating the outline of embodiment. It is a figure which shows typically the functional structure of the information processing apparatus which concerns on embodiment. It is a figure which shows typically the functional structure of the key matrix generation part which concerns on embodiment. It is a figure which shows typically an example of the use scene of the information processing apparatus which concerns on embodiment. It is a flowchart for demonstrating the flow of the information processing executed by the information processing apparatus which concerns on embodiment.

<Convolutional neural network>
The information processing apparatus according to the embodiment relates to a technique for concealing data input to a convolutional neural network (CNN). Therefore, as a prerequisite technology of the information processing apparatus according to the embodiment, first, a convolutional neural network will be briefly described.

FIG. 1 is a diagram schematically showing a general functional configuration of a convolutional neural network.

Currently, neural networks with various configurations have been proposed, but these basic configurations are common. The basic configuration of a neural network is represented by a superposition (or graph structure) of a plurality of types of layers. The neural network learns the model parameters so that the output result for the input data is an appropriate value. In other words, the neural network learns the model parameters to minimize the loss function defined so that the output result for the input data has an appropriate value.

In FIG. 1, as a feedforward neural network, it is composed of three layers, an input layer, a hidden layer, and an output layer, from the input layer to the output layer. Propagate in one direction. The hidden layer can be composed of a plurality of layers in a graph shape. Each layer has multiple units (neurons). In each layer, the parameter of the function that connects the unit in the front layer to the unit in the rear layer is called "weight". Learning in the present specification is to calculate an appropriate "weight" as a parameter of this function.

FIG. 1 illustrates a convolutional neural network. A convolutional neural network is composed of an input layer, a convolutional layer, a pooling layer, a full-connected layer, and an output layer. In FIG. 1, the first layer L1 is an input layer, and the second layer L2 is a convolution layer. Similarly, the nth layer Ln is the output layer. In a convolutional neural network, only specific units in the anterior layer are connected to units in the posterior layer. That is, in the convolutional neural network, not all the units in the front layer are connected to the units in the rear layer.

Learning in the neural network according to the embodiment means that the weight of each layer is optimally updated by using the error between the output value from the output layer and the label of the training data with respect to the training data. A "loss function" is defined to calculate the error. The error is propagated one after another from the output layer side to the input layer side by the "error backpropagation method", and the weight of each layer is updated little by little. Finally, a convergence calculation is performed that adjusts the weights of each layer to the appropriate values so that the error is small. Specifically, in learning in a neural network (that is, in the generation stage of a new model parameter), the model parameter is updated by the gradient in which the error is back-propagated.

FIG. 2 is a diagram for explaining the convolutional relationship between the input data and the feature map. The process in FIG. 2 is performed by the convolution layer and the fully connected layer. In the example shown in FIG. 2, a feature map is generated by applying one convolution filter to the input data. In FIG. 2, the sizes of the input data, the convolution filter, and the feature map are as follows.
Input data: 32 x 32 x 3 elements Convolution filter: 5 x 5 x 3 elements (model parameters)
Feature map: 28 x 28 elements

N convolution filters (N is an integer of 1 or more) are prepared, and these are model parameters. That is, the "weight" means N convolution filters. However, the bias term is not considered here.

FIG. 3 is a diagram for explaining the generation of the feature map using the convolution filter.
In the example shown in FIG. 3, one convolution filter composed of 5 × 5 × 3 elements is applied to the input data, and the sum of the products of the elements is used as the value of one element of the feature map. Then, by moving the same convolution filter with respect to the input data, one feature map is generated. Here, the number of elements (movement amount) for moving the convolution filter is referred to as "stride". A zero-padding area in which element 0 is filled is provided at the peripheral edge of the input data. As a result, the same number of convolution filters can be applied to the edge elements of the input data.

FIG. 4 is a diagram for explaining the relationship between N convolution filters and N-stage feature maps. In the example shown in FIG. 4, the number of convolution filters is N. 2 and 3 show an example in which one feature map generated by one convolution filter is generated. On the other hand, the example shown in FIG. 4 shows an example in which an N-stage feature map is generated by N convolution filters. In neural network learning, the feature map in one layer becomes the input data in the next layer. By executing the training of the neural network, the error based on the loss function is propagated one after another from the output layer side to the input layer side, and the weight of each layer is updated by the known backpropagation method.

<Outline of the embodiment>
On the premise of the above, the outline of the embodiment will be described.
The information processing apparatus according to the embodiment acquires the input data to be input to the learning model of the convolutional neural network in a concealed state, and processes it as it is without returning it to the state before concealment to obtain the output of the learning model. It is a device that can be used. The information processing apparatus according to the embodiment utilizes the fact that the convolution operation in the real space is multiplication in the frequency space to conceal both the input data and the convolution filter in the frequency space.

FIG. 5 is a diagram for explaining an outline of the embodiment. The input data received by the information processing apparatus according to the embodiment is data that can be represented by a matrix, for example, image data. In FIG. 5, the rectangle with d is the input data d. As described with reference to FIG. 1, in the convolutional neural network handled by the information processing apparatus according to the embodiment, the second layer L2 is a convolutional layer. In the convolutional layer, convolution of the input data d and the convolutional filter k is executed.

As is well known, the convolution of the convolution filter k and the input data d in the real space is the product of the convolution filter and the input data in the frequency space. In FIG. 5, the Fourier transform of the convolution filter k is shown by the transformation filter K, and the Fourier transform of the input data d in the real space is shown by the transformation input data D. When the input data d and the convolution filter k are represented by a matrix, the convolution of both in the frequency space is the Hadamard product of the conversion filter K and the conversion input data D. In FIG. 5, the convolution is represented by "*" and the Hadamard product is represented by "〇".

The information processing apparatus according to the embodiment generates a random number matrix E having the same size as the input data d and an inverse element 1 / E in the Hadamard product of the random number matrix E. Using the random number matrix E and its inverse element 1 / E, the Hadamard product of the conversion filter K and the conversion input data D can be expressed by the following equation (1).
K ○ D = K ○ E ○ 1 / E ○ D (1)

For convenience of explanation, it is assumed that the input data d is a matrix of 3 rows and 3 columns. In this case, the conversion input data D, the conversion filter K, the key matrix E, and the inverse key matrix 1 / E are all 3 rows and 3 columns. Therefore, the conversion input data D, the conversion filter K, the key matrix E, and the inverse key matrix 1 / E are referred to as the following equations (2), (3), (4), and (5), respectively.

From the equations (2) and (3), the left side of the equation (1) is the following equation (6).

Further, from the equations (3) and (4), the first two terms on the right side of the equation (1) are the following equations (7).

Further, from the equations (2) and (5), the two terms behind the right side of the equation (1) are the following equations (8).

The elements that make up the random number matrix E are random numbers. Therefore, unless the random number matrix E is known, it is difficult to estimate the conversion filter K from the equation (7), that is, the Hadamard product K ○ E of the conversion filter K and the random number matrix E. In this sense, the random number matrix E is a secret key for concealing the convolution filter k. Hereinafter, the random number matrix E may be referred to as a key matrix E. Further, the Hadamard product K ○ E of the conversion filter K and the random number matrix E may be described as a concealment filter. The equations (1) to (8) show the case where the input data d is a matrix of 3 rows and 3 columns, but in the case of other sizes, the equations (1) to (8) are changed only by changing the number of elements. ) Is the same formula.

The elements constituting 1 / E, which is the inverse element of the random number matrix E, are also random numbers. Therefore, unless the inverse element 1 / E of the random number matrix E is known, the conversion input data from the equation (8), that is, the conversion input data D, the inverse element 1 / E of the random number matrix E, and the Hadamard product 1 / E ○ D. It is difficult to estimate D. Therefore, the inverse element 1 / E of the random number matrix E is a secret key for concealing the conversion input data D. Hereinafter, the inverse element 1 / E of the random number matrix E may be described as the inverse key matrix 1 / E. Further, the inverse element 1 / E of the conversion input data D and the random number matrix E and the Hadamard product 1 / E ○ D may be described as concealed data.

The information processing apparatus according to the embodiment holds a concealment filter represented by the equation (7) in place of the conversion filter K in the real space. Further, the information processing apparatus according to the embodiment acquires the concealed data represented by the equation (8) in place of the input data d in the real space. The information processing apparatus according to the embodiment calculates the Hadamard product of the concealment filter and the concealed data, thereby calculating the Hadamard product of the conversion filter K before concealment and the conversion input data D before concealment. Can be obtained.

The information processing apparatus according to the embodiment obtains the result of the convolution operation of the input data d and the convolution filter k in the real space by Fourier inverse transforming the Hadamard product of the conversion filter K and the conversion input data D. be able to. As a result, the information processing apparatus according to the embodiment can input the acquired convolution operation result to the subsequent layers and acquire the result of the learning model in which the input data d is input.

As described above, the information processing apparatus according to the embodiment acquires the data to be processed of the learning model generated by using the convolutional neural network in a concealed state, and conceals the data without returning to the state before concealment. It can be processed as it is. As a result, the information processing apparatus according to the embodiment can conceal the data input to the learning model and suppress leakage.

Further, the information processing apparatus according to the embodiment can also conceal the convolution filter k that constitutes the learning model. As a result, according to the information processing apparatus according to the embodiment, the structure of at least a part of the learning model can be concealed.

<Functional configuration of the information processing device 1 according to the embodiment>
Hereinafter, the information processing apparatus according to the embodiment will be described in more detail.
FIG. 6 is a diagram schematically showing a functional configuration of the information processing device 1 according to the embodiment. The information processing device 1 includes a storage unit 2 and a control unit 3.

The information processing device 1 according to the embodiment includes a processor such as a CPU or GPU, a work memory such as a DRAM (Dynamic Random Access Memory), and a large-capacity storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Consists of. The information processing device 1 may be a single device such as a PC (Personal Computer), a workstation, or a server, or may be composed of a plurality of devices such as a cloud server.

The storage unit 2 includes a ROM (Read Only Memory) for storing the BIOS (Basic Input Output System) of the computer that realizes the information processing device 1, a RAM (Random Access Memory) that serves as a work area for the information processing device 1, and an OS (OS). An operating system), an application program, and a large-capacity storage device such as an HDD or SSD that stores various information referred to when the application program is executed.

The control unit 3 is a processor such as a CPU or GPU of the information processing device 1, and by executing a program stored in the storage unit 2, the model acquisition unit 30, the convolution filter acquisition unit 31, the conversion filter acquisition unit 32, It functions as a key matrix generation unit 33, a concealment filter generation unit 34, an inverse key matrix generation unit 35, a data acquisition unit 36, a filter execution unit 37, a model execution unit 38, and an inverse key matrix transmission unit 39.

The storage unit 2 holds a learning model generated by convolving training data and labels in advance and learning them with a neural network. The model acquisition unit 30 reads the learning model of the convolutional neural network using the matrix as the input data d from the storage unit 2 and acquires it.

The convolution filter acquisition unit 31 acquires the convolution filter k in the convolution layer in the learning model read by the model acquisition unit 30. In particular, the convolution filter acquisition unit 31 acquires the convolution filter k in at least the first convolution layer among the plurality of convolution layers in the learning model. As shown in FIG. 1, the second layer L2 of the learning model to be processed by the information processing apparatus 1 according to the embodiment, that is, the layer L adjacent to the input layer L1, is the first convolution layer. As a result, even if the concealed data is input to the input layer L1, it can be converted into real space data in the second layer L2, and the subsequent layers can be processed with the real space data.

The conversion filter acquisition unit 32 Fourier transforms the convolution filter k acquired by the convolution filter acquisition unit 31 to acquire a conversion filter K having the same size as the input data d. The key matrix generation unit 33 generates a key matrix E, which is a matrix having the same size as the input data d and is composed of elements other than 0.

FIG. 7 is a diagram schematically showing a functional configuration of the key matrix generation unit 33 according to the embodiment. The key matrix generation unit 33 includes a random number generation unit 330, a matrix parameter acquisition unit 331, and a matrix generation unit 332.

The random number generation unit 330 generates a random number in which both the absolute value and the reciprocal of the absolute value are equal to or greater than a predetermined value. Here, the "predetermined value" is a value in which the calculation error is within the allowable range when the CPU of the information processing device 1 calculates the reciprocal. The specific value of the predetermined value may be determined experimentally in consideration of the calculation accuracy imposed on the learning model. For example, when the random number generated by the random number generator 330 is r and the reciprocal is s (s is the numerical calculation result of 1 / r), | r × s-1 | is about the double precision real number type machine epsilon. A predetermined value may be set so as to be.

The matrix parameter acquisition unit 331 acquires the size of the input data d. The matrix generation unit 332 uses the random numbers generated by the random number generation unit 330 as elements, and generates a matrix having the same size as the input data d as the key matrix E. As a result, the key matrix generation unit 33 can generate a key matrix E having the same size as the input data d, whose elements are random numbers. Further, since each element constituting the key matrix E has a predetermined value or more, the error between the key matrix E and 1 of each element of the Hadamard product of the inverse key matrix 1 / E can be suppressed to about the machine epsilon.

Returning to the description of FIG. The concealment filter generation unit 34 generates a concealment filter which is the Hadamard product of the conversion filter K and the key matrix E. The inverse key matrix generation unit 35 generates an inverse key matrix 1 / E, which is the inverse element of the Hadamard product of the key matrix E. Specifically, the inverse key matrix generation unit 35 generates the inverse element of the Hadamard product by substituting each of the plurality of elements constituting the key matrix E with the reciprocal of each element.

The data acquisition unit 36 acquires concealment data which is the Hadamard product of the conversion input data D which is the Fourier transform of the input data d which is the processing target of the convolutional neural network and the inverse key matrix 1 / E. The filter execution unit 37 first calculates the Hadamard product of the concealment data and the concealment filter. The filter execution unit 37 performs an inverse Fourier transform on the Hadamard product of the calculated concealment data and the concealment filter. As a result, the filter execution unit 37 can acquire the convolution of the input data d and the convolution filter k in the real space.

The model execution unit 38 acquires the output of the learning model regarding the input data d, which is the processing target data, by inputting the execution result of the filter execution unit 37 into the layers after the convolutional layer from which the convolutional filter k has been acquired. As described above, the information processing apparatus 1 according to the embodiment acquires the input data d to be processed in a concealed state, and processes the acquired data as it is without returning it to the readable original state to obtain a learning model. You can get the output of.

<Usage scene of information processing device 1 according to the embodiment>
FIG. 8 is a diagram schematically showing an example of a usage scene of the information processing device 1 according to the embodiment. In the example shown in FIG. 8, the information processing device 1 is connected to the communication network N via the communication network N in a manner capable of communicating with the communication network N.

The user terminal T receives an input API (Application Program Interface) and an output API from the information processing device 1. By using the input API, the user terminal T can transmit the data to be processed to the information processing device 1 via the communication network N. Further, the user terminal T can receive the execution result of the learning model from the information processing device 1 via the communication network N by using the output API.

Specifically, the reverse key matrix transmission unit 39 transmits the reverse key matrix 1 / E to the user terminal T via the communication network N. At this time, the reverse key matrix transmission unit 39 includes the reverse key matrix 1 / E as a part of the input API and provides it to the user terminal T. The input API is provided in the form of a program that can be executed by the user terminal T. When the user terminal T executes the input API and processes the input data d to be processed, the input data d is first Fourier transformed into the converted input data D.

Subsequently, the input API executed by the user terminal T generates concealed data by taking the Hadamard product of the conversion input data D and the inverse key matrix 1 / E. Finally, the input API executed by the user terminal T transmits the confidential data to the information processing device 1 via the communication network N.

The data acquisition unit 36 acquires confidential data from the user terminal T via the communication network N. The filter execution unit 37 calculates the Hadamard product of the concealment data and the concealment filter, and Fourier inverse transforms the calculated Hadamard product of the concealment data and the concealment filter. The model execution unit 38 acquires the output of the learning model regarding the input data d, which is the processing target data, by inputting the execution result of the filter execution unit 37 into the layers after the convolutional layer from which the convolutional filter k has been acquired. When the inverse key matrix transmission unit 39 acquires the request from the output API executed by the storage unit 2, the inverse key matrix transmission unit 39 transmits the output of the learning model to the user terminal T via the communication network N. The user terminal T can acquire the output of the learning model regarding the input data d from the information processing device 1.

In this way, the user terminal T can delegate most of the operations required for applying the learning model to the information processing device 1. As a result, the learning model can be executed even when the user terminal T is a device having few calculation resources or the power consumption is limited due to reasons such as battery operation.

Further, the information processing device 1 can provide the user terminal T with an opportunity to use the learning model without providing the learning model itself to the user terminal T. As a result, the information processing device 1 can prevent the learning model from being leaked to a third party or being used illegally.

Further, since the user terminal T provides the data to be processed in the learning model to the information processing apparatus 1 in a concealed state, it is possible to prevent the data from being leaked to a third party during transmission of the communication network N.

<Processing flow of information processing executed by information processing device 1>
FIG. 9 is a flowchart for explaining the flow of information processing executed by the information processing apparatus 1 according to the embodiment. The process in this flowchart starts, for example, when the information processing device 1 is activated.

The model acquisition unit 30 reads out the learning model of the convolutional neural network using the matrix as the input data d from the storage unit 2 and acquires it (S2). The convolution filter acquisition unit 31 acquires the convolution filter k in the convolution layer in the learning model (S4).

The conversion filter acquisition unit 32 Fourier transforms the convolution filter k acquired by the convolution filter acquisition unit 31 to acquire a conversion filter K having the same size as the input data d (S6). The key matrix generation unit 33 generates a key matrix E which is a matrix having the same size as the input data d and each of which is composed of elements other than 0 (S8).

The concealment filter generation unit 34 generates a concealment filter which is the Hadamard product of the conversion filter K and the key matrix E (S10). The inverse key matrix generation unit 35 generates an inverse key matrix 1 / E, which is the inverse element of the Hadamard product of the key matrix E (S12). The data acquisition unit 36 acquires concealment data, which is the Hadamard product of the transform input data D, which is the Fourier transform of the input data d to be processed by the convolutional neural network, and the inverse key matrix 1 / E (S14).

The filter execution unit 37 acquires the Hadamard product of the concealment data and the concealment filter (S16). Subsequently, the filter execution unit 37 performs an inverse Fourier transform on the Hadamard product of the concealed data and the concealed filter (S18). The model execution unit 38 acquires the output of the training model for the input data d to be processed by using the matrix obtained by the filter execution unit 37 performing the inverse Fourier transform as the input of the layers after the convolution layer (S20).

When the model execution unit 38 acquires the output of the learning model related to the input data d, the process in this flowchart ends.

<Effects of the information processing device 1 according to the embodiment>
As described above, according to the information processing apparatus 1 according to the embodiment, it is possible to provide a technique for concealing the data input to the learning model of the convolutional neural network.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. is there. For example, the specific embodiment of the distribution / integration of the device is not limited to the above embodiment, and all or a part thereof may be functionally or physically distributed / integrated in an arbitrary unit. Can be done. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination has the effect of the original embodiment together.

<First modification>
In the above, the case where the convolution filter k acquired by the conversion filter acquisition unit 32 is singular has been mainly described. However, for example, as described with reference to FIG. 4, the number of convolution filters k may be N (N is an integer of 2 or more).

In this case, the convolution filter acquisition unit 31 acquires N convolution filters k in the convolution layer in the learning model. The conversion filter acquisition unit 32 Fourier transforms each of the N convolution filters to acquire N conversion filters K having the same size as the input data d.

The key matrix generation unit 33 generates N key matrices E, which are N different matrices of the same size as the input data d, each of which is a matrix composed of elements other than 0. The concealment filter generation unit 34 assigns a different key matrix E to each of the N conversion filters K, and generates a concealment filter which is the Hadamard product of each conversion filter K and the assigned key matrix E.

The inverse key matrix generation unit 35 generates N inverse key matrices 1 / E, which is the inverse element of the Hadamard product of each N key matrices E. The data acquisition unit 36 has N concealment data obtained by Hadamard products of the transform input data D, which is a Fourier transform of the input data d to be processed by the convolutional neural network, and each of the N inverse key matrices 1 / E. To get.

The filter execution unit 37 sets the concealment filter generated by using the key matrix E which is the inverse element of the inverse key matrix 1 / E used for generating each concealment data for each of the N concealment data. Inverse Fourier transform of the Hadamard product of. The model execution unit 38 acquires the output of the learning model related to the data to be processed by using the execution result of the filter execution unit 37 as the input of the layers after the convolutional layer.

As described above, even if the number of the convolution filters k is a plurality, the information processing apparatus 1 can execute the learning model in a state where the input data d is concealed as in the case of a single number.

<Second modification>
In the above, the case where the conversion filter acquisition unit 32 Fourier transforms the convolution filter k to acquire the conversion filter K having the same size as the input data d has been described. Here, when the convolution filter k is a filter that includes a large number of specific frequencies and does not include other frequencies, the elements of the conversion filter K contain a large number of 0s. In this case, when the concealment filter generation unit 34 generates the concealment filter which is the Hadamard product of the conversion filter K and the key matrix E, the part where the element of the conversion filter K is 0 also has the element of the concealment filter 0. It becomes. In such a case, the structure of the conversion filter K may be estimated from the concealment filter.

Therefore, the key matrix generation unit 33 generates two key matrices, a first key matrix E and a second key matrix F. Both the first key matrix E and the second key matrix F are matrices having the same size as the input data d, and each is a matrix composed of elements other than 0.

The concealment filter generation unit 34 generates a Hadamard product of the conversion filter K plus the second key matrix F and the first key matrix E as the concealment filter. That is, the concealment filter is (K + F) 〇E.

The inverse key matrix generation unit 35 generates an inverse key matrix 1 / E, which is the inverse element of the Hadamard product of the first key matrix E. The data acquisition unit 36 acquires the first concealment data which is the Hadamard product of the transformation input data D which is the Fourier transform of the input data d to be processed by the convolutional neural network and the inverse key matrix 1 / E. The data acquisition unit 36 also acquires the second concealment data, which is the Hadamard product of the transform input data D, which is the Fourier transform of the input data d to be processed by the convolutional neural network, and the second key matrix F. Specifically, the first concealment data is 1 / E ○ D, and the second concealment data is F ○ D. Therefore, the input API transmitted by the inverse key matrix transmission unit 39 to the user terminal T includes the second key matrix F in addition to the inverse key matrix 1 / E.

The filter execution unit 37 Fourier inverse transforms the matrix obtained by subtracting the second concealment data F ○ D from the Hadamard product of the first concealment data 1 / E ○ D and the concealment filter (K + F) 〇E. The matrix obtained by subtracting the second concealment data F ○ D from the Hadamard product of the first concealment data 1 / E ○ D and the concealment filter 1 / E ○ D is as shown in the following equation (9). Can be transformed.
(1 / E ○ D) ○ ((K + F) ○ E) -F ○ D = D ○ K (9)

The right side of the equation (9) is equal to the left side of the equation (1), and the convolution of the input data d and the convolution filter k is expressed in the frequency space. From the above, even when there are many 0s in the elements of the conversion filter K, the elements of the matrix can be non-zeroed by adding the second key matrix F to the conversion filter K.

1 ... Information processing device 2 ... Storage unit 3 ... Control unit 30 ... Model acquisition unit 31 ... Folding filter acquisition unit 32 ... Conversion filter acquisition unit 33 ... Key matrix generation Unit 330 ... Random number generation unit 331 ... Matrix parameter acquisition unit 332 ... Matrix generation unit 34 ... Concealment filter generation unit 35 ... Reverse key matrix generation unit 36 ... Data acquisition unit 37.・ ・ Filter execution unit 38 ・ ・ ・ Model execution unit 39 ・ ・ ・ Reverse key matrix transmission unit T ・ ・ ・ User terminal N ・ ・ ・ Communication network

Claims (7)

A model acquisition unit that acquires a learning model of a convolutional neural network that uses a matrix as input data,
A convolution filter acquisition unit that acquires a convolution filter in the convolution layer in the learning model, and a convolution filter acquisition unit.
A conversion filter acquisition unit that obtains a conversion filter having the same size as the input data by Fourier transforming the convolution filter.
A key matrix generator that generates a key matrix that is the same size as the input data and is a matrix composed of elements other than 0.
A concealment filter generator that generates a concealment filter that is the Hadamard product of the conversion filter and the key matrix,
An inverse key matrix generator that generates an inverse key matrix that is the inverse element of the Hadamard product of the key matrix,
A data acquisition unit that acquires the concealed data, which is the Hadamard product of the Fourier transform of the data to be processed by the convolutional neural network and the inverse key matrix.
A filter execution unit that Fourier inverse transforms the Hadamard product of the concealment data and the concealment filter,
A model execution unit that acquires the output of the learning model related to the processing target data by using the execution result of the filter execution unit as the input of the layers after the convolutional layer.
Information processing device equipped with. Further provided with a reverse key matrix transmitter that transmits the reverse key matrix to the user terminal via a communication network.
The data acquisition unit acquires the confidential data from the user terminal via the communication network.
The model execution unit transmits the output of the learning model to the user terminal via the communication network.
The information processing device according to claim 1. The key matrix generator
A random number generator that generates a random number in which both the absolute value and the reciprocal of the absolute value are greater than or equal to a predetermined value,
A matrix parameter acquisition unit that acquires the size of the input data, and
A matrix generator that uses the random numbers generated by the random number generator as elements and generates a matrix of the same size as the input data as the key matrix.
The information processing apparatus according to claim 1 or 2. The convolution filter acquisition unit acquires a convolution filter in at least the first convolution layer among the plurality of convolution layers in the learning model.
The information processing device according to any one of claims 1 to 3. The inverse key matrix generator generates the inverse element of the Hadamard product by substituting each of the plurality of elements constituting the key matrix with the reciprocal of each element.
The information processing device according to any one of claims 1 to 4. The processor
Steps to acquire a learning model of a convolutional neural network that uses a matrix as input data,
The step of acquiring the convolution filter in the convolution layer in the learning model, and
A step of Fourier transforming the convolution filter to obtain a transformation filter having the same size as the input data.
A step of generating a key matrix, which is a matrix of the same size as the input data and is a matrix composed of elements other than 0, respectively.
A step of generating a concealment filter which is a Hadamard product of the conversion filter and the key matrix,
The step of generating an inverse key matrix, which is the inverse element of the Hadamard product of the key matrix,
The Fourier transform of the data to be processed by the convolutional neural network, the step of acquiring the concealed data which is the Hadamard product of the inverse key matrix, and
The step of acquiring the Hadamard product of the concealment data and the concealment filter, and
A step of inverse Fourier transforming the Hadamard product of the concealment data and the concealment filter,
A step of acquiring the output of the learning model regarding the data to be processed by using the matrix obtained by the inverse Fourier transform as the input of the layers after the convolution layer, and
Information processing method to execute. On the computer
A function to acquire a learning model of a convolutional neural network that uses a matrix as input data,
A function to acquire a convolution filter in the convolution layer in the learning model, and
A function of Fourier transforming the convolution filter to obtain a conversion filter having the same size as the input data.
A function to generate a key matrix that is a matrix of the same size as the input data and is a matrix composed of elements other than 0.
A function to generate a concealment filter, which is the Hadamard product of each conversion filter and the assigned key matrix, and
A function to generate an inverse key matrix, which is the inverse element of the Hadamard product of the key matrix,
The Fourier transform of the data to be processed by the convolutional neural network, the function of acquiring the concealed data which is the Hadamard product of the inverse key matrix, and
A function to acquire the Hadamard product of the concealment data and the concealment filter, and
A function of inverse Fourier transforming the Hadamard product of the concealment data and the concealment filter,
A function of acquiring the output of the learning model related to the data to be processed by using the matrix obtained by the inverse Fourier transform as the input of the layers after the convolution layer.
A program that realizes.