JP7601205B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device , an information processing method, and a program.

Technology is known for generating a data generation model that generates data by machine learning. For example, Non-Patent Document 1 describes a technology for generating a data generation model that generates data by adversarially training the model against both a predictive model that predicts the class of data and a discriminative model that discriminates whether the data is true or false. Data generated by such a data generation model is used, for example, as training data for an inference model.

Mullick, Sankha Subhra, Shounak Datta, and Swagatam Das. "Generative Adversarial Minority Oversampling." Proceedings of the IEEE International Conference on Computer Vision. 2019.

However, in the technology described in Non-Patent Document 1 mentioned above, there is room for improvement in the diversity of data generated by the data generation model. For example, if learning data can be acquired from other information processing devices when there is a shortage of learning data, it may be possible to improve the learning efficiency of the data generation model. However, even in such a case, in practice, it may be difficult to acquire learning data held by other information processing devices, for example, due to privacy considerations.

One aspect of the present invention has been made in consideration of the above problems, and one example of the purpose of the present invention is to provide a technology for generating a data generation model that can generate more diverse data even in a situation where it is difficult to obtain learning data from other information processing devices.

An information processing device according to one aspect of the present invention comprises: a learning data acquisition means for acquiring learning data; a generation model information acquisition means for acquiring first generation model information relating to a first data generation model, the first generation model information reflecting second generation model information relating to a second data generation model different from the first data generation model; and a data generation model learning means for training the first data generation model, defined by reference to the first generation model information, using the learning data .

An information processing method according to one aspect of the present invention includes acquiring training data, acquiring first generation model information relating to a first data generation model, the first generation model information reflecting second generation model information relating to a second data generation model different from the first data generation model, and training the first data generation model, defined by reference to the first generation model information, using the training data .

A program according to one aspect of the present invention is a program for causing a computer to function as an information processing device, and causes the computer to function as: learning data acquisition means for acquiring learning data; generation model information acquisition means for acquiring first generation model information relating to a first data generation model, the first generation model information reflecting second generation model information relating to a second data generation model different from the first data generation model; and data generation model learning means for learning the first data generation model defined with reference to the first generation model information using the learning data .

According to one aspect of the present invention, even in a situation where it is difficult to obtain learning data from other information processing devices, it is possible to efficiently train a data generation model.

1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment. 1 is a flow chart showing the flow of an information processing method according to the first exemplary embodiment; FIG. 11 is a block diagram showing a configuration of an information processing device according to a second exemplary embodiment. FIG. 11 is a flow chart showing the flow of an information processing method according to the second exemplary embodiment. FIG. 11 is a block diagram showing a configuration of an information processing device according to an exemplary embodiment 3. FIG. 11 is a flow chart showing the flow of an information processing method according to an exemplary embodiment 3. FIG. 11 is a block diagram showing a configuration of an information processing system according to an exemplary embodiment 4. FIG. 11 is a flow chart showing the flow of an information processing method according to the fourth exemplary embodiment. FIG. 13 is a block diagram showing a configuration of an information processing system according to an exemplary embodiment 5. FIG. 11 is a flow diagram illustrating the flow of an information processing method according to the fifth exemplary embodiment. FIG. 13 is a flow chart illustrating the flow of another information processing method according to the fifth exemplary embodiment. FIG. 13 is a schematic diagram illustrating a specific example of a data generation model in exemplary embodiment 5. FIG. 13 is a schematic diagram illustrating another specific example of a data generation model in exemplary embodiment 5. FIG. 13 is a schematic diagram illustrating yet another specific example of a data generation model in exemplary embodiment 5. FIG. 13 is a block diagram showing a configuration of an information processing system according to a sixth exemplary embodiment. FIG. 13 is a flow diagram illustrating the flow of an information processing method according to the sixth exemplary embodiment. FIG. 13 is a block diagram showing a configuration of an information processing system according to an exemplary embodiment 7. FIG. 13 is a block diagram showing a configuration of an information processing system according to an eighth exemplary embodiment. FIG. 13 is a block diagram showing a configuration of an information processing system according to an exemplary embodiment 9. FIG. 2 is a diagram illustrating a configuration of a learning data acquisition unit in exemplary embodiment 10. FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device and a server in each exemplary embodiment of the present invention.

[Example embodiment 1]
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first exemplary embodiment of the present invention will be described in detail with reference to the accompanying drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

The information processing device 10 according to this exemplary embodiment trains a data generation model using training data (hereinafter also referred to as generation model training data), and can also be called a data generation model training device. Here, how the data generated by the data generation model according to this exemplary embodiment is used after generation is not limited to this exemplary embodiment, but as an example, the data can be used to train an inference model.

For example, due to privacy considerations, etc., a situation may arise in which it is difficult to obtain sufficient inference model training data for training an inference model. In such a situation, the information processing device 10 according to this exemplary embodiment can be used as a device that generates inference model training data for training the inference model.

On the other hand, even when acquiring generative model training data, situations may arise in which it is difficult to acquire sufficient generative model training data due to privacy considerations, etc. For example, consider a situation in which there are multiple information processing devices having the same configuration as the information processing device 10. Even if the learning efficiency of the data generation model can be improved if the generative model training data acquired by these multiple information processing devices can be directly used to train the data generation model, in reality, situations may arise in which, due to privacy considerations, the information processing device 10 should avoid acquiring generative model training data acquired by other information processing devices.

The information processing device 10 according to this exemplary embodiment is configured to be able to efficiently train a data generation model while respecting privacy even in such a situation.

The information processing device 10 according to this exemplary embodiment will be described in more detail below.

<Configuration of information processing device>
The configuration of an information processing device 10 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 10. As shown in Fig. 1, the information processing device 10 includes a learning data acquisition unit 101, a generation model information acquisition unit 102, and a data generation model learning unit 103.

Here, the learning data acquisition unit 101 is an example of a learning data acquisition means according to this exemplary embodiment. Also, the generative model information acquisition unit 102 is an example of a generative model information acquisition means according to this exemplary embodiment. Also, the data generation model learning unit 103 is an example of a data generation model learning means according to this exemplary embodiment.

(Learning data acquisition unit)
The learning data acquisition unit 101 acquires learning data (generative model learning data). The generative model learning data is data for training a first data generation model, which will be described later.

(Generative model information acquisition unit)
The generative model information acquisition unit 102 acquires first generative model information relating to a first data generation model, which will be described later.

(First Data Generation Model)
The first data generation model is a model that generates data. For example, the first data generation model inputs seed data and outputs data. The data generated by the first data generation model is used, for example, as data for learning the inference model described above. However, the use of the data generated by the first data generation model is not limited to this.

(First generative model information)
As an example, the first generation model information includes various parameters that define the first data generation model. In this exemplary embodiment, the first generation model information reflects second generation model information related to a second data generation model different from the first data generation model. This makes it possible to indirectly refer to other generation model learning data without directly acquiring generation model learning data held by other information processing devices described later. As an example, when the data generation model learning unit 103 described later is configured to sequentially learn the first data generation model, the first generation model information acquired by the generation model information acquisition unit 102 at a certain point in time includes information obtained by referring to the first generation model information that defined the first generation model information before the certain point in time and the second generation model information that defined the second generation model information before the certain point in time.

(Second Data Generation Model)
The second data generation model is a model different from the first data generation model, and is a model that generates data according to input data. For example, the second generation model is trained using other generation model training data in another information processing device different from the information processing device 10 that generates the first generation model. As an example, the other information processing device may be a device configured similarly to the information processing device 10. In addition, the second data generation model may be configured to be trained sequentially.

(Second Generative Model Information)
The second generative model information includes, as an example, various parameters that define the first data generation model.

The above-mentioned first generation model information may be configured to reflect other generation model information relating to another data generation model different from both the first data generation model and the second data generation model, or may be configured to reflect further other generation model information.

(Data generation model learning part)
The data generation model learning unit 103 learns a first data generation model defined by referring to the first model information acquired by the generation model information acquisition unit 102, using the generation model learning data as teacher data.

Here, as an example, the first generative model can be a generative model configured using a neural network, such as an autoencoder or a generative adversarial network, but this does not limit this exemplary embodiment, and generative models having other configurations can also be used.

The first model information that defines the first data generation model after learning by the data generation model learning unit 103 is used to generate updated model information by, for example, combining it with second model information that defines a second data generation model that is different from the first data generation model.

<Flow of information processing method>
The information processing method S10 executed by the information processing device 10 configured as above will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S10. As shown in Fig. 2, the information processing method S10 includes steps S101 to S103.

(Step S101)
In step S101, the learning data acquisition unit 101 acquires generative model learning data.

(Step S102)
In step S102, the generative model information acquisition unit 102 acquires first generative model information.

(Step S103)
In step S103, the data generation model learning unit 103 learns a first data generation model defined by referring to the first model information acquired by the generation model information acquisition unit 102, using the generation model learning data as teacher data.

The information processing device 10 may sequentially learn the first generative model by sequentially executing the above-mentioned information processing method S10. As an example, the first generative model information acquired by the generative model information acquisition unit 102 in step S102 of the information processing method S10 executed for the nth time (n is a natural number) includes information obtained by taking a weighted average of (i) the first generative model information that defined the first generative model information learned in step S103 before the n-1th time, and (ii) the second generative model information that defined the second generative model information at the time point before the n-1th time.

<Advantages of this exemplary embodiment>
As described above, in the information processing device 10 according to this exemplary embodiment,
Acquire training data for the first data generation model (generative model training data);
Acquire first generative model information regarding a first data generation model, the first generative model information reflecting second generative model information regarding a second data generation model different from the first data generation model;
A first data generation model defined by referring to the acquired first generation model information is trained using training data (generative model training data);
The following configuration has been adopted.

According to the above configuration, it is possible to generate a first data generation model by indirectly using the other generative model training data used by the other information processing device that generated the second generative model, without directly acquiring the other generative model training data. In other words, it is possible to generate a first data generation model that can generate diverse data in the same way as when learning by directly using both the generative model training data acquired by the device itself and the generative model training data acquired by the other information processing device.

As a result, even in a situation where the information processing device 10 should avoid acquiring other generation model training data used in other information processing devices, it is possible to generate a first data generation model capable of generating more diverse data. Also, as an example, when data generated by such a first data generation model is used to train an inference model, the diversity of the inference model training data can be increased, and an inference model that can handle more diverse data with higher accuracy can be obtained.

The learning performed by the information processing device 10 according to this exemplary embodiment is, for example, also called associative learning, but this term does not limit this exemplary embodiment.

Exemplary embodiment 2
A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. The information processing device 20 according to this exemplary embodiment is a device that generates data using the first data generation model generated by the information processing device 10 according to the first exemplary embodiment. As an example, the data generated by the information processing device 20 may be data to be used as teacher data in learning an inference model. However, the use of the data generated by the information processing device 20 is not limited to this.

The information processing device 20 according to this exemplary embodiment will be described in more detail below.

<Configuration of information processing device>
The configuration of the information processing device 20 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 20. As shown in Fig. 3, the information processing device 20 includes a seed data acquisition unit 201 and a data generation unit 202.

Here, the seed data acquisition unit 201 is an example of a seed data acquisition means according to this exemplary embodiment. Also, the data generation unit 202 is an example of a data generation means according to this exemplary embodiment.

(Seed Data Acquisition Department)
The seed data acquisition unit 201 acquires seed data. The seed data is data to be input to the first data generation model. For example, the seed data acquisition unit 201 may acquire the seed data by generating the seed data, or may acquire seed data generated by another functional block (not shown) included in the device or another device (not shown). A specific example of the seed data is not intended to limit the present exemplary embodiment, but may include random noise data as an example. In addition, the name "seed data" does not limit the present exemplary embodiment in any way.

(Data Generation Unit)
The data generation unit 202 generates data using seed data and a first data generation model. Specifically, the data generation unit 202 generates data by inputting the seed data into the first data generation model. The first data generation model is defined with reference to first generation model information. The first generation model information reflects second generation model information relating to a second data generation model different from the first data generation model. Details of the first data generation model, the first generation model information, the second data generation model, and the second generation model information are as described in the first exemplary embodiment.

<Flow of information processing method>
The information processing method S20 executed by the information processing device 20 configured as above will be described with reference to Fig. 4. Fig. 4 is a flow diagram showing the flow of the information processing method S20. As shown in Fig. 4, the information processing method S20 includes steps S201 to S203.

(Step S201)
In step S201, the seed data acquisition unit 201 acquires seed data. For example, the seed data acquisition unit 201 may acquire randomly generated data as the seed data.

(Step S202)
In step S202, the data generation unit 202 generates data by inputting seed data into a first data generation model. For example, first generation model information regarding the first data generation model generated by the information processing device 10 according to the first exemplary embodiment is stored in a memory (not shown) of the information processing device 20. The data generation unit 202 generates data using the first data generation model defined with reference to the first generation model information.

<Advantages of this exemplary embodiment>
As described above, in the information processing device 20 according to this exemplary embodiment,
・Get seed data,
- Perform data generation using the acquired seed data and a first data generation model.
The first data generative model is defined by first generative model information;
The following configuration is adopted:
The first generation model information is information about a first data generation model, and is information that reflects a second data generation model that is different from the first data generation model.

According to the above configuration, by using a first data generation model in which the second generation model information regarding the second generation model is reflected, it is possible to generate diverse data equivalent to both the first generation model and the generation model training data used to train the second generation model, without directly referring to other generation model training data used to train the second generation model. Also, as an example, when the data generated in this exemplary embodiment is used to train an inference model, the diversity of the inference model training data can be increased, and an inference model that can handle a wider variety of data with higher accuracy can be obtained.

Exemplary embodiment 3
A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. An information processing device 30 according to this exemplary embodiment is a device that generates updated generative model information by referring to generative model information that defines each of a plurality of data generation models.

The information processing device 30 according to this exemplary embodiment will be described in more detail below.

<Configuration of information processing device>
The configuration of the information processing device 30 according to this exemplary embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the information processing device 30. As shown in Fig. 5, the information processing device 30 includes a generative model information acquisition unit 301 and a generative model information update unit 302.

Here, the generative model information acquisition unit 301 is an example of generative model information acquisition means according to this exemplary embodiment. Also, the generative model information update unit 302 is an example of generative model information update means according to this exemplary embodiment.

(Generative model information acquisition unit)
The generative model information acquisition unit 301 acquires first generative model information relating to a first data generation model, and second generative model information relating to a second data generation model different from the first data generation model.

(First Data Generation Model and Second Data Generation Model)
The first data generation model is a model that generates data according to input data. The first data generation model is trained using generative model training data in an information processing device different from the information processing device 30. The second data generation model is a model different from the first data generation model, and is a model that generates data according to input data. The second data generation model is trained using generative model training data in an information processing device different from both the information processing device 30 and the information processing device that trains the first data generation model. The generative model training data used to train the first data generation model and the generative model training data used to train the second data generation model are, for example, data acquired independently by each information processing device, and may be different from each other.

(First Generative Model Information and Second Generative Model Information)
The first generative model information is information about the first data generation model, and includes, for example, various parameters that define the first data generation model. The second generative model information is information about the first data generation model, and includes, for example, various parameters that define the second data generation model.

Note that the first generative model information in this exemplary embodiment differs from the first generative model information in exemplary embodiments 1 and 2 and does not necessarily need to reflect the second generative model information.

(Generative model information update unit)
The generative model information update unit 302 references the first generative model information and the second generative model information to generate updated generative model information.

(Updated generative model information)
The updated generative model information reflects the first generative model information and the second generative model information. For example, the first data generation model and the second data generation model may be newly defined by referring to the updated generative model information.

<Flow of information processing method>
The information processing method S30 executed by the information processing device 30 configured as above will be described with reference to Fig. 6. Fig. 6 is a flow diagram showing the flow of the information processing method S30. As shown in Fig. 6, the information processing method S30 includes steps S301 to S203.

(Step S301)
In step S301, the generation model information acquisition unit 301 acquires first generation model information and second generation model information. For example, the generation model information acquisition unit 301 acquires the first generation model information from an information processing device that has trained a first data generation model. Also, for example, the generation model information acquisition unit 301 acquires the second generation model information from an information processing device that has trained a second data generation model.

(Step S302)
In step S302, the generative model information update unit 302 generates updated generative model information by referring to the first generative model information and the second generative model information. As an example, the generative model information update unit 302 may include information obtained by taking a weighted average of the first generative model information and the second generative model information in the updated generative model information.

<Advantages of this exemplary embodiment>
As described above, in the information processing device 30 according to this exemplary embodiment,
obtaining first generative model information relating to a first data generation model and second generative model information relating to a second data generation model different from the first data generation model;
generating updated generative model information by referring to the first generative model information and the second generative model information;
The following configuration has been adopted.

According to the above configuration, the data generation model defined by referring to the updated generative model information functions as a data generation model trained indirectly using (i) the generative model training data used to train the first data generation model and (ii) the generative model training data used to train the second data generation model.

As a result, even in a situation where it is necessary to avoid obtaining at least one of the above (i) and (ii) data for training the generation model, it is possible to generate a data generation model that can generate more diverse data by indirectly using both. In other words, it is possible to generate a data generation model that can generate data with the same diversity as when both (i) and (ii) are directly used for training. Also, as an example, when data generated using such a data generation model is used for training an inference model, the diversity of the data for training the inference model can be increased, and an inference model that can handle more diverse data with higher accuracy can be obtained.

Note that the information processing device 30 according to this exemplary embodiment functions as a server in federated learning, for example, but this wording does not limit this exemplary embodiment.

Exemplary embodiment 4
A fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to third exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

<Configuration of Information Processing System>
The configuration of the information processing system 1a according to this exemplary embodiment will be described with reference to FIG. 7. FIG. 7 is a block diagram showing the configuration of the information processing system 1a. As shown in FIG. 7, the information processing system 1a includes a first information processing device 10a and a server 30a. The first information processing device 10a and the server 30a are connected via a network N. Specific examples of the network N include a wireless LAN (Local Area Network), a wired LAN, a WAN, a public line network, a mobile data communication network, or a combination of these networks. However, the configuration of the network N is not limited to this.

(Configuration of the first information processing device)
The first information processing device 10 a includes a learning data acquisition unit 101 , a generation model information acquisition unit 102 , a data generation model learning unit 103 , and a transmission unit 110 .

The learning data acquisition unit 101 and the data generation model learning unit 103 are configured in the same manner as in exemplary embodiment 1.

The generative model information acquisition unit 102 is configured in a manner similar to that of exemplary embodiment 1. However, in this exemplary embodiment, the first generative model information is acquired by receiving it from the server 30a.

The transmission unit 110 transmits to the server 30a third generation model information regarding the first data generation model after it has been learned by the data generation model learning unit 103.

(Third generative model information)
The third generative model information is information about the first data generation model after training, and includes, for example, parameters that define the first data generation model after training.

(Fourth Generative Model Information)
The fourth generative model information is information about the second data generation model after training, and includes, for example, parameters that define the second data generation model after training.

(Server configuration)
The server 30 a includes a generative model information acquisition unit 301 , a generative model information update unit 302 , and a transmission unit 310 .

(Generative model information acquisition unit)
The generative model information acquisition unit 301 is configured in a manner similar to that of exemplary embodiment 3. However, this exemplary embodiment differs in that third generative model information and fourth generative model information are acquired instead of the first generative model information and the second generative model information.

(Generative model information update unit)
The generative model information update unit 302 is configured in a manner similar to that of exemplary embodiment 3. However, in this exemplary embodiment, the generative model information update unit 302 generates updated generative model information by referring to third and fourth generative model information instead of referring to the first and second generative model information.

The transmission unit 310 transmits the updated generative model information to the first information processing device 10a.

<Flow of information processing method>
The information processing method S1a executed by the information processing system 1a configured as above will be described with reference to FIG. 8. FIG. 8 is a flow diagram showing the flow of the information processing method S1a. As shown in FIG. 8, the information processing method S1a includes S101a to S104a executed by the first information processing device 10a, and S301a to S303a executed by the server 30a. The information processing system 1a executes the information processing method S1a sequentially. The information processing method S1a executed for the nth time will be described below. (n=1, 2, ...)
(Step S101a)
In step S101a, the learning data acquisition unit 101 acquires generative model learning data.

(Step S102a)
In step S102a, the generation model information acquisition unit 102 acquires the first generation model information by receiving it from the server 30a. Note that, when the information processing method S1a is executed for the first time (n=1), the first generation model information received from the server 30a does not necessarily have to be information reflecting the second generation model information. Also, when the information processing method S1a is executed for the second time or later (n≧2), the generation model information acquisition unit 102 acquires the "updated generation model information" transmitted by the server 30a in the previous (n-1th) step S303a (details will be described later) as the first generation model information. In this case, the acquired first generation model information (i.e., the previous updated generation model information) is information reflecting the second generation model information (i.e., the previous fourth generation model information).

(Step S103a)
In step S103a, the data generation model learning unit 103 uses the generation model learning data as teacher data to learn a first data generation model defined by referring to the first generation model information acquired by the generation model information acquisition unit 102. For example, the data generation model learning unit 103 executes an update process to update a parameter group defining the first data generation model using the generation model learning data. The data generation model learning unit 103 repeats the update process until the learning converges or a specified number of times is reached. As a result, the first data generation model is learned.

(Step S104a)
In step S104a, the transmission unit 110 transmits third generation model information regarding the first data generation model after learning to the server 30a.

(Step S301a)
In step S301a, the generation model information acquisition unit 301 of the server 30a acquires the third generation model information by receiving it from the first information processing device 10a. Also, the generation model information acquisition unit 301 acquires the fourth generation model information by receiving it from another information processing device that sequentially learns the second data generation model. The other information processing device is a device different from the first information processing device 10a and the server 30a.

(Step S302a)
In step S302a, the generation model information update unit 302 generates updated generation model information by referring to the third generation model information and the fourth generation model information. As an example, the generation model information update unit 302 may include information obtained by taking a weighted average of the third generation model information and the fourth generation model information in the updated generation model information.

(Step S303a)
In step S303a, the transmission unit 310 transmits the updated generative model information to the first information processing device 10a. The “updated generative model information” transmitted in this step is received by the first information processing device 10a as “first generative model information” in step S102a in the next (n+1) information processing method S1a.

<Advantages of this exemplary embodiment>
As described above, in the information processing system 1a according to this exemplary embodiment,
The first information processing device 10a
Acquire training data for the first data generation model (generative model training data);
Acquire, as first generation model information regarding the first data generation model, the first generation model information received from the server 30a, which reflects second generation model information regarding a second data generation model different from the first data generation model;
A first data generation model defined by referring to the acquired first generation model information is trained using training data (generative model training data);
Sending third generative model information regarding the first data generation model after learning to the server 30a;
The following configuration has been adopted.

In addition, the server 30a
Obtaining third generative model information and fourth generative model information regarding the second data generation model after training;
generating updated generative model information by referring to the third generative model information and the fourth generative model information;
Transmitting the updated generative model information to the first information processing device 10a;
The following configuration has been adopted.

According to the above configuration, the first information processing device 10a can acquire the first data generation model, which is defined by referring to the updated generation model information received from the server 30a, as a learning target. As a result, the first information processing device 10a can generate a first data generation model capable of generating more diverse data by indirectly using the other generation model learning data used by the other information processing device that generated the second generation model, without directly acquiring the other generation model learning data used by the other information processing device that generated the second generation model. In other words, the first information processing device 10a can generate a first data generation model capable of generating diverse data similar to the case of learning by directly using both the generation model learning data acquired by its own device and the generation model learning data acquired by the other information processing device. Also, as an example, when data generated by such a first data generation model is used for learning an inference model, the diversity of the inference model learning data can be increased, and an inference model that can handle more diverse data with higher accuracy can be obtained.

Exemplary embodiment 5
A fifth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to fourth exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

<Configuration of Information Processing System>
The configuration of an information processing system 1b according to this exemplary embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the configuration of the information processing system 1b. The information processing system 1b is a system that eliminates imbalances in a data group for learning an inference model and learns an inference model using the eliminated data group. In order to eliminate the imbalances, the information processing system 1b generates a data generation model by federated learning that generates data to be added to the data group.

As shown in FIG. 9, the information processing system 1b includes a first information processing device 10b-1, a second information processing device 10b-2, and a server 30b. The first information processing device 10b-1, the second information processing device 10b-2, and the server 30b are connected via a network N.

(Inference model)
In this exemplary embodiment, the inference model is a model that infers the label type of data. When data is input, the inference model outputs information indicating one of a plurality of label types. The first information processing device 10b-1 trains the first inference model using the learning data acquired by the device. The second information processing device 10b-2 trains the second inference model using the learning data acquired by the device. The first inference model and the second inference model are different from each other. Note that the configurations of the plurality of label types that can be indicated by the data output by the first inference model and the second inference model do not necessarily have to be the same. For example, the first inference model may output data indicating one of the labels L1, L3, L4, ..., Ln, and the second inference model may output data indicating one of the labels L2, L3, L4, ..., Ln. Hereinafter, when there is no need to particularly distinguish between the first inference model and the second inference model, they will simply be referred to as "inference models".

(Learning data, imbalanced data set)
The training data is a piece of data acquired for use in training an inference model, and is any one of a plurality of pieces of data included in a data group. Each piece of data included in the data group is labeled with any one of a plurality of label types, and the proportion of data pieces with a certain label type is equal to or less than a predetermined proportion. Hereinafter, such a data group is also referred to as an "imbalanced data group," and the "certain label type" is also referred to as a "minority label type."

In other words, an imbalanced data group includes multiple learning data (data fragments) each having one of multiple label types, and at least one of the multiple label types satisfies the condition of a minority label type. For example, as described above, the condition of a minority label type may be that the proportion of data fragments having that label type is equal to or less than a predetermined proportion, or that the number of data fragments having that label type is smaller than the number of other label types. However, the condition of a minority label type is not limited to the above.

The first imbalanced data group acquired by the first information processing device 10b-1 and the second imbalanced data group used by the second information processing device 10b-2 are at least partially different from each other. Here, the minority label type in the first imbalanced data group and the minority label type in the second imbalanced data group may or may not be the same. Note that when there is no need to particularly distinguish between the first imbalanced data group and the second imbalanced data group, they will continue to be referred to simply as "imbalanced data groups."

A specific example of a case where the minority label type in the first imbalanced data group and the second imbalanced data group are not the same will be described. A product image is a specific example of a data piece included in such an imbalanced data group. In this case, the first imbalanced data group and the second imbalanced data group are sets of product images obtained in different production factories. Each product image is labeled with one of "normal product", "defective product pattern A", and "defective product pattern B". In the first imbalanced data group, the proportion of product images labeled with "defective product pattern A" is equal to or less than a predetermined proportion. Or, the number of product images labeled with "defective product pattern A" is less than the number of product images labeled with "normal product" and less than the number of product images labeled with "defective product pattern B". In this case, the minority label type in the first imbalanced data group is "defective product pattern A". On the other hand, in the second imbalanced data group, the proportion of product images labeled with "defective product pattern B" is equal to or less than a predetermined proportion. Or, the number of product images labeled with "defective product pattern B" is less than the number of product images labeled with "normal products" and is less than the number of product images labeled with "defective product pattern A." In this case, the minority label type in the second imbalanced data group is "defective product pattern B."

A specific example of a case where the minority label type in the first imbalanced data group and the second imbalanced data group is the same will be described. A specific example of data fragments included in such an imbalanced data group is a communication data fragment in a financial system or a communication system. In this case, the first imbalanced data group and the second imbalanced data group are sets of communication data fragments obtained in different systems. There is an example where each communication data fragment is labeled as "normal" or "abnormal", with "abnormal" being the minority label type. Another specific example of data fragments is a medical image. In this case, the first imbalanced data group and the second imbalanced data group are sets of medical images obtained in different medical institutions. There is an example where each medical image is labeled as "no lesion" or "lesion", with "lesion" being the minority label type. However, the specific examples of data fragments and minority label types are not limited to those described above.

(Balanced Data Set)
A balanced data set includes multiple pieces of training data (data fragments) each labeled with one of multiple label types, and none of the label types meets the condition of a minority label type. For example, in a balanced data set, the proportion of training data labeled with each label type is equal to or greater than the proportion of processing, or the number of training data labeled with each label type is equal.

The first balanced data group generated by the first information processing device 10b-1 and the second balanced data group generated by the second information processing device 10b-2 are at least partially different from each other. However, when there is no need to particularly distinguish between the first balanced data group and the second balanced data group, they will continue to be referred to simply as "balanced data groups."

(Data generation model)
The data generation model generates data to be used as inference model training data. The first information processing device 10b-1 generates a first data generation model, and the second information processing device 10b-2 generates a second data generation model, but when there is no need to particularly distinguish between them, they will continue to be referred to simply as "data generation models." The data generation model is trained to generate training data that can become teacher data of at least a small number of label types.

(Data for training generative models)
The generative model training data is data for training a data generation model. A part or the whole of the imbalanced data group is used as the generative model training data.

(Data for learning inference models)
The inference model training data is data for training an inference model. In this exemplary embodiment, a balanced data group in which imbalance in an imbalanced data group is eliminated is used as the inference model training data.

(Configuration of the first information processing device)
The first information processing device 10b-1 includes a control unit 150-1 and a communication unit 160-1. The control unit 150-1 includes a learning data acquisition unit 101-1, a generation model information acquisition unit 102-1, a data generation model learning unit 103-1, a seed data acquisition unit 201-1, a data generation unit 202-1, an inference model learning unit 203-1, an inference unit 204-1, and a communication unit 160-1.

(Learning data acquisition unit)
The learning data acquisition unit 101-1 acquires a first imbalanced data group. The details of the first imbalanced data group are as described above.

(Generative model information acquisition unit)
The generative model information acquisition unit 102-1 is configured in the same manner as the generative model information acquisition unit 102 in the first exemplary embodiment.

(Data generation model learning part)
The data generation model training unit 103-1 trains the first data generation model using at least a part of the first imbalanced data group. For example, the data generation model training unit 103-1 may train at least one of an autoencoder including the first data generation model and a generative adversarial network including the first data generation model. Details of the first data generation model will be described later.

(Seed Data Acquisition Department)
The seed data acquisition unit 201-1 acquires seed data to be input to the first data generation model. For example, the seed data acquisition unit 201-1 may acquire seed data by generating the seed data, or may acquire seed data generated by another functional block (not shown) included in the device or another device (not shown).

(Data Generation Unit)
The data generating unit 202-1 generates data using seed data. Specifically, the data generating unit 202-1 generates learning data that can be teacher data of a minority label type by inputting the seed data into a first data generation model. For example, the data generating unit 202-1 may generate learning data that can be teacher data of a minority label type by inputting the seed data and a condition for identifying a minority label type into the first data generation model. Furthermore, the data generating unit 202-1 adds the generated learning data to a first imbalanced data group to generate a first balanced data group.

(Inference model learning part)
The inference model training unit 203-1 trains a first inference model by referring to the data generated by the data generation unit 202-1. Specifically, the inference model training unit 203-1 uses the first balanced data group as data for training the inference model.

(Inference section)
The inference unit 204-1 performs inference using the inference model generated by the inference unit 204-1.

(Communications Department)
The communication unit 160-1 transmits to the server 30b third generative model information regarding the first data generation model after being learned by the data generation model learning unit 103-1.

(Configuration of the second information processing device)
The second information processing device 10b-2 includes a control unit 150-2 and a communication unit 160-2. The control unit 150-2 includes a learning data acquisition unit 101-2, a generation model information acquisition unit 102-2, a data generation model learning unit 103-2, a seed data acquisition unit 201-2, a data generation unit 202-2, an inference model learning unit 203-2, and an inference unit 204-2.

The details of the configuration of the second information processing device 10b-2 can be explained in the same manner as in the explanation of the configuration of the first information processing device 10b-1, except that the suffixes "1" and "2" at the end of each reference number are interchangeable, "first" and "second" are interchangeable, and "third" and "fourth" are interchangeable.

(Server configuration)
The server 30b includes a control unit 350 and a communication unit 360. The control unit 350 includes a generative model information acquisition unit 301, a generative model information update unit 302, and the communication unit 360.

(Generative model information acquisition unit)
The generative model information acquisition unit 301 is configured in the same manner as the generative model information acquisition unit 301 in the fourth exemplary embodiment.

(Generative model information update unit)
The generative model information update unit 302 is configured similarly to the generative model information update unit 302 in exemplary embodiment 4. In particular, in this exemplary embodiment, the generative model information update unit 302 generates updated generative model information by taking the average of the third generative model information and the fourth generative model information.

(Communications Department)
The communication unit 360 distributes the updated generative model information to the first information processing device 10b-1 and the second information processing device 10b-2.

<Flow of information processing method for generating a data generation model>
The information processing method S1b executed by the information processing system 1b configured as above will be described with reference to Fig. 10. Fig. 10 is a flow diagram illustrating the flow of the information processing method S1b. The information processing system 1b generates a data generation model by sequentially executing the information processing method S1b. In Fig. 10, the reference characters including (n) at the end indicate each step of the information processing method S1b executed for the nth time, and the reference characters including (n+1) at the end indicate each step of the information processing method S1b executed for the n+1th time.

(Step S101-1)
In the n-th step S101-1, the learning data acquisition unit 101-1 of the first information processing device 10b-1 acquires a first imbalanced data group.

(Step S360)
In step S360 for the nth time, the communication unit 360 of the server 30b distributes the generative model information to the first information processing device 10b-1 and the second information processing device 10b-2. If n=1, predetermined information is applied as the generative model information to be distributed. If n≧2, the "updated generative model information" updated in step S302 for the (n-1)th time is applied as the generative model information to be distributed. Step S302 will be described later.

(Step S102-1)
In the n-th step S102-1, the generative model information acquisition unit 102-1 of the first information processing device 10b-1 acquires the generative model information distributed from the server 30b as first generative model information.

(Step S103-1)
In the n-th step S103-1, the data generation model learning unit 103-1 of the first information processing device 10b-1 learns a first data generation model defined by referring to the first generation model information acquired by the generation model information acquisition unit 102-1. For the learning, a part or all of the learning data included in the first imbalanced data group is used as teacher data. Specifically, the data generation model learning unit 103-1 repeats an update process of updating a parameter group using the learning data included in the first imbalanced data group so that the first data generation model outputs learning data that can be teacher data for at least a minority label type, until the learning converges or a specified number of times is reached.

(Step S160-1)
In the n-th step S160-1, the communication unit 160-1 of the first information processing device 10b-1 transmits third generation model information regarding the learned first data generation model to the server 30b.

(Steps S101-2 to S160-2)
The second information processing device 10b-2 executes steps S101-2, S102-2, S103-2, and S160-2 for the nth time. Details of these steps can be similarly explained by replacing the "1" and "2" at the end of each reference symbol with each other in the explanation of steps S101-1, S102-1, S103-1, and S160-1 executed by the first information processing device 10b-1, replacing "1" and "2" with each other, replacing "1st" and "2nd" with each other, and replacing "3rd" and "4th" with each other.

(Step S301)
In step S301 of the (n+1)th iteration, the generative model information acquisition unit 301 of the server 30b acquires the third generative model information by receiving it from the first information processing device 10b-1. Also, the generative model information acquisition unit 301 acquires the fourth generative model information by receiving it from the second information processing device 10b-2.

(Step S302)
In step S302 for the (n+1)th time, the generative model information update unit 302 generates updated generative model information by referring to the third generative model information and the fourth generative model information. Specifically, the generative model information update unit 302 includes information obtained by taking a weighted average of the third generative model information and the fourth generative model information in the updated generative model information.

(Step S360)
In step S360 of the (n+1)th time, the communication unit 360 distributes the updated generative model information to the first information processing device 10b-1 and the second information processing device 10b-2.

The first information processing device 10b-1 executes steps S101-1, S102-1, S103-1, and S160-1 for the n+1th time. The second information processing device 10b-2 executes steps S101-2, S102-2, S103-2, and S160-2 for the n+1th time. The details are as explained for each step of the nth time.

In this way, by sequentially executing the information processing method S1b, the first information processing device 10b-1 can indirectly use the learning data acquired by the second information processing device 10b-2 without directly acquiring it. As a result, the first information processing device 10b-1 can generate a first data generation model capable of generating more diverse data. Also, the second information processing device 10b-2 can indirectly use the learning data acquired by the first information processing device 10b-1 without directly acquiring it. As a result, the second information processing device 10b-2 can generate a second data generation model capable of generating more diverse data.

<Flow of information processing method for generating inference models>
The information processing method S20b executed by the first information processing device 10b-1 and the second information processing device 10b-2 will be described with reference to FIG. 11. FIG. 11 is a flow diagram illustrating the flow of the information processing method S20b. The first information processing device 10b-1 executes the information processing method S20b to generate a first inference model. The second information processing device 10b-2 executes the information processing method S20b to generate a second inference model. In the following, an example in which the first information processing device 10b-1 executes the information processing method S20b will be described. In the following, an example in which the second information processing device 10b-2 executes the information processing method S20b will be similarly described by replacing the "1" and "2" at the end of each reference symbol with each other, and replacing "first" and "second" with each other. As shown in FIG. 11, the information processing method S20b includes steps S201 to S203.

(Step S201)
In step S201, the seed data acquisition unit 201-1 acquires seed data. Specifically, the seed data acquisition unit 201-1 acquires at least as many pieces of seed data as are capable of resolving the imbalance in the first imbalanced data group. When acquiring a plurality of pieces of seed data, it is desirable that these pieces of seed data are different from each other. The number capable of resolving the imbalance is, for example, the difference between the number of pieces of learning data to which a minority label type is assigned in the first imbalanced data group and the number that is a predetermined ratio in the first balanced data group after the imbalance is resolved. For example, it is assumed that the number of pieces of learning data to which a minority label type is assigned in the first imbalanced data group is 20, the number of pieces of learning data included in the first imbalanced data group is 100, and the predetermined ratio is 0.3. In this case, the number capable of resolving the imbalance is 15. In addition, the number capable of resolving the imbalance is, for example, the difference between the number of pieces of learning data to which a minority label type is assigned and the number of pieces of learning data to which other label types are assigned in the first imbalanced data group. For example, assume that there are two label types, the number of learning data to which the minority label type is assigned is 20, and the number of learning data to which the other label type is assigned is 40. In this case, the number for which imbalance can be resolved is 20. Note that the number for which imbalance can be resolved is not limited to the above example.

(Step S202)
In step S202, the data generation unit 202-1 generates training data that can be teacher data of minority label types by inputting each seed data generated in step S201 into a first data generation model. The data generation unit 202-1 also adds the generated training data to a first imbalanced data group. This generates a first balanced data group in which the imbalance in the first imbalanced data group is eliminated.

(Step S203)
In step S203, the inference model training unit 203-1 trains a first inference model. For the training, each piece of training data included in the first balanced data group is used as inference model training data.

<Example of data generation model>
Here, specific examples of the first data generation model and the second data generation model will be described. In the following, when there is no need to particularly distinguish between the first data generation model and the second data generation model, they will simply be referred to as "data generation models." Furthermore, when there is no need to particularly distinguish between which information processing device each functional block included in the first information processing device 10b-1 and the second information processing device 10b-2 is included in, the "-1" or "-2" at the end of the reference numeral will be omitted.

(Specific Example 1)
A specific example 1 in which the data generation model is realized as a decoder included in a variational autoencoder will be described with reference to FIG. 12. FIG. 12 is a schematic diagram for explaining the specific example 1 of the data generation model. As shown in FIG. 12, the variational autoencoder trained by the data generation model training unit 103 includes an encoder EN1 and a decoder DE1. When data X is input, the encoder EN1 outputs a latent variable Z. When latent variable Z is input, the decoder DE1 outputs data X'. The data generation model training unit 103 inputs data X to which a minority label type is assigned among the training data X acquired by the training data acquisition unit 101 to the encoder EN1, and performs training so that the difference between data X' output by the decoder DE1 and data X becomes smaller. In detail, the data generation model training unit 103 uses the training data X to which a minority label type is assigned to perform an update process for updating a parameter group that defines the encoder EN1 and the decoder DE1 so that the difference between X and X' becomes smaller. Furthermore, the data generation model learning unit 103 repeats the update process until the learning converges or a specified number of times is reached. After learning, the decoder DE1 becomes a data generation model that generates learning data that can become teacher data of a minority label type.

The seed data acquisition unit 201 generates seed data S to be input to the decoder DE1 after learning. Specifically, the seed data acquisition unit 201 inputs learning data X with a minority label type to an encoder EN and generates data within the range indicated by the latent variable Z output as the seed data S. For example, the seed data acquisition unit 201 may generate data randomly sampled within the range indicated by the latent variable Z as the seed data S.

The data generation unit 202 generates learning data that can be used as teacher data for a minority label type by inputting the seed data S to the decoder DE2.

(Specific Example 2)
A second specific example in which the data generation model is realized as a decoder included in a conditional variational autoencoder will be described with reference to FIG. 13. FIG. 13 is a schematic diagram for explaining the second specific example of the data generation model. As shown in FIG. 13, the conditional variational autoencoder trained by the data generation model training unit 103 includes an encoder EN2 and a decoder DE2. When data X and a condition Y are input, the encoder EN2 outputs a latent variable Z. When latent variable Z and a condition Y are input, the decoder DE1 outputs data X'. Here, information specifying a label type is used as the condition Y.

The data generation model learning unit 103 inputs the learning data X acquired by the learning data acquisition unit 101 to the encoder EN2, and inputs a condition Y that specifies the label type attached to the learning data to the encoder EN2 and the decoder DE2. The data generation model learning unit 103 also performs learning so that the difference between the data X' output by the decoder DE2 and the data X becomes smaller. In detail, the data generation model learning unit 103 uses the learning data X acquired by the learning data acquisition unit 101 to perform an update process that updates a group of parameters that define the encoder EN2 and the decoder DE2 so that the difference between X and X' becomes smaller. The data generation model learning unit 103 also repeats the update process until the learning converges or until a specified number of times is reached. After learning, the decoder DE2 becomes a data generation model that generates learning data that can be teacher data for the label type specified by the condition Y.

The seed data acquisition unit 201 generates seed data S to be input to the decoder DE2 after learning. Specifically, the seed data acquisition unit 201 inputs learning data and a condition Y that specifies a minority label type to the encoder EN, references the output latent variable Z, and generates data within the range indicated by the latent variable Z as the seed data S. For example, the seed data acquisition unit 201 may generate data randomly sampled within the range indicated by the latent variable Z as the seed data S.

The data generation unit 202 generates learning data that can be used as teacher data for minority label types by inputting seed data S and a condition Y for identifying minority label types to the decoder DE2.

(Specific Example 3)
A specific example 3 in which the data generation model is realized as a generator included in a generative adversarial network will be described with reference to Fig. 14. Fig. 14 is a schematic diagram for explaining the specific example 3 of the data generation model. As shown in Fig. 14, the generative adversarial network trained by the data generation model training unit 103 includes a generator GEN and a discriminator DIS. When noise ND is input, the generator GEN outputs data GD. When data GD and true data X are input, the discriminator DIS determines whether the data GD is the true data X.

The data generation model learning unit 103 inputs noise ND to the generator GEN, and inputs data X with a minority label type from among the learning data X acquired by the learning data acquisition unit 101 to the discriminator DIS. The data generation model learning unit 103 also performs learning so that data GD generated by the generator GEN is determined to be true. In detail, the data generation model learning unit 103 uses the learning data X with a minority label type to perform an update process for updating a group of parameters that define the generator GEN and the discriminator DIS so that data GD is determined to be true. The data generation model learning unit 103 also repeats the update process until the learning converges or until a specified number of times is reached. After learning, the generator GEN becomes a data generation model that generates learning data that can become teacher data of a minority label type.

The seed data acquisition unit 201 generates seed data S to be input to the learned generator GEN. Specifically, the seed data acquisition unit 201 may generate randomly generated noise as the seed data S.

The data generation unit 202 generates learning data that can be used as teacher data for a minority label type by inputting the seed data S to the post-training generator GEN.

(Other specific examples)
As another specific example, the data generation model is realized as a generator included in a conditional generative adversarial network. In such a specific example, a condition Y is further input to the generator GEN and the discriminator DIS shown in FIG. 14. When the condition Y and the noise ND are input, the generator GEN outputs data GD. When the condition Y, the data GD, and the true data X are input, the discriminator DIS determines whether the data GD is the true data X.

In this case, the data generation model learning unit 103 inputs to the discriminator DIS any of the learning data X acquired by the learning data acquisition unit 101 and a condition Y that specifies the label type attached to the data X. The data generation model learning unit 103 also inputs the condition Y that specifies the label type and noise ND to the generator GEN. The data generation model learning unit 103 also performs learning so that the data GD generated by the generator GEN is determined to be true. In detail, the data generation model learning unit 103 uses the learning data X acquired by the learning data acquisition unit 101 to perform an update process that updates a group of parameters that define the generator GEN and the discriminator DIS so that the data GD is determined to be true. The data generation model learning unit 103 also repeats the update process until the learning converges or until a specified number of times is reached. After learning, the generator GEN becomes a data generation model that generates learning data that can be teacher data for the label type specified by the condition Y.

The seed data acquisition unit 201 generates seed data S to be input to the generator GEN after learning. The data generation unit 202 generates learning data that can be used as teacher data for the minority label type by inputting the seed data S and a condition Y that identifies the minority label type to the decoder DE2.

<Advantages of this exemplary embodiment>
As described above, in this exemplary embodiment,
Each of the first information processing device 10b-1 and the second information processing device 10b-2 acquires an imbalanced data set, and generates a data generation model for generating learning data that can be teacher data of a minority label type by associative learning using the acquired imbalanced data set.
The following configuration has been adopted.

According to the above configuration, even if the number of learning data with a minority label type is insufficient, it is possible to indirectly use learning data with a minority label type acquired by another information processing device. As a result, it is possible to generate a data generation model capable of generating more diverse data as learning data that can become teacher data for a minority label type.

In addition, according to the above configuration, by adding training data that can be teacher data of a minority label type generated using a data generation model, it is possible to generate a balanced data set that eliminates the imbalance in the imbalanced data set. As a result, by using such a balanced data set as data for training an inference model, it is possible to train the inference model with higher accuracy.

Exemplary embodiment 6
A sixth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to fifth exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

<Configuration of Information Processing System>
The configuration of an information processing system 1c according to this exemplary embodiment will be described with reference to Fig. 15. Fig. 15 is a block diagram showing the configuration of the information processing system 1c. The information processing system 1c is a modified version of the information processing system 1c according to the fifth exemplary embodiment, in which an inference model is generated by federated learning. That is, the information processing system 1c generates a data generation model for eliminating imbalance in an imbalanced data group used for learning the inference model by federated learning, and also generates the inference model itself by federated learning.

As shown in FIG. 15, the information processing system 1c includes a first information processing device 10c-1, a second information processing device 10c-2, and a server 30c. The first information processing device 10c-1, the second information processing device 10c-2, and the server 30c are connected via a network N.

(Configuration of the first information processing device)
The first information processing device 10c-1 includes an inference model information acquisition unit 205-1 in addition to the same configuration as the first information processing device 10b-1 according to the exemplary embodiment 5. In this exemplary embodiment, the configurations of the inference model learning unit 203-1 and the communication unit 160-1 are slightly different from those of the exemplary embodiment 5. The other functional blocks are similar to those of the exemplary embodiment 5, and therefore detailed description will not be repeated.

(Inference model information acquisition unit)
The inference model information acquisition unit 205-1 acquires first inference model information regarding a first inference model.

(First inference model information)
The first inference model information includes, as an example, various parameters that define the first inference model. In this exemplary embodiment, the first inference model information reflects second inference model information regarding a second inference model that is different from the first inference model.

(Inference model learning part)
The inference model learning unit 203-1 learns a first inference model defined by referring to the first inference model information, using the data generated by the data generating unit 202-1. Specifically, the inference model learning unit 203-1 uses a first balanced data group in which the imbalance in the first imbalanced data group is eliminated by adding the data generated by the data generating unit 202-1, for learning the first inference model.

(Communications Department)
In addition to being configured similarly to exemplary embodiment 5, communication unit 160-1 transmits third inference model information to server 30c.

(Third inference model information)
The third inference model information is information about the first inference model after training, and includes, for example, parameters that define the first inference model after training.

(Fourth inference model information)
The fourth inference model information is information about the second inference model after training, and includes, for example, parameters that define the second inference model after training.

(Configuration of the second information processing device)
The second information processing device 10c-2 includes an inference model information acquisition unit 205-2 in addition to the same configuration as the second information processing device 10b-2 according to exemplary embodiment 5. Details of the configuration of the second information processing device 10c-2 can be explained in the same manner as in the description of the configuration of the first information processing device 10c-1, except that the suffixes "1" and "2" of each reference sign are replaced with each other, "first" and "second" are replaced with each other, and "third" and "fourth" are replaced with each other.

(Server configuration)
In addition to the same configuration as the server 30b according to exemplary embodiment 5, the server 30c includes an inference model information acquisition unit 303 and an inference model information update unit 304. The configuration of the communication unit 360 is slightly different from that of exemplary embodiment 5. The other functional blocks are similar to those of exemplary embodiment 5, and therefore detailed description will not be repeated.

(Inference model information acquisition unit)
The inference model information acquisition unit 303 acquires third inference model information regarding the first inference model and fourth inference model information regarding the second inference model.

(Inference model information update unit)
The inference model information update unit 304 generates updated inference model information by referring to the third inference model information and the fourth inference model information.

(Updated inference model information)
The updated inference model information reflects the third generative model information and the fourth generative model information.

(Communications Department)
In addition to being configured similarly to exemplary embodiment 5, the communication unit 360 distributes updated inference model information to the first information processing device 10c-1 and the second information processing device 10c-2.

<Flow of information processing method for generating a data generation model>
The information processing system 1c configured as described above generates a first data generation model and a second data generation model by sequentially executing the information processing method S1b shown in FIG. 10 in the same manner as in exemplary embodiment 5.

<Flow of information processing method for generating inference models>
The information processing method S1c executed by the information processing system 1c will be described with reference to Fig. 16. Fig. 16 is a flow diagram illustrating the flow of the information processing method S1c. The information processing system 1c generates an inference model by sequentially executing the information processing method S1c. In Fig. 16, the reference characters including (n) at the end indicate each step of the information processing method S1c executed for the nth time, and the reference characters including (n+1) at the end indicate each step of the information processing method S1c executed for the n+1th time.

(Step S201-1)
In the n-th step S201-1, the seed data acquisition unit 201-1 of the first information processing device 10c-1 acquires seed data in an amount sufficient to eliminate the imbalance in the first imbalanced data group.

(Step S202-1)
In the n-th step S202-1, the data generation unit 202-1 of the first information processing device 10c-1 generates training data that can be teacher data of a minority label type by inputting each seed data into a first data generation model. As a result, a number of training data that can eliminate the imbalance is generated. In addition, the data generation unit 202-1 adds the generated training data to a first imbalanced data group. As a result, a first balanced data group is generated.

(Step S370)
In the n-th step S370, the communication unit 360 of the server 30c distributes the updated inference model information to the first information processing device 10c-1 and the second information processing device 10c-2. If n=1, predetermined information is applied as the inference model information to be distributed. If n≧2, the "updated inference model information" updated in the n-1th step S304 is applied as the inference model information to be distributed. Step S304 will be described later.

(Step S205-1)
In the nth step S205-1, the inference model information acquisition unit 205-1 of the first information processing device 10c-1 acquires the inference model information distributed from the server 30c as the first inference model information.

(Step S203-1)
In the n-th step S203-1, the inference model learning unit 203-1 of the first information processing device 10c-1 learns a first inference model defined by referring to the first inference model information acquired by the inference model information acquisition unit 205-1. The first balanced data group generated in step S202-1 is used for the learning. Specifically, the data generation model learning unit 103-1 repeats an update process for updating a parameter group of the first inference model using learning data included in the first balanced data group until the learning converges or a specified number of times is reached.

(Step S170-1)
In the nth step S170-1, the communication unit 160-1 of the first information processing device 10c-1 transmits third inference model information regarding the first inference model after learning to the server 30c.

(Steps S201-2, S202-2, S205-2, S203-2, S170-2)
The second information processing device 10c-2 executes steps S201-2, S202-2, S205-2, S203-2, and S170-2 for the nth time. Details of these steps will be explained in the same manner as in the explanation of steps S201-1, S202-1, S205-1, S203-1, and S170-1 executed by the first information processing device 10c-1, except that "1" and "2" at the end of each reference symbol are replaced with each other, "first" and "second" are replaced with each other, and "third" and "fourth" are replaced with each other.

(Step S303)
In step S303 of the (n+1)th iteration, the inference model information acquisition unit 303 of the server 30c acquires the third inference model information by receiving it from the first information processing device 10c-1. Also, the inference model information acquisition unit 303 acquires the fourth inference model information by receiving it from the second information processing device 10c-2.

(Step S304)
In step S304 for the (n+1)th time, the inference model information update unit 304 generates updated inference model information by referring to the third inference model information and the fourth inference model information. Specifically, the inference model information update unit 304 includes information obtained by taking a weighted average of the third inference model information and the fourth inference model information in the updated inference model information.

(Step S370)
In the (n+1)th step S370, the communication unit 360 distributes the updated inference model information to the first information processing device 10c-1 and the second information processing device 10c-2.

The first information processing device 10c-1 executes steps S201-1, S202-1, S205-1, S203-1, and S170-1 for the n+1th time. The second information processing device 10c-2 executes steps S201-2, S202-2, S205-2, S203-2, and S170-2 for the n+1th time. The details are as explained for each step of the nth time.

In this way, by sequentially executing the information processing method S1c, the first information processing device 10c-1 can indirectly use the learning data acquired or generated by the second information processing device 10c-2 without directly acquiring it, and can generate the first inference model accurately and efficiently. Also, the second information processing device 10c-2 can indirectly use the learning data acquired or generated by the first information processing device 10c-1 without directly acquiring it, and can generate the second inference model more accurately and efficiently.

<Advantages of this exemplary embodiment>
As described above, in this exemplary embodiment,
Each of the first information processing device 10c-1 and the second information processing device 10c-2 performs associative learning of an inference model using a balanced data set in which imbalance is eliminated using a data generation model generated by associative learning.

According to the above configuration, even if the number of training data included in the balanced data set in which the imbalance is eliminated using the data generation model is insufficient, it is possible to indirectly use the training data included in the balanced data set generated by another information processing device. As a result, even in a situation in which it is necessary to avoid directly acquiring the balanced data set generated by another information processing device, the increase in the amount of training data that can be indirectly referenced makes it possible to efficiently generate an inference model.

Exemplary embodiment 7
A seventh exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to sixth exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

<Configuration of Information Processing System>
The configuration of an information processing system 1d according to this exemplary embodiment will be described with reference to Fig. 17. Fig. 17 is a block diagram showing the configuration of the information processing system 1d. The information processing system 1d is a modified version of the information processing system 1c according to the sixth exemplary embodiment, which does not include the server 30c. In other words, the information processing system 1d performs federated learning of a data generation model and an inference model without the intervention of a server.

As shown in FIG. 17, the information processing system 1d includes a first information processing device 10d-1 and a second information processing device 10d-2. The first information processing device 10d-1 and the second information processing device 10d-2 are connected via a network N.

(Configuration of the first information processing device)
The first information processing device 10d-1 includes a model information update unit 260-1 in addition to the same configuration as the first information processing device 10c-1 according to the exemplary embodiment 6. In this exemplary embodiment, the configurations of the generation model information acquisition unit 102-1, the data generation model learning unit 103-1, the inference model information acquisition unit 205-1, and the inference model learning unit 203-1 are slightly different from those of the exemplary embodiment 6. The other functional blocks are similar to those of the exemplary embodiment 6, and therefore detailed description will not be repeated.

(Generative model information acquisition unit)
The generation model information acquisition unit 102-1 acquires fourth generation model information from the second information processing device 10d-2. The fourth generation model information is information on the second data generation model after learning by the second information processing device 10d-2.

(Inference model information acquisition unit)
The inference model information acquisition unit 205-1 acquires fourth inference model information from the second information processing device 10d-2. The fourth inference model information is information about the second inference model after learning that the second information processing device 10d-2 has learned.

(Model Information Update Department)
The model information update unit 206-1 generates updated generative model information by referring to the third generative model information and the fourth generative model information. Here, the third generative model information is information on the first data generation model after learning by the data generation model learning unit 103-1.

The model information update unit 206 also generates updated inference model information by referring to the third inference model information and the fourth inference model information. Here, the third inference model information is information about the first inference model after learning that has been trained by the inference model training unit 203-1.

(Data generation model learning part)
The data generation model learning unit 103-1 learns a first data generation model defined by the updated generative model information generated by the model information update unit 206-1, using the first imbalanced data group.

(Inference model learning part)
The inference model learning unit 203-1 learns a first inference model defined by the updated inference model information generated by the model information update unit 206-1, using a first balanced data group.

(Communications Department)
The communication unit 160-1 transmits the third generation model information and the third inference model information to the second information processing device 10d-2.

(Configuration of the second information processing device)
The second information processing device 10d-2 includes a model information update unit 260-2 in addition to the same configuration as the second information processing device 10c-2 according to exemplary embodiment 6. Details of the configuration of the second information processing device 10d-2 can be similarly described in the description of the configuration of the first information processing device 10d-1 by replacing the suffixes "1" and "2" of each reference character with each other, "first" and "second" with each other, and "third" and "fourth" with each other.

<Flow of information processing method for generating a data generation model>
The information processing system 1d configured as above generates a first data generation model and a second data generation model by modifying and sequentially executing the information processing method S1b shown in Fig. 10. Specifically, the information processing method S1b is modified so that step S302, which was executed by the server 30b, is executed by the first information processing device 10d-1 and the second information processing device 10d-2 instead. As an example, the first information processing device 10d-1 executes step S302-1 between steps S102-1 and S103-1. The modified information processing method S1b will be described below.

(Step S101-1)
Step S101-1 for the nth time is the same as in the fifth exemplary embodiment.

(Step S102-1)
Step S102-1 for the nth time is executed when n≧2. In step S102-1 for the nth time, the generative model information acquisition unit 102-1 of the first information processing device 10d-1 acquires fourth generative model information from the second information processing device 10d-2. The fourth generative model information is transmitted from the second information processing device 10d-2 in step S160-2 of the previous time (n−1th time). If n=1, execution of step S102-1 is omitted.

(Step S302-1)
In the n-th step S302-1, the model information update unit 206-1 of the first information processing device 10d-1 generates updated generative model information by referring to the third generative model information and the fourth generative model information. Note that, when n≧2, the third generative model information is information on the first data generation model after learning in the previous (n−1-th) step S103-1. When n=1, the model information update unit 206-1 generates predetermined information as the updated generative model information.

(Step S103-1)
In the n-th step S103-1, the data generation model learning unit 103-1 of the first information processing device 10d-1 learns a first data generation model defined by the updated generation model information using the first imbalanced data group.

(Step S160-1)
In the n-th step S160-1, the communication unit 160-1 of the first information processing device 10d-1 transmits new third generative model information regarding the learned first data generation model to the second information processing device 10d-2.

(Steps S101-2, S102-2, S302-2, S103-2, S160-2)
The second information processing device 10d-2 executes steps S101-2, S102-2, S103-2, and S160-2 for the nth time. Details of these steps will be explained in the same manner as in the explanation of steps S101-1, S102-1, S302-1, S103-1, and S160-1 executed by the first information processing device 10d-1, except that "1" and "2" at the end of each reference symbol are replaced with each other, "first" and "second" are replaced with each other, and "third" and "fourth" are replaced with each other.

<Flow of information processing method for generating inference models>
The information processing system 1d generates a first inference model and a second inference model by modifying and sequentially executing the information processing method S1c shown in Fig. 16. The information processing method S1c is modified so that step S304, which was executed by the server 30c, is executed by the first information processing device 10d-1 and the second information processing device 10d-2 instead. As an example, the first information processing device 10d-1 executes step S304-1 between steps S205-1 and S203-1. The modified information processing method S1c will be described below.

(Steps S201-1 to S202-1)
Steps S201-1 to S202-1 for the nth time are the same as those in the sixth exemplary embodiment.

(Step S205-1)
The nth step S205-1 is executed when n≧2. In the nth step S205-1, the inference model information acquisition unit 205-1 of the first information processing device 10d-1 acquires fourth inference model information from the second information processing device 10d-2. The fourth inference model information is transmitted from the second information processing device 10d-2 in the previous (n−1th) step S170-2. When n=1, execution of step S205-1 is omitted.

(Step S304-1)
In the nth step S304-1, the model information update unit 206-1 of the first information processing device 10d-1 generates updated inference model information by referring to the third inference model information and the fourth inference model information. Note that, when n≧2, the third inference model information is information on the first inference model after learning in the previous (n−1th) step S203-1. When n=1, the model information update unit 206-1 generates predetermined information as the updated inference model information.

(Step S203-1)
In the n-th step S203-1, the inference model learning unit 203-1 of the first information processing device 10d-1 learns a first inference model defined by the updated inference model information using the first balanced data group.

(Step S170-1)
In the nth step S170-1, the communication unit 160-1 of the first information processing device 10d-1 transmits new third inference model information regarding the learned first inference model to the second information processing device 10d-2.

(Steps S201-2, S202-2, S205-2, S304-2, S203-2, S170-2)
The second information processing device 10d-2 executes steps S201-2, S202-2, S205-2, S304-2, S203-2, and S170-2 for the nth time. Details of these steps can be similarly explained by replacing the "1" and "2" at the end of each reference symbol with each other, replacing "first" and "second", replacing "third" and "fourth" with each other in the explanation of steps S201-1, S202-1, S205-1, S304-1, S203-1, and S170-1 executed by the first information processing device 10d-1.

<Advantages of this exemplary embodiment>
As described above, in the first information processing device 10d-1 of this exemplary embodiment,
receiving fourth generation model information relating to the second data generation model after learning from the other information processing device;
generating updated generative model information by referring to third generative model information and fourth generative model information related to the first data generation model after learning in the own device;
Transmitting the third generative model information to another information processing device;
The following configuration has been adopted.

The above configuration achieves the same effects as exemplary embodiment 5 and exemplary embodiment 6 without requiring a server. Note that this exemplary embodiment acquires the fourth generative model information and the fourth inference model information from another information processing device. For this reason, this exemplary embodiment is suitable in a situation where there are few problems with multiple information processing devices sharing information that each other possesses.

Exemplary embodiment 8
An eighth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to seventh exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

<Configuration of Information Processing System>
The configuration of an information processing system 1e according to this exemplary embodiment will be described with reference to Fig. 18. Fig. 18 is a block diagram showing the configuration of the information processing system 1e. The information processing system 1e is an aspect in which the functions of the server 30c in the sixth exemplary embodiment are distributed and realized.

As shown in FIG. 18, the information processing system 1e includes a first information processing device 10e-1, a second information processing device 10e-2, a first server 30e, and a second server 40e. The first information processing device 10e-1, the second information processing device 10e-2, the first server 30e, and the second server 40e are connected via a network N.

(Configuration of the first information processing device)
The first information processing device 10e-1 is configured in a manner similar to that of the first information processing device 10c-1 according to the exemplary embodiment 6. However, in this exemplary embodiment, the configurations of the generative model information acquisition unit 102-1, the inference model information acquisition unit 205-1, and the communication unit 160-1 are slightly different from those of the exemplary embodiment 6. As the other functional blocks are similar to those of the exemplary embodiment 6, detailed description thereof will not be repeated.

(Generative model information acquisition unit)
The generative model information acquisition unit 102-1 is configured in a manner similar to that of the sixth exemplary embodiment, except that the source of the acquired first generative model information is the first server 30e.

(Inference model information acquisition unit)
The inference model information acquisition unit 205-1 is configured almost the same as in exemplary embodiment 6, except that the source of the first inference model information acquired is the second server 40e.

(Communications Department)
The communication unit 160-1 is configured almost the same as in the sixth exemplary embodiment, except that the destination of the third generation model information is the first server 30e. Also, the destination of the third inference model information is the second server 40e.

(Configuration of the second information processing device)
The second information processing device 10e-2 is configured in a manner substantially similar to that of the second information processing device 10c-2 according to the sixth exemplary embodiment. The details of the configuration of the second information processing device 10e-2 can be described in the same manner as in the description of the configuration of the first information processing device 10e-1, except that the suffixes "1" and "2" of each reference character are replaced with each other, "first" and "second" are replaced with each other, and "third" and "fourth" are replaced with each other. However, "first server" and "second server" are not replaced with each other.

(Configuration of the First Server)
The first server 30e functions as a server in federated learning of a data generation model. The first server 30e has a generation model information acquisition unit 301, a generation model information update unit 302, and a communication unit 360, which are included in the function blocks of the server 30c according to the sixth exemplary embodiment. The details of each of these function blocks are the same as those of the sixth exemplary embodiment.

(Configuration of the second server)
The second server 40e functions as a server in the federated learning of the inference model. The second server 40e is a device different from the first server 30e. The second server 40e has an inference model information acquisition unit 303, an inference model information update unit 304, and a communication unit 460, among the functional blocks included in the server 30c according to the sixth exemplary embodiment. The details of each of these functional blocks are the same as those in the sixth exemplary embodiment.

<Flow of information processing method for generating a data generation model>
The information processing system 1e configured as above generates a first data generation model and a second data generation model by sequentially executing the information processing method S1b shown in Fig. 10, similarly to the exemplary embodiment 5. However, in this exemplary embodiment, each step executed by the server 30b in the information processing method S1b is executed by the first server 30e.

<Flow of information processing method for generating inference models>
The information processing system 1e configured as described above generates a first inference model and a second inference model by sequentially executing the information processing method S1c shown in Fig. 16, similarly to the exemplary embodiment 6. However, in this exemplary embodiment, each step executed by the server 30c in the information processing method S1c is executed by the second server 40e.

<Advantages of this exemplary embodiment>
As described above, in this exemplary embodiment,
A server that performs federated learning of a data generation model and a server that performs federated learning of an inference model using a balanced data set that has eliminated imbalances are realized by different devices.
The following configuration has been adopted.

According to the above configuration, it is possible to distribute the load of the servers required to achieve the same effect as in the sixth exemplary embodiment. In addition, a device suitable for the respective loads can be applied as each server. For example, a case can be considered in which a plurality of information processing devices including a first information processing device 10e-1 and a second information processing device 10e-2 participate in the federated learning of the data generation model and the federated learning of the inference model. In such a case, the number of information processing devices participating in the federated learning of the data generation model and the number of information processing devices participating in the federated learning of the inference model may be significantly different. In this case, it is expected that the loads of the first server 30e and the second server 40e will be significantly different. In such a case, a device suitable for the respective loads can be applied as the first server 30e and the second server 40e.

Exemplary embodiment 9
A ninth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to eighth exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

<Configuration of Information Processing System>
The configuration of the information processing system 1f according to this exemplary embodiment will be described with reference to FIG. 19. FIG. 19 is a block diagram showing the configuration of the information processing system 1f. The information processing system 1f is an aspect in which the functions of the server 30b in the information processing system 1b according to the exemplary embodiment 5 are distributed and realized. Here, the exemplary embodiment 5 was an example of a form that takes into consideration a situation in which the clients (the first information processing device 10b-1 and the second information processing device 10b-2) of the federated learning should avoid directly referring to information acquired and generated by other information processing devices. By modifying this, this exemplary embodiment realizes an example of a form that further takes into consideration a situation in which the server should avoid directly referring to information acquired and generated by each client in the federated learning.

As shown in FIG. 19, the information processing system 1f includes a first information processing device 10f-1, a second information processing device 10f-2, a first server 30f-1, a second server 30f-2, and a third server 30f-3. The first information processing device 10f-1, the second information processing device 10f-2, the first server 30f-1, the second server 30f-2, and the third server 30f-3 are connected via a network N.

(Data fragment of generative model information)
In this exemplary embodiment, pieces of data generated from generative model information are transmitted and received between federated learning clients (first information processing device 10f-1 and second information processing device 10f-2) and servers (first server 30f-1, second server 30f-2, and third server 30f-3).

Here, it is described that three data fragments, A, B, and C, are generated from the generative model information, but the number of data fragments is not limited to this. Data fragments A, B, and C are different pieces of information from each other, and are information for generating generative model information. Data fragments A, B, and C cannot restore generative model information by referring to each of them individually or by referring to two of them, but they are information that can generate generative model information by referring to all of them. For example, data fragments A to C may be data fragments that have been randomly divided into three so that the sum of the generative model information does not change.

(Overview of federated learning using data fragments)
Each client generates data pieces A to C from generative model information related to the generative model trained by the client, and distributes and transmits the data pieces to each server. Since each server does not receive all of data pieces A, B, and C from one client, the original generative model information cannot be directly referenced. Each server integrates data pieces A (or data pieces B, C) received from each client to generate integrated data pieces A (or data pieces B, C). Each client can obtain all of integrated data pieces A, B, and C from two or more servers. Each client can restore updated generative model information by referring to integrated data pieces A, B, and C.

(Configuration of the first information processing device)
The first information processing device 10f-1 is configured in a manner similar to that of the first information processing device 10b-1 according to the exemplary embodiment 5. However, in this exemplary embodiment, the configurations of the generative model information acquisition unit 102-1 and the communication unit 160-1 are slightly different from those of the exemplary embodiment 5. As the other functional blocks are similar to those of the exemplary embodiment 5, detailed description thereof will not be repeated.

(Communications Department)
The communication unit 160-1 generates third data pieces A to C from third generation model information regarding the updated first data generation model. The communication unit 160-1 also transmits the third data pieces A and B to the first server 30f-1. The communication unit 160-1 also transmits the third data pieces B and C to the second server 30f-2. The communication unit 160-1 also transmits the third data pieces C and A to the third server 30f-3. The third data pieces A to C are applied in each server as "first data pieces A to C".

(Generative model information acquisition unit)
The generative model information acquisition unit 102-1 acquires the integrated data fragment A, the integrated data fragment B, and the integrated data fragment C by receiving them from at least two of the first server 30f-1, the second server 30f-2, and the third server 30f-3. The generative model information acquisition unit 102-1 also generates updated first generative model information by referring to the integrated data fragment A, the integrated data fragment B, and the integrated data fragment C. Here, the integrated data fragment A is the sum of the first data fragment A and the second data fragment A. The integrated data fragment B is the sum of the first data fragment B and the second data fragment B. The integrated data fragment C is the sum of the first data fragment C and the second data fragment C. The generative model information acquisition unit 102-1 obtains the average of the sum of the first data pieces A to C (i.e., the first generative model information) and the sum of the second data pieces A to C (i.e., the second generative model information) by dividing the sum of the integrated data pieces A, B, and C by the total number of information processing devices 10 (2 in this case). The generative model information acquisition unit 102-1 then determines this average as the updated first generative model information. The updated first generative model information reflects the second data pieces A to C (i.e., the second generative model information).

(Configuration of the second information processing device)
The second information processing device 10f-2 is configured in a manner substantially similar to that of the second information processing device 10b-2 according to the fifth exemplary embodiment. Details of the differences from the fifth exemplary embodiment will be described in the same manner as in the description of the configuration of the first information processing device 10f-1, except that the suffixes "1" and "2" of each reference character are replaced with each other, "first" and "second" are replaced with each other, and "third" and "fourth" are replaced with each other. However, the "first server", "second server", and "third server" are not replaced with each other.

(Configuration of the First Server)
The first server 30f-1 functions as one of a plurality of distributed servers in federated learning of a data generation model. The first server 30f-1 includes a control unit 350-1 and a communication unit 360-1. The control unit 350-1 includes a generation model information acquisition unit 301-1 and a generation model information update unit 302-1.

(Generative model information acquisition unit)
The generative model information acquisition unit 301-1 receives first data pieces A and B from the first information processing device 10f-1. Also, the generative model information acquisition unit 301-1 receives second data pieces A and B from the second information processing device 10f-2.

(Generative model information update unit)
The generative model information updating unit 302-1 integrates the first data fragment A and the second data fragment A to generate a post-integration data fragment A. The generative model information updating unit 302-1 also integrates the first data fragment B and the second data fragment B to generate a post-integration data fragment B. As described above, here, the sum of the first data fragment A (or data fragment B) and the second data fragment A (or data fragment B) is applied as the post-integration data fragment A (or data fragment B).

(Communications Department)
The communication unit 360-1 distributes the integrated data fragment A and the integrated data fragment B in response to a request from one or both of the first information processing device 10f-1 and the second information processing device 10f-2.

(Configuration of the second server)
The second server 30f-2 functions as one of a plurality of distributed servers in the federated learning of the data generation model. The second server 30f-2 includes a control unit 350-2 and a communication unit 360-2. The control unit 350-2 includes a generation model information acquisition unit 301-2 and a generation model information update unit 302-2.

(Generative model information acquisition unit)
The generative model information acquisition unit 301-2 receives the first data pieces B and C from the first information processing device 10f-1. Also, the generative model information acquisition unit 301-2 receives the second data pieces B and C from the second information processing device 10f-2.

(Generative model information update unit)
The generative model information updating unit 302-2 integrates the first data fragment B and the second data fragment B to generate a post-integration data fragment B. The generative model information updating unit 302-2 also integrates the first data fragment C and the second data fragment C to generate a post-integration data fragment C. As described above, here, the sum of the first data fragment B (or data fragment C) and the second data fragment B (or data fragment C) is applied as the post-integration data fragment B (or data fragment C).

(Communications Department)
The communication unit 360-2 distributes the integrated data fragment B and the integrated data fragment C in response to a request from one or both of the first information processing device 10f-1 and the second information processing device 10f-2.

(Configuration of the third server)
The third server 30f-3 functions as one of a plurality of distributed servers in the federated learning of the data generation model. The third server 30f-3 includes a control unit 350-3 and a communication unit 360-3. The control unit 350-3 includes a generation model information acquisition unit 301-3 and a generation model information update unit 302-3.

(Generative model information acquisition unit)
The generative model information acquisition unit 301-3 receives the first data pieces C and A from the first information processing device 10f-1. Also, the generative model information acquisition unit 301-3 receives the second data pieces C and A from the second information processing device 10f-2.

(Generative model information update unit)
The generative model information updating unit 302-3 references the first data fragment C and the second data fragment C to generate a post-integration data fragment C. The generative model information updating unit 302-3 also integrates the first data fragment A and the second data fragment A to generate a post-integration data fragment A. As described above, here, the sum of the first data fragment C (or data fragment A) and the second data fragment C (or data fragment A) is applied as the post-integration data fragment C (or data fragment A).

(Communications Department)
The communication unit 360-3 distributes the integrated data fragment C and the integrated data fragment A in response to a request from one or both of the first information processing device 10f-1 and the second information processing device 10f-2.

<Flow of information processing method for generating a data generation model>
The information processing system 1f configured as above generates a first data generation model and a second data generation model by modifying and sequentially executing the information processing method S1b shown in FIG.

Specifically, the information processing method S1b is modified so that the first server 30f-1, the second server 30f-2, and the third server 30f-3 execute steps S301, S302, and S360, respectively. The operations of the first server 30f-1, the second server 30f-2, and the third server 30f-3 in each of these steps can be similarly explained by replacing "third generation model information" with "first data piece," "fourth generation model information" with "second data piece," and "updated generation model information" with "integrated data piece" in the explanation of steps S301, S302, and S360 in FIG. 10. In addition, in the information processing method S1b, the operation of step S102-1 is modified so that the first generation model information is generated by referring to the integrated data pieces A to C. The operation of step S160-1 is modified so that two pieces of the third data pieces A to C generated from the third generative model information are sent to different servers (the first server 30f-1, the second server 30f-2, and the third server 30f-3). The operation of step S160-2 is modified so that two pieces of the fourth data pieces A to C generated from the fourth generative model information are sent to different servers (the first server 30f-1, the second server 30f-2, and the third server 30f-3).

<Advantages of this exemplary embodiment>
As described above, in this exemplary embodiment,
A server that performs federated learning of data generation models is distributed across multiple different devices.
The following configuration has been adopted.

Here, for example, due to consideration of privacy, there are cases where it is difficult for the server itself in the federated learning to obtain information held by each information processing device. In this case, it may not be preferable for the server to directly obtain the third generation model information and the fourth generation model information. According to the above configuration, the first server 30f-1, the second server 30f-2, and the third server 30f-3 do not directly obtain information on the first generation model after learning and information on the second generation model after learning. On the other hand, the first information processing device 10f-1 and the second information processing device 10f-2 can generate updated generation model information reflecting information on the first generation model after learning and information on the second generation model after learning by referring to information received from each server. Therefore, the same effect as in the exemplary embodiment 5 can be achieved.

In this exemplary embodiment, an example has been described in which the functions of server 30b in exemplary embodiment 5 are distributed across three servers. However, the present invention is not limited to this, and the functions of server 30b may be distributed across four or more servers. In this case, the first information processing device 10f-1 and the second information processing device 10f-2 each refer to the generation model information on the data generation model after learning, generate four or more pieces of data A, B, C, ..., and distribute and transmit a portion of each of the data to each server.

The method of dividing and restoring generative model information performed by the information processing system 1f according to this exemplary embodiment is also referred to as multi-party computation, for example, but this term does not limit this exemplary embodiment.

Exemplary embodiment 10
A tenth exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is an embodiment in which the learning data acquisition units 101-1 and 101-2 are modified in the first information processing device (10b-1, 10c-1, 10d-1, 10e-1, 10f-1) and the second information processing device (10b-2, 10c-2, 10d-2, 10e-2, 10f-2) according to any of the fifth to ninth exemplary embodiments. In the following description, the "-1" and "-2" at the end of the reference numerals will be omitted as appropriate when there is no need to distinguish between the first information processing device and the second information processing device. The learning data acquisition unit 101 in this exemplary embodiment will be described with reference to FIG. 20. FIG. 20 is a diagram showing the configuration of the learning data acquisition unit 101. Since the other configurations and the flow of the information processing method are as described in each exemplary embodiment, detailed description will be omitted.

(Learning Data Selection Unit)
20, the learning data acquisition unit 101 includes a learning data selection unit 111. The learning data selection unit 111 is an example of a selection means according to this exemplary embodiment. The learning data selection unit 111 selects an imbalanced data group from a plurality of data group candidates.

For example, the learning data selection unit 111 may select data sets A, B, C, and so on stored in the storage unit SR that satisfy the condition of being an imbalanced data group. For example, such a condition may be the condition of the minority label type described above. The storage unit SR may be included in the information processing device 10b, 10c, 10d, 10e, or 10f, or may be included in an external device.

<Advantages of this exemplary embodiment>
As described above, in this exemplary embodiment,
Select an imbalanced data set to be used for training the data generation model from multiple data set candidates;
The following configuration has been adopted.

According to the above configuration, each information processing device can select its own data set so as to eliminate the imbalance overall, allowing the data generation model to be trained more stably.

[Modifications]
Each of the above exemplary embodiments has been described as including one or two information processing devices. However, each exemplary embodiment can be modified to include three or more information processing devices. This allows each information processing device to generate a data generation model by indirectly using learning data used in more other information processing devices without directly acquiring the learning data.

In the above-described exemplary embodiments 6 to 9, the functional blocks included in each of the two information processing devices (the first information processing device, the second information processing device) may be distributed among a plurality of devices. For example, the functional blocks included in the first information processing device (or the second information processing device) may be distributed among (i) a device including a functional block related to learning the data generation model (a learning data acquisition unit, a generation model information acquisition unit, and a data generation model learning unit), (ii) a device including a functional block related to data generation (a seed data acquisition unit, and a data generation unit), (iii) a device including a functional block related to learning the inference model (an inference model information acquisition unit, and an inference model learning unit), and (iv) a device including a functional block related to inference (an inference unit).

[Software implementation example]
Some or all of the functions of the information processing devices 10, 10a, 10b-1, 10b-2, 10c-1, 10c-2, 10d-1, 10d-2, 10e-1, 10e-2, 10f-1, 10f-2, 20, and the servers 30, 30a, 30b, 30c, 30d, 30e, 30f, 40e, and 50f (hereinafter referred to as the information processing devices 10 to 20 and the servers 30 to 50) may be realized by hardware such as an integrated circuit (IC chip), or by software.

In the latter case, the information processing devices 10-20 and the servers 30-50 described above are realized, for example, by a computer that executes program instructions, which are software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 21. Computer C has at least one processor C1 and at least one memory C2. Memory C2 stores program P for operating computer C as each of information processing devices 10-20 and each of servers 30-50. In computer C, processor C1 reads and executes program P from memory C2 to realize each function of information processing devices 10-20 and each of servers 30-50.

The processor C1 may be, for example, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a combination of these.

The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input/output interface for connecting input/output devices such as a keyboard, mouse, display, and printer.

The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communications network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[Additional Note 1]
The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.

[Additional Note 2]
Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the following described aspects.

(Appendix 1)
A learning data acquisition means for acquiring learning data;
a generation model information acquisition means for acquiring first generation model information relating to a first data generation model, the first generation model information reflecting second generation model information relating to a second data generation model different from the first data generation model;
a data generation model learning means for learning the first data generation model, the first data generation model being defined by referring to the first generation model information, using the learning data;
An information processing device comprising:

According to the above configuration, the first data generation model is trained by indirectly using other learning data used to generate the second data generation model in another information processing device without directly acquiring the other learning data. As a result, even in a situation where acquiring other learning data used in another information processing device should be avoided, a first data generation model capable of generating more diverse data can be generated.

(Appendix 2)
2. The information processing device according to claim 1, further comprising a transmitting means for transmitting third generation model information regarding the first data generation model after it has been learned by the data generation model learning means.

According to the above configuration, the third data generation model information regarding the first data generation model after learning can be provided to another device. As a result, it becomes possible for the other device to indirectly use the learning data used to learn the first data generation model without directly acquiring it, thereby performing learning.

(Appendix 3)
The generative model information acquisition means
Obtaining fourth generative model information regarding the second data generation model after training;
The information processing device includes:
third generation model information regarding the first data generation model after being trained by the data generation model training means;
2. The information processing device according to claim 1, further comprising a generative model information update means for generating updated generative model information by referring to the fourth generative model information.

According to the above configuration, by referring to the fourth generation model information, it is possible to indirectly use other learning data used to generate the second data generation model in another information processing device. As a result, even in a situation where it is necessary to avoid obtaining other learning data used in another information processing device, it is possible to generate a first data generation model capable of generating more diverse data.

(Appendix 4)
The learning data acquisition means includes:
The information processing device according to any one of appendices 1 to 3, characterized in that a data group including a plurality of data fragments, each of which is labeled with one of a plurality of label types, and a proportion of data fragments having a certain label type is equal to or less than a predetermined proportion, is acquired as the learning data.

According to the above configuration, an imbalanced data group in which the number of data pieces with label types is imbalanced is used as training data to train the first data generation model. As a result, the first data generation model can be used to generate data for eliminating the imbalance in such a data group.

(Appendix 5)
The learning data acquisition means includes:
5. The information processing device according to claim 4, further comprising a selection means for selecting the data group from a plurality of data group candidates.

According to the above configuration, by selecting an imbalanced data group, the effect described in Supplementary Note 4 can be more reliably achieved.

(Appendix 6)
The data generation model learning means includes:
an autoencoder including the first data generation model; and
6. The information processing device according to any one of claims 1 to 5, further comprising: training at least one generative adversarial network including the first data generation model.

The above configuration makes it possible to realize a first data generation model that achieves the above-mentioned effects.

(Appendix 7)
The autoencoder may include
7. The information processing device according to claim 6, comprising at least one of a variational autoencoder and a conditional variational autoencoder.

The above configuration makes it possible to realize a first data generation model that achieves the above-mentioned effects.

(Appendix 8)
a data generation means for generating data using the first data generation model;
An information processing device described in any one of appendices 1 to 7, further comprising an inference model learning means for learning a first inference model by referring to the data generated by the data generation means.

According to the above configuration, the first data generation model that achieves the above-mentioned effects references the data generated, so that the first inference model can be trained with high accuracy.

(Appendix 9)
The system further includes an inference model information acquisition means for acquiring first inference model information regarding the first inference model, the first inference model information reflecting second inference model information regarding a second inference model different from the first inference model,
The information processing device described in Appendix 8, characterized in that the inference model learning means learns the first inference model defined by reference to the first inference model information using data generated by the data generation means.

According to the above configuration, the first inference model is trained by indirectly using other learning data used to generate the second inference model in another information processing device without directly acquiring the other learning data. As a result, even in a situation where acquiring other learning data used in another information processing device should be avoided, the first inference model can be trained with high accuracy.

(Appendix 10)
A seed data acquisition means for acquiring seed data;
a data generating means for generating data using the seed data,
The data generating means
An information processing device characterized in that data generation is performed using a first data generation model defined by reference to first generation model information relating to a first data generation model, the first generation model information reflecting second generation model information relating to a second data generation model different from the first data generation model.

According to the above configuration, by using the first data generation model that is trained to reflect the second generative model information regarding the second generative model, it is possible to generate more diverse data.

(Appendix 11)
An information processing device as described in Appendix 10, further comprising an inference model learning means for learning a first inference model by referring to the data generated by the data generation means.

According to the above configuration, the first data generation model that achieves the above-mentioned effects references the data generated, so that the first inference model can be trained with high accuracy.

(Appendix 12)
The system further includes an inference model information acquisition means for acquiring first inference model information regarding the first inference model, the first inference model information reflecting second inference model information regarding a second inference model different from the first inference model,
The information processing device described in Appendix 11, characterized in that the inference model learning means learns the first inference model defined by reference to the first inference model information using data generated by the data generation means.

According to the above configuration, the first inference model is trained by indirectly using other learning data used to generate the second inference model in another information processing device without directly acquiring the other learning data. As a result, even in a situation where acquiring other learning data used in another information processing device should be avoided, the first inference model can be trained with high accuracy.

(Appendix 13)
a generation model information acquisition means for acquiring first generation model information relating to a first data generation model and second generation model information relating to a second data generation model different from the first data generation model;
a generative model information update means for generating updated generative model information by referring to the first generative model information and the second generative model information;
An information processing device comprising:

According to the above configuration, it is possible to generate a trained data generation model indirectly using (i) the training data used to train the first data generation model and (ii) the training data used to train the second data generation model. As a result, even in a situation where it is necessary to avoid obtaining at least one of the training data (i) and (ii), it is possible to train the data generation model so that it can generate a wider variety of data.

(Appendix 14)
The information processing device described in Appendix 13, further comprising a transmission means for transmitting the updated generative model information to a first information processing device implementing the first data generation model and a second information processing device implementing the second data generation model.

According to the above configuration, even in a situation where the first information processing device and the second information processing device should avoid providing each other with the learning data used by each other, a data generation model trained by indirectly using the learning data used by both devices can be provided to the first information processing device and the second information processing device.

(Appendix 15)
The generation model information update means
The information processing device according to claim 13 or 14, characterized in that the updated generative model information is generated by taking an average of the first generative model information and the second generative model information.

The above configuration realizes learning that indirectly uses the learning data used to train the first data generation model and the learning data used to train the second data generation model.

(Appendix 16)
An information processing system including a first information processing device and a server,
The first information processing device,
A learning data acquisition means for acquiring learning data;
a generation model information acquisition means for acquiring, from the server, first generation model information relating to a first data generation model, the first generation model information reflecting second generation model information relating to a second data generation model different from the first data generation model;
a data generation model learning means for learning the first data generation model defined by referring to the first generation model information, using the learning data acquired by the learning data acquisition means;
a transmission means for transmitting third generation model information regarding the first data generation model after it has been trained by the data generation model training means;
The server,
The third generative model information; and
a generation model information acquisition means for acquiring fourth generation model information regarding the second data generation model after learning;
a generation model information update means for generating updated generation model information by referring to the third generation model information and the fourth generation model information;
An information processing system comprising a transmission means for transmitting the updated generative model information to the first information processing device.

According to the above configuration, a system that achieves the same effect as that of Supplementary Note 1 can be realized using an information processing device and a server.

(Appendix 17)
Obtaining training data;
acquiring first generative model information relating to a first data generation model, the first generative model information reflecting second generative model information relating to a second data generation model different from the first data generation model; and
An information processing device method comprising: training the first data generation model, defined by reference to the first generation model information, using the training data.

The above configuration provides the same effect as that described in Supplementary Note 1.

(Appendix 18)
obtaining seed data; and generating data using the seed data;
Including,
In the step of generating data,
An information processing method characterized in that data generation is performed using a first data generation model defined by reference to first generation model information relating to a first data generation model, the first generation model information reflecting second generation model information relating to a second data generation model different from the first data generation model.

The above configuration achieves the same effect as in Appendix 10.

(Appendix 19)
obtaining first generative model information relating to a first data generation model and second generative model information relating to a second data generation model different from the first data generation model; and
generating updated generative model information by referring to the first generative model information and the second generative model information;
An information processing method comprising:

The above configuration provides the same effect as that described in Supplementary Note 13.

(Appendix 20)
A program for causing a computer to function as an information processing device, the program comprising:
A learning data acquisition means for acquiring learning data;
a generation model information acquisition means for acquiring first generation model information relating to a first data generation model, the first generation model information reflecting second generation model information relating to a second data generation model different from the first data generation model;
a data generation model learning means for learning the first data generation model, the first data generation model being defined by referring to the first generation model information, using the learning data;
A program that functions as a

The above configuration provides the same effect as that described in Supplementary Note 1.

(Appendix 21)
A program for causing a computer to function as an information processing device, the program comprising:
A seed data acquisition means for acquiring seed data;
A program that functions as a data generation means that generates data using the seed data, the data generation means generating data using a first data generation model that is defined by referring to first generation model information that relates to a first data generation model and reflects second generation model information relating to a second data generation model that is different from the first data generation model.

The above configuration achieves the same effect as in Appendix 10.

(Appendix 22)
A program for causing a computer to function as an information processing device, the program comprising:
a generation model information acquisition means for acquiring first generation model information relating to a first data generation model and second generation model information relating to a second data generation model different from the first data generation model;
a generative model information update means for generating updated generative model information by referring to the first generative model information and the second generative model information;
A program that functions as a

The above configuration provides the same effect as that described in Supplementary Note 13.

[Additional Note 3]
A part or all of the above-described embodiments can be further expressed as follows.

(Appendix 23)
At least one processor, the processor comprising:
A learning data acquisition process for acquiring learning data;
a generation model information acquisition process for acquiring first generation model information related to a first data generation model, the first generation model information reflecting second generation model information related to a second data generation model different from the first data generation model;
a data generation model learning process for learning the first data generation model, which is defined by referring to the first generation model information, using the learning data.

The information processing device may further include a memory, and the memory may store a program for causing the processor to execute the learning data acquisition process, the generative model information acquisition process, and the data generation model learning process. The program may also be recorded on a computer-readable, non-transitory, tangible recording medium.

(Appendix 24)
At least one processor, the processor comprising:
A seed data acquisition process for acquiring seed data;
An information processing device that executes a data generation process that generates data using the seed data, the data generation process generating data using a first data generation model that is defined by referring to first generation model information that reflects first generation model information regarding a first data generation model, the first generation model information reflecting second generation model information regarding a second data generation model that is different from the first data generation model.

The information processing device may further include a memory, and the memory may store a program for causing the processor to execute the seed data acquisition process and the data generation process. The program may also be recorded on a computer-readable, non-transitory, tangible recording medium.

(Appendix 25)
At least one processor, the processor comprising:
a generation model information acquisition process for acquiring first generation model information on a first data generation model and second generation model information on a second data generation model different from the first data generation model;
and an information processing device that executes a generative model information update process that generates updated generative model information by referring to the first generative model information and the second generative model information.

The information processing device may further include a memory, and the memory may store a program for causing the processor to execute the generative model information acquisition process and the generative model information update process. The program may also be recorded on a computer-readable, non-transitory, tangible recording medium.

1a to 1f Information processing systems 10, 10a to 10f, 20 Information processing devices 30, 30a to 30f, 40e, 50f Information processing devices (servers)
101 Learning data acquisition unit 102, 301, 501 Generative model information acquisition unit 103, Data generation model learning unit 160, 360, 460, 560 Communication unit 110, 310 Transmission unit 111 Learning data selection unit 150, 350 Control unit 201 Seed data acquisition unit 202 Data generation unit 203 Inference model learning unit 204 Inference unit 205, 303 Inference model information acquisition unit 206 Model information update unit 302, 502 Generative model information update unit 304 Inference model information update unit C Computer C1 Processor C2 Memory

Claims (9)

A learning data acquisition means for acquiring learning data;
a generation model information acquisition means for acquiring first generation model information relating to a first data generation model, the first generation model information reflecting second generation model information relating to a second data generation model different from the first data generation model;
a data generation model learning means for learning the first data generation model, the first data generation model being defined by referring to the first generation model information, using the learning data;
Equipped with
The learning data acquisition means includes:
A data group including a plurality of data fragments, each of which is labeled with one of a plurality of label types, and the proportion of data fragments to which a certain label type is attached is equal to or less than a predetermined proportion, is acquired as the learning data.
23. An information processing apparatus comprising : The information processing apparatus according to claim 1 , further comprising a transmitting unit configured to transmit third generation model information regarding the first data generation model after being trained by the data generation model training unit. The generative model information acquisition means
Obtaining fourth generative model information regarding the second data generation model after training;
The information processing device includes:
third generation model information regarding the first data generation model after being trained by the data generation model training means;
The information processing apparatus according to claim 1 , further comprising: a generative model information update means for generating updated generative model information by referring to the fourth generative model information. The learning data acquisition means includes:
4. The information processing apparatus according to claim 1, further comprising a selection unit for selecting the data group from a plurality of data group candidates. The data generation model learning means includes:
an autoencoder including the first data generation model; and
The information processing apparatus according to claim 1 , further comprising: training at least one of a generative adversarial network including the first data generation model. The autoencoder may include
6. The information processing apparatus according to claim 5 , further comprising at least one of a variational autoencoder and a conditional variational autoencoder. a data generation means for generating data using the first data generation model;
7. An information processing apparatus according to claim 1, further comprising an inference model learning means for learning a first inference model by referring to the data generated by the data generating means. one or more processors,
Obtaining training data;
acquiring first generative model information relating to a first data generation model, the first generative model information reflecting second generative model information relating to a second data generation model different from the first data generation model; and
The method includes training the first data generation model, which is defined by referring to the first generation model information, using the training data;
In acquiring the learning data,
A data group including a plurality of data fragments, each of which is labeled with one of a plurality of label types, and the proportion of data fragments to which a certain label type is attached is equal to or less than a predetermined proportion, is acquired as the learning data.
23. An information processing method comprising: A program for causing a computer to function as an information processing device, the program comprising:
A learning data acquisition means for acquiring learning data;
a generation model information acquisition means for acquiring first generation model information relating to a first data generation model, the first generation model information reflecting second generation model information relating to a second data generation model different from the first data generation model;
a data generation model learning means for learning the first data generation model, the first data generation model being defined by referring to the first generation model information, using the learning data;
Function as a
The learning data acquisition means includes:
A data group including a plurality of data fragments, each of which is labeled with one of a plurality of label types, and the proportion of data fragments to which a certain label type is attached is equal to or less than a predetermined proportion, is acquired as the learning data.
A program characterized by :

Images