JP7561005B2 - Model learning method, model learning system, and computer program - Google Patents

Description

The present invention relates to a model learning method, a model learning system, and a computer program for the practical application of distributed learning.

Judgment, recognition, etc. using learning models based on deep learning have been put to practical use. It has been confirmed that learning models can be used in a variety of technical fields. Even if a huge amount of training data is used to improve the accuracy of the learning models to a level that can be used in various fields, it takes a long time for the parameters to converge, and there is no guarantee that the accuracy will improve.

Patent Document 1 discloses a method in which multiple replicas of a learning model to be learned are prepared, and these multiple model replicas learn independently and asynchronously. In Patent Document 1, a parameter server is fragmented into multiple parts, and each of the multiple replicas of the learning model asynchronously acquires parameters from the fragmented parameter servers, learns from them, and returns the parameters to each parameter server, repeating this process. This type of distributed processing is said to allow the parameters of the learning model to converge quickly.

U.S. Pat. No. 8,768,870

Even in the deep learning method using distributed processing as disclosed in Patent Document 1, training data is aggregated. However, data in fields such as medicine, finance, and authentication is personal data and is highly confidential. In order to aggregate data as training data to improve the accuracy of the model, each individual's consent is required to provide the data, and even if consent is obtained, there is always a risk to the security of data management.

The present invention has been made in consideration of the above circumstances, and aims to provide a model learning method, a model learning system, and a computer program that promote the practical application of learning models based on distributed learning.

A model learning method according to an embodiment of the present disclosure includes the steps of distributing global model data stored in a server to a plurality of nodes, acquiring local model data learned based on the global model by local data processed by the plurality of nodes, updating the global model data based on the learned local model data acquired from each of the plurality of nodes, storing the updated global model data in association with update information corresponding to the number of updates, calculating a performance evaluation value of the global model, storing the calculated performance evaluation value in association with the update information of the global model, accepting a reference request for the global model together with a specification of the update information of the global model, and outputting the global model data and performance evaluation value associated with the specified update information.

The model learning system of one embodiment of the present disclosure includes a plurality of nodes each processing local data, and a server connected to the plurality of nodes for communication, the server distributes global model data to the plurality of nodes, and the plurality of nodes each proceeds with learning of the local model from the global model using the local data to be processed, and a processing unit provided in the server acquires the learned local model data from the plurality of nodes, updates the global model data based on the learned local model data acquired from each of the plurality of nodes, stores the updated global model data in association with update information corresponding to the number of updates, calculates a performance evaluation value of the global model, stores the calculated performance evaluation value in association with the update information of the global model, accepts a reference request for the global model together with a designation of the update information of the global model, and outputs the global model data and performance evaluation value associated with the designated update information.

A computer program according to an embodiment of the present disclosure causes a computer capable of communication with a plurality of nodes to execute a process of distributing stored global model data to the plurality of nodes, acquiring local model data learned by each of the plurality of nodes, updating the global model data based on the learned local model data acquired from each of the plurality of nodes, storing the updated global model data in association with update information corresponding to the number of updates, calculating a performance evaluation value of the global model, storing the calculated performance evaluation value in association with the update information of the global model, accepting a reference request for the global model together with a specification of the update information of the global model, and outputting the global model data and performance evaluation value associated with the specified update information.

A computer program according to an embodiment of the present disclosure causes a computer connected to a server to execute a process of learning a local model using local data to be processed based on a global model distributed from the server, transmitting data of the learned local model to the server, receiving screen data from the server including a list of updated global models associated with different update information, displaying the list on a display unit provided in the computer based on the received screen data, accepting a designation of any one of the update information in the list, and acquiring and displaying a performance evaluation value of the global model corresponding to the designated update information from the server.

In the model learning method, model learning system, and computer program disclosed herein, when a global model is updated, the model data and the performance evaluation value at each stage are stored as history in association with update information corresponding to the number of updates. The global model data, together with the performance evaluation value, are acquired from a node or another information processing device, making it possible to display a list.

In one embodiment of the model learning method of the present disclosure, the server repeats multiple rounds of distributing global model data, acquiring local model data, and updating the global model data, and each time the global model data is updated, stores the global model data and performance evaluation value in association with the update information.

During the update process, which involves multiple rounds of global model data distribution, global model data distribution, local model data acquisition, and global model data updates, the data and performance evaluation values of the global model are stored in association with different update information. This makes it possible to trace back to past models in the update history and visualize a comparison of performance with past models.

In one embodiment of the model learning method of the present disclosure, the server creates and outputs screen data in response to the reference request, the screen data showing the performance of the global model based on the performance evaluation value.

The performance evaluation of the global model is visualized as screen data. The operator can check the update process and decide to select a model, change the update method, stop the update, etc.

In one embodiment of the model learning method disclosed herein, the server receives a designation of the number of times to update the data of the global model and an instruction to stop updating the data of the global model.

The performance evaluation value of the model being updated can be output, and data on the global model at each stage is stored and can be obtained from the node or other information processing devices. The number of updates can be specified with reference to the performance evaluation value, and an instruction to stop updating can be accepted. Distributed learning can be carried out appropriately according to changes in the performance evaluation value.

In one embodiment of the model learning method of the present disclosure, the server acquires an evaluation value of the accuracy of the global model calculated based on local data from each of the multiple nodes, and calculates a performance evaluation value of the global model based on the evaluation value.

The performance evaluation value may be calculated based on the accuracy using local data at each node. The performance evaluation value of the global model may be the average of the accuracy calculated at each node, or may be a weighted average based on the amount of local data. Distribution of global model data, training of local models, and updating of global model data based on the trained local models may be repeated until the accuracy at each node is sufficient for practical use.

A model learning method according to one embodiment of the present disclosure accepts the selection of a global model to be redistributed, and redistributes the data of the selected global model to the multiple nodes regardless of the progress of learning.

It is possible to select one of the models from the update history. The operator can compare the performance evaluation values and then select the model to redistribute, i.e., proceed with learning.

In one embodiment of the model learning method of the present disclosure, the multiple nodes store data of the local model in association with update information, calculate an evaluation value of the local model, and store the calculated evaluation value in association with the update information of the local model.

The performance history of the local models trained on each node is also aggregated on the server. By making the performance and history of the local models aggregated from each node available for viewing on the server, operators can select the nodes to participate in distributed learning.

The model learning method of one embodiment of the present disclosure causes the multiple nodes to store the amount of local data used to learn the local model or the characteristics of the local data in association with update information for the local model, and the server acquires the amount of data or the characteristics of the data from the multiple nodes in association with the data of the local model.

The amount of data, amount of learning, or characteristics of the data used for learning of the local models used to update the global model are aggregated on the server. It becomes possible to select which local model to use to update the data of the global model, depending on the characteristics of the local model.

According to the model learning method, model learning system, and computer program disclosed herein, the performance evaluation value of the model is visualized and can be confirmed by an operator. It is also possible to compare the performance with a model in the process of being updated, select a model that is predicted to have better performance, and proceed with learning.

FIG. 1 is a schematic diagram of a model learning system according to a first embodiment. FIG. 2 is a block diagram showing a configuration of a node. FIG. 2 is a block diagram showing a configuration of a server. FIG. 2 is a block diagram showing the configuration of a monitor. 13 is a flowchart showing an example of a learning process procedure in the model learning system. 13 is a flowchart illustrating an example of a learning process procedure for a local model based on a distributed global model. 13 is a flowchart illustrating an example of a global model update process in the server. 13 is a flowchart showing an example of a reference processing procedure in a learning process. 13 shows an example of an update history screen for a global model. An example of an individual performance screen that visualizes the individual performance of a global model is shown. 13 is a flowchart illustrating an example of a learning process procedure in the model learning system of the second embodiment. 13 is a flowchart illustrating an example of a learning process procedure in the model learning system of the second embodiment. 13 shows another example of an individual performance screen in which the individual performance of a global model is visualized. 13 shows an example of a detailed screen of an evaluation value. 13 shows an example of a learning condition editing screen. FIG. 13 is a schematic diagram of model learning according to the second embodiment; 13 is a flowchart illustrating an example of a learning process procedure for a local model of a node according to the third embodiment. 13 is a flowchart illustrating an example of a global model update process according to the third embodiment. The performance history screen of the local model is shown. 13 is a flowchart illustrating an example of a learning process procedure of a local model according to the fourth embodiment. 13 is a flowchart illustrating an example of a global model update process according to the fourth embodiment; The screen for editing the weight of each node is shown. 13 shows an example of an individual performance screen in which individual performance of a global model is visualized in the fourth embodiment. FIG. 13 is a schematic diagram of a model learning system according to a fifth embodiment. 13 is a flowchart illustrating an example of a learning process procedure in the model learning system of the fifth embodiment. 13 is a flowchart illustrating an example of a learning process procedure of a local model according to the fifth embodiment. 23 is a flowchart illustrating an example of a global model update process according to the fifth embodiment. 13 shows a weight editing screen in the fifth embodiment.

This disclosure will be specifically described with reference to drawings showing embodiments thereof.

First Embodiment
1 is a schematic diagram of a model learning system 100 according to a first embodiment. The model learning system 100 includes one or more nodes 1 provided for a storage device 2 that stores data, a server 3, and a communication network N that connects the nodes 1 and the server 3 for communication.

The storage device 2 is capable of inputting and outputting data between devices that input and output data to be learned, such as sensors that measure physical quantities and cameras that take images, and stores this data. The storage device 2 may be connected to a computer for a specific purpose that outputs data according to data input by operation. The storage device 2 may be a storage device of an information terminal used by a user. The storage device 2 may be a storage device used in a server device that collects data from client devices.

When node 1 is input with data of the same type as the data stored in storage device 2, node 1 executes deep learning of the model so as to output a recognition result, a judgment result, or new data based on the data. Server 3 is a computer that provides a model to node 1, and also realizes model learning system 100 that cooperates with node 1 to learn the model. In order to execute model learning using the data stored in storage device 2 (hereinafter referred to as local data) as training data, it is necessary to be able to access these. In the model learning system 100 of this embodiment, learning can proceed with a state in which access to the local data from server 3 is disabled.

The server 3 initially obtains a zeroth-order global model 51. The server 3 distributes the zeroth-order global model 51 to the node 1 via the communication network N. The entity (data) of the global model 51 distributed from the server 3 to the node 1 is only the trained parameters, or both the trained parameters and the program. The global model 51 may be definition data (including network definitions, losses, and pre-set hyperparameters) that defines the model configuration, and parameters such as weight coefficients to be trained.

The model to be trained can have any architecture as long as it is the subject of what is called deep learning. The type of deep learning model should be appropriately selected according to the contents of the input data and output data. The model to be trained described below may be a classification system, detection system, or generation system using a CNN (Convolutional Neural Network) that includes a convolutional layer, or it may be a RNN (Recurrent Neural Network) that learns by taking into account time series elements.

The communication network N includes a public communication network, such as the Internet, and a carrier network. The communication network N may be a dedicated line for the model learning system 100.

Node 1 can access local data stored in the storage device 2 based on the local network LN between node 1 and storage device 2. Node 1 performs deep learning using the accessible local data. It is preferable that the local data has already been annotated by an operator at the location where node 1 is installed. Node 1 obtains the 0th global model 51 distributed from server 3. Node 1 proceeds with learning based on the 0th global model 51 and the local data as training data, and obtains the 1st local model 52.

Node 1 transmits the first local model 52 to server 3. Because local data is not transmitted to server 3, processing such as abstraction and anonymization of local data is not required.

The server 3 receives multiple first local models 52 from each of the multiple nodes 1, and performs statistical processing on the multiple received first local models 52 to create a first global model 51. The server 3 redistributes the first global model 51 to the multiple nodes 1. The redistributed global model 51 may be only the weight coefficient. The redistributed weight coefficient may be the learning target or may be the entire model. The redistributed global model 51 may correspond to the difference from the previous update.

The model learning system 100 repeats the following steps: distributing an nth-order global model 51 from the server 3 to node 1, learning the nth-order global model 51 using local data at node 1, transmitting the (n+1)th-order local model 52 obtained by learning to the server 3, collecting the (n+1)th-order local model 52 at the server 3, and creating (updating) the (n+1)th-order global model 51.

This enables distributed learning without allowing access to local data from the server 3. In the model learning system 100, the learning process in which the distribution of the global model 51 and the learning of the local model 52 are repeated can be confirmed from the monitor 4. It is preferable that the process can be confirmed not only by the monitor 4 but also by the operator of the node 1. In the model learning system 100, the operator can interrupt learning and select an appropriate global model 51 from the monitor 4.

The server 3 may accept communication connections to the monitor 4 only from within the same local network (on-premise type), or may accept communication connections from the monitor 4 via the communication network N, which is a public communication network, as shown in Figure 1 (cloud type).

The global model 51, which has been trained through distributed learning to an accuracy that allows it to be distributed to devices other than node 1, is distributed and used not only to each node 1 but also to information processing devices 6 outside the system that do not participate in the creation of the local model 52.

The configuration of the model learning system 100 that realizes this learning method will be described in detail below.

Figure 2 is a block diagram showing the configuration of node 1. Node 1 is a personal computer or a server computer. Node 1 includes a processing unit 10, a memory unit 11, a communication unit 12, a display unit 13, and an operation unit 14.

The processing unit 10 is a processor that uses a CPU (Central Processing Unit) and/or a GPU (Graphics Processing Unit). Based on the node program 1P stored in the memory unit 11, the processing unit 10 executes processing including reading data from the storage device 2, sending and receiving models to and from the server 3, and model learning.

The storage unit 11 uses a non-volatile memory such as a hard disk, a flash memory, or an SSD (Solid State Drive). The storage unit 11 stores data referenced by the processing unit 10. The storage unit 11 stores a node program 1P. The storage unit 11 stores a library 1L for deep learning. The node program 1P and/or the library 1L for deep learning may be a node program 8P and/or a library 8L for deep learning stored in a recording medium 8 that is read by the processing unit 10 and copied to the storage unit 11. The storage unit 11 stores a global model 51 acquired from the server 3 and a local model 52 that is learned using local data.

The communication unit 12 realizes data communication via the communication network N and communication with the storage device 2 via the local network LN. Specifically, the communication unit 12 is, for example, a network card. The processing unit 10 reads data from the storage device 2 via the communication unit 12 and transmits and receives data between the server 3.

The display unit 13 is a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 displays a screen including information based on data stored in the memory unit 11 or data provided from the server 3. The display unit 13 may be a display with a built-in touch panel.

The operation unit 14 is a user interface such as a keyboard and a pointing device that can input and output data to and from the processing unit 10. The operation unit 14 may be a voice input unit. The operation unit 14 may be a touch panel of the display unit 13. The operation unit 14 may be physical buttons. The operation unit 14 notifies the processing unit 10 of operation data by the operator of the node 1.

FIG. 3 is a block diagram showing the configuration of server 3. Server 3 is a server computer. Server 3 includes a processing unit 30, a storage unit 31, and a communication unit 32. In the following description, server 3 is described as being configured as one server computer, but it may also be configured in a manner in which multiple server computers are communicatively connected via a network for distributed processing. Server 3 may be a cloud type that can be communicatively connected from each node 1 via a communication network N, or it may be an on-premise type that is communicatively connected to each node 1 via a virtual private network.

The processing unit 30 is a processor using a CPU and/or a GPU. The processing unit 30 executes the learning process of the global model 51 based on the server program 3P stored in the memory unit 31.

The storage unit 31 uses a non-volatile memory such as a hard disk or SSD. The storage unit 31 stores data referenced by the processing unit 30. The storage unit 31 stores a server program 3P. The server program 3P includes a web server program. The storage unit 31 stores a global model 51 and a local model 52 transmitted from multiple nodes 1. The server program 3P may be a server program 9P stored in a recording medium 9 that is read by the processing unit 30 and copied to the storage unit 31.

The communication unit 32 realizes data communication via the communication network N. Specifically, the communication unit 32 is, for example, a network card. The processing unit 30 transmits and receives data between multiple nodes 1 via the communication unit 32.

Figure 4 is a block diagram showing the configuration of the monitor 4. The monitor 4 is a personal computer. The monitor 4 may be a tablet terminal or a smartphone. The monitor 4 includes a processing unit 40, a storage unit 41, a communication unit 42, a display unit 43, and an operation unit 44.

The processing unit 40 is a processor using a CPU and/or a GPU. The processing unit 40 executes a process of displaying information provided by the server 3 on the display unit 43 based on a web browser program stored in the storage unit 41.

The storage unit 41 uses a non-volatile memory such as a hard disk, a flash memory, or an SSD (Solid State Drive). The storage unit 41 stores data referenced by the processing unit 40. The storage unit 41 stores various programs including a web browser program.

The communication unit 42 realizes communication via the communication network N. Specifically, the communication unit 42 is, for example, a network card. The processing unit 40 transmits and receives data to and from the server 3 via the communication unit 42.

The display unit 43 is a display such as a liquid crystal display or an organic EL display. The display unit 43 displays a screen including data provided by the server 3. The display unit 43 may be a display with a built-in touch panel.

The operation unit 44 is a user interface such as a keyboard and a pointing device that can input and output data to and from the processing unit 40. The operation unit 44 may be a voice input unit. The operation unit 44 may be a touch panel of the display unit 43. The operation unit 44 may be a physical button. The operation unit 44 notifies the processing unit 40 of operation data by the operator of the node 1.

The learning process procedure in the model learning system 100 configured in this manner will be described. Figure 5 is a flowchart showing an example of the learning process procedure in the model learning system 100.

The server 3 acquires an initial (0th order) global model 51 that has been prepared in advance (step S1). The initial global model 51 may be a model created as a 0th order model at a specific node 1, or may be a model learned at a specific location other than node 1, and may be stored in advance in the storage unit 31. The acquisition in step S1 includes reading out the global model 51 that has been stored in advance in the storage unit 31.

The server 3 distributes the acquired zeroth global model 51 to node 1 (step S2).

The server 3 obtains the local model 52 to be learned at the node 1 based on the global model 51 distributed to the node 1 (step S3).

The server 3 performs statistical processing on the acquired local model 52 and updates it to the next-generation global model 51 (step S4). In step S4, the server 3 may increment and store the number of updates (rounds).

The server 3 calculates a performance evaluation value for the updated global model 51 (step S5). In step S5, the server 3 may calculate, as an evaluation value, the accuracy with which corresponding output data is output when input data is input to the updated global model 51 for a specific node 1 or information processing device 6. The performance evaluation value may be calculated using evaluation values from each node 1, as described below.

The server 3 associates the calculated performance evaluation value with round information corresponding to the number of updates (rounds) and stores it in the storage unit 31 (step S6). The round information may be not only the number of updates but also the date and time, etc. The server 3 may omit the processing of steps S5 and S6 for the first few rounds from the 0th round.

The server 3 determines whether the updated global model 51 satisfies the learning completion conditions (step S7).

For example, in step S7, the server 3 determines whether the number of updates has reached a predetermined number. The server 3 may determine that the learning completion condition is met when the performance evaluation value calculated in step S5 meets a pre-stored reference value. The server 3 may determine that the learning completion condition is met in step S7 when it is determined that the performance evaluation value has not improved compared to before the update.

If it is determined that the learning completion condition is not met (S7: NO), the server 3 redistributes the updated global model 51 to multiple nodes 1 (step S8) and returns the process to step S3.

In step S8, the server 3 may redistribute only parameters such as weighting coefficients, rather than redistributing the global model 51 as is.

In step S8, the server 3 may also transmit data indicating the number of updates to the global model 51, i.e., "n" in the nth global model 51. In step S8, the server 3 may transmit the round information stored in step S6 in association with the global model 51. As described above, the round information is information indicating the number of times distribution to nodes 1 and statistical processing were performed until it is determined in step S7 that the learning completion condition is satisfied. The server 3 may also transmit data indicating the type of global model 51, for example, the type of architecture that was used for deep learning.

If it is determined that the learning completion condition is met (S7: YES), the server 3 stores the updated global model 51 in association with the version information as a global model 51 that can be distributed to devices other than the node 1 (step S9). The version information corresponds to the number of times that the learning of the local model 52 and the updating of the global model 51 were performed until it was determined in step S7 that the learning completion condition is met. The version information may include round information as minor information.

The server 3 transmits the stored global model 51 to multiple nodes 1 or other information processing devices 6 (step S10), and ends the process. The other information processing devices 6 are devices that, like the node 1, use the same type of data as the learning target, but do not provide local data for training (see FIG. 1).

The server 3 executes the processing procedure shown in the flowchart in Figure 5 at regular intervals, for example once a month. Each time, the version of the global model is upgraded, making it a more practical model.

Figure 6 is a flowchart showing an example of a learning process procedure for a local model 52 based on a distributed global model 51. The process shown in the flowchart in Figure 6 is executed by each of the multiple nodes 1 when the server 3 distributes the global model 51 in step S2 or step S8.

The processing unit 10 of node 1 receives the distributed global model 51 and stores it in the memory unit 11 (step S301).

The processing unit 10 of node 1 loads the stored global model 51 as an instance (step S302). The processing unit 10 acquires the local data stored in the storage device 2 as training data (step S303) and provides this to the global model 51 to perform learning (step S304).

In step S304, the processing unit 10 inputs the input data of the local data to the loaded global model 51. The processing unit 10 calculates the loss for the output data and the result data corresponding to the input data included in the local data. The processing unit 10 learns parameters including weight coefficients in the distributed global model 51 based on the calculated loss.

The processing unit 10 of node 1 determines whether the learning completion condition is met (step S305). In step S305, the processing unit 10 sets the learning completion condition to be that the number of learning (updates) meets a predetermined number (one or more). The processing unit 10 may determine that the learning completion condition is met when the output accuracy of the global model 51 after learning is equal to or greater than a stored predetermined value. The processing unit 10 may determine that the learning completion condition is met when the change in output accuracy falls within a predetermined range and can be determined to have converged.

If it is determined that the learning completion condition is not met (S305: NO), the processing unit 10 returns the process to step S304. This allows learning to continue.

If it is determined that the learning completion condition is met (S305: YES), the processing unit 10 ends the learning and stores the global model 51 with updated parameters as the local model 52 (step S306).

The processing unit 10 of the node 1 transmits the stored local model 52 to the server 3 (step S307) and ends the process. This enables the server 3 to obtain the local model 52 trained with the local data from each of the multiple nodes 1.

In step S307, the processing unit 10 may also transmit data indicating "n" indicating whether the local model 52 is of the nth order or whether the original global model 51 is of the nth order. In step S307, if the node 1 obtains round information corresponding to the number of updates until one learning is completed, as described below, the node 1 may transmit the round information in association with the local model 52.

It should be noted that the process shown in the flowchart of Figure 6 does not result in local data being sent from node 1 to server 3. No anonymization of the local data is performed. The data sent from node 1 is the model itself. The characteristics of the local data are reflected, but no data is sent.

Figure 7 is a flowchart showing an example of a process for updating the global model 51 in the server 3. The process procedure shown in the flowchart in Figure 7 corresponds to the details of step S4 in the process procedure shown in the flowchart in Figure 5.

The processing unit 30 of the server 3 acquires the local model 52 transmitted from the node 1 (step S401) and stores the local model 52 in association with the identification data of the node 1 (step S402). In step S401, the processing unit 30 of the server 3 acquires the local model 52 transmitted asynchronously from each node 1.

The processing unit 30 determines whether the global model 51 should be updated with the acquired local model 52 (step S403). In step S403, the processing unit 30 may determine that the global model 51 should be updated if local models 52 can be acquired from all of the nodes 1 to which the global model 51 has been distributed. In step S403, the processing unit 30 may determine that the global model 51 should be updated if local models 52 can be acquired from a number of representative nodes 1 that have been determined in advance.

If it is determined that an update is not necessary (S403: NO), the processing unit 30 of the server 3 returns the process to step S401. The local models 52 sent from each node 1 are acquired and aggregated until it is determined that an update is necessary.

If it is determined that an update is necessary (S403: YES), the processing unit 30 of the server 3 calculates the average of the local models 52 from the multiple nodes 1 (step S404). The processing unit 30 updates the average as a new global model 51 (step S405).

The processing unit 30 stores the updated global model 51 in association with round information, for example, data "n" indicating the number of updates (nth) (step S406), and ends the update process for the global model 51. As a result, the (n-1)th global model 51 is updated to the nth global model 51.

As described above, in the model learning system 100 of the first embodiment, the local data is not sent to the server 3, and the global model 51 is put into practical use based on the learning results using the local data stored in each location. Even in cases where the amount of data is insufficient using only the local data from each location, it is possible to provide a model that is more accurate and can be put into practical use sooner than learning using a large amount of data aggregated in one location.

In the first embodiment, the server 3 stores the global model 51 in association with round information and version information during the process of learning (updating) and revising (upgrading) the global model 51. Therefore, the server 3 can provide information on the update process to the monitor 4 or node 1 for each version and each round.

Figure 8 is a flowchart showing an example of a reference process for the learning process. Based on the Web server program included in the server program 3P, the server 3 provides information about the learning process in response to a reference request from the monitor 4 or node 1 according to the process steps shown below.

The operator of monitor 4 or node 1 attempts to access a web page provided by server 3 based on the stored web browser program.

The processing unit 30 of the server 3 receives the connection request by functioning as a web server (step S601), and reads out the main (portal) web page from the memory unit 31 according to the account information included in the request and transmits it to the request source (step S602).

An account for accessing the server 3 is issued to each of the administrators of monitor 4 and node 1. The account includes an account ID, authentication information, and authority data. The server 3 stores the account data in the memory unit 31. The account for the administrator of monitor 4 is associated with authority to specify the number of learning times and interrupt learning. The account for the administrator of node 1 is associated with authority that does not affect the contents of the global model 51, such as viewing performance evaluation values during the learning process.

The processing unit 30 receives the specification of the global model 51 sent from the monitor 4 or node 1 (step S603), and transmits a web page including a list of the update history of the specified global model 51 (step S604). As shown in the flowchart of FIG. 5, each time the global model 51 is updated, the global model 51 and the performance evaluation value are stored in association with the round information and version information. Therefore, the processing unit 30 can create a list of the update history based on the stored data.

When the processing unit 30 receives a performance reference request including a specification of version information included in the list of update history (step S605), it creates a web page including an image showing the performance of the global model 51 during the update process that was stored in association with the specified version information (step S606). The image showing the performance may include, for example, a graph showing the progress of performance, the number of data items used for learning, etc.

The processing unit 30 outputs the created screen data to the monitor 4 or node 1 that originated the request (step S607) and ends the process.

Figures 9 and 10 show example screens displayed on the monitor 4 or node 1. Figure 9 shows an example of an update history screen 430 for the global model 51. The update history screen 430 in Figure 9 is displayed on the display unit 43 of the monitor 4. The update history screen 430 includes a list 432 showing performance evaluation values and update dates and times in a table format in association with the version names corresponding to the version information of the global model 51. The text showing the version names in the list 432 includes links to individual pages and is selectable. When the text showing the version name is selected, the processing unit 10 of the node 1 accepts the specification of the version information. When the latest version of the global model 51 is being updated (rounded), an icon showing that the update is being performed is displayed next to the version name.

Figure 10 shows an example of an individual performance screen 431 that visualizes the individual performance of the global model 51. The individual performance screen 431 in Figure 10 is displayed when any version name text is selected from the list 432 of the update history screen 430 of the global model 51. Figure 10 shows an example of the individual performance screen 431 when the version "ver4.0" is selected in Figure 9. The individual performance screen 431 includes a graph 434 that shows the progress of the performance evaluation value calculated for each update round of the global model 51 of the specified version information. The individual performance screen 431 includes a details screen 435 that shows information such as the local model 52 used in learning the selected version of the global model 51, i.e., the number of nodes 1 that participated in the learning. The operator of the monitor 4 can check the learning (update) process of the global model 51 by the contents of the details screen 435 and the image 434 in which the performance progress is visualized.

In response to a reference request from an account of the administrator of monitor 4 that has been given the authority to interrupt and terminate learning, an interface 433 for instructing to interrupt the update is displayed on the individual performance screen 431 as shown in FIG. 10. When interface 433 is selected, the rounds up to the point where the learning completion condition is met are interrupted in the processing procedure shown in the flowchart of FIG. 5. It is preferable that a log of the interruption of the update is recorded. This allows the administrator of monitor 4 to check the accuracy status of each round, and interrupt the learning if continued updates are not expected to improve the model's performance any further.

For a reference request from an account of the administrator of monitor 4 that has been given the authority to select a global model, an interface that accepts the selection of the global model to be redistributed may be displayed on the individual performance screen 431. When this interface is selected, the processing unit 30 interrupts the processing procedure shown in the flowchart of FIG. 5, redistributes the selected global model 51 in step S8, returns the process to step S2, and continues the update process.

In this way, in the first embodiment, the local model 52 is learned based on the local data, and as shown in Figures 9 and 10, the operator can visually check the learning process of the global model 51 from the monitor 4 or node 1. As shown in Figures 9 and 10, if the operator has administrator authority for the monitor 4, the administrator can also stop the learning, i.e., the update of the global model.

Second Embodiment
The model learning system 100 in the second embodiment differs from the first embodiment in the judgment process for satisfying the learning completion condition. The configuration of the model learning system 100 in the second embodiment is the same as that of the model learning system 100 in the first embodiment except for the details of the judgment process described above, so the same reference numerals are used for the common configuration and detailed description will be omitted.

Figures 11 and 12 are flowcharts showing an example of a learning process procedure in the model learning system 100 of the second embodiment. Among the process procedures shown in the flowcharts of Figures 11 and 12, the same step numbers are used for the steps common to the process procedures shown in the flowchart of Figure 5, and detailed descriptions are omitted.

When the global model 51 is updated in step S4 (S4), the processing unit 30 of the server 3 transmits the updated global model 51 to multiple nodes 1 (step S501).

Each of the multiple nodes 1 that receive the updated global model 51 reads local data from the storage device 2 using the transmitted global model 51 and performs accuracy evaluation on the input data. The processing unit 10 of the node 1 calculates the accuracy of the local data when the input data is input, and transmits it to the server 3. The processing unit 10 may calculate the matching rate between the input data in the local data and the output data corresponding to the input data as the accuracy, or may accept an accuracy evaluation from the operator of the node 1 via the display unit 13 and the operation unit 14.

The processing unit 30 of the server 3 acquires the evaluation value sent from each node 1 (step S502), and calculates a total evaluation value from the evaluation values acquired from the multiple nodes 1 (step S503). In step S503, the processing unit 30 may calculate the average accuracy, the average matching rate, or the average evaluations received from the operators.

The processing unit 30 stores the evaluation values obtained from each node in step S502 for the updated global model 51 and the total evaluation value calculated in step S503 in association with the round information of the global model 51 (step S504).

The processing unit 30 transmits a web page including the update history including the evaluation value for the global model 51 stored for each round information and the total evaluation value to the monitor 4 and outputs it (step S505).

The processing unit 30 accepts the selection of the global model 51 to be redistributed based on the update history (step S506) and determines whether or not the selection to stop learning has been accepted (step S507).

In step S507, the processing unit 30 includes an interface for ending learning in the web page sent in step S505, and determines that a selection to stop learning has been accepted when "end" is selected in the interface for ending learning. The processing unit 30 may also include an interface for continuing learning in the same web page, and determine that a selection to continue learning has been accepted when "continue learning" is selected.

If it is determined in step S507 that the selection to stop learning has not been accepted (S507: NO), it is determined whether there have been any changes to the learning conditions (step S508).

In step S508, the processing unit 30 includes an interface for accepting changes to the learning conditions in the web page sent in step S505, and determines that there is a change to the learning conditions when the interface for changing the learning conditions is selected.

If it is determined that there is a change in the learning conditions (S508: YES), the processing unit 30 accepts the change in conditions (step S509). In step S509, the processing unit 30 may accept a change in the nodes participating in the learning of the local model 52 on a web page described later, or may accept a change in the number of updates to the local model 52. It may also accept changes to other conditions. The order of the processing procedures of steps S509 and S508 may be reversed, and the processing unit 30 may determine that there is a change in the learning conditions when a change is accepted.

The processing unit 30 redistributes the global model 51 corresponding to the round information selected in step S506 to multiple nodes 1 based on the accepted changed learning conditions (step S510), and returns the process to step S3.

If it is determined in step S508 that the learning conditions have not changed (S508: NO), the processing unit 30 redistributes the global model 51 corresponding to the round information selected in step S506 to the node 1 that has been the transmission target up to that point (S510), and returns the process to step S3.

If it is determined in step S507 that the selection to stop learning has been accepted (S507: YES), the processing unit 30 stores the global model 51 selected in step S506 in association with the version information as a global model 51 that can be distributed to other devices (step S511).

The processing unit 30 transmits the stored global model 51 to multiple nodes or other information processing devices 6 (S10) and ends the processing.

FIGS. 13 to 15 are diagrams showing example screens displayed on the display unit 43 of the monitor 4. FIG. 13 shows another example of an individual performance screen 431 that visualizes the individual performance of the global model 51. The individual performance screen 431 in FIG. 13 is displayed when the text of the version information of the global model 51 being updated is selected from the update history screen 430 shown in FIG. 9.

As shown in FIG. 13, the individual performance screen 431 in the second embodiment includes a graph 434 showing the progress of the total evaluation value calculated for each update round for the global model 51 of the specified version information, on top of the graph showing the progress of the performance evaluation value, similar to FIG. 10 shown in the first embodiment.

The graph 434 includes a mark 436 for selecting the number of rounds. The details screen 435 of the individual performance screen 431 shows the evaluation of the global model 51 for the number of rounds to which the mark 436 is attached.

The individual performance screen 431 includes an interface 437 for continuing learning thereafter. The interface 437 is, for example, a button icon, as shown in FIG. 13. The operator on the monitor 4 selects the interface 437 for continuing learning for the round global model 51 selected with the mark 436 on the individual performance screen 431. In response to this operation, the processing unit 30 of the server 3 determines that there have been no changes to the learning conditions (S508: NO), distributes the selected global model 51 to the node (S510), and continues processing.

In the individual performance screen 431, the number of rounds may be selected in the details screen 435 instead of the mark 436.

The individual performance screen 431 includes an interface 438 for stopping updates and terminating learning. The interface 438 is, for example, a button icon, as shown in FIG. 13. The interface 438 may also be linked text. The operator of the monitor 4 selects the interface 438 for terminating learning with the global model 51 selected with the mark 436 on the individual performance screen 431, based on the total evaluation value. In response to this operation, the processing unit 30 of the server 3 stores the selected global model 51 as a distributable global model 51 (S511).

The individual performance screen 431 includes an interface 439 for displaying details of the evaluation value for each round of the selected version information. When the operator of the monitor 4 selects the interface 439, the total evaluation value and specific numerical values of the evaluation value from each node 1 are output. FIG. 14 shows an example of an evaluation value details screen 440. The details screen 440 in FIG. 14 includes a list 441 that shows the total evaluation value and the evaluation value from each node (S504) calculated each time an update (round) is performed, in association with the round information for each round. The list 441 may also be associated with the update date and time.

Returning to the individual performance screen 431 in FIG. 13, the explanation will continue. The individual performance screen 431 includes an interface 442 for interrupting learning. As shown in FIG. 13, the interface 442 is a button icon. When the operator of the monitor 4 confirms on the individual performance screen 431 that further learning does not seem to satisfy the conditions for distribution, the operator selects the interface 442 for interrupting learning. In response to this operation, the processing unit 30 of the server 3 interrupts the processing procedure shown in the flowcharts of FIG. 11 and FIG. 12.

The individual performance screen 431 of the second embodiment includes a node screen 443 that visualizes the evaluation value at each node for the global model 51 of that version. The node screen 443 includes a rounded rectangular graphic that corresponds to the global model 51 and includes text indicating the round information selected by the mark 436. The graphic corresponding to the global model 51 is displayed with text indicating the total evaluation value for that round number. The node screen 443 includes a rounded rectangular graphic that includes text (A, B, C, ...) identifying each node 1 so as to connect to the graphic corresponding to the global model 51 via an edge. The graphic corresponding to each node 1 is displayed with text indicating the evaluation value for the global model 51 for that round number at each node 1. The graphic corresponding to each node 1 is associated with a link to the performance history of the local model 52 at each node 1.

In this way, the evaluation value of each node 1 is visualized, allowing the operator to grasp the overall learning situation in the model learning system 100.

Figure 15 shows an example of an editing screen for learning conditions. Figure 15 shows an example of an editing screen 444 that is displayed when any node 1 is selected on the node screen 443 of the individual performance screen 431 in Figure 14. The editing screen 444 corresponds to an interface that accepts changes to the learning conditions. In Figure 15, the editing screen 444 is displayed like a menu screen when node 1 is selected, and it is possible to select whether or not to exclude the selected node 1 from the nodes 1 that learn the local model 52 used to update the global model 51. In the example of Figure 15, the editing screen 444 is displayed in the form of a menu screen, but is not limited to this.

The operator of monitor 4 can change the learning conditions on edit screen 444, which is displayed by operating node screen 443, and continue learning on interface 437 of individual performance screen 431 under the changed learning conditions, for example, excluding local model 52 by node 1 of "node D".

The learning condition editing screen 444 may include a menu for accepting settings of conditions for terminating learning, such as the number of learning times when learning the local model 52, as shown in FIG. 15.

Figure 16 is a schematic diagram of model learning in the second embodiment. The schematic diagram in Figure 16 shows the update history when the nth global model 51 after each update (round) is selected by the operator via the monitor 4 when updating the n+1th global model 51 after the update. Compared to the schematic diagram of the model learning system in Figure 1, it shows that the global model 51 is not always adopted in succession as it is updated. This makes it possible to proceed with distributed learning so as to improve accuracy.

Third Embodiment
In the third embodiment, a performance evaluation is calculated while the local model 52 is being trained at the node 1 and transmitted to the server 3, so that the accuracy of the local model 52 at each node 1 can be viewed from the monitor 4 via the server 3.

The configuration of the model learning system 100 in the third embodiment is similar to that in the first and second embodiments, so the same reference numerals are used for the configuration common to the model learning system 100 in the first and second embodiments, and detailed descriptions are omitted.

In the third embodiment, each node 1 stores the learning process and the history of the local model 52. FIG. 17 is a flowchart showing an example of a learning process procedure for the local model 52 of a node in the third embodiment. Among the process procedures shown in the flowchart of FIG. 17, the same step numbers are used for the steps common to the process procedures shown in the flowchart of FIG. 6 of the first embodiment, and detailed descriptions thereof are omitted.

The processing unit 10 of node 1 executes learning based on the distributed global model 51 (S301-S304), and stores the performance evaluation obtained as a result of the learning in association with data indicating the progress of the learning of the local model 52 (step S311) until YES is determined in S305 or each time the number of times steps S301-S304 are repeated reaches a predetermined number of times. In step S311, the processing unit 10 may calculate the accuracy with which the output data when local data is input matches the training data as the performance evaluation. In step S311, the processing unit 10 may store the number of times steps S301-S304 are repeated for the round information of the distributed global model 51, or the accuracy, as data indicating the progress of the learning (minor update).

If the processing unit 10 determines in step S305 that the learning completion condition is met (S305: YES), it stores the learned local model 52 in association with the round information of the distributed global model 51 (step S312).

The processing unit 10 transmits the stored local model 52 and the performance evaluation stored in step S311 to the server 3 in association with the round information (step S313), and ends the process. This enables the server 3 to acquire the local model 52 learned from the local data in each round and its individual performance evaluation from each of the multiple nodes 1.

The calculation and storage of the performance evaluation in step S311 may be performed after the learning is determined to satisfy the learning completion condition and is terminated (S305: YES), rather than each time the learning in step S304 is performed.

The server 3 stores information such as round information and performance evaluation transmitted from the node 1 together with the local model 52. FIG. 18 is a flowchart showing an example of the update process of the global model 51 in the third embodiment. The server 3 in the third embodiment executes the processing procedures shown in the flowcharts of FIG. 11 and FIG. 12, as in the second embodiment. Among these, the server in the third embodiment executes the processing procedures shown below in step S4. Among the processing procedures shown in the flowchart of FIG. 18, the steps common to the processing procedures shown in the flowchart of FIG. 7 of the first embodiment are assigned the same step numbers and detailed explanations are omitted.

The processing unit 30 of the server 3 acquires the local model 52 and performance evaluation transmitted from the node 1 in association with the round information (step S411). The processing unit 30 stores the acquired local model 52 and performance evaluation in association with the round information (step S412). The processing unit 30 may also store the local model 52 and performance evaluation in association with the identification data of the node 1. The processing unit 30 executes the process of step S403.

In this way, when the processing unit 30 of the server 3 obtains the local model 52 from the node 1, it can obtain the performance evaluation and store it by version and by round.

The data including the progress of the learning accuracy in the local model 52 aggregated from these nodes 1 is output by the server 3 to the monitor 4 in the processing of step 505 in the flowcharts of Figures 11 and 12, and is output in the update history of the updated global model 51.

Figure 19 shows the performance history screen 445 of the local model 52. Figure 19 is displayed when one of the graphics of multiple nodes 1 included in the node screen 443 is selected on the individual performance screen 431 of the global model 51 shown in Figure 13. Figure 19 shows an example that is displayed when the graphic corresponding to node 1 of "node A" is selected.

The performance history screen 445 of the local model 52 includes a list of performance evaluation values calculated for different versions of the global model 51. The performance history screen 445 outputs a graph showing the progress of the accuracy (performance) for each version of the local model 52 in the selected node 1. The performance history screen 445 outputs the evaluation value for the latest round for the local model 52 corresponding to the global model 51 being updated. In the example of FIG. 19, it is shown that in the node 1 of "node A", the evaluation value of the local model 52 increases smoothly from "0.29" to "0.80" as it progresses from version 1.0 to version 4.0. The performance history screen 445 may output the performance evaluation value for each round for the local model 52 in each node 1 by version in a table format as shown in FIG. 14.

This allows the operator to refer to the evaluation values at each node 1 and proceed with distributed learning to improve accuracy.

Fourth Embodiment
In the fourth embodiment, the method of updating the global model 51 based on the local model 52 learned in each node 1 differs from the methods described in the first to third embodiments. The configuration of the model learning system 100 in the fourth embodiment is similar to that of the model learning system 100 in the first embodiment except for the details of the update process of the global model 51, so the same reference numerals are used for the common configuration and detailed description will be omitted.

In the model learning system 100 of the fourth embodiment, the server 3 executes one of the learning processing procedures shown in the flowchart of FIG. 5 of the first embodiment and the processing procedures shown in the flowcharts of FIG. 11 and FIG. 12 of the second embodiment. In the fourth embodiment, among these processing procedures, the update processing of step S4 is different. In addition, in response to the difference in the update processing, the contents of the data transmitted from the node 1 are also different from those described in the first embodiment.

Figure 20 is a flowchart showing an example of a learning process procedure for the local model 52 of the fourth embodiment. Among the process procedures shown in the flowchart of Figure 20, the steps common to the process procedures shown in the flowchart of Figure 6 in the first embodiment are given the same step numbers and detailed descriptions are omitted.

The processing unit 10 of node 1 acquires the local data stored at that time from the storage device 2 (S303), calculates the amount of local data (number of data items) (step S323), and proceeds to step S304.

Each time learning is performed (S304), the processing unit 10 of node 1 calculates the amount of learning (step S324) and determines whether the learning completion condition is met (step S305). In step S324, the processing unit 10 may calculate the number of times the learning is repeated as the amount of learning, may calculate the degree of improvement in accuracy from the previous version of the local model 52 as the amount of learning, or may calculate the amount of change in the local model 52 as the amount of learning.

In step S305, the processing unit 10 may determine that the learning completion condition is met when, as a result of the learning, the accuracy of the output data from the neural network for the local data satisfies a predetermined condition, for example, the accuracy is equal to or greater than a predetermined value. The processing unit 10 may determine that the learning completion condition is met when the amount of learning (number of times of learning) is equal to or greater than a predetermined amount (predetermined number of times).

The processing unit 10 of the node 1 transmits the amount of data calculated in step S323 or the amount of learning counted together with the trained local model 52 to the server 3 (step S327), and ends the process. This enables the server 3 to obtain the amount of data or the amount of learning of the original local data together with the local model 52 trained with the local data from each of the multiple nodes 1.

Even in the fourth embodiment, node 1 transmits the amount of local data, but the data itself is not transmitted to server 3.

In the fourth embodiment, the operator can assign weights that take into account the amount of learning of the local model 52 at each node 1, the amount of data used for learning, and the like. FIG. 21 is a flowchart showing an example of a process for updating the global model 51 in the fourth embodiment. Among the process steps shown in the flowchart in FIG. 21, those steps that are common to the process steps shown in the flowchart in FIG. 7 of the first embodiment are given the same step numbers and detailed descriptions are omitted.

The processing unit 30 of the server 3 acquires the local model 52 and the amount of data or learning transmitted from each node 1 (step S421), and stores the local model 52 and the amount of data or learning in association with the identification data of the node 1 (step S422). In step S411, the processing unit 30 of the server 3 asynchronously acquires the local model 52 transmitted from each node 1.

When the processing unit 30 determines that it is time to update (S403: YES), the processing unit 30 of the server 3 initially calculates weights based on the amount of data or the amount of learning for the local models 52 from multiple nodes 1 (step S423).

The processing unit 30 of the server 3 outputs the amount of data and/or the amount of learning at each node 1, as well as the initially calculated weights (step S424), and accepts the setting (editing) of the weights (step S425).

The processing unit 30 uses the initially calculated weights or the weights received in step S425 to calculate a weighted average of the local models 52 learned at each node 1 (step S426). The processing unit 30 updates the weighted average of the local models 52 as a new global model 51 (step S427).

The processing unit 30 stores the updated global model 51 in association with the data indicating the version information (S406), and ends the update process of the global model 51.

The server 3 continues the update process shown in the flowchart of FIG. 21 until the update has been performed a predetermined number of times or until the accuracy of the updated global model 51 meets a predetermined standard. When repeatedly performing the update process, the reception process of step S425 may be skipped after being performed once.

Figure 22 shows an editing screen 446 for the weights of each node. The editing screen 446 is displayed on the display unit 43 of the monitor 4 by the processing of step S424 of the server 3 during the update of the global model 51 shown in the flowchart of Figure 21. The weight editing screen 446 includes the amount of local data (number of data items) in each node 1. The weight editing screen 446 includes an editable weight that is initially calculated for each node 1 based on the amount of data. In the example of Figure 22, the operator operating the monitor 4 can adjust the weights for calculating the weighted average while referring to the local data in each node 1. At this time, it is also possible to exclude a specific node 1 by setting the weight to 0 (zero) and have the server 3 calculate the weighted average.

In the fourth embodiment, the weighting coefficients are output and changes can be accepted each time the global model 51 is updated (round), but the editing screen 446 may be displayed only once in one version. The weighting coefficients do not need to be changed within the same version.

The update method in the fourth embodiment makes it possible to handle models trained using local data with different amounts of data according to the amount of data and to appropriately evaluate them. The update method in the fourth embodiment can also be applied to the second or third embodiment.

In the fourth embodiment, each time the local model 52 is learned from each node 1, the server 3 can obtain the amount of local data or the amount of learning at that time, and can output this data when outputting the performance of the global model 51. Figure 23 shows an example of an individual performance screen 431 that visualizes the individual performance of the global model 51 in the fourth embodiment. In the individual performance screen 431 shown in Figure 23, unlike the screen shown in Figure 13, the details screen 435 showing the performance of the selected version of the global model 51 includes information on the number of data items used for learning.

In this way, the server 3 can obtain information such as data volume from each node 1, so that even in an environment with a small amount of data, it is expected that the global model 51 can be optimized using the learning information of the local model 52 based on the operator's knowledge.

Fifth Embodiment
The model learning system 100 can be applied to deep learning models including neural networks. For example, the input data is image data, and the output data is a detection result of a subject appearing in the image of the image data, or a judgment result for the subject. In another example, the input data is cash receipts and withdrawals data, and the output data is a judgment result such as an evaluation, or a predicted value regarding a change in a company's business condition (growth or deterioration) or a predicted value regarding an economic forecast. In another example, the input data is measurement data from various sensors installed in a factory or production equipment, and the output data is data regarding production management including abnormality/normality. In another example, the input data is text data, and the output data is a judgment result or predicted data.

In the fifth embodiment, an example will be described in which the model learning system 100 is applied to learning a model that outputs data that assists in diagnosing whether or not a patient is experiencing symptoms of a specific disease, based on medical data obtained about the patient at a medical facility. In the following description, the medical data is, for example, image data of a fundus photograph taken during an examination.

Figure 24 is a schematic diagram of the model learning system 100 of the fifth embodiment. The configuration of the model learning system 100 in the fifth embodiment is basically the same as that of the model learning system 100 of the first embodiment. In the model learning system 100 of the fifth embodiment, the configuration common to the model learning system 100 of the first embodiment is given the same reference numerals and detailed description is omitted.

In the fifth embodiment, the node 1 and the information processing device are provided in a medical facility. The node 1 and the information processing device can acquire image data obtained from an imaging device 7 that takes fundus photographs of patients. In the medical facility where the node 1 is provided, the imaging device 7 outputs image data to the storage device 2. The imaging device includes different types of devices.

In the fifth embodiment, the global model 51, the local model 52, and the model stored as distributable are models that are trained to output data and accuracy that assists in the diagnosis of glaucoma when image data is input. The local data used as training data is image data as input data, and image data resulting from segmenting the disc and cup portions of a fundus photograph as output data. The output data may also be a data set including the results of a doctor or technician's determination of whether or not symptoms are present. The image data that is input data for the local data is associated with device data that indicates the type of imaging device. The device data may be a model number or data that identifies the device manufacturer.

In the fifth embodiment of the model learning system 100, the server 3 acquires an initial global model (zeroth-order global model) 51 that has been created in advance with multiple different architectures at a specific medical facility. The server 3 distributes the zeroth-order global models 51 with different architectures to each of the nodes 1 of the medical facilities that cooperate in the training.

Each node 1 receives the distributed multiple 0th order global models 51, and proceeds with learning based on different 0th order global models 51, each using local data as training data, to obtain multiple first order local models 52. Node 1 transmits the first order local models 52 trained with different architectures to server 3.

The server 3 creates a first global model 51 by weighting the local models 52 acquired from each node 1 for each different architecture. The server 3 redistributes the created first global model 51 to multiple nodes 1.

The server 3 repeatedly obtains the (n+1)th order local model 52 created from the distributed nth order global model 51 and updates the (n+1)th order global model 51 from the (n+1)th order local model 52 for each different architecture.

The server 3 compares the global models 51 obtained repeatedly for each different architecture, selects the global model 51 of the architecture with higher accuracy, and provides it to the node 1 and the information processing device as a distributable model.

This makes it possible to learn using a large amount of data from different medical facilities without consolidating image data of test results, which is personal information itself, on server 3.

Figure 25 is a flowchart showing an example of a learning process procedure in the model learning system 100 of the fifth embodiment.

The server 3 acquires an initial (0th order) global model 51 created with a different architecture at a specific medical facility (step S201). The acquisition in step S201 includes reading out the global model 51 previously stored in the memory unit 31.

The server 3 distributes the acquired multiple zeroth-order global models 51 to each node 1 (step S202).

Based on the global model 51 distributed to node 1, server 3 obtains local models 52 to be learned at node 1 for each different architecture (step S203).

The server 3 performs statistical processing on the acquired local model 52 for each different architecture, and updates it to the next-generation global model 51 (step S204). In step S204, the server 3 may increment the number of updates.

The server 3 calculates a performance evaluation value for the updated global model 51 for each different architecture (step S205). In step S205, when input data of local data in a specific node 1 is input to a global model 51 of a different architecture, the server 3 may calculate the accuracy of the corresponding output data as an evaluation value. In step S205, the server 3 may transmit the updated global model 51 to each node 1, obtain evaluation values from each node 1, and calculate a total evaluation value, as shown in the second embodiment.

The server 3 stores the calculated performance evaluation value in the storage unit 31 in association with round information corresponding to the number of updates and data identifying the architecture (step S206). The version information may be not only the number of updates but also the date and time, etc. The server 3 may omit the processing of steps S205 and S206 in the first few rounds from the 0th round.

The server 3 determines whether the learning completion condition is met (step S207).

In step S207, the server 3 makes a judgment based on, for example, whether the number of updates (number of rounds) has reached a predetermined number. When input data of local data in a specific node 1 is input to a global model 51 of a different architecture, the server 3 may make a judgment based on whether the accuracy of outputting corresponding output data in any of a plurality of global models 51 satisfies a predetermined condition. The condition for ending learning may be, as shown in the second embodiment, that a total evaluation value is calculated from the evaluation values for the global model 51 sent to each node 1, and the total evaluation value satisfies a predetermined criterion.

If it is determined that the learning completion condition is not met (S207: NO), the server 3 redistributes the updated global model 51 of each architecture to multiple nodes 1 (step S208) and returns the process to step S203.

If it is determined that the learning completion condition is met (S207: YES), the server 3 selects one or more global models 51 that can be distributed from the global models 51 of each architecture after the update (step S209). In step S209, the server 3 selects one or more global models 51 that have relatively high accuracy.

The server 3 transmits the selected global model 51 to multiple nodes 1 or other information processing devices (step S210) and ends the processing.

This allows each medical facility to use a highly accurate and practical architecture model for their image data.

Figure 26 is a flowchart showing an example of a learning process procedure for the local model 52 in the fifth embodiment. The process shown in the flowchart in Figure 26 is executed by each of the multiple nodes 1 when the server 3 distributes the global model 51 in step S202 or step S206. Among the process procedures shown in the flowchart in Figure 26, steps that are common to the process procedures shown in the flowchart in Figure 6 of the first embodiment are given the same reference numerals and detailed explanations are omitted.

When the processing unit 10 of node 1 receives and stores the distributed global model 51 (S301), it executes the processes from S302 to S305 for each architecture. The processing unit 10 stores the learned global model 51 as a local model 52 in association with data identifying the architecture (step S336).

The processing unit 10 determines whether learning has been completed for all architectures (step S337).

If it is determined that learning is not complete (S337: NO), the processing unit 10 returns the process to step S302 and executes the processes from step S302 to S336 for the next architecture.

If it is determined that learning has been completed for all architectures (S337: YES), the processing unit 10 of node 1 transmits the stored local model 52 to the server 3 in association with the device data, which is a characteristic of the local data (step S338). The processing unit 10 then ends the n-th learning process in node 1.

This allows the server 3 to obtain a local model 52 trained with local data from each of the multiple nodes 1. The local data itself is not transmitted to the server 3, but the quality of the local data can be distinguished by the server 3.

Figure 27 is a flowchart showing an example of the update process of the global model 51 in the fifth embodiment. The process procedure shown in the flowchart of Figure 27 corresponds to the details of step S204 in the process procedure shown in the flowchart of Figure 25.

The processing unit 30 of the server 3 acquires the local model 52 for each architecture and the device data transmitted from the node 1 (step S431). The processing unit 30 associates and stores the acquired local model 52 for each architecture and the device data (step S432). In step S432, the processing unit 30 may store the local model 52 for each identification code of the node 1.

The processing unit 30 determines whether or not an update is necessary (step S433). In step S433, the processing unit 30 may determine that an update is necessary if local models 52 can be acquired from all of the nodes 1 to which the global model 51 has been distributed. In step S433, the processing unit 30 may determine that an update is necessary if local models 52 can be acquired from a number of representative nodes 1 that have been determined in advance.

If it is determined that an update is required (S433: YES), the processing unit 30 of the server 3 outputs the data volume and device data as the characteristics of the data in the multiple nodes 1 (step S434) and accepts the setting (editing) of the weights (step S435).

The processing unit 30 uses the weights received in step S425 to calculate the weighted average of the local models 52 learned by each node 1 for each architecture (step S436). The processing unit 30 updates the weighted average of the local models 52 for each architecture as a new global model 51 (step S437).

The processing unit 30 stores the updated global model 51 in association with data indicating round information for each architecture (step S438), and ends the update process of the global model 51.

Figure 28 shows a weight editing screen 446 in the fifth embodiment. The editing screen 446 is displayed on the display unit 43 of the monitor 4 by the process of step S434 of the server 3 during the update of the global model 51 shown in the flowchart of Figure 27. The weight editing screen 446 includes the amount of local data (number of data items) in each node 1 and the model name of the device that outputs the local data. In the example of Figure 28, the operator operating the monitor 4 can adjust the weights for calculating the weighted average while referring to the amount of local data in each node 1 and the model number of the imaging device 7 that outputs the local data used in learning. At this time, it is also possible to exclude a specific node 1 by setting the weight to 0 (zero) and have the server 3 calculate the weighted average.

In the fifth embodiment, the weighting coefficients are output and changes can be accepted each time the global model 51 is updated (round), but the editing screen 446 may be displayed only once in one version. The weighting coefficients do not need to be changed within the same version.

The update method in the fifth embodiment makes it possible to appropriately evaluate a model trained using local data with different data characteristics according to the characteristics of the data. In addition, it is expected that the performance of the global model 51 will be improved compared to a simple average of local models 52 with different data characteristics. The update method in the fifth embodiment can also be combined with the second to fourth embodiments.

As another data characteristic, an evaluation of the accuracy of annotations for local data may be stored at each node 1, and the server 3 may calculate a weighted average with weights according to the high or low of the evaluation. The evaluation of the accuracy of annotations may be performed by an administrator of the monitor 4.

In the fifth embodiment, since the local data is medical data, the weights used in the weighted average in step S424 may be assigned according to the attributes of the medical data. For example, weights may be assigned according to the male/female ratio of patients, age distribution, regionality, etc. These abstracted data may be reflected in the local model 52.

In addition, the update method in the fifth embodiment allows the evaluation of which of the different CNN architectures is most appropriate for the global model 51, making it practical to use.

The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is defined by the claims, and includes all modifications within the meaning and scope of the claims.

Reference Signs List 1 Node 10 Processing section 13 Display section 14 Operation section 2 Storage device 3 Server 30 Processing section 31 Storage section 3P Server program 4 Monitor 40 Processing section 43 Display section 44 Operation section 51 Global model 52 Local model

Claims (11)

Distributing global model data stored in the server to multiple nodes;
acquiring data of a local model learned based on the global model by local data processed by the plurality of nodes;
updating data of the global model based on data of the local model after learning acquired from each of the plurality of nodes;
The updated global model data is stored in association with update information corresponding to the number of updates;
Calculating a performance evaluation value of the global model;
Repeating the distribution of the data of the global model, the acquisition of the data of the local model, and the update of the data of the global model for multiple rounds;
Each time the data of the global model is updated, the data and the performance evaluation value of the global model are stored in association with update information;
receiving a request to reference the global model together with a specification of update information for the global model;
creating screen data for displaying a screen showing a transition of performance of the global model for each update based on the calculated performance evaluation value in response to the reference request;
outputting the screen to a request source of the reference request . The model learning method according to claim 1 , wherein the server accepts, on a screen showing a transition in performance of the global model, a designation of the number of updates to the data of the global model and an instruction to stop updating the data of the global model. The server,
obtaining an evaluation value of accuracy of the global model calculated based on local data for each of the plurality of nodes;
calculating an overall performance evaluation value of the global model based on the evaluation value ;
outputting, to a source of the reference request, a performance screen showing the evaluation values calculated at each of the plurality of nodes in association with the update information of the global model and the overall evaluation value;
The model training method according to claim 1 . The performance screen includes a screen in which the evaluation values calculated for each of the plurality of nodes are visualized by connecting a graphic corresponding to the global model with a graphic corresponding to each of the plurality of nodes connected via an edge.
The model learning method according to claim 3 . The server causes the node to output an edit screen that accepts selection of a local model to be used for updating the global model.
The model learning method according to claim 3 . The server,
outputting to the node an edit screen for accepting settings of weights for each of a plurality of local models used for updating the global model;
Using the set weighting, the data of the global model is updated with the data of the plurality of local models.
The model learning method according to claim 3 . The server,
Accepting a selection of a global model to be redistributed on the screen ;
The model learning method according to claim 1 , further comprising the step of redistributing data of a selected global model to the plurality of nodes regardless of progress of learning. The server,
causing the plurality of nodes to calculate an evaluation value of a local model each time an update is performed;
storing, in the plurality of nodes, an amount of local data used in learning the local model or characteristics of the local data in association with update information of the local model together with data of the local model ;
acquiring a data amount or a data characteristic from the plurality of nodes in association with data of a local model ;
The node outputs an evaluation value associated with the update information of the local model.
The model learning method according to any one of claims 1 to 6 . The system includes a plurality of nodes each processing local data, and a server connected to the plurality of nodes for communication;
The server distributes data of a global model to the plurality of nodes;
each of the plurality of nodes learns a local model from the global model using local data to be processed;
The processing unit included in the server includes:
acquiring data of the local model after training from the plurality of nodes;
updating data of a global model based on data of a local model after learning acquired from each of the plurality of nodes;
The updated global model data is stored in association with update information corresponding to the number of updates;
Calculating a performance evaluation value of the global model;
Repeating the distribution of the data of the global model, the acquisition of the data of the local model, and the update of the data of the global model for multiple rounds;
Each time the data of the global model is updated, the data and the performance evaluation value of the global model are stored in association with update information;
receiving a reference request for the global model from a node ;
creating screen data for displaying a screen showing a transition of performance of the global model for each update based on the calculated performance evaluation value in response to the reference request;
A model learning system that outputs the screen from the request source of the reference request . A computer that can communicate with multiple nodes.
Distributing the stored global model data to the plurality of nodes;
Acquire data of a local model trained at each of the plurality of nodes;
updating data of a global model based on data of a local model after learning acquired from each of the plurality of nodes;
The updated global model data is stored in association with update information corresponding to the number of updates;
Calculating a performance evaluation value of the global model;
Repeating the distribution of the data of the global model, the acquisition of the data of the local model, and the update of the data of the global model for multiple rounds;
Each time the data of the global model is updated, the data and the performance evaluation value of the global model are stored in association with update information;
receiving a request to reference the global model;
creating screen data for displaying a screen showing a transition of performance of the global model for each update based on the calculated performance evaluation value in response to the reference request;
a computer program that causes the screen to be output from a request source of the reference request . The computer that is connected to the server
learning a local model using the local data to be processed based on the global model distributed from the server;
Transmitting data of the local model after training to the server;
receiving screen data including a list of updated global models associated with different update information from the server;
displaying the list on a display unit provided in the computer based on the received screen data;
Accepting a designation of any update information in the list,
acquiring from the server screen data of a screen showing a transition of performance of the global model for each update based on a performance evaluation value of the global model corresponding to the specified update information ;
A computer program that executes a process to display the screen based on the acquired screen data .

Images