JP7280935B2 - Server Device, Trained Model Providing Program, Trained Model Providing Method, and Trained Model Providing System - Google Patents

Description

TECHNICAL FIELD The present invention relates to a technique for introducing and optimizing a trained model at low cost to an industrial device that performs processing such as judgment and classification using a trained model by deep learning or the like. .

2. Description of the Related Art Conventionally, in a device such as a machine tool or an abnormality detection device for a finished product, a trained model generated by deep learning or the like has been used to identify a work target, perform abnormality detection processing, or the like. In these devices, work accuracy and abnormality detection accuracy are improved by performing specialized learning for the work environment, work conditions, etc. of each device.

Devices using such a trained model include, for example, Patent Document 1 and Patent Document 2. The evolutionary automatic image classification device described in Patent Document 1 is a device that classifies images using a learning device from various feature quantities, and the metal surface quality evaluation device described in Patent Document 2 is a metal surface photographed. This is a device for evaluating the surface quality of metals with a learner based on images.

Japanese Patent Application Laid-Open No. 2007-213480 JP 2011-191252 A

In the case of using a learner trained by machine learning, etc., including Patent Document 1 and Patent Document 2, to perform determination, classification, etc., the configuration of the learner specialized for the work environment, work conditions, etc. of the device is required. It is necessary to set and learn. It takes a considerable cost to make such settings and learn from scratch until it becomes possible to perform highly accurate judgment and classification. Even if a trained model is obtained at such cost, the same trained model cannot be used on devices with different working environments and working conditions, etc., so it is necessary to start learning from scratch. There was a problem that there is

In order to solve this problem, a method of preparing a general-purpose learning model that can correspond to various work environments and work conditions is conceivable. However, general-purpose learning models have the advantage of being able to be applied to various situations because they can handle various work environments and conditions. There was a problem that the accuracy was lower than that of the model. In addition, the complexity of the model increases, the amount of information required to realize it increases, and there is a problem that the computation cost and the memory cost increase. Furthermore, when each device has characteristics unique to it, there is also a problem that versatility must be ensured so as to absorb individual differences.

The present invention has been made in view of the above problems. An object of the present invention is to provide a model providing method and a trained model providing system.

A server device according to the present invention is a server device that can communicate with at least one or more devices equipped with a learning device that performs processing using a trained model via a communication network. A storage unit that stores a plurality of pre-learned shared models, a device data acquisition unit that acquires device data including environment and condition information from the device, and an optimal device for the device based on the acquired device data. and a transmission unit for transmitting the selected sharing model to the device.

Further, the server device according to the present invention stores and manages an additional learning processing unit that performs additional learning on the shared model using sample data for performing additional learning of the shared model, and an additionally trained model. an additionally trained model management unit, wherein the transmission unit transmits the additionally trained model to the device when additional learning is performed on the shared model.

Further, in the server device according to the present invention, the target shared model selection unit stores the content of the device data acquired by the device data acquisition unit as additionally learned models based on other devices stored by the additionally learned model management unit. If the model is applicable, the additionally trained model is selected with priority over the shared model, and the transmitting unit transmits the selected additionally trained model to the device. It is characterized by

In addition, the server device according to the present invention includes an additionally trained model management unit that receives an additionally trained model transmitted from a device having a function of performing additional learning processing on a shared model and stores it in a storage unit. characterized by

Further, in the server device according to the present invention, the target sharing model selection unit calculates a score obtained by evaluating the degree of conformity of each sharing model to the device based on the device data obtained from the device, and shares according to the score. It is characterized in that a model is selected.

Further, in the server device according to the present invention, the target shared model selection unit selects an optimum shared model using machine learning based on device data. It is characterized by

A trained model providing program according to the present invention executes a trained model selection process on a server device that can communicate with at least one or more devices equipped with a learning device that performs processing using a trained model via a communication network. A learned model providing program for realizing each function to make the shared model store a plurality of shared models that have been pre-learned according to the environment and conditions of various devices in the server device by a storage means. , a device data acquisition function for acquiring device data including environment and condition information from the device, a target sharing model selection function for selecting an optimal sharing model for the device based on the acquired device data, and a selected sharing and a transmission function for transmitting the model to the device.

A trained model providing method according to the present invention provides a trained model for executing a process of selecting and providing an optimum trained model for a device equipped with a learning device that performs processing using the trained model. A method comprising: storage processing for storing a plurality of shared models pre-learned according to various device environments and conditions by a storage means; and device data for acquiring device data including environment and condition information from the device. characterized by including acquisition processing, target sharing model selection processing for selecting an optimal sharing model for the device based on the acquired device data, and transmission processing for transmitting the selected sharing model to the device. .

A trained model providing system according to the present invention includes at least one device equipped with a learning device that performs processing using a trained model, and at least one server capable of communicating with the device via a communication network. wherein the server device and/or the device is provided with a storage unit that stores at least one shared model that has been pre-learned according to the environment and conditions of various devices. , a device data acquisition unit that acquires device data including environment and condition information from a device that requires a trained model, and a target sharing that searches and selects the optimal sharing model for the device based on the acquired device data and a model selection unit in the server device, and a transmission unit that transmits the selected shared model to the device that requires the trained model in the server device and/or the device.

Further, in the trained model providing system according to the present invention, the target shared model selection unit includes a score obtained by evaluating the fitness of each shared model for the device based on device data obtained from the device requiring the trained model. are calculated respectively, and the sharing model is selected according to the score.

Further, in the trained model providing system according to the present invention, the device has a function of performing additional learning processing on the shared model, and the server apparatus receives the additionally trained model transmitted from the device. An additionally trained model management unit for receiving and storing in a storage unit is provided, and the target shared model selection unit of the server device selects the additionally trained model in addition to the shared model. It is characterized by

Further, in the trained model providing system according to the present invention, the device has a function of performing additional learning processing on the shared model, a storage unit for storing the additionally trained model, and a an additional learned model information transmission unit that transmits information necessary for selection to the server device, and the target shared model selection unit of the server device is stored in the storage unit of the device in addition to the shared model. It is characterized in that the additionally trained model is also included in the options for selection.

According to the present invention, a plurality of shared models pre-learned according to the environment and conditions of various devices are classified according to the environment and conditions and stored in a plurality of server devices. By sending to , it is possible to achieve highly accurate discrimination/classification according to the situation, compared to the case of using the conventional general-purpose learning model described above, while reducing the complexity expressed by the learning model. Advantageously, computation and memory costs are reduced due to the reduction. In addition, there is an advantage that the introduction cost can be greatly reduced compared to the case of generating a trained model independently on the device. In addition, by providing an additional learning processing function, it is possible to obtain an additional trained model that is more specialized according to the environment and conditions of the device, so it is possible to additionally perform highly accurate inference processing on the device. In this additional learning process, by performing additional learning based on an appropriate shared model according to the environment and conditions of the device, it is possible to obtain many effects of an action called transfer learning. Transfer learning is expected to enable efficient learning in environments where additional learning is desired by skillfully using the weights of shared models created in other environments between environments where device environments and conditions are not exactly the same. is. In addition, by storing and managing the additionally trained model in the server device as well, it is possible to immediately provide the additionally trained model when there is a request from another device with the same environment and conditions. Become. This makes it possible to reduce the computation cost and memory cost for additional learning compared to using a general-purpose learning model. Furthermore, by constructing a trained model providing system comprising at least one or more devices and at least one or more server devices, the shared model stored in the storage units of a plurality of server devices and/or devices can be optimized. Since it is possible to select the most suitable sharing model and provide it to the device, it becomes possible to select the optimal sharing model from a larger amount of data options.

1 is a block diagram showing the configuration of a server device 10 according to the present invention; FIG. FIG. 10 is a flow chart diagram showing a flow of learning processing of additional learning; FIG. 10 is a flow chart diagram showing a flow until an inference process is performed in the device 20;

[First embodiment]
An example of the server device according to the first embodiment will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a server device 10 according to the present invention. The server device 10 is communicably connected to a plurality of devices 201, 202, . The server device 10 and the devices 201 to 20n may be devices designed as dedicated machines, but they can be realized by general computers. In this case, the server device 10 and the devices 201 to 20n include a CPU (Central Processing Unit), GPU (Graphics Processing Unit), memory , a storage such as a hard disk drive (not shown). Further, it is needless to say that various processes are executed by a program in order to cause these general computers to function as the server device 10 of this example.

The server device 10 includes at least a device data acquisition unit 11 , a target shared model selection unit 12 , an additional learning processing unit 13 , an additional learned model management unit 14 , and a storage unit 15 .

The device data acquisition unit 11 has a function of acquiring device data including device environment and condition information generated in any of the devices 201 to 20n. Here, device data refers to data necessary to define attributes such as the device environment, conditions, and data units, sample data with label information necessary for additional learning, and sensor data in actual devices. It includes various data that can be acquired by the device, such as data, network log data, etc. At least, the device data should contain the data necessary to select the sharing model. Specifically, the position data and actuator torque amount of factory robots, acceleration sensor data, image data including or not including depth acquired by in-vehicle cameras and laser radars, displacement sensor data, various process data for process automation, Sensor data such as various data in infrastructure, agriculture, bio-healthcare, etc., network log data, product photo data including normal and abnormal products, voice data, machine type, workpiece type, sensor type, Various data can be used as device data, such as geographic information.

In addition, the environment and conditions of the device are required for the device, for example, when the device is a work machine that performs picking and there are several types of workpiece shapes to be picked. The environment, conditions, etc. to be used differ individually. In addition, the functions required of the learning device differ from device to device, such as whether it is a device that determines whether a product is defective or not, or a device that classifies a product into multiple items. Therefore, it is necessary to acquire information such as individual environments and conditions that differ from device to device as device data. Information such as the environment and conditions may be input on the device side according to a format, or may be determined by the server apparatus 10 from various data and defined as information such as the environment and conditions. . In that case, a method of specifying the definition of information such as the environment and conditions by machine learning using the acquired data may be used.

The target sharing model selection unit 12 has a function of selecting the optimal sharing model for the device based on the device data acquired by the device data acquisition unit 11 . Here, the shared model is a model that has been learned in advance according to various device environments and conditions. The degree of pre-learning may be set to any level, but at least the learning should be done to the extent that it is more efficient and contributes to cost reduction than learning from scratch on a device. is preferred. Selection by the target sharing model selection unit 12 is performed based on the acquired device data, and it is possible to appropriately determine which of the acquired device data is used for selecting the sharing model. As for the method of selecting the sharing model, it may be automatically selected from the degree of matching of each item of the device data, or a plurality of sharing models with a high degree of matching may be presented to the user to allow the user to select. may The degree of matching between items is determined, for example, by determining whether each item of the device data matches for each item and by the number of matching items. If a shared model matching the definition of the device environment, conditions, etc. is not found, a new model having a neural network structure suitable for the definition may be generated. The method of selecting a shared model to be targeted may be a method of selecting a shared model based on a preset rule, or a method of selecting a shared model for selecting an optimal shared model. A method of selecting a shared model by using a learned model for selection (a learned model different from the shared model and the additional learned model and having learned the selection behavior of the shared model) may be used.

In addition, as a method for selecting the optimum shared model in the target shared model selection unit 12, the score evaluated for each shared model is calculated based on the environment and conditions obtained from the device, and the selection is made according to the score. You can do it. The score, which is an evaluation of the goodness of fit of the shared model, is based on the device data on the environment and conditions such as the type of machine, the type of workpiece, the type of sensor, and geographical information, as well as the location data of factory robots, Actuator torque, acceleration sensor data, image data with or without depth acquired by in-vehicle cameras and laser radar, displacement sensor data, various process data for process automation, various data for infrastructure, agriculture, bio-healthcare, etc. sensor data, network log data, product photo data including normal and abnormal products, and more detailed device data such as voice data. How each of these items is evaluated and scored is set in advance, and a total score is calculated by totaling the points for each item for each sharing model. As for the actual selection of the shared model, the shared model with the highest score may be automatically selected, or a plurality of shared models with high scores may be presented to the user for selection. It should be noted that a method of calculating a score, which is a degree of fitness evaluation, learning a learning model for selecting an optimal shared model, and selecting a shared model based on the learned model may also be used. In this case, since the learning model also learns how to score each device data, it is possible to select an optimal shared model.

The additional learning processing unit 13 has a function of performing additional learning on the shared model selected by the target shared model selection unit 12 . Although the shared model has been trained in advance, it has not been trained in the environment and conditions specific to the device. should be fine-tuned by learning the Therefore, in the device data acquisition unit 11, sample data to be used as input data in additional learning is also acquired, and additional learning of the shared model is performed using the acquired sample data. Here, additional learning is not limited to re-learning weights for all layers of the neural network, but also freezing some layers and re-learning only other layers, or adding more layers. It also includes doing As a result, it is possible to generate an additionally trained model that has been fine-tuned as a more optimal model by adding learning content in an environment and conditions that are specific to the device. In order to function as the additional learning processing unit 13, the server device 10 is required to have a configuration for functioning as a learning device.

The additionally trained model management unit 14 has a function of storing the additionally trained model generated by the additional learning processing unit 13 in the storage unit 15, which will be described later, and transmitting the additionally trained model to the target device. have In addition, it has a function to set and manage definition information such as environment and conditions so that the additionally trained model can be used by other devices that meet the conditions. In this way, by defining definition information such as the environment and conditions for additionally trained models, when the target shared model selection unit 12 selects a shared model suitable for a device, The additionally trained model generated based on this can also be used as an option candidate.

The storage unit 15 has a function of storing a plurality of shared models that have been pre-learned according to various device environments and conditions. The storage unit 15 also stores an additionally trained model that is learned by applying sample data for learning device-specific environments and conditions to the shared model. Note that the storage unit 15 does not necessarily have to be in the server device 10, and may be a system that is provided on the device side. In that case, the server device 10 holds information about where the target sharing model is stored, and transfers it from the storage location to the device as needed.

Next, a flow of processing from selecting a shared model in the server device 10 to performing additional learning will be described. FIG. 2 is a flowchart showing the flow of learning processing for additional learning. In FIG. 2, first, device data is collected in order to select a sharing model suitable for the device (S11). Specifically, the device data acquisition unit 11 receives the device data transmitted from the device 20 and collects the device data. Attributes of the device data are defined based on the collected device data (S12). Attributes of device data are information such as device environment and conditions necessary for selecting a sharing model, and are defined. Then, based on the attribute of the defined device data, a shared model is searched (S13). The search target at this time may also include additionally trained models generated by performing additional learning on other devices. As a result of the search, it is determined whether or not the corresponding sharing model exists (S14). If the relevant shared model exists, select the relevant shared model and proceed to the next step (S16). A learning model composed of the configuration is newly generated (S15), and the process proceeds to the next step (S16).

After selecting a shared model or generating a new learning model, the learner performs additional learning on the shared model or the new learning model (S16). Additional learning is performed using sample data for additional learning collected from the device 20 . After the additional learning is completed, the generated additionally trained model is stored in the storage unit 15 (S17). Server device 10 transmits the generated additionally trained model to device 20 .

Note that if the device 20 has a function of performing additional learning processing, or if the selected shared model matches the conditions of the device 20 without the need for additional learning, the Steps (S16) and (S17) may be omitted and the selected sharing model may be transmitted to the device 20 as it is.

Next, a flow from downloading a shared model to performing inference processing in the device 20 will be described. FIG. 3 is a flow chart diagram showing the flow up to performing inference processing in the device 20 . In FIG. 3, the device 20 for which inference processing is desired first collects device data (S21). Attributes of the device data are defined based on the collected device data (S22). Note that the definition of the attributes of the device data may be performed on the server device 10 side. Then, in order to search for the optimum sharing model using the device data, the device data is transmitted to the server device 10 (S23). The server apparatus 10 that has received the device data selects an optimum sharing model, and performs additional learning as necessary. Then, in the device 20, the shared model or the additionally trained model selected by the server device 10 is downloaded to the learning device and stored (S24). Finally, in the device 20, in a state in which the shared model or the additionally trained model is stored in the learning device, inference processing is performed in the learning device using the device data, and an inference result is obtained as output data (S25). .

The output data is completely different depending on the inference process to be performed. The output data includes the name of the object in the video (classification process), characteristics such as race and gender of the person in the video, and photos, voices, and texts processed based on specific rules.

Note that if the device 20 has a function of performing additional learning processing, additional learning may be performed on the shared model after step (S24) in FIG. If the additionally trained model is uploaded to the server device 10 when additional learning is performed on the device 20 side, the additionally trained model subjected to the additional learning on the device 20 side can also be used by another device. available.

A specific operation example of the present invention will be described using the state of FIG. 1 as an example. Assume that an additionally trained model obtained by performing additional learning based on sample data included in device data 201 is “model A′”. The shared model selected by the device 202 transmitting device data to the server apparatus 10 is "model B", which is obtained by performing additional learning based on sample data included in the device data of the device 202. Assume that the additionally trained model is "model B'". In this way, each of the devices 201 and 202 can acquire an optimal and additionally trained model simply by transmitting device data including information such as the environment and conditions of its own device to the server device 10 . , 202, the introduction cost can be greatly reduced compared to the case where the learned model is generated independently.

Also, in FIG. 1, when the device 20n transmits device data to the server apparatus 10 and requests a sharing model, the environment, conditions, etc. defined from the device data of the device 20n are the same as those of the device 201. When the server apparatus 10 determines that the same trained model can be applied, it does not perform additional learning based on the "model A", but transmits the additionally trained model "model A'" to the device 20n. For example, the device 20n can perform inference processing as it is. In this way, if there is an additional trained model generated based on another device with the same environment and conditions, it is possible to directly use it, so that the introduction cost can be further reduced and the introduction It is possible to shorten the time to In addition, since an optimal neural network size can be applied compared to the case of using a general-purpose learning model, it is possible to reduce the computation cost and memory cost for additional learning.

In addition, in a situation where the products handled in the same factory are changed, conventionally, it was necessary to learn from scratch each time the product was changed. All you have to do is search for and download the optimal sharing model again at the right time. In other words, there is an advantage that it is easy to introduce the optimum sharing model when the processing contents of the same device 20 are changed. And if an additional trained model is generated by another device with the same processing content, there is an advantage that the additional trained model that can perform highly accurate inference processing can be immediately introduced without the need for additional learning processing. . Thus, another advantage of the server apparatus 10 of the present invention is that a large number of devices 201 to 20n access the server apparatus 10 and accumulate data of additionally trained models.

In the first embodiment, the shared model and the additionally trained model were explained separately, but they are both trained models, just because the degree of learning is different. . That is, when viewed from another device, if the shared model and the additionally trained model can be appropriately selected according to the degree of learning, they are necessarily stored separately as in the storage unit 15 of FIG. No need. If information for retrieving the optimum model is attached to the shared model and the additionally trained model at the time of retrieval, it can be said that they may be treated as the same trained model. In this case, it can be said that the server device 10 of the present invention functions without the additionally trained model management unit 14 .

In the first embodiment, the server device 10 is provided with the additional learning processing unit 13, but the present invention is not limited to this. It may be provided on the side. In this case, the additionally trained model generated on the device 20 side may be transmitted to the server device 10. The configuration may be such that only information required for selection is transmitted to the server device 10 . Only when another device 20 needs the same additionally trained model, it can be configured to directly transmit it to the server device 10 or the device 20 that needs it. This makes it possible to reduce the data area required for the server device 10 .

In the first embodiment, as shown in FIG. 1, a configuration in which one server device 10 and a plurality of devices 201 to 20n are connected via a communication network 30 has been described as an example. but not limited to, for example, a shared model (including additionally trained models) stored in a plurality of server devices 10 can be communicated via the communication network 30 in a state of mutual recognition, A shared model may be retrieved from another server device 10 and provided to the device. By configuring a learned model providing system composed of a plurality of server devices and a plurality of devices in this way, it is possible to provide a shared model stored in any of the plurality of server devices 10 to the device 20. It will be possible to select the optimal sharing model from a larger amount of data options.

In the first embodiment, the target data for additional learning in the additional learning processing unit 13 is learned using the device data acquired only by the device. Data acquired by another device under the same conditions may be used, or a learned model generated by another device under the same environment and conditions may be used for updating. Further, a mixed learned model may be generated by mixing additionally trained models respectively generated in a plurality of devices under the same environment and conditions. Various known techniques can be applied to the mixing of learning models.

The present invention is a technology that can be applied to any field that requires inference processing using a trained model, and can be operated as a database of trained models.

REFERENCE SIGNS LIST 10 server device 11 device data acquisition unit 12 target shared model selection unit 13 additional learning processing unit 14 additional learned model management unit 15 storage unit 20, 201 to 20n device 30 communication network

Claims (23)

at least one storage for storing a plurality of models ;
at least one processor;
The at least one processor
selecting at least one model from the plurality of models based on a score calculated using data obtained from a first device;
generating a first model for the first device by additionally learning the selected at least one model using model learning data included in data acquired from the first device;
transmitting the first model;
and run
The data for model learning is data acquired by the first device,
model generator. wherein the first device is at least one of a robot, an automobile, a process automation device, an infrastructure device, an agricultural device, a healthcare device, a machine that performs work, an anomaly detection device, a classification device;
The model generation device according to claim 1. The first device is a machine for performing work,
The data for model learning is data obtained during work performed by the first device,
The model generation device according to claim 1. use of the first model for the first device is determined by the device itself, not by the first device;
The model generation device according to any one of claims 1 to 3. The plurality of models stored in the at least one storage includes models that have undergone additional learning using data for model learning acquired by a device other than the first device.
The model generation device according to any one of claims 1 to 4 . 6. The plurality of models stored in the at least one storage includes a model additionally trained using data for model learning acquired by the first device. 1. The model generating device according to 1 . The at least one processor generates the first model by updating some parameters of the selected at least one model.
The model generation device according to any one of claims 1 to 6 . The at least one processor adds new parameters to the at least one selected model to generate the first model.
The model generation device according to any one of claims 1 to 7 . The at least one processor generates the first model using data from devices other than the first device.
The model generation device according to any one of claims 1 to 8 . The first model is a neural network,
The model generation device according to any one of claims 1 to 9 . at least one storage;
at least one processor;
The at least one processor
transmitting data to at least one model generator;
receiving from the at least one model generator a first model generated based on the transmitted data;
generating a second model by additionally learning the first model using the acquired data;
and run
The transmitted data includes at least data for model learning,
device. is at least one of: a robot, an automobile, a process automation device, an infrastructure device, an agricultural device, a healthcare device, a machine that performs work, an anomaly detection device, a classification device;
Device according to claim 11 . A machine that performs work,
The data for model learning is data acquired in the work,
13. Device according to claim 12 . utilization of the first model is determined by the at least one model generator;
14. A device according to any one of claims 11-13 . The first model is a model generated by additional learning based on the transmitted data,
15. A device according to any one of claims 11-14 . the at least one processor sends the second model to the at least one model generator;
16. A device according to any one of claims 11-15 . the at least one processor sends information about the second model to the at least one model generator without sending the second model to the at least one model generator;
16. A device according to any one of claims 11-15 . by at least one processor
selecting at least one model from a plurality of models based on a score calculated using data obtained from the first device;
generating a first model for the first device by additionally learning the selected at least one model using model learning data included in data acquired from the first device;
transmitting the first model;
and run
The data for model learning is data acquired by the first device,
Model generation method. at least one processor,
selecting at least one model from a plurality of models based on a score calculated using data obtained from the first device;
generating a first model for the first device by additionally learning the selected at least one model using model learning data included in data acquired from the first device;
transmitting the first model;
A program that executes
The data for model learning is data acquired by the first device,
program. a device having at least one processor;
another device different from the device;
with
The at least one processor
selecting at least one model from a plurality of models based on a score calculated using data obtained from the other device;
generating a first model for the other device by additionally learning the selected at least one model using model learning data included in data acquired from the other device;
transmitting the first model;
and run
The data for model learning is data acquired by the other device,
model generation system. by at least one processor
transmitting data to at least one model generator;
receiving from the at least one model generator a first model generated based on the transmitted data;
generating a second model by additionally learning the first model using the acquired data;
and run
The transmitted data includes at least data for model learning,
Method. at least one processor,
transmitting data to at least one model generator;
receiving from the at least one model generator a first model generated based on the transmitted data;
generating a second model by additionally learning the first model using the acquired data;
A program that executes
The transmitted data includes at least data for model learning,
program. a device having at least one processor;
at least one model generator;
with
The at least one processor
transmitting data to the at least one model generator;
receiving from the at least one model generator a first model generated based on the transmitted data;
generating a second model by additionally learning the first model using the acquired data;
and run
The transmitted data includes at least data for model learning,
system.

Images