JP7615817B2 - Mathematical formula specification device, mathematical formula specification method, estimation device and estimation method - Google Patents

Description

The present invention relates to an apparatus and method for formulating the computational process of a trained model, and also to an apparatus and method for estimating the output of a trained model using such a formula.

The use of third-generation AI that uses deep neural networks (hereinafter referred to as "DNN") is being promoted in society. DNN is used, for example, for image recognition of still and video images, voice recognition, natural language processing such as machine translation, and anomaly detection processing that detects signs of abnormalities from time-series data from sensors attached to industrial equipment.

A DNN is a neural network that is compatible with deep learning and has a deeper network hierarchy. A DNN contains multiple layers, including an input layer, multiple hidden layers, and an output layer, and each layer is structured with multiple nodes connected by edges. Each layer has a function called an activation function, and edges can have weights. The output value of each node is calculated from the output value of the node in the previous layer that is connected to that node. In other words, the output value of each node is calculated from the output value of the node in the previous layer, the weight value of the connecting edge, and the activation function of the layer.

However, because DNNs have a multi-layered learning process for the perceptron layer, when an output value is obtained from an input value, humans may not understand why that output value was obtained. Because there are various types of calculation methods in DNNs and the content is complex, it is difficult to visualize the calculation process in a way that users can understand.

To solve this problem, for example, Patent Document 1 describes a technique for visualizing the calculation process of a DNN, in which a linear expression is obtained by applying a dimensionality reduction algorithm, such as a principal component analysis algorithm or a least squares method, to a group of output values. The technique described in Patent Document 1 allows humans to explicitly understand what calculations are being performed inside a DNN, so the user can know which parameters and weights to adjust the next time they use it.

JP 2019-46453 A (Published on March 22, 2019)

However, the method described in Patent Document 1 uses a dimensionality reduction algorithm, which is a linear algorithm, and the range of computational expression, mathematical formulation, and visualization is narrow, so it may be difficult to mathematically formulate and visualize the internal computational process of a DNN.

One aspect of the present invention was made in consideration of the above problems, and aims to realize a technology that mathematically converts the computational process of a trained model.

In order to solve the above problem, a mathematical formula identification device according to one aspect of the present invention includes one or more processors that execute an identification step. Also, a mathematical formula identification method according to one aspect of the present invention includes an identification step. The identification step is a step of identifying a nonlinear mathematical formula that represents the relationship between a numerical value representing input data of a trained model and a numerical value representing output data of the trained model.

The estimation device according to one aspect of the present invention includes one or more processors that execute a second estimation step. The estimation method according to one aspect of the present invention includes a second estimation step. The second estimation step is a step of estimating a numerical value representing the output data by substituting a numerical value representing the input data into the nonlinear mathematical formula identified by the above-mentioned mathematical formula identification method and performing a calculation.

According to a mathematical formula identification device and a mathematical formula identification method of one aspect of the present invention, the calculation process of a trained model can be mathematically formulated. Furthermore, according to an estimation device and an estimation method of one aspect of the present invention, the output of a trained model can be estimated using a mathematical formula identified by such a mathematical formula identification device and a mathematical formula identification method.

FIG. 1 is a diagram showing a configuration of an arithmetic formula system according to an embodiment of the present invention. 2 is a block diagram showing a configuration of a mathematical expression specifying device included in the mathematical expression generating system of FIG. 1 . 3 is a flowchart showing the flow of a mathematical formula specifying method implemented by the mathematical formula specifying device of FIG. 2 . FIG. 1 illustrates an example of a trained model and a genetic algorithm. 4 is a flowchart showing the flow of a specific step in FIG. 3 . 10 is a flowchart showing the flow of processing performed by the mathematical formula identification device. FIG. 13 is a diagram illustrating an example of the contents of a crossover. FIG. 1 illustrates the contents of a subtree mutation. FIG. 1 illustrates the content of a hoist mutation. FIG. 1 illustrates the contents of point mutations. FIG. 1 illustrates an example of a trained model and a genetic algorithm. 4 is a flowchart showing the flow of a specific step in FIG. 3 . FIG. 2 is a block diagram showing a configuration of an estimation device included in the arithmetic formula system of FIG. 1 . 14 is a flowchart showing the flow of an estimation method implemented by the estimation device of FIG. 13 . FIG. 2 is a block diagram showing a configuration of a machine learning device included in the arithmetic formula system of FIG. 1 . 16 is a flowchart showing the flow of a machine learning method implemented by the machine learning device of FIG. 15 . FIG. 2 is a diagram showing a configuration of a casting phase. FIG. 1 illustrates an example of a trained model and a genetic algorithm. FIG. 1 illustrates an example of a trained model and a genetic algorithm. FIG. 1 illustrates an example of a trained model and a genetic algorithm. FIG. 1 illustrates an example of a trained model and a genetic algorithm.

[Embodiment 1]
[Calculation Formula System]
An operation formula system S according to an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a diagram showing the configuration of the operation formula system S.

The arithmetic formula system S is a system that identifies a nonlinear formula that represents the relationship between the input and output of the trained model LM1. As shown in FIG. 1, the arithmetic formula system S includes a formula identification device 1, a machine learning device 2, and an estimation device 3.

The mathematical formula identification device 1 is a device that implements the mathematical formula identification method M1. The mathematical formula identification method M1 includes an estimation step M11 in which the mathematical formula identification device 1 estimates output data from input data using a learned model LM1 constructed by machine learning, and an identification step M12 in which the mathematical formula identification device 1 identifies a nonlinear mathematical formula f that represents the relationship between a numerical value representing the input data and a numerical value representing the output data by nonlinear regression using a genetic algorithm GA. As the learned model LM1, for example, an algorithm such as a neural network model such as a convolutional neural network or a recurrent neural network, a regression model such as a linear regression, or a tree model such as a regression tree can be used. The configuration of the mathematical formula identification device 1 and the flow of the mathematical formula identification method M1 will be described in detail below with reference to the drawings.

Various types of data can be used as input to the trained model LM1. Examples of input to the trained model LM1 include image data representing still or moving images, audio data representing audio, document data representing documents, and time-series data from sensors attached to industrial equipment. In addition, examples of input to the trained model LM1 include various parameters collected on a casting line.

Various data can be used as the output of the trained model LM1. The output of the trained model LM1 can be, for example, image recognition results, voice recognition results, machine translation results, and the presence or absence of abnormalities. In addition, the output of the trained model LM1 can be, for example, the target value for the CB value (compactability) of the molding sand in the casting line.

The mathematical expression identification device 1 includes a data collection device 111. The data collection device 111 temporarily holds input data to the learned model LM1 and output data from the learned model LM1, and inputs the held input data and output data to the next stage genetic algorithm GA. The data collection device 111 is, for example, a buffer equipped with information storage means such as a semiconductor memory or a hard disk, and input/output means. Note that, although the example of FIG. 1 illustrates a configuration in which the data collection device 111 is included in the mathematical expression identification device 1, the data collection device 111 may be configured as a device separate from the mathematical expression identification device 1. The data collection device 111 may be configured as a hardware unit, or may be configured as hardware and software.

The machine learning device 2 is a device for implementing the machine learning method M2. The machine learning method M2 is a method for constructing a learning dataset DS using input data and constructing a trained model LM1 by machine learning (supervised learning) using the learning dataset DS. Details of the configuration of the machine learning device 2 and the flow of the machine learning method M2 will be described later with reference to the accompanying drawings.

The estimation device 3 is a device for implementing the estimation method M3. The estimation method M3 includes a second estimation step in which the estimation device 3 estimates a numerical value representing the output data by substituting a numerical value representing the input data into the nonlinear mathematical formula f identified by the mathematical formula identification method M1 and performing a calculation. The configuration of the estimation device 3 and the flow of the estimation method M3 will be described in detail below with reference to the drawings.

The computational formula system S executes a learning phase, a formula specification phase, and an estimation phase. The learning phase, formula specification phase, and estimation phase can be briefly explained as follows.

(1) Learning Phase In the learning phase, a trained model LM1 is constructed. The machine learning device 2 creates training data from input data and output data, and adds the created training data to the training dataset DS. The machine learning device 2 also constructs the trained model LM1 by machine learning using the training data DS. The constructed trained model LM1 is transferred from the machine learning device 2 to the mathematical formula identification device 1.

(2) Mathematical Formula Identification Phase In the mathematical formula identification phase, the mathematical formula identification device 1 estimates output data based on input data using the learned model LM1. In addition, the mathematical formula identification device 1 identifies a nonlinear mathematical formula that represents the relationship between the numerical values representing the input data and the numerical values representing the output data of the learned model LM1 by nonlinear regression using a genetic algorithm.

(3) Estimation Phase In the estimation phase, the estimation device 3 estimates a numerical value representing the output data by performing a calculation by substituting a numerical value representing the input data into the mathematical formula identified by the mathematical formula identification device 1. That is, the estimation device 3 estimates the output data from the input data without using the learned model LM1.

[Configuration of the mathematical formula identification device]
The configuration of the mathematical expression identification device 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the mathematical expression identification device 1.

The mathematical expression identification device 1 is realized using a general-purpose computer, and includes a processor 11, a primary memory 12, a secondary memory 13, an input/output interface 14, a communication interface 15, and a bus 16. The processor 11, the primary memory 12, the secondary memory 13, the input/output interface 14, and the communication interface 15 are interconnected via the bus 16.

The secondary memory 13 stores the formula identification program P1 and the learned model LM1. The processor 11 expands the formula identification program P1 and the learned model LM1 stored in the secondary memory 13 onto the primary memory 12. The processor 11 then executes each step included in the formula identification method M1 according to the instructions included in the formula identification program P1 expanded onto the primary memory 12. The learned model LM1 expanded onto the primary memory 12 is used when the processor 11 executes the estimation step M11 (described later) of the formula identification method M1. The formula identification program P1 being stored in the secondary memory 13 means that the source code or an executable file obtained by compiling the source code is stored in the secondary memory 13. The learned model LM1 being stored in the secondary memory 13 means that parameters defining the learned model LM1 are stored in the secondary memory 13.

Devices that can be used as processor 11 include, for example, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a microcontroller, or a combination of these. Processor 11 is sometimes called a "computing device."

In addition, a device that can be used as the primary memory 12 can be, for example, a semiconductor RAM (Random Access Memory). The primary memory 12 is sometimes called a "main memory". In addition, a device that can be used as the secondary memory 13 can be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), an optical disk drive (ODD), a floppy (registered trademark) disk drive (FDD), or a combination thereof. The secondary memory 13 is sometimes called an "auxiliary memory". The secondary memory 13 may be built into the mathematical expression identification device 1, or may be built into another computer (for example, a computer constituting a cloud server) connected to the mathematical expression identification device 1 via the input/output interface 14 or the communication interface 15. In this embodiment, the storage in the mathematical expression identification device 1 is realized by two memories (the primary memory 12 and the secondary memory 13), but is not limited to this. In other words, the storage in the mathematical expression identification device 1 may be realized by one memory. In this case, for example, one storage area of the memory can be used as the primary memory 12, and another storage area of the memory can be used as the secondary memory 13.

An input device and/or an output device are connected to the input/output interface 14. Examples of the input/output interface 14 include interfaces such as USB (Universal Serial Bus), ATA (Advanced Technology Attachment), SCSI (Small Computer System Interface), and PCI (Peripheral Component Interconnect). Examples of input devices connected to the input/output interface 14 include a group of sensors provided in a casting line. Data acquired from the group of sensors in the formula identification method M1 is input to the formula identification device 1 and stored in the primary memory 12. Examples of input devices connected to the input/output interface 14 include a keyboard, a mouse, a touchpad, a microphone, or a combination of these. Data acquired from a user in the formula identification method M1 is input to the formula identification device 1 through these input devices and stored in the primary memory 12. Examples of output devices connected to the input/output interface 14 include a display, a projector, a printer, a speaker, a headphone, or a combination of these. Information provided to a user in the formula identification method M1 is output from the formula identification device 1 through these output devices. The mathematical expression identification device 1 may incorporate a keyboard that functions as an input device and a display that functions as an output device, like a laptop computer. Alternatively, the mathematical expression identification device 1 may incorporate a touch panel that functions as both an input device and an output device, like a tablet computer.

The communication interface 15 is connected to other computers via a network in a wired or wireless manner. Examples of the communication interface 15 include interfaces such as Ethernet (registered trademark) and Wi-Fi (registered trademark). Available networks include a PAN (Personal Area Network), a LAN (Local Area Network), a CAN (Campus Area Network), a MAN (Metropolitan Area Network), a WAN (Wide Area Network), a GAN (Global Area Network), or an internetwork including these networks. The internetwork may be an intranet, an extranet, or the Internet. Data (e.g., the trained model LM1) that the formula identification device 1 acquires from other computers (e.g., the machine learning device 2) in the formula identification method M1, and data that the formula identification device 1 provides to other computers in the formula identification method M1 are transmitted and received via these networks.

In this embodiment, a configuration is adopted in which the mathematical formula identification method M1 is executed using a single processor (processor 11), but the present invention is not limited to this. In other words, a configuration may be adopted in which the mathematical formula identification method M1 is executed using multiple processors. In this case, the multiple processors that cooperate to execute the mathematical formula identification method M1 may be provided in a single computer and configured to be able to communicate with each other via a bus, or may be provided in a distributed manner in multiple computers and configured to be able to communicate with each other via a network. As an example, a processor built in a computer that constitutes a cloud server and a processor built in a computer owned by a user of the cloud server may cooperate to execute the mathematical formula identification method M1.

In addition, in this embodiment, a configuration is adopted in which the learned model LM1 is stored in a memory (secondary memory 13) built into the same computer as the processor (processor 11) that executes the mathematical formula identification method M1, but the present invention is not limited to this. In other words, a configuration may be adopted in which the learned model LM1 is stored in a memory built into a computer different from the processor that executes the mathematical formula identification method M1. In this case, the computer with the built-in memory that stores the learned model LM1 is configured to be able to communicate with the computer with the built-in processor that executes the mathematical formula identification method M1 via a network. As an example, a mode can be considered in which the learned model LM1 is stored in a memory built into a computer that constitutes a cloud server, and a processor built into a computer owned by a user of the cloud server executes the mathematical formula identification method M1.

In addition, in this embodiment, a configuration is adopted in which the learned model LM1 is stored in a single memory (secondary memory 13), but the present invention is not limited to this. That is, a configuration may be adopted in which the learned model LM1 is distributed and stored in multiple memories. In this case, the multiple memories that store the learned model LM1 may be provided in a single computer (which may or may not be a computer with a built-in processor that executes the formula identification method M1), or may be distributed and provided in multiple computers (which may or may not include a computer with a built-in processor that executes the formula identification method M1). As an example, a configuration may be considered in which the learned model LM1 is distributed and stored in memories built into each of the multiple computers that make up the cloud server.

When the data collection device 111 is configured as a device separate from the mathematical expression identification device 1, the data collection device 111 includes, for example, a memory (not shown) and a processor (not shown).

[Flow of formula identification method]
The flow of the mathematical expression identification method M1 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the mathematical expression identification method M1. The mathematical expression identification method M1 includes an estimation step M11 and an identification step M12.

The estimation step M11 is a step in which the processor 11 estimates output data from input data using the learned model LM1. In the estimation step M11, the processor 11 obtains output data by inputting the input data to the learned model LM1. The input data input to the learned model LM1 and the output data output from the learned model LM1 are temporarily stored in the data collection device 111. The data stored in the data collection device 111 is used in the subsequent identification step M12.

In the identification step M12 in FIG. 3, the processor 11 identifies a nonlinear formula that expresses the relationship between the numerical values representing the input data and the numerical values representing the output data by nonlinear regression using the genetic algorithm GA. The data collection device 111 inputs the temporarily held input data and output data of the trained model LM1 in temporal synchronization with the genetic algorithm GA. In this way, the data collection device 111 acts as a kind of buffer that collects the data of the trained model LM1 (in certain information units) and inputs it to the next stage of the genetic algorithm GA.

The genetic algorithm GA is an algorithm that prepares multiple individuals i, each of which represents a candidate solution by genes, and searches for a solution by preferentially selecting an individual i with high fitness Di and repeating operations such as crossover and mutation. In this embodiment, the individual i is a nonlinear formula (relational equation) represented in a tree structure, and the operators and arguments contained in the formula are represented by the nodes of the tree. The fitness Di is given by a fitness function.

[Example 1 of trained model and genetic algorithm]
4 is a diagram illustrating a trained model LM1 and a genetic algorithm GA. The trained model LM1 is a deep neural network composed of multiple layers. The trained model LM1 includes multiple layers including an input layer LX to which input data is input, a hidden layer LY, and an output layer LZ to which output data is output. Each layer has a structure in which multiple nodes are connected by edges.

Each layer has a function called an activation function, and edges can have weights. The output value of each node is calculated from the output value of the node in the previous layer that is connected to that node. In other words, the output value of each node is calculated from the output value of the node in the previous layer, the weight value of the connecting edge, and the activation function of the layer.

In the example of FIG. 4, the input layer LX includes nodes X1, X2, X3, and X4. That is, in this example, the input data input to the trained model LM1 includes output values x1, x2, x3, and x4 of nodes X1, X2, X3, and X4.

The hidden layer LY includes nodes Y1, Y2, and Y3. The output values y1, y2, and y3 of nodes Y1, Y2, and Y3 are calculated from the output values x1, x2, x3, and x4 of nodes in the input layer LX, which is the layer preceding the hidden layer LY.

The output layer LZ includes node Z1. The output value z1 of node Z1 is calculated from the output values y1, y2, and y3 of the nodes in the hidden layer LY, which is the layer before the output layer LZ. That is, in this example, the output data of the trained model LM1 is the output value z1 of node Z1.

The data collection device 111 temporarily holds the input data to the learned model LM1 and the output data of the learned model LM1, and sequentially inputs the input data and output data to the next stage of the genetic algorithm GA.

The genetic algorithm GA includes a first generation G1 to a fourth generation G4. In the example of Fig. 4, a nonlinear formula which is an individual i is expressed by, for example, the following formula (1).
z1=f ₀ (x1, x2, x3, x4)...(1)

In equation (1), z1 is a numerical value representing the output value of node Z1 of learned model LM1, i.e., the output data. x1, x2, x3, and x4 are numerical values representing the output values of each node of input layer LX, i.e., the input data. That is, equation (1) is an equation that represents the relationship between the output values x1, x2, x3, and x4 of each node of input layer LX and the output value z1 of node Z1 of output layer LZ. As shown in equation (1), processor 11 identifies a nonlinear equation by nonlinear regression using a genetic algorithm that treats a nonlinear equation (relational equation) representing the relationship between input data and output data as a candidate solution.

Figure 5 is a flow chart illustrating the flow of the identification step M12 executed by the processor 11. The processor 11 executes the identification step M12 using a specified module (hereinafter referred to as "Module A"). Module A is a module that executes a genetic algorithm. In Module A, the processor 11 starts by creating a population of simple random formulas that represent the relationships between known independent variables and their dependent variable targets in order to predict new data. The processor 11 then evolves the population to generate the next generation of the population by selecting the most fit individuals from the population to be subjected to genetic manipulation. Through the above operations, a formula that best represents the above relationships is identified.

In this specific example, a module that executes genetic programming is used as module A. Genetic programming is an extension of the genetic algorithm, and uses a tree structure to represent gene types. Note that the flow of the identification step M12 shown in FIG. 5 is an example, and the method of identifying a formula using the genetic algorithm GA is not limited to the method shown in FIG. 5. Various other methods can be adopted as a method of identifying a formula using the genetic algorithm GA.

In step M121, the processor 11 acquires input data and output data of the learned model LM1 from the data collection device 111. The input data includes output values x1, x2, x3, and x4 of each node of the input layer LX of the learned model LM1. The output data includes the value z1 of node Z1 in the output layer LZ.

In step M122, the processor 11 acquires parameters to be used in the genetic algorithm GA. The parameters include, for example, the number of individuals to be generated N, the tournament size Nt, the crossover probability Pc, the mutation probability Pms, the number of evolutionary generations Ng, the operator Oj to be used in the syntax tree, the maximum depth d of the syntax tree, and the event occurrence probabilities Pk1 to Pk5. The values of each parameter are input to the mathematical formula identification device 1, for example, by the user.

The number of individuals to be generated N represents the number of individuals i to be included in the set. The tournament size Nt is the number of individuals i to be randomly selected from the set of the current generation. The mutation probability Pms is the probability that a gene will mutate. Operators Oi used in the syntax tree are, for example, Max, Min, sqrt (root), log (natural logarithm), +, -, ×, ÷, sin (radians), cos (radians), tan (radians), abs, neg, and inv. Max is an operator that selects the maximum value. Min is an operator that selects the minimum value. neg is an operator that makes the sign negative. inv is an operator that sets arguments close to zero to 0.

The event occurrence probabilities Pk1 to Pk5 are the probabilities that operations m1 to m5 are selected as operations for evolving the next generation set. The processor 11 evolves the next generation set using one of the operations m1 to m5. The sum of the event occurrence probabilities Pk1 to Pk5 is 1. As an example, the values of the event occurrence probabilities Pk1, Pk2, Pk3, Pk4, and Pk5 are "0.1", "0.2", "0.3", "0.4", and "0.1", respectively. Operations m1 to m5 will be described later with reference to different drawings.

In step M123, processor 11 randomly generates N individuals i based on the specified parameters (operator Oi to be used in the syntax tree, maximum depth d of the syntax tree, etc.) and generates a set of N individuals i that will become the initial current generation.

In step M124, the processor 11 calculates the fitness Di of each individual i included in the set of the current generation. The fitness Di is given by a fitness function.

In step M125, processor 11 randomly selects individuals i of tournament size Nt from the current generation set, selects the individual i with the highest fitness Di among them, and adds it to the next generation set. The individual i selected in step M125, i.e., the individual i added to the next generation set, is also called the "winner tree."

Processor 11 repeats the process of step M125 until the number of individuals in the next generation becomes N, the same as the current generation, that is, while the number of individuals in the next generation has not yet reached N (step M126; NO). When the number of individuals in the next generation reaches N (step M126; YES), processor 21 executes the process of step M127.

In step M127, processor 11 executes a process to evolve the next generation set. Details of step M127 will be described later with reference to a different drawing.

In step M130, processor 11 overwrites the next-generation set onto the current-generation set. In step M131, processor 11 determines whether the number of evolutionary generations Ng has been reached. If the number of evolutionary generations Ng has not been reached (step M131; NO), processor 11 returns to the processing of step M124. On the other hand, if the number of evolutionary generations Ng has been reached (step M131; YES), processor 11 proceeds to the processing of step M132.

In step M132, processor 11 identifies an individual i with the highest fitness Di from among the individuals i included in the set of the current generation. In step M133, processor 11 outputs the identified individual i as the optimal solution. For example, processor 11 displays on a display a formula that is the selected individual i, or a figure (such as a graph) that represents the formula.

FIG. 6 is a flow chart illustrating the flow of step M127 executed by the processor 11. In step M201, the processor 11 selects one of operations m1 to m5 based on the event occurrence probabilities Pk1 to Pk5 set by the user. If operation m1 is selected (step M201; "operation m1"), the processor 11 proceeds to processing of step M202. If operation m2 is selected (step M201; "operation m2"), the processor 11 proceeds to processing of step M211. If operation m3 is selected (step M201; "operation m3"), the processor 11 proceeds to processing of step M221. If operation m4 is selected (step M201; "operation m4"), the processor 11 proceeds to processing of step M231. If operation m5 is selected (step M201; "operation m5"), the processor 11 ends the processing.

Operation m1 is a crossover. Crossover is a method of mixing genetic material between individuals. In the case of crossover, in step M202, processor 11 randomly selects subtrees in each winning tree for the winning trees in the next generation set.

In step M203, processor 11 generates a next-generation set for donors. The processing content of step M203 is the same as the processing content of steps M123 to M125 in FIG. 6. That is, processor 11 first randomly generates N individuals i based on individual parameters specified by the user, and generates a set of N individuals i (hereinafter referred to as the "donor set"). Next, processor 11 calculates the fitness Di of each individual i included in the donor set. Next, processor 11 randomly extracts individuals i in a number equal to the tournament size Nt from the donor set, selects the individual i with the highest fitness Di among them, and adds it to the donor set of the next generation. The individual i selected by this processing is also referred to as the "donor tree". Processor 11 repeats the donor tree selection processing until the number of donor trees of the next generation reaches N.

In step M204, processor 11 randomly selects a subtree included in the donor tree (hereinafter referred to as the "donor subtree").

In step M205, processor 11 replaces the subtrees in the winning tree. In this embodiment, processor 11 removes the subtree selected in step M202 from the winning tree, and transplants the donor subtree selected in step M203 into the location where the subtree was. In other words, processor 11 replaces the subtree included in the winning tree with the donor subtree. The winning tree with the replaced subtree becomes the descendant (individual) of the next generation.

Operation m2 is an operation for mutating a subtree. By mutating a subtree, extinct functions and operators can be reintroduced into the population and diversity can be maintained. In this case, in step M211, processor 11 randomly selects a subtree to be included in the winning tree.

In step M212, processor 11 randomly generates a subtree. In step M213, processor 11 replaces the subtrees in the winning tree. In this embodiment, processor 11 removes the subtree selected in step M202 from the winning tree, and transplants the subtree generated in step M212 into the location of the removed subtree. In other words, processor 11 replaces the subtree included in the winning tree with the subtree generated in step M212. The winning tree with the replaced subtree becomes the descendant (individual) of the next generation.

Operation m3 is a hoist mutation. Hoist mutation is a mutation operation that fights tree bloat. In step M221, the processor randomly selects a subtree to be included in the winning tree. In step M222, processor 11 randomly selects a subtree to be included in the subtree selected in step M221.

In step M223, processor 11 rolls up the subtree selected in step M222 to the position of the original subtree (the subtree selected in step M221). The winning tree into which this subtree has been rolled up becomes the descendant (individual) of the next generation.

Operation m4 is a point mutation. Point mutation is an operation that reintroduces extinct formulas and operators into a population in order to maintain diversity. In step M231, processor 11 randomly selects a node of the winning tree. In step M232, processor 11 replaces the node selected in step M231 with another node. As a result, the formula represented by the winning tree is replaced with another formula that requires the same number of arguments as the original node. The winning tree obtained by the replacement becomes the descendant (individual) of the next generation.

Operation m5 is regeneration, in which the winning tree is replicated and included in the next generation without modification.

Figures 7 to 10 are diagrams illustrating the contents of operations performed on the winning tree. Figure 7 is a diagram illustrating the contents of operation m1 (crossover). In the example of Figure 7, subtree tr111 of winner tree tr11 is replaced with subtree tr121 of donor tree tr12, resulting in winner tree tr13. Winner tree tr13 becomes the descendant (individual) of the next generation.

Figure 8 is a diagram illustrating the contents of operation m2 (subtree mutation). In the example of Figure 8, subtree tr111 of winner tree tr11 is replaced with subtree tr22 to become winner tree tr23. Winner tree tr23 becomes the descendant (individual) of the next generation.

Figure 9 is a diagram illustrating the contents of operation m3 (hoist mutation). In the example of Figure 9, subtree tr1121 of winner tree tr11 is rolled up to the position of subtree tr112, becoming winner tree tr31. Winner tree tr31 becomes the descendant (individual) of the next generation.

Figure 10 is a diagram illustrating the contents of operation m4 (point mutation). In the example of Figure 10, nodes n21 and n34 included in the winner tree tr11 are replaced with nodes n421 and n434, resulting in a winner tree tr41. The winner tree tr41 becomes the descendant (individual) of the next generation.

[Specific example 2 of trained model and genetic algorithm]
11 is a diagram showing another example of a trained model LM1 and a genetic algorithm GA. In the example of FIG. 11, an individual i of the genetic algorithm GA includes, for example, the following four equations (2) to (5).
z1=f ₁ (y1, y2, y3)...(2)
y1=f ₂ (x1, x2, x3, x4)...(3)
y2=f ₃ (x1, x2, x3, x4)...(4)
y3=f ₄ (x1, x2, x3, x4)...(5)

Equations (3) to (5) are nonlinear equations (first equation) that express the relationship between the output values x1, x2, x3, and x4 of each node in the input layer LX and the output values y1, y2, and y3 of each node in the hidden layer LY. Equation (2) is a nonlinear equation (second equation) that expresses the relationship between the value z1 of node Z1 in the output layer LZ and the output values y1, y2, and y3 of each node in the hidden layer LY.

That is, in this example, the processor 11 identifies the first and second formulas by nonlinear regression using a genetic algorithm with a first nonlinear formula that represents the relationship between a numerical value representing the input data and each output value of all nodes belonging to the hidden layer LY1, and a second nonlinear formula that represents the relationship between each output value of each node of the hidden layer LY1 and a numerical value representing the output data as solution candidates.

FIG. 12 is a flowchart illustrating the flow of specific step M12 executed by processor 11 in the example of FIG. 11. The flowchart of FIG. 12 executes steps M151, M152, and M153 instead of steps M121, M123, and M132 of the flowchart of FIG. 5.

In step M151, the processor 11 acquires input data and output data of the learned model LM1. The acquired input data includes the output values x1, x2, and x3 of each node of the input layer LX. The output data includes the output value z1 of the node Z1 of the output layer LZ, as well as the output values y1, y2, and y3 of each node of the hidden layer LY.

In step M152, processor 11 randomly generates N individuals i based on specified parameters (operator Oi to be used in the syntax tree, maximum depth d of the syntax tree, etc.), and generates a set of N individuals i that will become the initial current generation. Individual i includes the four formulas (2) to (5) described above. After completing the processing of step M152, processor 11 proceeds to the processing of step M124.

The processor 11 executes the processing of steps M124 to M131. As a result, the optimal solution (equations (2) to (5)) is obtained by the genetic algorithm. After completing the processing of step M131, the processor 11 proceeds to the processing of step M153.

In step M153, the processor 11 identifies a nonlinear formula expressing the relationship between the input data and the output data using the first formula (formulas (3) to (5)) and the second formula (formula (2)) identified by the genetic algorithm. In this example, the processor 11 identifies a formula expressing the relationship between the input data and the output data of the trained model LM1 by solving the simultaneous equations (2) to (5) included in the individual i selected as the optimal solution. The formula expressing the relationship between the input data and the output data is expressed, for example, by the above-mentioned formula (1). In step S133, the processor 11 outputs the identified formula, for example, by displaying it on a display.

As described above, according to this embodiment, the mathematical formula identification device 1 identifies a nonlinear mathematical formula that represents the relationship between the numerical values representing the data input to the learned model LM1 and the numerical values representing the output data output from the learned model LM1 by nonlinear regression using a genetic algorithm GA. The user can easily understand the calculation process of the learned model by the nonlinear mathematical formula or a figure such as a graph represented by this mathematical formula.

[Configuration of the Estimation Device]
The configuration of the estimation device 3 will be described with reference to Fig. 13. Fig. 13 is a block diagram showing the configuration of the estimation device 3.

The estimation device 3 is realized using a general-purpose computer, and includes a processor 31, a primary memory 32, a secondary memory 33, an input/output interface 34, a communication interface 35, and a bus 36. The processor 31, the primary memory 32, the secondary memory 33, the input/output interface 34, and the communication interface 35 are interconnected via the bus 36.

An estimation program P3 is stored in the secondary memory 33. The processor 21 expands the estimation program P3 stored in the secondary memory 23 onto the primary memory 32. The processor 31 then executes each step included in the estimation method M3 (an example of a second estimation step) according to instructions included in the estimation program P3 expanded onto the primary memory 32. The formula f stored in the secondary memory 33 is used in the second estimation step M31 (described below) of the estimation method M3. Note that the estimation program P3 being stored in the secondary memory 33 refers to the source code, or an executable file obtained by compiling the source code, being stored in the secondary memory 33.

Devices that can be used as processor 31 include, for example, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a microcontroller, or a combination of these. Processor 31 is sometimes called a "computing device."

In addition, a device that can be used as the primary memory 32 can be, for example, a semiconductor RAM (Random Access Memory). The primary memory 32 is sometimes called a "main memory". In addition, a device that can be used as the secondary memory 33 can be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), an optical disk drive (ODD), a floppy disk drive (FDD), or a combination thereof. The secondary memory 33 is sometimes called an "auxiliary memory". The secondary memory 33 may be built into the estimation device 3, or may be built into another computer (for example, a computer constituting a cloud server) connected to the estimation device 3 via the input/output interface 34 or the communication interface 35. In this embodiment, the storage in the estimation device 3 is realized by two memories (the primary memory 32 and the secondary memory 33), but is not limited to this. In other words, the storage in the estimation device 3 may be realized by one memory. In this case, for example, one storage area of the memory can be used as the primary memory 32, and another storage area of the memory can be used as the secondary memory 33.

An input device and/or an output device are connected to the input/output interface 34. Examples of the input/output interface 34 include interfaces such as USB (Universal Serial Bus), ATA (Advanced Technology Attachment), SCSI (Small Computer System Interface), and PCI (Peripheral Component Interconnect). Examples of input devices connected to the input/output interface 34 include a group of sensors provided in the casting line. Data acquired from the group of sensors in the estimation method M3 is input to the estimation device 3 and stored in the primary memory 32. Examples of input devices connected to the input/output interface 34 include a keyboard, a mouse, a touchpad, a microphone, or a combination of these. Data acquired from the user in the estimation method M3 is input to the estimation method M3 via these input devices and stored in the primary memory 32. Examples of output devices connected to the input/output interface 34 include a display, a projector, a printer, a speaker, a headphone, or a combination of these. Information provided to the user in the estimation method M3 is output from the estimation device 3 via these output devices. The estimation device 3 may incorporate a keyboard that functions as an input device and a display that functions as an output device, like a laptop computer. Alternatively, the estimation device 3 may incorporate a touch panel that functions as both an input device and an output device, like a tablet computer.

The communication interface 35 is connected to other computers via a network in a wired or wireless manner. Examples of the communication interface 35 include interfaces such as Ethernet (registered trademark) and Wi-Fi (registered trademark). Available networks include a PAN (Personal Area Network), a LAN (Local Area Network), a CAN (Campus Area Network), a MAN (Metropolitan Area Network), a WAN (Wide Area Network), a GAN (Global Area Network), or an internetwork including these networks. The internetwork may be an intranet, an extranet, or the Internet. Data (e.g., formula f) provided to the estimation device 3 from other computers (e.g., the formula identification device 1) is transmitted and received via these networks.

In this embodiment, a configuration is adopted in which the estimation method M3 is executed using a single processor (processor 31), but the present invention is not limited to this. That is, a configuration in which the estimation method M3 is executed using multiple processors may be adopted. In this case, the multiple processors that cooperate to execute the estimation method M3 may be provided in a single computer and configured to be able to communicate with each other via a bus, or may be provided in a distributed manner in multiple computers and configured to be able to communicate with each other via a network. As an example, a processor built in a computer that constitutes a cloud server and a processor built in a computer owned by a user of the cloud server may cooperate to execute the estimation method M3.

In addition, in this embodiment, a configuration is adopted in which formula f is stored in a memory (secondary memory 33) built into the same computer as the processor (processor 31) that executes estimation method M3, but the present invention is not limited to this. That is, a configuration may be adopted in which formula f is stored in a memory built into a computer different from the processor that executes estimation method M3. In this case, the computer with the built-in memory that stores formula f is configured to be able to communicate with the computer with the built-in processor that executes estimation method M3 via a network. As an example, a mode can be considered in which formula f is stored in a memory built into a computer that constitutes a cloud server, and a processor built into a computer owned by a user of the cloud server executes estimation method M3.

In addition, in this embodiment, a configuration is adopted in which formula f is stored in a single memory (secondary memory 33), but the present invention is not limited to this. That is, a configuration may be adopted in which formula f is distributed and stored in multiple memories. In this case, the multiple memories that store formula f may be provided in a single computer (which may or may not be a computer with a built-in processor that executes estimation method M3), or may be distributed and provided in multiple computers (which may or may not include a computer with a built-in processor that executes estimation method M3). As an example, a configuration may be considered in which formula f is distributed and stored in memories built into each of the multiple computers that make up the cloud server.

[Flow of estimation method]
The flow of the estimation method M3 will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the flow of the estimation method M3. The estimation method M3 includes a second estimation step M31 and an output step M32.

The second estimation step M31 is a step in which the processor 11 estimates a numerical value representing the output data by substituting a numerical value representing the input data into the nonlinear formula identified by the estimation device 3 and performing a calculation. The output data estimated in the second estimation step M31 is data that is estimated to be output when the input data is input to the learned model LM1.

The output step M32 is a step in which the processor 11 outputs the estimated output data. The processor 11 outputs the output data, for example, by displaying an image representing the output data on a display.

According to this embodiment, the estimation device 3 estimates the output of the learned model LM1 using a nonlinear formula identified by nonlinear regression using the genetic algorithm GA. In other words, the estimation device 3 can estimate the output of the learned model LM1 from the input data without using the learned model LM1. Therefore, for example, even if the calculation device of the estimation device 3 does not support advanced calculation processing, the estimation device 3 can estimate the output result of the learned model LM1.

In addition, in this embodiment, the data collection device 111 buffers the input data and output data of the learned model LM1 and inputs them to the genetic algorithm GA with temporal synchronization. Since it is possible to achieve temporal synchronization between the output from the learned model LM1 and the input to the genetic algorithm GA, it is possible to predict the time until the next stage of the genetic algorithm GA outputs the final solution.

In addition, in this embodiment, the data collection device 111 acts as a kind of buffer that collects data of the learned model LM1 (in certain information units) and inputs it to the next stage of the genetic algorithm GA. This makes it possible to reduce wasted memory space and wasted calculation processing in the generation calculation processing of the next stage of the genetic algorithm GA, and to make the generation calculation processing of the genetic algorithm more efficient.

[Configuration of machine learning device]
The configuration of the machine learning device 2 will be described with reference to Fig. 15. Fig. 15 is a block diagram showing the configuration of the machine learning device 2.

The machine learning device 2 is realized using a general-purpose computer, and includes a processor 21, a primary memory 22, a secondary memory 23, an input/output interface 24, a communication interface 25, and a bus 26. The processor 21, the primary memory 22, the secondary memory 23, the input/output interface 24, and the communication interface 25 are interconnected via the bus 26.

The secondary memory 23 stores the machine learning program P2 and the learning dataset DS. The learning dataset DS is a collection of teacher data DS1, DS2, etc. The processor 21 expands the machine learning program P2 stored in the secondary memory 23 onto the primary memory 22. Then, the processor 21 executes each step included in the machine learning method M2 according to the instructions included in the machine learning program P2 expanded onto the primary memory 22. The learning dataset DS stored in the secondary memory 23 is constructed in a learning dataset construction step M21 (described later) of the machine learning method M2, and is used in a trained model construction step M22 (described later) of the machine learning method M2. In addition, the trained model LM1 constructed in the trained model construction step M22 of the machine learning method M2 is also stored in the secondary memory 23. Note that the machine learning program P2 being stored in the secondary memory 23 refers to the source code, or the executable file obtained by compiling the source code, being stored in the secondary memory 23. In addition, the learned model LM1 being stored in the secondary memory 23 means that the parameters that define the learned model LM1 are stored in the secondary memory 23.

Devices that can be used as the processor 21 include, for example, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a microcontroller, or a combination of these. The processor 21 is sometimes called a "computing device."

In addition, a device that can be used as the primary memory 22 can be, for example, a semiconductor RAM (Random Access Memory). The primary memory 22 is sometimes called a "main memory". In addition, a device that can be used as the secondary memory 23 can be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), an optical disk drive (ODD), a floppy disk drive (FDD), or a combination thereof. The secondary memory 23 is sometimes called an "auxiliary memory". The secondary memory 23 may be built into the machine learning device 2, or may be built into another computer (for example, a computer constituting a cloud server) connected to the machine learning device 2 via the input/output interface 24 or the communication interface 25. In this embodiment, the memory in the machine learning device 2 is realized by two memories (the primary memory 22 and the secondary memory 23), but is not limited to this. In other words, the memory in the machine learning device 2 may be realized by one memory. In this case, for example, one storage area of the memory can be used as the primary memory 22, and another storage area of the memory can be used as the secondary memory 23.

An input device and/or an output device are connected to the input/output interface 24. Examples of the input/output interface 24 include interfaces such as USB (Universal Serial Bus), ATA (Advanced Technology Attachment), SCSI (Small Computer System Interface), and PCI (Peripheral Component Interconnect). Examples of input devices connected to the input/output interface 24 include a group of sensors provided in the casting line. Data acquired from the group of sensors in the machine learning method M2 is input to the machine learning device 2 and stored in the primary memory 22. Examples of input devices connected to the input/output interface 24 include a keyboard, a mouse, a touchpad, a microphone, or a combination thereof. Data acquired from a user in the machine learning method M2 is input to the machine learning device 2 via these input devices and stored in the primary memory 22. Examples of output devices connected to the input/output interface 24 include a display, a projector, a printer, a speaker, a headphone, or a combination thereof. Information provided to the user in the machine learning method M2 is output from the machine learning device 2 via these output devices. The machine learning device 2 may incorporate a keyboard that functions as an input device and a display that functions as an output device, like a laptop computer. Alternatively, the machine learning device 2 may incorporate a touch panel that functions as both an input device and an output device, like a tablet computer.

Other computers are connected to the communication interface 25 via a network in a wired or wireless manner. Examples of the communication interface 25 include interfaces such as Ethernet (registered trademark) and Wi-Fi (registered trademark). Available networks include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a global area network (GAN), or an internetwork including these networks. The internetwork may be an intranet, an extranet, or the Internet. Data (e.g., the trained model LM1) provided by the machine learning device 2 to other computers (e.g., the mathematical formula identification device 1) is transmitted and received via these networks.

In this embodiment, a configuration is adopted in which the machine learning method M2 is executed using a single processor (processor 21), but the present invention is not limited to this. That is, a configuration may be adopted in which the machine learning method M2 is executed using multiple processors. In this case, the multiple processors that cooperate to execute the machine learning method M2 may be provided in a single computer and configured to be able to communicate with each other via a bus, or may be provided in a distributed manner in multiple computers and configured to be able to communicate with each other via a network. As an example, a processor built into a computer that constitutes a cloud server and a processor built into a computer owned by a user of the cloud server may cooperate to execute the machine learning method M2.

In addition, in this embodiment, a configuration is adopted in which the training dataset DS is stored in a memory (secondary memory 23) built into the same computer as the processor (processor 21) that executes the machine learning method M2, but the present invention is not limited to this. That is, a configuration may be adopted in which the training dataset DS is stored in a memory built into a computer other than the processor that executes the machine learning method M2. In this case, the computer with the built-in memory that stores the training dataset DS is configured to be able to communicate with the computer with the built-in processor that executes the machine learning method M2 via a network. As an example, a configuration is conceivable in which the training dataset DS is stored in a memory built into a computer that constitutes a cloud server, and a processor built into a computer owned by a user of the cloud server executes the machine learning method M2.

In addition, in this embodiment, a configuration is adopted in which the training dataset DS is stored in a single memory (secondary memory 23), but the present invention is not limited to this. That is, a configuration may be adopted in which the training dataset DS is distributed and stored in multiple memories. In this case, the multiple memories that store the training dataset DS may be provided in a single computer (which may or may not be a computer with a built-in processor that executes the machine learning method M2), or may be distributed and provided in multiple computers (which may or may not include a computer with a built-in processor that executes the machine learning method M2). As an example, a configuration can be considered in which the training dataset DS is distributed and stored in memories built into each of the multiple computers that make up the cloud server.

In addition, in this embodiment, a configuration is adopted in which the formula identification method M1, the machine learning method M2, and the estimation method M3 are executed using different processors (processor 11, processor 21, and processor 31), but the present invention is not limited to this. That is, the formula identification method M1, the machine learning method M2, and the estimation method M3 may be executed using the same processor. In this case, by executing the machine learning method M2, the learned model LM1 is stored in a memory built into the same computer as this processor. Then, when executing the formula identification method M1, this processor uses the learned model LM1 stored in this memory to identify the formula. Furthermore, when executing the estimation method M3, this processor estimates output data using the identified formula.

[Machine learning method flow]
The flow of the machine learning method M2 will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the flow of the machine learning method M2.

The machine learning method M2 includes a learning dataset construction step M21 and a trained model construction step M22.

The learning dataset construction step M21 is a step in which the processor 21 constructs a learning dataset DS, which is a collection of teacher data DS1, DS2, ....

Each teacher data DSi (i = 1, 2, ...) includes first data and second data. The first data included in the teacher data DSi is the same as the input data input to the trained model LM1. In the learning dataset construction step M21, the processor 21 acquires this data in a similar manner to the mathematical formula identification device 1. The teacher data DSi also includes second data as a label. In the learning dataset construction step M21, the processor 21 associates the first data with the second data and stores them in the secondary memory 23. The processor 21 repeats the above process to construct the learning dataset DS.

The trained model construction step M22 is a step in which the processor 21 constructs the trained model LM1. In the trained model construction step M22, the processor 21 constructs the trained model LM1 by supervised learning using the training dataset DS. Then, the processor 21 stores the constructed trained model LM1 in the secondary memory 23.

Example 1
The arithmetic formula system S according to this embodiment is applied to, for example, a casting system. The casting phase C1 will be described with reference to Fig. 17. Fig. 17 is a diagram showing the configuration of the casting phase C1 in the casting system.

The casting phase C1 is a phase in which casting is performed using casting sand. For example, as shown in FIG. 17, the casting phase C1 can be composed of a kneading process C11, a molding process C12, a demolding process C13, a mold coating process C14, a frame alignment process C15, a pouring process C16, a cooling process C17, and a frame removal process C18.

The kneading process C11 is a process in which additives including resin and a hardener are added to the molding sand and kneaded. The molding process C12 is a process in which a mold is formed by filling a flask with the molding sand kneaded in the kneading process C11. Here, an upper mold corresponding to the upper part of the mold and a lower mold corresponding to the lower part of the mold are each formed. The mold removal process C13 is a process in which the upper mold and the lower mold formed in the mold removal process C12 are separated from the flask. The mold coating process C14 is a process in which a mold wash is applied to the product surfaces of the upper mold and the lower mold removed in the mold removal process C13. The frame alignment process C15 is a process in which the upper mold and the lower mold to which the mold wash has been applied in the mold coating process C14 are combined to obtain a mold. The pouring process C16 is a process in which molten metal is poured into the mold obtained in the frame alignment process C15. The cooling process C17 is a process for cooling the molten metal poured into the mold in the pouring process C16. The cooled molten metal solidifies inside the mold to become a casting. The flask-removing process C18 is a process for vibrating the mold to dismantle it into a sand block and remove the casting obtained in the cooling process C17.

The sand blocks obtained in the casting phase C1 are recycled into casting sand through, for example, a crushing process, a separation process, a pre-recycling cooling process, a sand recycling process, etc.

In the casting system according to this embodiment, the input data in the formula determined using the genetic algorithm includes, for example, sensor output values detected by various sensors installed on the casting line. The output data includes, for example, the CB value, mold strength, a label indicating whether the mold is chipped, and a label indicating whether the product is good or bad.

According to this embodiment, for example, the formula identification device 1 identifies a nonlinear formula expressing the relationship between the sensor output value input to the learned model LM1 and the CB value output from the learned model LM1 by nonlinear regression using a genetic algorithm GA. The user can easily grasp the relationship between the sensor output value and the CB value by using this nonlinear formula or a figure such as a graph represented by this formula.

Example 2
The arithmetic formula system S according to this embodiment is applied to, for example, a sales forecasting system. In the sales forecasting system, the input data in the formula specified using a genetic algorithm includes, for example, condition data indicating temperature, day of the week, traffic volume, population, number of households, store area, unit price, number of products available, distance from a station, advertising expenses, SNS hashtag statistics, keyword search statistics, number of people infected with infectious diseases, number of foreign visitors to Japan, exchange rates, stock prices, interest rates, prices, employment statistics, the Bank of Japan Tankan, and household survey. The output data includes, for example, data indicating sales amounts.

According to this embodiment, for example, the formula identification device 1 identifies a nonlinear formula that expresses the relationship between the condition data input to the trained model LM1 and the sales amount output from the trained model LM1 by nonlinear regression using a genetic algorithm GA. The user can easily understand the relationship between the conditions and sales in the sales forecasting system by using this nonlinear formula or a figure such as a graph represented by this formula.

Example 3
The arithmetic formula system S according to the present embodiment is applied to, for example, a drug discovery system. In the drug discovery system, input data in a formula specified using a genetic algorithm includes, for example, molecular model parameters and data indicating the binding affinity of each component. Output data includes, for example, data indicating a drug.

According to this embodiment, for example, the formula identification device 1 identifies a nonlinear formula that expresses the relationship between the input data input to the learned model LM1 and the data indicating the drug output from the learned model LM1 by nonlinear regression using a genetic algorithm GA. The user can easily understand the relationship between the input data and the drug by using this nonlinear formula or a figure such as a graph represented by this formula.

Example 4
The mathematical formula system S according to the present embodiment is applied to, for example, a material development system. In the material development system, input data in a mathematical formula specified using a genetic algorithm includes, for example, the mixing ratio and physical property values of alloy components. Output data includes, for example, characteristic data representing the characteristics (strength, etc.) of the material to be produced.

According to this embodiment, for example, the formula identification device 1 identifies a nonlinear formula expressing the relationship between the mixture ratio and physical property values input to the learned model LM1 and the characteristic data output from the learned model LM1 by nonlinear regression using a genetic algorithm GA. The user can easily grasp the relationship between the mixture ratio and physical property values and the characteristic data by using this nonlinear formula or a figure such as a graph represented by this formula.

Example 5
The arithmetic formula system S according to the present embodiment is applied to, for example, an image recognition system. In the image recognition system, input data in a formula specified using a genetic algorithm includes, for example, data indicating image feature quantities. Output data includes, for example, data indicating image recognition results.

According to this embodiment, for example, the formula identification device 1 identifies a nonlinear formula expressing the relationship between the feature amount input to the trained model LM1 and the image recognition result output from the trained model LM1 by nonlinear regression using a genetic algorithm GA. The user can easily grasp the relationship between the feature amount of the image and the image recognition result by using this nonlinear formula or a figure such as a graph represented by this formula.

Example 6
The arithmetic formula system S according to the present embodiment is applied to, for example, an equipment malfunction prediction system. In the equipment malfunction prediction system, input data in the formula specified using the genetic algorithm includes, for example, detection values detected by each sensor installed in the equipment. Output data includes, for example, data indicating an estimation result of whether or not a malfunction of the equipment occurs.

According to this embodiment, for example, the formula identification device 1 identifies a nonlinear formula that expresses the relationship between the detection value input to the learned model LM1 and the estimated result of the occurrence or non-occurrence of a malfunction output from the learned model LM1 by nonlinear regression using a genetic algorithm GA. The user can easily grasp the relationship between the detection value of the sensor and the occurrence or non-occurrence of a malfunction in the equipment by using this nonlinear formula or a figure such as a graph represented by this formula.

[Embodiment 2]
Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

The computation formula system according to this embodiment differs from the computation formula system S according to the above-described first embodiment in the structure of the trained model. The device configurations of the mathematical formula identification device 1, the machine learning device 2, and the estimation device 3 according to this embodiment are similar to the device configurations of the mathematical formula identification device 1, the machine learning device 2, and the estimation device 3 according to the above-described first embodiment.

While the trained model LM1 in the above-mentioned first embodiment includes one hidden layer LY, the trained model LM2 in this embodiment includes multiple hidden layers LY1 and LY2.

[Example 3 of trained model and genetic algorithm]
18 is a diagram illustrating a trained model LM2 and a genetic algorithm GA according to the second embodiment. In the diagram, the trained model LM2 includes an input layer LX, hidden layers LY1 and LY2, and an output layer LZ. Each layer has a structure in which a plurality of nodes are connected by edges. The trained model LM2 includes two hidden layers LY1 and LY2.

The input layer LX includes nodes X1, X2, X3, and X4. That is, input data including the values x1, x2, x3, and x4 of nodes X1, X2, X3, and X4, respectively, is input to the input layer LX.

Hidden layer LY1 includes nodes Y1, Y2, and Y3, and hidden layer LY2 includes nodes Y11, Y12, and Y13. The values y1, y2, and y3 of nodes Y1, Y2, and Y3 are calculated from the output values x1, x2, x3, and x4 of the nodes in input layer LX, which is the layer before hidden layer LY1. The values y11, y12, and y13 of nodes Y11, Y12, and Y13 are calculated from the output values y1, y2, and y3 of the nodes in hidden layer LY1, which is the layer before hidden layer LY2.

The output layer LZ includes a node Z1. The value z1 of node Z1 is calculated from the output values y11, y12, y13, and y14 of the nodes in the hidden layer LY2, which is the layer before the output layer LZ. The output layer LZ outputs the value z1 of node Z1 as output data.

In the example of Figure 18, the nonlinear equation representing individual i of the genetic algorithm is expressed, for example, by the above equation (1).

In this example, the identification step M12 executed by the processor 11 is similar to the process of FIG. 5 shown in the above-mentioned embodiment 1. That is, the processor 11 executes the processes of steps M121 to M132 in FIG. 5 to identify and output a nonlinear formula that expresses the relationship between the input data and the output data.

[Specific example 4 of trained model and genetic algorithm]
FIG. 19 is a diagram illustrating a trained model LM2 and a genetic algorithm GA according to this embodiment.

In the example of FIG. 19, an individual i of the genetic algorithm GA includes, for example, the following seven equations (6) to (12).
z1=f ₅ (y1, y2, y3, y11, y12, y13)...(6)
y1=f ₂ (x1, x2, x3, x4)...(7)
y2=f ₃ (x1, x2, x3, x4)...(8)
y3=f ₄ (x1, x2, x3, x4)...(9)
y11=f ₆ (x1, x2, x3, x4)...(10)
y12=f ₇ (x1, x2, x3, x4)...(11)
y13=f ₈ (x1, x2, x3, x4)...(12)

In this example, the specific step M12 executed by the processor 11 is similar to the process shown in FIG. 12 in the first embodiment described above. That is, in step M151, the processor 11 obtains input data and output data for mathematically formulating the relationship.

The input data includes the values x1, x2, and x3 of each node in the input layer LX. The output data includes the value z1 of node Z1 in the output layer LZ, as well as the output values y1, y2, y3, y11, y12, and y13 of each node in the hidden layers LY1 and LY2.

Processor 11 executes the processes of steps M152 to M131 in FIG. 12 and selects individual i which is the optimal solution. In step M153, processor 11 identifies a formula expressing the relationship between the input data and output data of trained model LM2 by solving the simultaneous equations (6) to (12) contained in the selected individual i. The formula expressing the relationship between the input data and the output data is expressed, for example, by the above-mentioned formula (1). Processor 11 outputs the identified formula, for example, by displaying it on a display.

[Example]
Examples 1 to 6 described in the above-mentioned embodiment 1 can also be applied as examples of this embodiment. That is, the arithmetic formula system according to this embodiment is applied to, for example, a casting system, a sales forecasting system, a drug discovery system, a material development system, and an image recognition system. Among the above-mentioned embodiments 1 to 6, examples 3 to 5 are more suitable for embodiment 2, while examples 1, 2, and 6 are more suitable for embodiment 1.

[Modifications]
[Modification 1]
20 is a diagram illustrating a trained model LM3 and a genetic algorithm GA according to Modification 1. In the diagram, the trained model LM3 includes an input layer LX, multiple hidden layers LY11 and LY12, and an output layer LZ. Each layer has a structure in which multiple nodes are connected by edges.

The input layer LX includes nodes X1, X2, X3, and X4. That is, input data including the values x1, x2, x3, and x4 of nodes X1, X2, X3, and X4, respectively, is input to the input layer LX.

Hidden layer LY11 includes nodes Y1, Y2, and Y3, and hidden layer LY12 includes nodes Y11 and Y12. The values y1, y2, and y3 of nodes Y1, Y2, and Y3 are calculated from the output values x1, x2, x3, and x4 of the nodes in input layer LX, which is the layer before hidden layer LY11. The values y11 and y12 of nodes Y11 and Y12 are calculated from the output values y1, y2, and y3 of the nodes in hidden layer LY11, which is the layer before hidden layer LY12.

The output layer LZ includes a node Z1. The value z1 of node Z1 is calculated from the output values y11 and y12 of the nodes in the hidden layer LY12, which is the layer before the output layer LZ. The output layer LZ outputs the value z1 of node Z1 as output data.

In FIG. 20, an individual i of the genetic algorithm GA includes, for example, the four nonlinear formulas (2) to (5) described above. That is, the processor 11 identifies, by nonlinear regression using the genetic algorithm, a first nonlinear formula (formulas (3) to (5)) that represents the relationship between the input data and the output values of each node belonging to a part of a hidden layer (one hidden layer (hidden layer LY11)), and a second nonlinear formula (formula (2)) that represents the relationship between the data of each node and the output data.

The specific step M12 executed by the processor 11 is similar to the process of FIG. 12 shown in the first embodiment described above. That is, in step M151, the processor 11 acquires input data and output data for mathematically formulating the relationship. The input data includes the values x1, x2, and x3 of each node in the input layer LX. The output data includes the value z1 of the node Z1 in the output layer LZ, as well as the output values y1, y2, and y3 of each node in the hidden layer LY11.

Processor 11 executes the processes of steps M152 to M131 in FIG. 12 and selects individual i which is the optimal solution. In step M153, processor 11 identifies a formula expressing the relationship between the input data and output data of trained model LM2 by solving simultaneous equations (2) to (5) included in the selected individual i. The formula expressing the relationship between the input data and the output data is expressed, for example, by the above-mentioned formula (1). Processor 11 outputs the identified formula, for example, by displaying it on a display.

The hidden layers of the trained model LM3 may include layers that are slow to learn and layers that are fast to learn. A layer that is slow to learn is a layer with many perceptrons and multiple weightings. A layer that is fast to learn is a layer with few perceptrons and no weighting or only one weighting. In this modified example, for a hidden layer that is slow to learn (e.g., hidden layer LY11), processor 11 includes the values of each node in that layer as variables in the formula. On the other hand, for a layer that is fast to learn (e.g., hidden layer LY12), processor 11 does not include the values of each node in that layer in the formula.

By including the values of each node in layers where learning is slower as variables in the formula, it is possible to grasp the features in complex phenomena such as image recognition. It is also easier to understand how each node works. Furthermore, by not including the values of each node in layers where learning is faster in the formula, calculation costs can be reduced and the formula can be simplified.

[Modification 2]
21 is a diagram illustrating a trained model LM3 and a genetic algorithm GA according to Modification 1. In the diagram, the trained model LM3 is similar to the trained model LM3 shown in FIG.

In FIG. 21, an individual i of the genetic algorithm GA includes, for example, the following three nonlinear expressions (13) to (15).
z1=f ₈ (y11, y12)...(13)
y11=f ₆ (x1, x2, x3, x4)...(14)
y12=f ₇ (x1, x2, x3, x4)...(15)

The processor 11 uses nonlinear regression using the genetic algorithm to determine a first nonlinear equation (equations (14) to (15)) that represents the relationship between the input data and the output values of each node belonging to the hidden layer LY12 that precedes the output layer LZ, and a second nonlinear equation (equation (13)) that represents the relationship between the output values of each node and the output data.

In this example, the specific step M12 executed by the processor 11 is similar to the process shown in FIG. 12 in the first embodiment described above. That is, in step M151, the processor 11 obtains input data and output data for mathematically formulating the relationship.

The input data includes the values x1, x2, and x3 of each node in the input layer LX. The output data also includes the value z1 of node Z1 in the output layer LZ, as well as the output values y11 and y12 of each node in the hidden layer LY12.

Processor 11 executes the processes of steps M152 to M131 in FIG. 12 and selects individual i which is the optimal solution. In step M153, processor 11 identifies an equation expressing the relationship between the input data and output data of trained model LM2 by solving the simultaneous equations (13) to (15) contained in the selected individual i. The equation expressing the relationship between the input data and the output data is expressed, for example, by the above-mentioned equation (1). Processor 11 outputs the identified equation, for example, by displaying it on a display.

In the above-described second embodiment, a trained model LM2 including two hidden layers LY1 and LY2 is used, but the number of hidden layers included in the trained model may be two or more.

〔summary〕
The mathematical formula identification device according to aspect 1 includes one or more processors that execute an identification step of identifying a nonlinear mathematical formula that represents a relationship between a numerical value representing input data of a trained model and a numerical value representing output data of the trained model.

With the above configuration, the mathematical formula identification device can mathematically express the computational process of the trained model.

The mathematical formula identification device according to aspect 2 has the following features in addition to the features of the mathematical formula identification device according to aspect 1. That is, in the mathematical formula identification device according to aspect 2, the processor further executes an estimation step of estimating the output data from the input data using the trained model constructed by machine learning, and in the identification step, identifies the nonlinear mathematical formula by nonlinear regression using a genetic algorithm.

With the above configuration, the mathematical formula identification device can mathematically express the computational process of the trained model.

The mathematical formula identification device according to aspect 3 has the following features in addition to the features of the mathematical formula identification device according to aspect 2. That is, in the mathematical formula identification device according to aspect 3, the trained model is a deep neural network composed of multiple layers, and in the identification step, the processor identifies the mathematical formula by nonlinear regression using the genetic algorithm, in which a nonlinear mathematical formula that represents the relationship between a numerical value representing the input data input to an input layer and a numerical value representing the output data output from an output layer is used as a solution candidate.

According to the above configuration, the mathematical formula identification device identifies a nonlinear mathematical formula that represents the relationship between the numerical values representing the input data and the numerical values representing the output data of the trained model by nonlinear regression using a genetic algorithm, which treats the nonlinear mathematical formula that represents the relationship between the numerical values representing the input data and the output data of the trained model as candidate solutions. This makes it possible to mathematically express the computational process of the trained model.

The mathematical formula identification device according to aspect 4 has the following features in addition to the features of the mathematical formula identification device according to aspect 2. That is, in the mathematical formula identification device according to aspect 4, the trained model is a deep neural network composed of multiple layers, and in the identification step, the processor identifies the first mathematical formula and the second mathematical formula by nonlinear regression using the genetic algorithm in which a nonlinear first mathematical formula expressing the relationship between the numerical value expressing the input data and each output value of some or all of the nodes belonging to a hidden layer, and a nonlinear second mathematical formula expressing the relationship between the output value of each of the nodes and the numerical value expressing the output data, are set as solution candidates, and the processor identifies a nonlinear mathematical formula expressing the relationship between the numerical value expressing the input data and the numerical value expressing the output data using the identified first mathematical formula and second mathematical formula.

According to the above configuration, the formula identification device identifies, by nonlinear regression using a genetic algorithm, a formula that expresses the relationship between the output values of nodes in the input layer of a trained model including multiple layers and the output values of some or all of the nodes in the hidden layer, and a formula that expresses the relationship between the output values of nodes in the hidden layer and the output values of nodes in the output layer. This makes it possible to mathematically express the calculation process of the trained model.

The mathematical formula identification device according to aspect 5 has the following features in addition to the features of the mathematical formula identification device according to aspect 4. That is, in the mathematical formula identification device according to aspect 5, the trained model includes a plurality of hidden layers, and in the identification step, the processor identifies a first nonlinear mathematical formula that represents the relationship between a numerical value representing the input data and an output value of each node belonging to one hidden layer, and a second nonlinear mathematical formula that represents the relationship between the output value of each node and a numerical value representing the output data, by nonlinear regression using the genetic algorithm.

According to the above configuration, the formula identification device identifies, by nonlinear regression using a genetic algorithm, a formula that expresses the relationship between the output value of a node in an input layer of a trained model including multiple layers and the output value of a node in one hidden layer, and a formula that expresses the relationship between the output value of a node in the one hidden layer and the output value of a node in the output layer. This makes it possible to mathematically express the calculation process of the trained model.

The mathematical formula identification device according to aspect 6 has the following features in addition to the features of the mathematical formula identification device according to aspect 4. That is, in the mathematical formula identification device according to aspect 6, the trained model includes a plurality of hidden layers, and in the identification step, the processor identifies a first nonlinear mathematical formula that represents the relationship between a numerical value representing the input data and an output value of each node belonging to a hidden layer preceding the output layer, and a second nonlinear mathematical formula that represents the relationship between the data of each node and a numerical value representing the output data, by nonlinear regression using the genetic algorithm.

According to the above configuration, the formula identification device identifies, by nonlinear regression using a genetic algorithm, a formula that expresses the relationship between the output values of nodes in the input layer of a trained model including multiple layers and the output values of nodes in the hidden layer preceding the output layer, and a formula that expresses the relationship between the output values of nodes in the hidden layer and the output values of nodes in the output layer. This makes it possible to mathematically express the computational process of the trained model.

The method for identifying a mathematical formula according to aspect 7 includes a step in which one or more processors identify a nonlinear mathematical formula that represents a relationship between a numerical value representing input data of a trained model and a numerical value representing output data of the trained model.

The estimation device according to aspect 8 includes one or more processors that execute a second estimation step of estimating a numerical value representing the output data by substituting a numerical value representing the input data into the nonlinear mathematical formula identified by the mathematical formula identification device according to any one of aspects 1 to 6 and performing a calculation.

With the above configuration, it is possible to estimate the output of the trained model using a nonlinear formula that mathematically represents the computational process of the trained model.

The method for identifying a mathematical formula according to aspect 9 includes a second estimation step in which one or more processors estimate a numerical value representing the output data by substituting a numerical value representing the input data into the nonlinear mathematical formula identified by the method for identifying a mathematical formula according to aspect 7 and performing a calculation.

With the above configuration, it is possible to estimate the output of the trained model using a nonlinear formula that mathematically represents the computational process of the trained model.

[Additional Note 1]
In each of the above-described embodiments and modifications, trained models trained by machine learning through supervised learning are used as the trained models LM1, LM2, and LM3, but trained models trained by machine learning through unsupervised learning may also be used. As in each of the above-described embodiments and modifications, for trained models trained by machine learning through unsupervised learning, the processor 11 identifies a nonlinear formula expressing the relationship between the input and output of the trained model by regression using a genetic algorithm.

[Additional Note 2]
In each of the above-described embodiments and modified examples, the machine learning device 2 constructs the trained models LM1, LM2, and LM3, but the trained models LM1, LM2, and LM3 may be constructed in advance by a device other than the machine learning device 2. In this case, the mathematical formula identification device 1 performs the above-described mathematical formula identification method M1 using the trained models constructed in advance by the other device.

[Additional Note 3]
In the above-described embodiments and modifications, the processor 11 identifies the nonlinear formula expressing the relationship between the input data and the output data by nonlinear regression using a genetic algorithm. The algorithm for identifying the nonlinear formula expressing the relationship between the input data and the output data is not limited to the genetic algorithm, and may be another nonlinear algorithm.

As an example, processor 11 may identify a nonlinear formula expressing the relationship between input data and output data using the Monte Carlo method. In this case, as an example, processor 11 creates multiple formulas by randomly selecting the length of the formula and the elements of the formula (variables, operators, etc.), and constructs a relational formula that derives output data from input data by selecting the formula with the smallest error from the multiple formulas created.

As another example, the processor 11 may identify a nonlinear formula using a trained model (hereinafter referred to as a "formula derivation model") that derives a nonlinear formula from a dataset. In this case, the formula derivation model is a trained model that takes a dataset as input and outputs a label corresponding to the nonlinear formula. The input dataset is a set of input data and output data of the trained models LM1, LM2, and LM3 described above. The output label is a label corresponding to the nonlinear formula.

In the learning phase of the formula derivation model, an information processing device such as the machine learning device 2 constructs a formula derivation model by machine learning using learning data that links a dataset and a label.

[Additional Note 4]
The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Other embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

Reference Signs List 1: Mathematical formula identification device 2: Machine learning device 3: Estimation device 11, 21: Processor 12, 22: Primary memory 13, 23: Secondary memory 14, 24: Input/output interface 15, 25: Communication interface 16, 26: Bus LM1, LM2: Trained model M1: Mathematical formula identification method M11: Estimation step M12: Identification step M2: Machine learning method M21: Training data set construction step M22: Trained model construction step S: Arithmetic formula system

Claims (8)

An estimation step of estimating output data from input data using a trained model constructed by machine learning;
A step of identifying a nonlinear formula that represents a relationship between a numerical value representing the input data of the trained model and a numerical value representing the output data of the trained model;
and one or more processors executing
The trained model is a deep neural network composed of multiple layers,
In the identifying step, the processor identifies a nonlinear equation that represents a relationship between a numerical value representing the input data input to an input layer and a numerical value representing the output data output from an output layer by nonlinear regression using a genetic algorithm, the nonlinear equation being a candidate solution.
A mathematical expression identification device. An estimation step of estimating output data from input data using a trained model constructed by machine learning;
A step of identifying a nonlinear formula that represents a relationship between a numerical value representing the input data of the trained model and a numerical value representing the output data of the trained model;
and one or more processors executing
The trained model is a deep neural network composed of multiple layers,
In the identifying step, the processor identifies a first nonlinear formula expressing a relationship between a numerical value expressing the input data and each output value of a part or all of the nodes belonging to a hidden layer, and a second nonlinear formula expressing a relationship between the output value of each of the nodes and the numerical value expressing the output data, by nonlinear regression using a genetic algorithm with the first formula and the second formula as solution candidates, and identifies a nonlinear formula expressing the relationship between the numerical value expressing the input data and the numerical value expressing the output data, by using the identified first formula and second formula.
A mathematical expression identification device. The trained model includes a plurality of hidden layers,
In the identifying step, the processor identifies a first nonlinear formula expressing a relationship between a numerical value expressing the input data and an output value of each node belonging to one hidden layer, and a second nonlinear formula expressing a relationship between data of each node and a numerical value expressing the output data, by nonlinear regression using the genetic algorithm.
3. The mathematical expression specifying device according to claim 2. The trained model includes a plurality of hidden layers,
In the identifying step, the processor identifies a first nonlinear formula expressing a relationship between a numerical value expressing the input data and an output value of each node belonging to a hidden layer preceding the output layer, and a second nonlinear formula expressing a relationship between data of each node and a numerical value expressing the output data, by nonlinear regression using the genetic algorithm.
3. The mathematical expression specifying device according to claim 2. one or more processors,
An estimation step of estimating output data from input data using a trained model constructed by machine learning;
A step of identifying a nonlinear formula that represents a relationship between a numerical value representing the input data of the trained model and a numerical value representing the output data of the trained model;
Including,
The trained model is a deep neural network composed of multiple layers,
In the identifying step, the processor identifies a nonlinear equation that represents a relationship between a numerical value representing the input data input to an input layer and a numerical value representing the output data output from an output layer by nonlinear regression using a genetic algorithm, the nonlinear equation being a candidate solution.
A method for identifying a mathematical expression. one or more processors,
An estimation step of estimating output data from input data using a trained model constructed by machine learning;
A step of identifying a nonlinear formula that represents a relationship between a numerical value representing the input data of the trained model and a numerical value representing the output data of the trained model;
Including,
The trained model is a deep neural network composed of multiple layers,
In the identifying step, the processor identifies a first nonlinear formula expressing a relationship between a numerical value expressing the input data and each output value of a part or all of the nodes belonging to a hidden layer, and a second nonlinear formula expressing a relationship between the output value of each of the nodes and the numerical value expressing the output data, by nonlinear regression using a genetic algorithm with the first formula and the second formula as solution candidates, and identifies a nonlinear formula expressing the relationship between the numerical value expressing the input data and the numerical value expressing the output data, by using the identified first formula and second formula.
A method for identifying a mathematical expression. a second estimation step of estimating a numerical value representing the output data by substituting a numerical value representing the input data into the nonlinear mathematical formula identified by the mathematical formula identification device according to any one of claims 1 to 4 and performing an operation;
and one or more processors executing
An estimation device comprising: a second estimation step in which one or more processors estimate a numerical value representing the output data by substituting a numerical value representing the input data into the nonlinear mathematical formula identified by the mathematical formula identification method according to claim 5 or 6, and performing a calculation;
The estimation method according to claim 1,

Images