JP7484653B2 - Information processing device, information processing method, operation management device, and operation management method - Google Patents

Description

The present invention relates to an information processing device, an information processing method, an operation management device, and an operation management method.

Inferences using neural networks, such as classification, regression, or the output of a policy function, have a certain degree of instability in the inference results due to overlearning due to the high degree of freedom of the neural network. Therefore, various regularization methods have been proposed to reduce overlearning. However, regularization to reduce overlearning is difficult to adjust parameters, and depending on the settings, the learning time may become enormous or generalization performance may not be obtained. Furthermore, neural networks are generally known to be vulnerable to noise. In practice, neural networks may output erroneous inference results due to the influence of measurement errors, etc. Non-Patent Documents 1 and 2 disclose techniques for adding learning noise during learning in order to pad the learning data and improve generalization performance.

JP 2019-032659 A

“Noisy Networks for Exploration” Meire Fortunato et al. (2018) “Creating artificial neural networks that generalize” Jocelyn Sietsma, Robert J.F.Dow (1991)

However, the above-mentioned conventional technology has limitations on how much accuracy can be improved. Therefore, there is a need to develop technology that can further improve the accuracy of inference results obtained by inputting input data into a neural network.

The present invention has been made in view of the above, and its purpose is to provide an information processing device, an information processing method, an operation management device, and an operation management method that can improve the accuracy of inference results obtained by inputting input data into a neural network.

In order to solve the above-mentioned problems and achieve the object, an information processing device according to one aspect of the present invention is an information processing device including a control unit that acquires at least one input data and outputs inference result data based on the input data using a neural network, the control unit stores the acquired input data in a memory unit, generates multiple noise-added data by adding multiple noises different from each other to one input data read from the memory unit, inputs the multiple noise-added data to the neural network, outputs multiple noise-added inference result data corresponding to the multiple noise-added data, and outputs the inference result data based on the multiple noise-added inference result data.

In the information processing device according to one aspect of the present invention, the plurality of mutually different noises are noises selected based on a normal distribution.

In the information processing device according to one aspect of the present invention, in the above invention, the control unit performs a simulation for each of the plurality of noise-added inference result data, outputs a plurality of evaluation values corresponding to each of the plurality of noise-added inference result data, and outputs at least one noise-added inference result data selected from the plurality of noise-added inference result data based on the plurality of evaluation values as the inference result data.

In the information processing device according to one aspect of the present invention, in the above invention, the control unit performs ensemble learning on the multiple noise-added inference result data, and outputs the output obtained by the ensemble learning as the inference result data.

In the information processing device according to one aspect of the present invention, the input data is data including discrete values.

An operation management device according to one aspect of the present invention is an operation management device for a storage facility that includes at least one storage tank for storing liquid and at least one liquid delivery unit capable of supplying the liquid to the storage tank, and that includes an operation control unit that controls the liquid delivery unit to manage the liquid level in the storage facility, and the operation control unit acquires liquid level information including the liquid level in the storage facility from the storage facility, inputs the acquired information including the liquid level information as storage input data to the information processing device of the above invention, acquires inference result data output from the information processing device in response to the storage input data as storage inference result data, and controls the liquid delivery unit based on the storage inference result data.

In the operation management device according to one aspect of the present invention, in the above invention, the storage inference result data includes information on the liquid level that switches on and off the liquid delivery unit.

In the operation management device according to one aspect of the present invention, in the above invention, the storage input data includes information on a predicted value along with time of the amount of the liquid to be discharged from the storage section at a given time.

In the operation management device according to one aspect of the present invention, in the above invention, the liquid delivery unit is composed of a pump whose on/off switching is controlled, and the storage input data includes information on the on/off of the pump at a specified time.

In the operation management device according to one aspect of the present invention, in the above invention, the liquid delivery unit is composed of a pump driven by electricity, and the storage input data includes information on the unit price of the electricity and the amount of power consumed at a specified time.

An information processing method according to one aspect of the present invention is an information processing method executed by an information processing device that acquires at least one input data and outputs inference result data based on the input data using a neural network, storing the acquired input data in a storage unit, generating multiple noise-added data by adding multiple noises different from each other to one piece of input data read from the storage unit, inputting the multiple noise-added data to the neural network, outputting multiple noise-added inference result data corresponding to the multiple noise-added data, and outputting the inference result data based on the multiple noise-added inference result data.

An operation management method according to one aspect of the present invention is an operation management method for a storage facility including at least one storage tank for storing liquid and at least one liquid delivery unit capable of supplying the liquid to the storage facility, the operation management device controls the liquid delivery unit to manage the liquid level in the storage facility, and obtains liquid level information including the liquid level in the storage facility from the storage facility, and uses the obtained information including the liquid level information as storage input data to obtain storage inference result data obtained in response to the storage input data based on the information processing method according to the above invention, and controls the liquid delivery unit based on the storage inference result data.

The information processing device, information processing method, operation management device, and operation management method of the present invention make it possible to improve the accuracy of inference results obtained by inputting input data into a neural network.

FIG. 1 is a block diagram showing the configuration of an operation management system to which an information processing device according to an embodiment of the present invention is applied. FIG. 2 is a diagram illustrating a method for deriving inference results from input data according to an embodiment of the present invention. FIG. 3 is a flowchart for explaining a method for managing an operation of a water reservoir system according to an embodiment of the present invention. FIG. 4 is a graph showing the control duration along with the operation time according to the method for managing the operation of a water-distributing reservoir system according to one embodiment of the present invention. FIG. 5 is a graph showing the control duration along with the operation time according to the conventional method for managing the operation of a water reservoir system. FIG. 6 is a graph showing the control score along with the operation time according to the operation management method for a water reservoir system according to an embodiment of the present invention. FIG. 7 is a graph showing the control score along with the operation time according to the conventional method for managing the operation of a reservoir system. FIG. 8 is a diagram for explaining a method for deriving an inference result from input data according to a modified embodiment of the present invention. FIG. 9 is a diagram showing a schematic diagram of the overall configuration of a neural network according to the prior art.

One embodiment of the present invention will be described below with reference to the drawings. Note that in all the drawings of the embodiment below, the same or corresponding parts are given the same reference numerals. Furthermore, the present invention is not limited to the embodiment described below.

Figure 1 is a block diagram showing the configuration of an operation management system to which an information processing device according to one embodiment of the present invention is applied. As shown in Figure 1, an operation management system 1 according to one embodiment of the present invention has a learning device 10 and an operation management device 20 that can communicate with each other via a network 2. The operation management device 20 may also be equipped with the learning device 10. The operation management device 20 is configured to be able to collect various information from a water reservoir system 30 via the network 2.

Network 2 is, for example, a public communication network such as the Internet, and is composed of one or more combinations of, for example, a LAN (Local Area Network), a WAN (Wide Area Network), a telephone communication network such as a mobile phone, a public line, a VPN (Virtual Private Network), and a dedicated line. Network 2 appropriately combines wired communication and wireless communication.

(Water reservoir system)
The reservoir system 30 as a storage facility includes a plurality of reservoirs 31, 32 and a plurality of distribution pumps 33, 34. One or more of the distribution pumps 33, 34 are provided for each of the reservoirs 31, 32. Specifically, for example, when there are about 30 reservoirs 31, 32 as storage units, about 40 distribution pumps 33, 34 as liquid delivery units are provided. Water is supplied from the reservoirs 31, 32 to consumers 35, 36 that require water.

In this embodiment, the learning device 10 as a machine learning device performs reinforcement learning using predetermined data such as environmental conditions such as the transition of the reservoir system 30, the on/off status and power consumption of the distribution pumps 33, 34, the electricity charges per unit time of the consumers 35, 36 at a specified time (hereinafter, the electricity rate), and time as input parameters, and the water level at which the multiple distribution pumps 33, 34 in the reservoir system 30 are switched on and off as an output parameter.

(Learning device)
The learning device 10 executes a data collection process to collect various information transmitted from a plurality of reservoir systems 30 having a communication unit (not shown) via a network 2. The learning device 10 is capable of performing machine learning using the various collected information. The learning device 10 includes a control unit 11, a memory unit 12, a communication unit 13, an output unit 14, and an input unit 15.

Specifically, the control unit 11 includes a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field-Programmable Gate Array), and a main memory unit such as a RAM (Random Access Memory) or a ROM (Read Only Memory) (none of which are shown).

The storage unit 12 is physically composed of a storage medium selected from a volatile memory such as a RAM, a non-volatile memory such as a ROM, an erasable programmable ROM (EPROM), a hard disk drive (HDD), a solid state drive (SSD), and a removable medium. The removable medium is, for example, a universal serial bus (USB) memory, or a disk recording medium such as a compact disc (CD), a digital versatile disc (DVD), or a Blu-ray (registered trademark) disc (BD). The storage unit 12 may also be configured using a computer-readable recording medium such as an externally mountable memory card. The storage unit 12 can store an operating system (OS), various programs, various tables, various databases, and the like for executing the operation of the learning device 10. The various programs include the learning model and neural network according to this embodiment. These various programs can also be recorded on computer-readable recording media such as hard disks, flash memory, CD-ROMs, DVD-ROMs, and flexible disks, and widely distributed.

In this embodiment, the control unit 11 loads a program stored in the memory unit 12 into the working area of the main memory unit, executes it, and controls each component unit through the execution of the program, thereby realizing functions that meet a predetermined purpose. Specifically, the control unit 11 can realize the functions of a learning unit 111, a judgment unit 112, an action selection unit 113, a state input unit 114, an environment unit 115, a calculation unit 116, and a simulator unit 117 by executing the program.

The function of the learning unit 111 is executed by the control unit 11 executing the program. The learning unit 111 can perform machine learning based on the input/output data set received by the learning device 10. The learning unit 111 writes and stores the learned results as a learning model 121 in the memory unit 12. The learning unit 111 may store the latest learning model at a predetermined timing in the memory unit 12 separately from the neural network being trained. When storing in the memory unit 12, an update may be performed in which the old learning model 121 is deleted and the latest learning model 121 is stored, or an accumulation may be performed in which the latest learning model 121 is stored while preserving part or all of the old learning model 121.

The communication unit 13 is, for example, a LAN (Local Area Network) interface board or a wireless communication circuit for wireless communication. The LAN interface board and the wireless communication circuit are connected to the network 2. The communication unit 13 is connected to the network 2 and communicates with the operation management device 20 and the water reservoir system 30.

The output unit 14 as an output means is configured to notify the outside of predetermined information by displaying characters, figures, etc. on the screen of a display such as a liquid crystal display or plasma display, or by outputting sound from a speaker, according to the control of the control unit 11. Furthermore, the output unit 14 includes a printer that outputs predetermined information by printing it on printing paper, etc. The various information stored in the memory unit 12 can be confirmed, for example, on a monitor of the output unit 14 installed in a predetermined office, etc.

The input unit 15 as an input means is composed of, for example, a keyboard or a touch panel keyboard that is built into the output unit 14 and detects touch operations on the display panel, or a voice input device that enables calls to be made with the outside. The output unit 14 and the input unit 15 may be integrated together to form an input/output unit such as a touch panel display or a speaker/microphone.

(Operation management device)
The operation management device 20 has a configuration capable of communicating via the network 2. The operation management device 20 includes a control unit 21, a memory unit 22, a communication unit 23, an output unit 24, and an input unit 25. The control unit 21, the memory unit 22, the communication unit 23, the output unit 24, and the input unit 25 have the same physical and functional configurations as the above-mentioned control unit 11, the memory unit 12, the communication unit 13, the output unit 14, and the input unit 25, respectively.

In this embodiment, the control unit 21 can specifically realize the functions of the operation plan unit 211 and the operation control unit 212 by loading a program stored in the memory unit 22 into the working area of the main memory unit and executing it.

The memory unit 22 stores an operation plan database 221 in which operation plan information generated by the operation plan unit 211 can be searched, and an operation information database 222 in which operation information of the water distribution reservoir system 30 acquired by the operation control unit 212 can be searched. The operation plan information stored in the operation plan database 221 includes, for example, plan information on the timing of starting and stopping the water distribution pumps 33, 34. The operation control unit 212 controls the starting and stopping of the water distribution pumps 33, 34 in the water distribution reservoir system 30 based on the operation plan information. The operation information in the operation information database 222 includes, for example, on/off information of the water distribution pumps 33, 34 at a specified time (hereinafter, on/off information) and information on the water level of the water stored in the water distribution reservoirs 31, 32 (hereinafter, water level information). Note that the information on the water distribution reservoir system 30 is not limited to these pieces of information.

The communication unit 23 is connected to the network 2 and is capable of communicating with the learning device 10 and the reservoir system 30. The communication unit 23 collects various information, such as water level information as liquid level information of the reservoirs 31, 32 and start/stop information, by command signals output based on the control of the control unit 21. Note that the information transmitted and received by the communication unit 23 is not limited to these pieces of information. In this embodiment, the operation management device 20 can also function as a cloud server capable of communicating via the network 2.

Here, the present inventor has studied an information processing method using a neural network according to the prior art for a learning device 10 as an information processing device. FIG. 9 is a diagram showing a schematic diagram of the overall configuration of a neural network according to the prior art. The neural network 300 shown in FIG. 9 is, for example, a forward propagation type neural network, and has an input layer 301, an intermediate layer 302, and an output layer 303. The input layer 301 is composed of a plurality of nodes, and different input parameters are input to each node. The intermediate layer 302 receives the output from the input layer 301. The intermediate layer 302 has a multi-layer structure including a layer composed of a plurality of nodes that receive the input from the input layer 301. The output layer 303 receives the output from the intermediate layer 302 and outputs an output parameter. Machine learning using a neural network in which the intermediate layer 302 has a multi-layer structure, for example, a 3-5 layer structure, is called deep learning. In the example shown in FIG. 9, the input parameter is a predetermined input data 310, and the output parameter is the desired inference result data 320.

When generating a predetermined learning model by supervised learning or semi-supervised learning in the neural network 300, an input/output data set of learning input parameters and learning output parameters is used. The learning unit 111 in the control unit 11 that outputs predetermined inference result data 120 for input data 110 can generate a learning model 121 by machine learning such as deep learning using the neural network shown in FIG. 9, using the learning input parameters and learning output parameters as teacher data. The control unit 11 of the learning device 10 derives inference result data 320 from the input data 310 based on the learning model generated by the learning unit 111.

When the neural network 300 is used for reinforcement learning, input data 310 such as state s and action a are input to the neural network 300, and inference result data 320 is output. As the inference result data 320, state s, action a, state action value (Q value), etc. are output.

After various studies on the conventional neural network 300, the inventor came up with a method of adding multiple different noises to the input data during inference, i.e., when inputting the data into the neural network 300, in order to improve the accuracy of the inference results that are output. That is, the inventor came up with the idea of adding noise to the input data during inference to generate multiple patterns of input data, and selecting or generating at least one waterway result based on multiple inference results obtained by inputting each of the multiple generated input data into the neural network 300, thereby obtaining an output with high accuracy and a high evaluation value. The present invention and one embodiment thereof described below were devised by the inventor through the above-mentioned intensive studies.

2 is a diagram for explaining an information processing method executed by the learning device 10 as an information processing device according to one embodiment of the present invention. As shown in FIG. 2, in the information processing method according to one embodiment, the calculation unit 116 generates first noise-added data 311-1 by adding a first noise to one input data 310. Similarly, the calculation unit 116 generates second noise-added data 311-2 by adding a second noise different from the first noise to one input data 310. Furthermore, the calculation unit 116 generates third noise-added data 311-3 by adding a third noise different from both the first noise and the second noise to the input data. The addition of these mutually different noises is performed until n noises (n is a natural number of 2 or more) are added to one input data 310. The n noises from the first noise to the nth noise added to the input data 310 are mutually different noises that are not correlated with each other. These multiple noises from the first noise to the nth noise that are mutually different are selected based on, for example, a normal distribution, so-called Gaussian noise, but it is also possible to select noises based on other distributions. As a result, the calculation unit 116 generates the first noise-added data 311-1 to the nth noise-added data 311-n, which are different from each other, based on the same input data 310. The generated first noise-added data 311-1 to the nth noise-added data 311-n are stored in the storage unit 12.

The learning unit 111 reads the program of the neural network 122 from the storage unit 12 and executes the processing of the neural network 300. That is, the learning unit 111 inputs each of the first noise-added data 311-1 to the nth noise-added data 311-n to the neural network 300. The neural network 300 outputs the first inference result data 321-1, the second inference result data 321-2, the third inference result data 321-3, ..., the nth inference result data 321-n corresponding to each of the first noise-added data 311-1 to the nth noise-added data 311-n. The first inference result data 321-1 to the nth inference result data 321-n output as the multiple noise-added inference result data are stored in the storage unit 12.

Then, the simulator 340 executed by the simulator unit 117 of the control unit 11 individually executes a simulation for each of the first inference result data 321-1 to the n-th inference result data 321-n output from the neural network 300. The simulator 340 outputs a first evaluation value 330-1 as a result of executing a simulation based on the first inference result data 321-1. Similarly, the simulator unit 117 outputs a second evaluation value 330-2, a third evaluation value 330-3, ..., and an n-th evaluation value 330-n as a result of executing a simulation based on the second inference result data 321-2, the third inference result data 321-3, ..., and the n-th inference result data 321-n, respectively. The output first evaluation value 330-1 to the n-th evaluation value 330-n are stored in the memory unit 12.

The first evaluation value 330-1 through the nth evaluation value 330-n output by the simulator unit 117 are input to the judgment unit 112 of the control unit 11, which has the function of a selector 350 that selects an evaluation value. The judgment unit 112 selects the optimal evaluation value using the selector 350. The judgment unit 112 of the control unit 11 selects and outputs inference result data corresponding to the evaluation value selected from the first evaluation value 330-1 through the nth evaluation value 330-n. As a result of the above, inference result data 320 for the input data 310 is output.

As described above, in the case of the neural network 300 that outputs so-called control methods and evaluation values such as reinforcement learning, and that can be simulated in advance by the simulator unit 117, the accuracy of the inference result data can be improved by obtaining multiple evaluation values (1st to nth evaluation values) from multiple inference result data (1st to nth inference result data) and selecting the inference result data with the optimal evaluation value. In addition, by adding multiple different noises to the input data, the data input to the neural network 300 can be increased by the amount of added noise, so that in reinforcement learning and the like that outputs a policy by padding the input by adding noise, a control policy with a higher evaluation value can be obtained by combining it with a simulator.

A predetermined threshold value may be set for the evaluation values, and if all of the first evaluation value 330-1 through the nth evaluation value 330-n are below the predetermined threshold value, further noise may be added to the input data 310 until at least one or more of the first evaluation value 330-1 through the nth evaluation value 330-n become equal to or greater than the predetermined threshold value. The information processing method according to the embodiment described above is applicable to any process in which information processing can be performed using a neural network.

(Operation and management method for water reservoir system)
Next, a method for managing operation of a water reservoir based on the information processing method according to the above embodiment will be described.

First, in order to facilitate understanding of the operation management method according to one embodiment, the problems of the conventional technology in the operation management method of a water reservoir system will be described. That is, conventionally, the water distribution pumps 33, 34 for transporting liquid tap water between the water purification plant, the water distribution reservoirs 31, 32, and the consumers 35, 36 can often only be controlled to be on and off. In this case, it was difficult to generate an operation plan for reducing the amount of power used by the water distribution pumps 33, 34 and the number of times the water distribution pumps 33, 34 are started and stopped. This is because, while a linear programming problem that allows for a solution of continuous values can calculate the optimal value in a realistic time, an integer programming problem that allows for only a solution of discrete values (integers) and a mixed integer programming problem that mixes discrete and continuous values require the calculation of a huge number of patterns, and the calculation cannot be completed in a realistic processing time. In other words, optimization problems that allow only discrete values as solutions for some or all variables, rather than continuous values, require trying a huge number of patterns to derive the optimal solution, so it is generally difficult to arrive at the optimal solution, and as an alternative, a solution close to the optimal solution is derived within an acceptable time.

However, according to the inventor's knowledge, problems that allow only discrete values require a huge number of case distinctions, and when special circumstances exist for each target, changing the pruning search method that distinguishes between cases for each target becomes a very cumbersome task. For this reason, there has been a demand for technology that can reduce the amount of complicated processing and perform optimal operation management in water reservoir systems, etc.

The inventors therefore investigated a method of employing reinforcement learning in machine learning, such as DQN (Deep Q-Network), in generating operation plans. In other words, they came up with the idea of employing reinforcement learning in machine learning as a control method that uses evaluation of the results of operations, rather than evaluation of the operations themselves. This eliminates the need for a huge number of case distinctions, and when creating a simulator, if special circumstances for each target are taken into account, they can be reflected in the control.

Furthermore, the inventor has come up with the idea of a reward in reinforcement learning that corresponds to the magnitude of deviation from the upper or lower limit of the water level tolerance range, which is the constraint condition in the water reservoirs 31 and 32, that is, the deviation of the water level from the tolerance range. Specifically, the greater the deviation of the water level from the tolerance range, the smaller the reward is, and the larger the negative absolute value of the reward is. This makes it easier to progress in learning by increasing the reward if the water level of the water reservoirs 31 and 32 falls within the tolerance range. This is because, in the above-mentioned control method, if a uniform penalty is given when a constraint condition is deviated and an episode, which is a unit of repeated learning in reinforcement learning, is stopped, the same penalty is given whether the degree of deviation is large or small. Therefore, when updating the control method, it is not clear in which direction the control method should be updated. Therefore, by applying a penalty using, for example, a monotonically increasing function depending on the degree of deviation from the constraints, a gradient can be generated in which, when the constraints are deviated from, the reward for a smaller degree of deviation is greater than the reward for a larger degree of deviation. By performing reinforcement learning according to the gradient, it is possible to learn a policy that satisfies the constraints.

In addition, instead of starting and stopping the distribution pumps 33, 34, the water level threshold for turning the distribution pumps 33, 34 on or off is optimized, changing the problem of optimizing discrete values to a problem of optimizing continuous values, making it possible to complete the calculation in a realistic processing time. In other words, when simply planning the start and stop of the distribution pumps 33, 34 at a set time, one of the operations calculated as the power of the number of time increments and the number of distribution pumps 33, 34 is selected for the two ways of turning the distribution pumps 33, 34 on and off. However, for the distribution pumps 33, 34 in a specified start/stop state, the water level of the distribution reservoirs 31, 32 at which the distribution pumps 33, 34 are next started and stopped (hereinafter, the start/stop water level) is controlled to be determined. This allows the optimization problem of the discrete values of on/off to be changed to the optimization problem of the continuous values of the water levels of the distribution reservoirs 31, 32. Therefore, since it is only necessary to determine the water level for each inference by the neural network 300, a solution close to the optimal solution can be derived in a shorter time than in the past. Furthermore, by setting the water levels in each of the reservoirs 31 and 32 within a predetermined tolerance range, deviation from the constraints can be reduced. Furthermore, when a predetermined control is selected from continuous values, there is less variation in each inference compared to when a predetermined control is selected from discrete values, making it possible to create a stable operation plan.

The method for managing operation of a water reservoir according to this embodiment, which is described below, was devised by the inventor through the above-mentioned intensive study. FIG. 3 is a flowchart for explaining the method for managing operation of a water reservoir according to this embodiment. The operation management device 20 constantly or at appropriate times acquires water level information from the water reservoirs 31, 32 via the network 2, and acquires start/stop information from the water distribution pumps 33, 34, and stores the acquired water level information and start/stop information in the memory unit 22. The transmission, supply, or acquisition of information between the learning device 10, the operation management device 20, and the water reservoir system 30 is also performed via the network 2, but a detailed explanation of each is omitted.

As shown in FIG. 3, in step ST1, the state input unit 114 in the control unit 11 of the learning device 10 acquires information about the water reservoir system 30 as storage input data. The storage input data, which is an example of input data, includes discrete values such as the on/off status of the water distribution pumps 33, 34. Specifically, the state input unit 114 acquires water level information and start/stop information from the operation management device 20. The state input unit 114 acquires information on the predicted value of the amount of tap water used by each consumer 35, 36 in a predetermined time, for example, 48 hours (hereinafter, demand forecast information) from the operation plan database 221 by the operation plan unit 211 in the control unit 21 of the operation management device 20. The demand forecast information is, in other words, a predicted value of the amount of tap water discharged from the water reservoirs 31, 32 according to time. The status input unit 114 acquires information on the electricity rate unit price at a specified time (hereinafter, rate information) from a business server of a specified power supply business, specifically from the business server (not shown) of the power supply business supplying power to the water distribution pumps 33, 34, as appropriate. The status input unit 114 acquires current time information (hereinafter, time) from a specified time server (not shown) all the time or as appropriate. The status input unit 114 may acquire all of the information, including water level information, start/stop information, demand forecast information, rate information, and time, or may acquire only a portion of the information including the water level information. It is also possible to acquire other information.

Next, the process proceeds to step ST2, and the calculation unit 116 of the control unit 11 adds multiple different noises to the acquired data. Specifically, as shown in FIG. 2, the calculation unit 116 adds multiple different noises to at least one of the multiple water level information pieces in the multiple water level information pieces in the multiple water distribution reservoirs 31 and 32 acquired, for example. In this embodiment, these noises are selected based on normal distribution, or so-called Gaussian noise, but it is also possible to select noises based on other distributions, such as uniform distribution, or other distributions. By adding multiple different noises to the water level information, multiple noise-added data are generated for one piece of water level information as noise-added water level information. Similarly, multiple different noises are added to at least one piece of on/off information pieces in the multiple on/off information pieces in the multiple distribution pumps 33 and 34 acquired. As a result, multiple noise-added data are generated for one piece of on/off information as noise-added on/off information. Furthermore, similarly, multiple noise-added demand forecast information is generated for at least one piece of demand forecast information. The calculation unit 116 may add noise to all of the water level information, start/stop information, and demand forecast information, or may add noise to only selected information from among these. The calculation unit 116 may also add multiple different noises to the fee information and time as necessary to generate multiple noise-added fee information and multiple noise-added times. In other words, it is sufficient to add noise to at least one of the various pieces of information acquired by the status input unit 114.

Next, moving to step ST3 in FIG. 3, the learning unit 111 in the control unit 11 of the learning device 10 reads out the neural network 122 from the memory unit 12 and inputs the multiple noise-added data to the neural network 122. The learning unit 111 may also read out the learning model 121 generated by machine learning using reinforcement learning and input the multiple noise-added data to the learning model 121. Using the learning model 121 or the neural network 122, the learning unit 111 of the control unit 11 outputs multiple inference result data 321 that respectively correspond to the multiple noise-added data that have been input (see FIG. 2).

Then, the process proceeds to step ST4, where the multiple inference result data 321 are input to the simulator 340 (see FIG. 2) executed by the simulator unit 117. The simulator 340 outputs multiple evaluation values 330 corresponding to the multiple inference result data 321. The multiple evaluation values 330 are input to the selector 350 executed by the judgment unit 112 of the control unit 11. The selector 350 selects at least one evaluation value 330 from the multiple evaluation values 330 based on a predetermined condition, such as the evaluation value 330 with the highest evaluation value or the evaluation value 330 closest to a predetermined evaluation value. If the multiple evaluation values 330 do not satisfy the predetermined condition, the process may return to step ST2 and add noise again. Next, the action selection unit 113 of the control unit 11 selects at least one inference result data 321 corresponding to the evaluation value selected by the selector 350 from the multiple inference result data 321 as the final inference result data 320. Here, the final inference result data 320 may be the timing for switching on and off the respective distribution pumps 33, 34, the water levels of the distribution reservoirs 31, 32 at which the switching on and off is performed, or the on and off pattern of the respective distribution pumps 33, 34.

Next, proceeding to step ST5, the communication unit 13 of the learning device 10 transmits the final inference result data 320 derived by the control unit 11 to the operation management device 20 as storage inference result data. The operation control unit 212 of the operation management device 20 controls the operation, i.e., start/stop, of the distribution pumps 33, 34 of the distribution reservoir system 30 based on the acquired storage inference result data (final inference result data 320). The water levels of the distribution reservoirs 31, 32 obtained by starting and stopping the distribution pumps 33, 34 in the distribution reservoir system 30 are transmitted to the operation management device 20 as water level information at all times or as needed. Similarly, the on/off operation of the distribution pumps 33, 34 is also transmitted to the operation management device 20 as start/stop information at all times or as needed. The water level information of the distribution reservoirs 31, 32 and the start/stop information of the distribution pumps 33, 34 acquired by the operation management device 20 are stored in the operation information database 222 in the memory unit 22 of the operation management device 20.

Next, proceeding to step ST6, the operation control unit 212 determines whether the water levels of the respective reservoirs 31, 32 are within the allowable range based on the acquired water level information of the respective reservoirs 31, 32. The allowable range of the water levels of the respective reservoirs 31, 32 is set for each reservoir 31, 32. The operation control unit 212 checks for each reservoir 31, 32 whether the water levels in the respective reservoirs 31, 32 according to the start and stop of the distribution pumps 33, 34 deviate from the allowable range. If the water levels deviate from the allowable range, the operation control unit 212 derives the deviation level, i.e., how much the water levels of the respective reservoirs 31, 32 are higher than the upper limit of the allowable range or how much lower the water levels are than the lower limit of the allowable range. Similarly, the operation control unit 212 derives the deviation level depending on how much the number of starts and stops of the distribution pumps 33, 34 and the electricity usage fee deviate from the allowable range. The deviation level derived by the operation control unit 212 is transmitted from the operation management device 20 to the learning device 10.

The environment unit 115 of the learning device 10 derives a penalty related to the reward for reinforcement learning based on the acquired deviation level. Specifically, the environment unit 115 sets the reward so that the greater the acquired deviation level, the greater the linear deduction, in other words, the smaller the reward. The environment unit 115 also sets a positive reward when operation can be continued for a predetermined time, for example, one hour, while maintaining the allowable range of the water level, which is a constraint condition of the distributing reservoirs 31, 32. Furthermore, the environment unit 115 sets the reward so that the penalty increases as the deviation level increases, depending on the number of starts and stops of the distributing pumps 33, 34 of the distributing reservoir system 30 and the electricity usage fee.

Next, the process proceeds to step ST7, where the learning unit 111 obtains the reward set by the environment unit 115 and sets it as the reward in the reinforcement learning. This advances the machine learning in the reinforcement learning of the learning unit 111, and more appropriate inference results can be obtained for the operational control of the water reservoir system 30, i.e., the start/stop pattern of the water distribution pumps 33, 34 can be optimized.

Figure 4 is a graph showing the control duration over operation time according to an embodiment of the operation management method for a water reservoir system in which noise is added to the input data described above. Figure 5 is a graph showing the control duration over operation time according to a conventional operation management method for a water reservoir system using reinforcement learning without adding noise to the input data as a comparative example. The control duration is the time during which operation can be continued while maintaining the water levels of the water reservoirs 31 and 32 within the allowable water level range, and in the examples shown in Figures 4 and 5, the upper limit of the operation duration is set to 24 hours (one day).

From FIG. 4, it can be seen that in the operation management method according to one embodiment, the operation of the reservoir system 30 could be continued for approximately 1,900 hours while maintaining the water levels of the reservoirs 31, 32 within a predetermined allowable range. In the example shown in FIG. 4, it was confirmed that operation was possible while maintaining the constraints of the reservoirs 31, 32, i.e., the allowable range of the water levels, during approximately 1,900 hours of operation. In contrast, from FIG. 5, it can be seen that in the operation management method according to the conventional technology, the water levels of the reservoirs 31, 32 deviate from the allowable range of the water levels. In the example shown in FIG. 5, it was confirmed that operation deviated from the allowable range of the water levels for approximately 110 hours during approximately 1,900 hours of operation, resulting in operation deviating from the constraints of the reservoirs 31, 32.

In other words, in the conventional technology, stable operation was achieved about 94.2% of the time ((1900-110)/1900=). Thus, even if a learning model generated by reinforcement learning or a learning model generated by supervised learning or semi-supervised learning is constructed with higher accuracy, it is extremely difficult to maintain stable operation 100% of the time. In contrast, the operation management method according to the present embodiment, which adds noise to the input data, allows stable operation of the reservoir system 30. Note that the target for improving the accuracy of the inference results output by adding noise to the input data is not limited to the reservoir system 30, but can be applied to various targets such as waste treatment facilities equipped with storage pits and incinerators, and architectural structures such as bridges.

Figure 6 is a graph showing the control score along the operation time according to the operation management method of the water reservoir system according to one embodiment. In the operation management method according to one embodiment, the selector 350 eliminates inference results of evaluation values less than 20, for example. Figure 7 is a graph showing the control score along the operation time according to the operation management method of the water reservoir system according to the conventional technology as a comparative example. The control scores shown in Figures 6 and 7 are calculated based on the operation continuity, which is whether or not operation could be continued within the allowable range of the water level, which is the above-mentioned constraint condition, the power usage fee, and the number of starts and stops of the water distribution pumps 33 and 34. For example, the operation continuity is given 1 point when operation within the allowable range of the water level is continued for one hour, and if it cannot be continued, the points according to the amount of deviation from the allowable range of the water level are deducted. In addition, if the number of starts and stops exceeds a predetermined number of times or the power usage fee exceeds a predetermined fee, the points according to the number of times or amount of the exceedance are deducted. The deduction is preferably a linear score according to the deviation of the water level from the allowable range, but it may also be an exponential score, a logarithmic score, or a polynomial score. The control score is also called a reward or an evaluation value.

From Figure 6, it can be seen that in the operation management method of the reservoir system 30 according to one embodiment, it is possible to keep the control score in the range of 20 points or more and 23 points or less. In contrast, from Figure 7, it can be seen that in the management method of the reservoir system 30 according to the conventional technology, the control score falls below 20, or even to around -10. In other words, it can be seen that the operation management method according to the above-mentioned embodiment makes it possible to further optimize the operation of the reservoir system 30 compared to the conventional method.

(Modification)
Next, a modified example of the information processing method according to the embodiment described above will be described. FIG. 8 is a diagram for explaining a method of deriving an inference result from input data according to a modified example of the embodiment. As shown in FIG. 8, in the operation management method according to the modified example, unlike the embodiment described above, after the neural network outputs the first inference result data 321-1 to the n-th inference result data 321-n, these first inference result data 321-1 to the n-th inference result data 321-n are input to the ensemble 360. The ensemble 360 is executed by the learning unit 111 based on an algorithm such as bagging, boosting, or stacking, for example, and performs inference by majority rule by combining multiple inference result data 321. In estimation problems such as classification and regression, accuracy can be improved by ensemble learning using multiple outputs. In the case of the modified example of the embodiment, multiple inference result data 321-1 to 321-n corresponding to multiple noise-added input data 311-1 to 311-n to which noise is added are obtained, and the final inference result data 320 that obtains a more accurate inference result by ensemble learning can be obtained. Other information processing methods are the same as those in the first embodiment. Note that it is also possible to use a learning type object recognition algorithm other than the ensemble learning.

According to the embodiment described above, multiple noises different from each other are added to input data to generate multiple noise-added data, the multiple noise-added data are input to a neural network to obtain multiple inference result data corresponding to each of the multiple noise-added data, and final inference result data is output based on the obtained multiple inference result data. This makes it possible to further improve the accuracy of the inference result data obtained by inputting input data into a neural network.

(recoding media)
In the above-described embodiment, a program for executing the processing method executed by the learning device 10 or the operation management device 20 can be recorded on a recording medium that can be read by a computer or other machine or device such as a wearable device (hereinafter referred to as a computer, etc.). By having a computer, etc. read and execute the program from the recording medium, the computer, etc. functions as a mobile body control device. Here, a recording medium that can be read by a computer, etc. refers to a non-transient recording medium that accumulates information such as data and programs by electrical, magnetic, optical, mechanical, or chemical action and can be read from a computer, etc. Among such recording media, those that can be removed from a computer, etc. include, for example, a flexible disk, a magneto-optical disk, a CD-ROM, a CD-R/W, a DVD, a BD, a DAT, a magnetic tape, and a memory card such as a flash memory. In addition, a hard disk, a ROM, etc. are examples of recording media fixed to a computer, etc. Furthermore, an SSD can be used as a recording medium that can be removed from a computer, etc., or as a recording medium fixed to a computer, etc.

In addition, the programs executed by the learning device 10 and the operation management device 20 in one embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading via the network.

Other Embodiments
In one embodiment, the above-mentioned "unit" can be read as a "circuit" etc. For example, the control unit can be read as a control circuit.

Note that in the explanation of the flowcharts in this specification, the order of processing between steps is clearly indicated using expressions such as "first," "next," "then," and "continue." However, the order of processing required to implement this embodiment is not uniquely determined by these expressions. In other words, the order of processing in the flowcharts described in this specification can be changed as long as there are no contradictions.

Further advantages and modifications may be readily derived by those skilled in the art. The broader aspects of the present disclosure are not limited to the specific details and representative embodiments shown and described above. Thus, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof.

1 Operation management system 2 Network 10 Learning device 11, 21 Control unit 12, 22 Memory unit 13, 23 Communication unit 14, 24 Output unit 15, 25 Input unit 20 Operation management device 30 Reservoir system 31, 32 Reservoir 33, 34 Distribution pump 35, 36 Consumer 111 Learning unit 112 Judgment unit 113 Action selection unit 114 State input unit 115 Environment unit 116 Calculation unit 117 Simulator unit 121 Learning model 122, 300 Neural network 211 Operation plan unit 212 Operation control unit 221 Operation plan database 222 Operation information database 301 Input layer 302 Intermediate layer 303 Output layer 310 Input data 311-1 First noise-added data 311-2 Second noise-added data 311-3 Third noise-added data 311-n nth noise-added data 320, 321 Inference result data 321-1 First inference result data 321-2 Second inference result data 321-3 Third inference result data 321-n nth inference result data 330 Evaluation value 330-1 First evaluation value 330-2 Second evaluation value 330-3 Third evaluation value 330-n nth evaluation value 340 Simulator 350 Selector 360 Ensemble

Claims (12)

An information processing device including a control unit that acquires at least one input data and outputs inference result data using a neural network based on the input data,
The control unit is
The acquired input data is stored in a storage unit;
generating a plurality of noise-added data by adding a plurality of noises different from each other to the single input data read from the storage unit;
inputting the plurality of noise-added data into the neural network, and outputting a plurality of noise-added inference result data corresponding to the plurality of noise-added data;
and an information processing device that outputs the inference result data based on the plurality of noise-added inference result data. The information processing apparatus according to claim 1 , wherein the plurality of noises different from each other are noises selected based on a normal distribution. The control unit is
performing a simulation for each of the plurality of noise-added inference result data, and outputting a plurality of evaluation values corresponding to each of the plurality of noise-added inference result data;
The information processing apparatus according to claim 1 , further comprising: outputting, as the inference result data, at least one noise-added inference result data selected from the plurality of noise-added inference result data based on the plurality of evaluation values. The control unit is
performing ensemble learning on the plurality of noise-added inference result data;
The information processing device according to claim 1 , wherein an output obtained by the ensemble learning is output as the inference result data. The information processing device according to any one of claims 1 to 4, wherein the input data is data including discrete values. An operation management device for a storage facility including at least one storage tank for storing a liquid and at least one liquid delivery unit capable of supplying the liquid to the storage tank, the operation management device including an operation control unit that controls the liquid delivery unit to manage a liquid level in the storage unit,
The operation control unit is
Acquiring liquid level information including a liquid level in the storage unit from the storage facility;
Inputting information including the acquired liquid level information as stored input data into an information processing device according to any one of claims 1 to 5, and acquiring inference result data output from the information processing device in response to the stored input data as stored inference result data;
An operation management device that controls the liquid delivery unit based on the storage inference result data. The operation management device according to claim 6 , wherein the storage inference result data includes information on a liquid level that switches on/off the liquid delivery unit. The operation management device according to claim 6 or 7, wherein the storage input data includes information on a predicted value along with time of an amount of the liquid to be discharged from the storage portion at a predetermined time. The liquid delivery unit is composed of a pump whose on/off switching is controlled,
The operation management device according to any one of claims 6 to 8, wherein the storage input data includes information on starting and stopping of the pump at a predetermined time. The liquid delivery unit is composed of a pump driven by electricity,
The operation management device according to any one of claims 6 to 9, wherein the stored input data includes information on the unit price of electricity and power consumption at a predetermined time. 1. An information processing method executed by an information processing device that acquires at least one input data and outputs inference result data using a neural network based on the input data, comprising:
The acquired input data is stored in a storage unit;
generating a plurality of noise-added data by adding a plurality of noises different from each other to the single input data read from the storage unit;
inputting the plurality of noise-added data into the neural network, and outputting a plurality of noise-added inference result data corresponding to the plurality of noise-added data;
and outputting the inference result data based on the plurality of noise-added inference result data. An operation management method for a storage facility including at least one storage tank for storing a liquid and at least one liquid delivery unit capable of supplying the liquid to the storage tank, the operation management method being performed by an operation management device that controls the liquid delivery unit to manage a liquid level in the storage unit,
Acquiring liquid level information including a liquid level in the storage unit from the storage facility;
The information including the acquired liquid level information is used as storage input data, and storage inference result data obtained corresponding to the storage input data is acquired based on the information processing method according to claim 11 .
The operation management method further comprises controlling the liquid delivery unit based on the storage inference result data.

Images