JP7638146B2 - Computer system and method for controlling computer system - Google Patents

Description

The present invention relates to a system in which objects functioning as autonomous entities work in cooperation with one another.

The technology described in Patent Document 1 is known as a technology related to a system in which objects functioning as autonomous entities work in cooperation with each other.

Patent document 1 states, "An autonomous body system having a plurality of autonomous bodies, each of which has a situation grasping unit that grasps a situation, an operation decision unit that decides an operation based on the grasped situation, and an operation execution unit that executes the determined operation, the plurality of autonomous bodies of the autonomous body system include one or more first autonomous bodies and two or more second autonomous bodies, the situation grasped by the situation grasping unit of the first autonomous body includes the situation of the second autonomous body, the situation grasped by the situation grasping unit of the second autonomous body includes the result of the operation executed by the operation execution unit of the first autonomous body, and the operation decision unit of the second autonomous body decides the operation based on the result of the operation executed by the operation execution unit of the first autonomous body."

JP 2020-87206 A

Alexander Sasha Vezhnevets, Simon Osindero, Tom Schaul, Nicolas Heess, Max Jaderberg, David Silver, Koray Kavukcuoglu, "FeUdal Networks for Hierarchical Reinforcement Learning", [online], March 3, 2017, [Retrieved April 16, 2021], Internet <https://arxiv.org/abs/1703.01161>

An object that functions as an autonomous entity holds a policy (model) for determining the actions to be taken to accomplish a task. The policy differs depending on the task and the system (environment) in which the task is to be performed. The policy is generally generated and updated through a learning process.

When the task and/or system changes, there are two possible ways to acquire a new policy:

One way is to acquire new strategies by trial and error of actions based on the current strategies. With this method, there is a risk that the actions required to complete the task will not be performed.

The other method is to initialize the current policy and acquire a new policy by trial and error. This method has the problem of high learning costs.

The present invention aims to provide a method and system that supports efficient acquisition of strategies for an object functioning as an autonomous entity.

A representative example of the invention disclosed in the present application is as follows: That is, a computer system including a plurality of autonomously operating objects constituting a control system and a control system communicably connected to the plurality of objects, the control system including at least one computer, being a part of the control system, holding initial information including a definition of a subsystem including the objects and a definition of a task to be performed in the subsystem, and a plan of the task to be performed in the control system, acquiring monitoring data for grasping the state of the entire control system, generating time-series data from the monitoring data, determining identification information of the subsystem and the task to be performed in the subsystem using the initial information and the plan, generating a monitoring history in which the identification information of the subsystem, the identification information of the task, and the time-series data are associated with each other, and storing the monitoring history in a first database. and when there is a target object for which a new measure for determining an action to be performed in order to accomplish the task is required, the plan is referred to to identify the subsystem including the target object and the task to be performed in the subsystem , the monitoring history that matches the combination of the identified subsystem and the identified task is obtained from the first database , and the obtained monitoring history is transmitted to the target object, the object determines an action to be performed based on the state of the subsystem and the measure, performs the determined action, and stores an action history relating to the performed action in a second database, and when the monitoring history is received from the computer, the new measure is generated using the action history and the received monitoring history.

According to the present invention, the control system can support efficient acquisition of strategies for an object functioning as an autonomous entity by transmitting useful monitoring history. Problems, configurations, and effects other than those described above will become clear from the description of the following embodiment.

FIG. 1 is a diagram illustrating an example of the configuration of a computer system according to a first embodiment. FIG. 2 is a diagram illustrating an example of a configuration of a computer included in the control system of the first embodiment. FIG. 1 illustrates an example of a configuration of an apparatus according to a first embodiment. FIG. 4 is a diagram illustrating an example of a data structure of a monitoring history according to the first embodiment. FIG. 4 is a diagram illustrating an example of a data structure of a behavior history according to the first embodiment. 4 is a flowchart illustrating an example of a control process executed by the control system according to the first embodiment. FIG. 4 is a diagram showing an example of a screen presented by the control system according to the first embodiment. 11 is a flowchart illustrating a monitoring history transmission process executed by the control system according to the first embodiment. 11 is a flowchart illustrating a behavior decision process executed by the device according to the first embodiment. 11 is a flowchart illustrating an action plan generation process executed by the device according to the first embodiment. 13 is a flowchart illustrating a monitoring history transmission process executed by the control system according to the second embodiment. 13 is a flowchart illustrating a monitoring history transmission process executed by a control system according to a third embodiment.

The following describes an embodiment of the present invention with reference to the drawings. However, the present invention should not be interpreted as being limited to the description of the embodiment shown below. It will be easily understood by those skilled in the art that the specific configuration can be changed without departing from the concept or spirit of the present invention.

In the configuration of the invention described below, the same or similar configurations or functions are given the same reference symbols, and duplicate explanations are omitted.

In this specification, the terms "first," "second," "third," and the like are used to identify components and do not necessarily limit the number or order.

The position, size, shape, range, etc. of each component shown in the drawings, etc. may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not limited to the position, size, shape, range, etc. disclosed in the drawings, etc.

Figure 1 is a diagram showing an example of the configuration of a computer system in the first embodiment.

The computer system includes a control system 100 and a plurality of devices 101. The control system 100 and the plurality of devices 101 constitute one system (control system). The control system 100 and the devices 101 are connected via a network. The network may be, for example, a WAN (Wide Area Network) or a LAN (Local Area Network), and the connection method may be either wired or wireless.

The control system 100 and the device 101 function as an autonomous entity. Here, an autonomous entity refers to an object that grasps the state of the system including itself, selects an action according to the state, and executes the selected action. Note that an object in which a human judges the state, selects an action based on the judgment result, and performs an operation on the device 101 to execute the action can also be treated as an autonomous entity. Moreover, a human being itself can also be treated as an autonomous entity.

The control system 100 monitors the state of the control system and transmits an output to the devices 101 to encourage cooperation in order to optimize the entire control system. The output is, for example, control information and control instructions. The control system 100 monitors the control system from a bird's-eye view without acquiring information such as the state of the devices 101 and the actions performed. For example, if a logistics warehouse is the control system, the control system 100 monitors the positions of items in the logistics warehouse, the positions of the devices, and the operating status of the devices.

The control system 100 is an autonomous entity that optimizes the entire control system, and may be called holistic intelligence.

The control system 100 has a control unit 110 and a monitoring unit 111, and also holds a monitoring history database 120. The monitoring unit 111 acquires monitoring data for grasping the state of the control system, and stores the data in the monitoring history database 120 as a monitoring history 400 (see FIG. 4). The control unit 110 determines an output for optimizing the entire control system based on the monitoring history 400, and transmits the output to the target device 101. For example, the control unit 110 transmits an output for improving a KPI (Key Performance Indicator) set for the entire control system.

In the space in which the device 101 exists, cameras, sensors, etc. that acquire monitoring data are installed. As information for identifying the state of the device 101, images, videos, and monitoring data such as measured values of temperature, humidity, etc. are acquired. Note that sensors may be installed in the device 101. The sensors installed in the device 101 transmit the monitoring data to the control system 100 using a communication means different from the communication means of the device 101.

The device 101 performs a task to achieve a predetermined purpose. Specifically, the device 101 grasps the state of the system including itself using sensors, etc., and selects an action for performing the task based on the state and the output from the control system 100, and executes the action. The device 101 is part of the control system, and selects an action autonomously under the control of the system (subsystem) surrounding the device 101.

The device 101 is, for example, a picking robot, a transport vehicle, a forklift, etc. When the device 101 is a picking robot, in order to perform the task of "grasping and moving item A", the picking robot performs the following actions: extending an arm to a predetermined angle according to the storage state of item A, grasping item A at a predetermined position of item A, and placing item A at a predetermined position in the placement area.

Note that device 101 is an autonomous entity that optimizes a subsystem and may also be called an autonomous individual.

The device 101 has an action decision unit 130, an execution unit 131, and a learning unit 132, and also holds an action history database 140. The action decision unit 130 selects an action based on an action policy 150 for deciding an action, the state of the subsystem, and the output of the control system 100. For example, the action decision unit 130 decides an action that improves a KPI set for the subsystem. In the action policy 150, the subsystem is treated as an environment. The execution unit 131 executes the selected action, and stores an action history 500 (see FIG. 5) related to the executed action in the action history database 140. The learning unit 132 executes a learning process for generating the action policy 150 using the action history 500. The learning process is, for example, reinforcement learning.

Figure 2 is a diagram showing an example of the configuration of the computer 200 included in the control system 100 of Example 1.

The control system 100 includes at least one computer 200. The computer 200 has a processor 201, a memory 202, and a network interface 203. The computer 200 may have a storage device such as a hard disk drive (HDD) and a solid state drive (SSD). The computer 200 may also have input devices such as a keyboard, a mouse, and a touch panel, and an output device such as a display.

The processor 201 executes a program stored in the memory 202. The processor 201 executes processing according to the program, thereby operating as a specific functional unit (module). In the following explanation, when processing is explained with a functional unit as the subject, this indicates that the processor 201 is executing a program that realizes the functional unit.

The memory 202 stores the programs executed by the processor 201 and the information used by the programs. In this embodiment, the memory 202 stores the programs that realize the control unit 110 and the monitoring unit 111, and also stores the monitoring history database 120.

The network interface 203 is an interface for communicating with external devices via a network.

Note that each functional unit of the computer 200 may be a single functional unit consisting of multiple functional units, or a single functional unit may be divided into multiple functional units for each function.

The control system 100 may also include a storage system that manages the monitoring history database 120.

Figure 3 is a diagram showing an example of the configuration of device 101 in Example 1.

The device 101 has a processor 301, a memory 302, a network interface 303, and a drive device 304. The device 101 may also have a storage device, an input device, and an output device.

The processor 301, memory 302, and network interface 303 are the same hardware elements as the processor 201, memory 202, and network interface 203. The driving device 304 is a device that realizes an operation according to an action, and is a roller, belt, sensor, motor, lift, arm, tire, etc.

The memory 302 in this embodiment stores programs that realize the behavior decision unit 130, the execution unit 131, and the learning unit 132, and also stores the behavior history database 140.

In addition, each functional unit of the device 101 may be configured such that multiple functional units are combined into one functional unit, or one functional unit may be divided into multiple functional units for each function.

Figure 4 shows an example of the data structure of the monitoring history 400 in Example 1.

The monitoring history 400 includes a subsystem ID 401, a task ID 402, a period 403, and time series data 404.

Subsystem ID 401 is a field that stores an environment ID for identifying a subsystem. Task ID 402 is a field that stores a task ID for identifying a task performed in the subsystem.

The subsystems and tasks may be identified by a single piece of identification information. For example, a bit string may be used in which the higher bits represent the subsystems and the lower bits represent the tasks.

Time series data 404 is a field that stores time series data generated from monitoring data. If the monitoring data is images or measurement values, time series data generated from multiple images and measurement values that are continuous in time is stored in time series data 404. If the monitoring data is video, one video is stored in time series data 404. Period 403 is a field that stores the period of the time series data. For example, "2020/04/01 09:30-2020/04/01 09:40" is stored.

Figure 5 is a diagram showing an example of the data structure of the behavior history 500 in Example 1.

The behavior history 500 includes a task ID 501, a time 502, a state 503, a behavior 504, and a transition state 505.

Task ID 501 is a field that stores a task ID for identifying a task performed by device 101. Time 502 is a field that stores the time when a selected action was performed. State 503 is a field that stores data regarding the state of the subsystem before the action was performed. Action 504 is a field that stores data regarding the selected action. Transition state 505 is a field that stores data regarding the state of the subsystem after the action is performed.

FIG. 6 is a flowchart illustrating an example of a control process executed by the control system 100 of the first embodiment. FIG. 7 is a diagram illustrating an example of a screen presented by the control system 100 of the first embodiment.

The monitoring unit 111 acquires monitoring data from the control system (step S101).

At this time, the monitoring unit 111 temporarily accumulates the monitoring data until the monitoring data of the time span required for management as the monitoring history 400 (e.g., 10 minutes) is acquired. The monitoring unit 111 may also hold a plan for the tasks to be performed in the control system and determine the time span for generating the time series data based on the plan.

Next, the monitoring unit 111 generates a monitoring history 400 from the monitoring data (step S102) and stores the monitoring history 400 in the monitoring history database 120 (step S103).

Specifically, the monitoring unit 111 generates time series data from the monitoring data and determines the subsystem ID and task ID to be assigned. The monitoring unit 111 also generates a monitoring history 400 that includes the subsystem ID, the task ID, the time series data, and the period of the time series data.

Here, the following methods can be considered for assigning subsystem IDs and task IDs:

(Method 1) Set the definition of a subsystem and the correspondence between the subsystem ID and the definition of a task and the correspondence between the task ID in the control system 100 as initial information. Also, set a task plan in the control system 100. The monitoring unit 111 determines the subsystem ID and task ID based on the acquisition position of the monitoring data, the task plan, and the initial information.

(Method 2) The monitoring unit 111 presents the user with a screen 700 for inputting a partial system ID and a task ID. The screen 700 includes a time series data display field 701, a partial system ID input field 702, a task ID input field 703, and a register button 704. The time series data display field 701 displays the time series data included in the monitoring history 400. Note that the user may be able to select the time series data to be included in the monitoring history 400. The partial system ID input field 702 and the task ID input field 703 are fields for inputting a partial system ID and a task ID. The user refers to the time series data and inputs values into the partial system ID input field 702 and the task ID input field 703. The register button 704 is an operation button for registering the values input into the partial system ID input field 702 and the task ID input field 703.

Return to the explanation of Figure 6.

The control unit 110 grasps the state of the entire control system based on the monitoring history 400, determines an output for optimizing the entire control system, and transmits the determined output to the device 101 (step S104). For example, the control unit 110 determines the output using a measure (not shown) and the monitoring history 400. In the measure of the control unit 110, the control system is treated as the environment. The control unit 110 may grasp the state of the entire control system based on the monitoring data.

In this way, the control system 100 transmits an output to the device 101 that encourages optimization of the entire control system without understanding the characteristics of each device 101, such as the KPIs of the subsystems.

The control system 100 may hold an action history database separate from the monitoring history database 120. In this case, the control unit 110 generates an action history including the time when the output was transmitted, the state of the control system before the output was transmitted, the determined output, and the state of the control system after the output was transmitted, and stores the action history in the action history database. The control system 100 may use the action history database to execute a learning process for generating or updating a measure.

Figure 8 is a flowchart explaining the monitoring history transmission process executed by the control system 100 in the first embodiment.

When the control unit 110 of the control system 100 determines that a new action plan 150 is required for a certain device 101 due to a change in either a subsystem or a task, the control unit 110 executes a monitoring history transmission process. For example, the need for a new action plan 150 can be determined based on input from a user.

In the following description, the device 101 for which a new action plan 150 is required is referred to as the target device 101.

The control unit 110 identifies the subsystems and tasks to which the new behavioral policy 150 is to be applied (step S201). For example, the control unit 110 identifies the subsystems and tasks to which the new behavioral policy 150 is to be applied based on the task plan or user input.

Next, the control unit 110 selects the monitoring history 400 corresponding to the confirmed subsystem and task from the monitoring history database 120 (step S202). Specifically, the control unit 110 selects the monitoring history 400 whose subsystem ID 401 and task ID 402 values match the identification information of the confirmed subsystem and task.

Next, the control unit 110 sends a learning instruction including the selected monitoring history 400 to the target device 101 (step S203).

Figure 9 is a flowchart explaining the action decision process executed by the device 101 of Example 1.

The device 101 uses a sensor or the like to acquire status data to grasp the status of the subsystem (step S301).

The action decision unit 130 of the device 101 decides on an action based on the state data and the action plan 150, and the execution unit 131 of the device 101 executes the decided action (step S302). When an output is received from the control system 100, the execution unit 131 decides on an action based on the state data, the action plan 150, and the output. The execution unit 131 does not use the output received from the control system 100 as is, but autonomously decides on an action to improve the KPIs, etc. of the subsystem.

The execution unit 131 of the device 101 generates an action history 500 regarding the performed action and stores the action history 500 in the action history database 140 (step S303).

In this way, device 101 performs actions to optimize the subsystem without understanding the characteristics of other devices 101.

Figure 10 is a flowchart explaining the action plan generation process executed by the device 101 of Example 1.

The learning unit 132 of the device 101 executes an action plan generation process when an execution trigger occurs. For example, the execution trigger occurs when a certain amount of time has passed, when a certain number of action histories 500 have been accumulated, or when a learning instruction is received from the control system 100. Here, the action plan generation process that is executed when a learning instruction is received from the control system 100 will be described.

The action plan generation process may be executed before the start of a task, or may be executed while the task is being performed. By generating a new action plan 150 while the task is being performed, it is possible to quickly respond to task switching.

The learning unit 132 reads the behavior history 500 stored in the behavior history database 140 and the monitoring history 400 received from the control system 100 (step S401).

Next, the learning unit 132 executes a learning process using the monitoring history 400 and the behavior history 500 (step S402). In this embodiment, reinforcement learning is executed. Since reinforcement learning is a well-known technology, a detailed description is omitted.

Next, the learning unit 132 saves the new behavioral policy 150 generated by the learning process (step S403).

The learning unit 132 may update the current behavior policy 150 to the new behavior policy 150, or may store the new behavior policy 150 as stock without updating the current behavior policy 150.

The effect of the invention according to the first embodiment will be explained using a concrete example. Here, a picking robot is considered as the device 101. The picking robot is assumed to be performing task A, which is to "independently grasp item A on the transport lane and move it to a specified position." In addition, the picking robot is assumed to be newly assigned task B, which is to "grasp item A in cooperation with two picking robots and move it to a specified position." However, it is assumed that the subsystems are the same.

When trial and error of behavior is performed using the current behavior policy 150, there is a possibility that the picking robots will try to avoid collisions and will not be able to successfully grasp the item. In this case, a new policy cannot be generated. One possible method is for the picking robot to initialize the behavior policy 150, perform trial and error of behavior, and update the behavior policy 150. However, this method has the problem of high learning processing costs.

The control system 100 of the first embodiment can narrow down the trial-and-error behavior by transmitting the monitoring history 400 of the picking robot that has the same subsystem and is performing task B. Therefore, the cost of the learning process can be reduced.

As described above, according to the first embodiment, the control system 100 can transmit to the device 101 a monitoring history 400 that is useful for generating a new action plan 150. The device 101 can efficiently generate the action plan 150 by using the action history 500 and the monitoring history 400.

In the second embodiment, the control system 100 divides the monitoring history 400 into subtask units and transmits the divided monitoring history 400 to the device 101. Below, the second embodiment will be described, focusing on the differences from the first embodiment.

The configuration of the computer system in Example 2 is the same as that in Example 1. The hardware configuration and software configuration of the control system 100 and the device 101 in Example 2 are the same as those in Example 1. The control process, the action decision process, and the action plan generation process in Example 2 are the same as those in Example 1.

In the second embodiment, the monitoring history transmission process is partially different. FIG. 11 is a flowchart explaining the monitoring history transmission process executed by the control system 100 in the second embodiment.

The control unit 110 identifies the subsystems and tasks to which the new behavioral strategy 150 is to be applied (step S201).

Next, the control unit 110 selects the monitoring history 400 corresponding to the confirmed subsystem and task identification information (subsystem ID and task ID) from the monitoring history database 120 (step S202).

Next, the control unit 110 generates a partial monitoring history by dividing the selected monitoring history 400 into subtask units (step S211). The following methods can be used as a division method.

(Method 1) The control unit 110 divides the monitoring history 400 into subtask units using the method described in Non-Patent Document 1. Note that by using the method described in Non-Patent Document 1, causal relationships between subtasks, such as the transition order between subtasks, can also be obtained.

(Method 2) The control unit 110 identifies the time range in which each subtask is performed based on the task plan, and divides the monitoring history 400 into subtask units based on the identified time range.

Next, the control unit 110 transmits a learning instruction including the partial monitoring history to the target device 101 (step S212). At this time, the control unit 110 may include only the partial monitoring history of important subtasks in the learning instruction. In addition, the control unit 110 may include only the partial monitoring history of subtasks that are related based on the causal relationship between the subtasks in the learning instruction.

According to the second embodiment, by transmitting a partial monitoring history in which the monitoring history 400 is divided into subtask units, it is possible to narrow down the trial-and-error behavior on a subtask basis. Therefore, the learning unit 132 can generate the behavior policy 150 more efficiently.

The monitoring unit 111 may generate a partial monitoring history from the monitoring history 400 using the method described in Non-Patent Document 1, and store the partial monitoring history in the monitoring history database 120. In this case, the monitoring unit 111 includes identification information of the subtask in the partial monitoring history. The control unit 110 selects the partial monitoring history generated by the monitoring unit 111, and transmits it to the target device 101.

In Example 3, the control system 100 acquires an action history 500 from a device 101 that is the same subsystem as the subsystem to which the new action policy 150 is applied and that performs the same task as the task to which the new action policy 150 is applied, and transmits the action history 500 to the target device 101. Below, Example 3 will be described, focusing on the differences from Example 1.

The configuration of the computer system in Example 3 is the same as that in Example 1. The hardware configuration and software configuration of the control system 100 and the device 101 in Example 3 are the same as those in Example 1. The control process, the action decision process, and the action plan generation process in Example 3 are the same as those in Example 1.

In the third embodiment, the monitoring history transmission process is partially different. FIG. 12 is a flowchart explaining the monitoring history transmission process executed by the control system 100 in the third embodiment.

The control unit 110 identifies the subsystems and tasks to which the new behavioral strategy 150 is to be applied (step S201).

Next, the control unit 110 identifies a source device 101 that holds the behavior history 500 corresponding to the confirmed subsystem and task (step S221). For example, the control unit 110 identifies the source device 101 based on the task plan.

Next, the control unit 110 acquires the behavior history 500 from the source device 101 (step S222).

Next, the control unit 110 sends a learning instruction including the behavior history 500 to the target device 101 (step S223).

The control unit 110 may obtain the monitoring history 400 from the monitoring history database 120, as in the first embodiment.

According to Example 3, by transmitting the behavioral history 500 of the other device 101 to the target device 101, the learning unit 132 can generate the behavioral policy 150 more efficiently.

The present invention is not limited to the above-described embodiments, but includes various modified examples. For example, the above-described embodiments are provided to explain the present invention in detail, and are not necessarily limited to those including all of the described configurations. In addition, it is possible to add, delete, or replace part of the configuration of each embodiment with another configuration.

The above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example by designing them as integrated circuits. The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.

In addition, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C/C++, perl, Shell, PHP, Python, Java (registered trademark), etc.

Furthermore, the program code of the software that realizes the functions of the embodiment may be distributed over a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or storage medium.

In the above examples, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

100 Control system 101 Device 110 Control unit 111 Monitoring unit 120 Monitoring history database 130 Action decision unit 131 Execution unit 132 Learning unit 140 Action history database 150 Action policy 200 Computer 201, 301 Processor 202, 302 Memory 203, 303 Network interface 304 Driving device 400 Monitoring history 500 Action history 700 Screen 701 Time series data display field 702 Subsystem ID input field 703 Task ID input field 704 Registration button

Claims (12)

A computer system including a plurality of autonomously operating objects constituting a control system, and a control system communicatively connected to the plurality of objects,
The control system includes:
at least one computer;
a part of the control system, which holds initial information including a definition of a subsystem including the object and a definition of a task to be performed in the subsystem, and a plan of the task to be performed in the control system;
Obtaining monitoring data for grasping the overall state of the control system;
generating time-series data from the monitoring data, determining identification information of the subsystem and a task to be performed in the subsystem using the initial information and the plan, generating a monitoring history in which the identification information of the subsystem, the identification information of the task, and the time-series data correspond to each other, and storing the generated monitoring history in a first database;
When there is a target object for which a new policy needs to be generated to determine an action to be taken to accomplish the task, referring to the plan, the subsystem including the target object and the task to be accomplished in the subsystem are identified;
obtaining , from the first database, the monitoring history that matches the combination of the identified subsystem and the identified task ;
Transmitting the acquired monitoring history to the target object;
The object is
determining an action to take based on the state of the subsystem and the policy;
Carrying out said determined action;
storing an action history relating to the performed action in a second database;
a computer system, when receiving the monitoring history from the computer, generating the new measure using the action history and the received monitoring history. 2. The computer system of claim 1,
The control system includes:
generating a partial monitoring history by dividing the monitoring history acquired from the first database into subtask units constituting the task corresponding to the monitoring history;
A computer system which transmits the partial monitoring history to the target object. 3. The computer system of claim 2,
The control system includes:
Analyzing causal relationships between the subtasks;
A computer system characterized in that the partial monitoring history related by causal relationships is transmitted. 2. The computer system of claim 1,
The monitoring history includes time information,
The control system includes :
generating a partial monitoring history by dividing the monitoring history acquired from the first database into subtask units constituting the task corresponding to the monitoring history based on the time information and the plan included in the monitoring history;
A computer system which transmits the partial monitoring history to the target object. 2. The computer system of claim 1,
A computer system, wherein the object generates the new policy during the execution of the task using the policy. 6. The computer system of claim 5,
A computer system, wherein the object stores the new policy or updates the policy currently in use to the new policy. A method for controlling a computer system including a plurality of autonomously operating objects constituting a control system, and a control system communicatively connected to the plurality of objects, comprising:
The control system includes:
at least one computer;
a part of the control system, which holds initial information including a definition of a subsystem including the object and a definition of a task to be performed in the subsystem, and a plan of the task to be performed in the control system;
The method for controlling a computer system includes:
A first step in which the control system acquires monitoring data for grasping a state of the entire control system;
a second step in which the control system generates time-series data from the monitoring data, determines identification information of the subsystem and a task to be performed in the subsystem using the initial information and the plan, generates a monitoring history in which the identification information of the subsystem, the identification information of the task, and the time-series data correspond to each other, and stores the generated monitoring history in a first database;
a third step of, when a target object for which the control system needs to newly generate a strategy for determining an action to be performed in order to accomplish the task exists, identifying the subsystem including the target object and a task to be performed in the subsystem by referring to the plan;
a fourth step of acquiring, from the first database, the monitoring history that matches the combination of the identified subsystem and the identified task;
a fifth step of the control system transmitting the acquired monitoring history to the target object;
a sixth step of determining an action to be taken by the object based on the state of the subsystem and the policy;
a seventh step in which the object performs the determined action;
an eighth step of storing an action history relating to the action performed by the object in a second database;
A method for controlling a computer system, comprising: a ninth step of, when the object receives the monitoring history from the computer, generating the new measure using the behavior history and the received monitoring history. 8. A method for controlling a computer system according to claim 7, comprising:
The fifth step includes:
a tenth step of generating a partial monitoring history by dividing the monitoring history acquired from the first database into subtask units constituting the task corresponding to the monitoring history by the control system;
an eleventh step in which the control system transmits the partial monitoring history to the target object. 9. A method for controlling a computer system according to claim 8, comprising:
the tenth step includes a step in which the control system analyzes a causal relationship between the subtasks;
The method for controlling a computer system, wherein the eleventh step includes a step in which the control system transmits the partial monitoring history associated with a causal relationship. 8. A method for controlling a computer system according to claim 7, comprising:
The monitoring history includes time information,
The fifth step includes:
a step in which the control system generates a partial monitoring history by dividing the monitoring history acquired from the first database into subtask units constituting the task corresponding to the monitoring history, based on the time information and the plan included in the monitoring history;
a step of the control system transmitting the partial monitoring history to the target object. 8. A method for controlling a computer system according to claim 7, comprising:
A method for controlling a computer system, wherein the seventh step includes a step of generating the new policy by the object during execution of the task using the policy. A method for controlling a computer system according to claim 11, comprising:
A method for controlling a computer system, wherein the eighth step includes a step in which the object saves the new policy or updates the policy currently being used to the new policy.

Images