JP6809705B2 - Simulation system - Google Patents

Description

The present invention relates to a simulation system, and more particularly to a simulation system that simulates a robot or a human agent in a coexistence environment between a robot such as a communication robot and a human.

Conventional simulation devices of this type are disclosed in Non-Patent Documents 1 and 2. In these simulation devices of Non-Patent Documents 1 and 2, by simulating and visualizing sensors and actuators, it is possible to more easily simulate the adaptation of a robot in a coexistence environment with humans.

https://www.openrobots.org/morse/doc/stable/morse.html http://gazebosim.org/

In order to debug a robot that operates in such a coexisting environment, it is necessary to repeatedly debug the behavior of people who cause problems. However, it is difficult to reproduce a specific situation (a situation that occurs in a limited time zone, a situation that occurs only when there are a large number of people, etc.) because the prior art such as non-patent documents merely simulates the sensor. However, there is a problem that the necessary debugging cannot be performed, and finally it is necessary to check the operation in the real space.

Therefore, the main object of the present invention is to provide a novel simulation system.

Another object of the present invention is to provide a simulation system that facilitates the necessary debugging.

The present invention has adopted the following configuration in order to solve the above problems. The reference numerals and supplementary explanations in parentheses indicate the correspondence with the embodiments described later in order to assist the understanding of the present invention, and do not limit the present invention in any way.

The first invention is a simulation system that simulates the behavior of a human agent input into a virtual simulation space, a human generation module that inputs a human agent into a virtual simulation space, and a human agent according to the first instruction set for the existence of a robot. It is a simulation system including a moving module for performing a reaction action, and a situation reproduction means for reproducing a specific situation in a virtual simulation space by partially changing the first instruction set by the second instruction set.

In the first invention, the simulation system (10: a reference reference numeral exemplifying a corresponding portion in the embodiment; the same applies hereinafter) includes a man-generated module (110) and a mobile module (114), and the man-generated module (110). ) Generates a human agent according to, for example, the appearance rate data (118) and inputs it into the virtual simulation space (22). The mobile module (114) causes a human agent to perform a routine reaction action to the robot according to a first instruction set such as an objective. The situation reproducing means (122) reproduces a specific situation in the virtual simulation space by partially changing the first instruction set.

According to the first invention, a specific situation can be reproduced in a virtual simulation space by partially modifying the first instruction set that causes a human agent to generate a routine robot reaction behavior, so that debugging is easy. Become.

The second invention is a simulation system that simulates the behavior of a human agent input into a virtual simulation space, a human generation module that inputs a human agent into a virtual simulation space, and causes the human agent to perform a reaction action to the existence of a robot. A display that displays the virtual simulation space including the first memory that sets the first instruction set, the movement module that controls the movement of the human agent according to the first instruction set of the first memory, and the human agent that is moved by the movement module. It is a simulation system to be equipped.

In the second invention, the simulation system (10) comprises a human simulator (26) including a human generation module (110) and a mobile module (114). The human generation module (110) generates a human agent according to, for example, the appearance rate data (118) and inputs it into the virtual simulation space (22). The mobile module (114) causes a human agent to perform a routine reaction action to the robot according to a first instruction set preset in the first memory, such as an objective. The display (105) connected to the human simulator (26) displays the virtual simulation space including the human agent moved by the mobile module.

According to the second invention, it is possible to visually confirm the reaction behavior to the robot according to the first instruction set of the human agent in the virtual simulation space, so that the simulation can be performed efficiently.

The third invention is subordinate to the second invention and further comprises a second memory that sets a second instruction set that is higher than the first instruction set and can partially change the first instruction set, and is man-generated. A simulation system in which at least one of the modules and the mobile module is controlled by a second instruction set.

In the third invention, the man-generating module (110) and / or the moving module (114) creates a human agent and / or moves the human agent according to a second instruction set described in the second memory.

According to the third invention, since the first instruction set can be partially rewritten by the second instruction set, the behavior of the person agent according to the first instruction set can be partially changed. Therefore, the specific situation required for debugging can be easily reproduced, and the efficiency of debugging can be expected.

The fourth invention is subordinate to the third invention , further comprising means for the human agent to appear in the virtual simulation space by the human generation module according to the appearance rate, and the second instruction set is by the human generation module regardless of the appearance rate. It is a simulation system that makes a human agent appear in the virtual simulation space.

In the fourth invention, for example, the human generation module (110) generates a human agent according to the appearance rate from the appearance rate data (118). The second instruction set can generate any kind of human agent at any timing by the human generation module (110) regardless of its appearance rate.

According to the fourth invention, for debugging, a large number of human agents can be put into the virtual simulation space at a time to easily reproduce a specific situation, and the efficiency of debugging can be described.

According to the present invention, necessary debugging can be easily performed by using a test code that can easily reproduce the behavior of a human agent on a robot.

The above-mentioned object, other object, feature and advantage of the present invention will become more apparent from the detailed description of the following examples made with reference to the drawings.

FIG. 1 is a block diagram showing a simulation system according to an embodiment of the present invention. FIG. 2 is an illustrated diagram showing an example of a robot to which the present invention is applied. FIG. 3 is a block diagram showing an electrical configuration of the robot shown in FIG. FIG. 4 is an illustrated diagram showing an example of the virtual simulation space of FIG. FIG. 5 is a functional block diagram showing details of the human simulator of FIG. FIG. 6 is an illustrated diagram showing an example of an objective model of a human agent. FIG. 7 is a schematic diagram showing another example of an objective model of a human agent. FIG. 8 is a schematic diagram showing another example of an objective model of a human agent. FIG. 9 is an illustrated diagram showing an example of movement of a human agent in a part of the virtual simulation space of FIG. FIG. 10 is a flow chart showing an example of operation according to the objective of the simulation system of the embodiment of FIG. FIG. 11 is a table showing an example of the script. FIG. 12 is a flow chart showing another example of the operation according to the script of the simulation system of the embodiment of FIG.

With reference to FIG. 1, the simulation system 10 of this embodiment includes a robot 12, an operating device 14 that remotely controls the robot 12, and a simulation device 16 that simulates a robot agent or a human (pedestrian) agent that simulates the robot 12. Including. In the simulation system 10, it is realized by one or two or more computers (processors) except for the robot 12.

The robot 12 of this embodiment is a robot that operates in a coexistence environment with humans, such as the communication robot shown in FIG. Since such a robot 12 is affected by the behavior of a person existing in the environment, the simulation device 16 of this embodiment is used when simulating the behavior of the robot 12 according to a robot application program (robot behavior determination program). It also supports the development of safe robot application programs for robot 12 by simulating human behavior.

Here, the configuration of the robot 12 will be described with reference to FIGS. 2 and 3 to the extent necessary for understanding the present invention. The robot 12 includes a carriage 30, and two wheels 32 and one trailing wheel 34 for moving the robot 12 are provided on the lower surface of the carriage 30. The two wheels 32 are independently driven by a wheel motor 36 (see FIG. 3), and the carriage 30, that is, the robot 12 can be moved in any direction in the front-rear, left-right, and left-right directions.

A cylindrical sensor mounting panel 38 is provided on the trolley 30, and a large number of distance sensors 40 are mounted on the sensor mounting panel 38. These distance sensors 40 measure the distance to an object (such as a person or an obstacle) around the robot 12 by using, for example, infrared rays or ultrasonic waves.

A body 42 is provided upright on the sensor mounting panel 38. Further, the above-mentioned distance sensor 40 is further provided in the upper front center of the body 42 (position corresponding to the chest of a person), and measures the distance in front of the robot 12 mainly to a person. Further, the body 42 is provided with a support column 44 extending from substantially the center of the upper end portion on the side surface side thereof, and an omnidirectional camera 46 is provided on the support column 44. The omnidirectional camera 46 captures the surroundings of the robot 12 and is distinguished from the eye camera 70 described later. As the omnidirectional camera 46, a camera using a solid-state image sensor such as a CCD or CMOS can be adopted.

Upper arms 50R and upper arms 50L are provided at the upper ends of both side surfaces of the body 42 (positions corresponding to human shoulders) by shoulder joints 48R and shoulder joints 48L, respectively. Although not shown, the shoulder joint 48R and the shoulder joint 48L each have three orthogonal degrees of freedom. That is, the shoulder joint 48R can control the angle of the upper arm 50R around each of the three orthogonal axes. One axis (yaw axis) of the shoulder joint 48R is an axis parallel to the longitudinal direction (or axis) of the upper arm 50R, and the other two axes (pitch axis and roll axis) are orthogonal to the axis from different directions. It is the axis to do. Similarly, the shoulder joint 48L can control the angle of the upper arm 50L around each of the three orthogonal axes. One axis (yaw axis) of the shoulder joint 48L is an axis parallel to the longitudinal direction (or axis) of the upper arm 50L, and the other two axes (pitch axis and roll axis) are orthogonal to the axes from different directions. It is the axis to do.

Further, an elbow joint 52R and an elbow joint 52L are provided at the tips of the upper arm 50R and the upper arm 50L, respectively. Although not shown, the elbow joint 52R and the elbow joint 52L each have a degree of freedom of one axis, and the angles of the forearm 54R and the forearm 54L can be controlled around the axis of this axis (pitch axis).

A hand 56R and a hand 56L corresponding to a human hand are provided at the tips of the forearm 54R and the forearm 54L, respectively. These hands 56R and 56L are configured to be openable and closable, although detailed illustration is omitted, so that the robot 12 can grip or hold an object using the hands 56R and 56L. However, the shapes of the hands 56R and 56L are not limited to the shapes of the embodiments, and may have shapes and functions that closely resemble those of human hands.

Although not shown, the front surface of the trolley 30, the portion corresponding to the shoulder including the shoulder joint 48R and the shoulder joint 48L, the upper arm 50R, the upper arm 50L, the forearm 54R, the forearm 54L, the hand 56R and the hand 56L, respectively. , A contact sensor 58 (shown comprehensively in FIG. 3) is provided. The contact sensor 58 on the front surface of the trolley 30 detects the contact of a person or other obstacle with the trolley 30. Therefore, when the robot 12 comes into contact with an obstacle during its own movement, it can detect the contact and immediately stop driving the wheels 32 to suddenly stop the movement of the robot 12. In addition, the other contact sensor 58 detects whether or not the respective parts have been touched.

A neck joint 60 is provided at the upper center of the body 42 (a position corresponding to a person's neck), and a head 62 is provided above the neck joint 60. Although not shown, the neck joint 60 has three degrees of freedom, and the angle can be controlled around each of the three axes. A certain axis (yaw axis) is an axis that goes directly above the robot 12 (vertically upward), and the other two axes (pitch axis, roll axis) are axes that are orthogonal to each other in different directions.

A speaker 64 is provided on the head 62 at a position corresponding to a person's mouth. The speaker 64 is used by the robot 12 to communicate with people around it by voice. Further, a microphone 66R and a microphone 66L are provided at positions corresponding to human ears. Hereinafter, the right microphone 66R and the left microphone 66L may be collectively referred to as a microphone 66. The microphone 66 captures ambient sounds, especially human voice, which is the object of communication.

Further, a right eyeball portion 68R and a left eyeball portion 68L are provided at positions corresponding to the human eye. The right eyeball portion 68R and the left eyeball portion 68L include a right eye camera 70R and a left eye camera 70L, respectively. Hereinafter, the right eyeball portion 68R and the left eyeball portion 68L may be collectively referred to as an eyeball portion 68. Further, the right eye camera 70R and the left eye camera 70L may be collectively referred to as an eye camera 70.

The eye camera 70 captures a face of a person approaching the robot 12, other parts or objects, and captures a corresponding video signal. In this embodiment, the robot 12 detects the line-of-sight directions (vectors) of the left and right eyes of the person by the video signal from the eye camera 70.

Further, as the eye camera 70, the same camera as the omnidirectional camera 46 described above can be used. For example, the eye camera 70 is fixed in the eyeball portion 68, and the eyeball portion 68 is attached to a predetermined position in the head 62 via an eyeball support portion (not shown). Although not shown, the eyeball support portion has two degrees of freedom, and the angle can be controlled around each of these axes. For example, one of these two axes is an axis (yaw axis) in the upward direction of the head 62, and the other is orthogonal to one axis and orthogonal to the front side (face) of the head 62. It is the axis (pitch axis) in the direction of By rotating the eyeball support portion around each of these two axes, the tip (front) side of the eyeball portion 68 or the eye camera 70 is displaced, and the camera axis, that is, the line-of-sight direction is moved. The installation positions of the speaker 64, the microphone 66, and the eye camera 70 are not limited to the relevant parts, and may be provided at appropriate positions.

As described above, the robot 12 of this embodiment has independent 2-axis drive of the wheels 32, 3 degrees of freedom of the shoulder joint 48 (6 degrees of freedom on the left and right), 1 degree of freedom of the elbow joint 52 (2 degrees of freedom on the left and right). It has a total of 17 degrees of freedom, including 3 degrees of freedom for the neck joint 60 and 2 degrees of freedom for the eyeball support (4 degrees of freedom on the left and right).

FIG. 3 is a block diagram showing an electrical configuration of the robot 12. With reference to FIG. 3, the robot 12 includes one or more processor 80s. The processor 80 is connected to the memory 84, the motor control board 86, the sensor input / output board 88, and the voice input / output board 90 via the bus 82.

The memory 84 includes a ROM, an HDD, and a RAM, although not shown. Various programs are stored in advance in the ROM and HDD.

The motor control board 86 is composed of, for example, a DSP, and controls the drive of each axis motor such as each arm, neck joint 60, and eyeball portion 68. That is, the motor control board 86 receives control data from the processor 80 and controls the angles of the two axes of the right eyeball portion 68R (in FIG. 3, collectively referred to as “right eyeball motor 92”). Control the rotation angle of. Similarly, the motor control board 86 receives control data from the processor 80 and controls the angles of the two axes of the left eyeball portion 68L (in FIG. 3, collectively referred to as the "left eyeball motor 94"). The rotation angle of) is controlled.

Further, the motor control board 86 receives control data from the processor 80, and is a total of three motors that control the angles of the three orthogonal axes of the shoulder joint 48R and one motor that controls the angles of the elbow joint 52R. The rotation angles of the four motors (collectively referred to as "right arm motor 96" in FIG. 3) are controlled. Similarly, the motor control board 86 receives control data from the processor 80, and has three motors that control the angles of the three orthogonal axes of the shoulder joint 48L and one motor that controls the angles of the elbow joint 52L. The rotation angles of a total of four motors (collectively referred to as "left arm motor 98" in FIG. 3) are controlled.

Further, the motor control board 86 receives control data from the processor 80 and controls the angles of the three orthogonal axes of the neck joint 60 (in FIG. 3, collectively referred to as “head motor 100”). ) Rotation angle is controlled. Then, the motor control board 86 receives control data from the processor 80 and controls the rotation angles of the two motors (collectively referred to as “wheel motors 36” in FIG. 3) that drive the wheels 32.

A hand actuator 101 is further coupled to the motor control board 86, and the motor control board 86 receives control data from the processor 80 and controls the opening and closing of the hands 56R and 56L.

Like the motor control board 86, the sensor input / output board 88 is composed of a DSP, and takes in signals from each sensor and gives them to the processor 80. That is, data regarding the reflection time from each of the distance sensors 40 is input to the processor 80 through the sensor input / output board 88. Further, the video signal from the omnidirectional camera 46 is input to the processor 80 after being subjected to predetermined processing on the sensor input / output board 88 as needed. The video signal from the eye camera 70 is also input to the processor 80 in the same manner. Further, signals from the plurality of contact sensors 58 (collectively referred to as “contact sensor 58” in FIG. 3) described above are given to the processor 80 via the sensor input / output board 88. Similarly, the voice input / output board 90 is also composed of a DSP, and a voice or a voice according to the voice synthesis data given from the processor 80 is output from the speaker 64. Further, the voice input from the microphone 66 is given to the processor 80 via the voice input / output board 90.

Further, the processor 80 is connected to the communication LAN board 102 via the bus 82. The communication LAN board 102 is composed of, for example, a DSP, and supplies transmission data given by the processor 80 to the wireless communication module 104, and the wireless communication module 104 transmits transmission data to a server (not shown) or the like via a network. To do. Further, the communication LAN board 102 receives data via the wireless communication module 104, and gives the received data to the processor 80.

Returning to FIG. 1, the operating device 14 is basically composed of one or more computers or processors, and the robot application program 18 is set in the memory (not shown) of the processor, and the robot application is set. The program is given from the computer, for example, wirelessly to the robot 12 or the simulation device 16. The robot application program 18 is an application program for the robot 12. For example, when the robot 12 is operated as a guide robot, the robot 12 executes the operation for that purpose.

Environmental data 20 is further stored in the memory of the operating device 14 in advance. The environment data 20 mainly includes the map data of the virtual simulation space 22 shown in FIG. 4 and the appearance rate data of the human agent in the virtual simulation space 22. The environment and the objects in the environment are represented as 3D model data and visualized as 3D virtual space.

The virtual simulation space 22 shown in FIG. 4 assumes a three-dimensional closed space having an entrance / exit 22a indicated by a dotted line rectangle in the figure. In this virtual simulation space 22, a passage 22b indicated by a thick black line is set, and the robot 12 and the human agent pass through the passage 50, the open space, and the like, or exist there. These passages 22b and the plaza are partitioned by objects such as buildings and walls, and branch or merge at the places indicated by circles in the figure.

The simulation device 16 is basically composed of one or more processors, and includes a robot simulator 24 that simulates a robot 12 and a human simulator 26 that simulates a human agent such as a pedestrian, and the simulation results of each are included. Is input to the physics engine 28.

The robot simulator 24 is, for example, the MORSE simulator exemplified as Non-Patent Document 1 above, and this MORSE simulator provides many models of sensors, actuators and robots similar to the robot 12, and for such models. API (Application Programming Interface) can also be used.

In the sensor simulation, for example, in the case of a camera image, optical calculation is performed from the camera viewpoint to generate image information for each camera viewpoint. In the case of a laser range finder, the distance is measured from the sensor in each direction by imitating a scan by a laser, and the result is saved as a sensor value.

When developers prepare robot application programs for robots, they access these APIs to send sensor data (eg, read distance from a laser range meter) and commands to actuators (eg, read). Movement speed).

The human simulator 26 periodically generates new pedestrians, updates their positions, and removes them as they leave the simulated environment (virtual simulation space 22). A pedestrian or person is represented as a three-dimensional object, placed in the virtual simulator space 22, and its walking motion is animated using an animation engine.

However, since the human simulator 26 is given the environmental data 20 as shown in FIG. 1, the human simulator 26 periodically generates new pedestrians with a predetermined probability called the appearance rate included in the environmental data. To do. Then, the pedestrian is assigned to one of the doorways 22a shown in FIG. 4 and is set to go to the other one.

However, pedestrians are often generated as groups (like families and couples), and when they are displayed in groups, the size is defined by the distribution of group members, assuming that all members share the same purpose.

The physics engine 28 calculates what kind of movement path a robot or a human agent follows based on a physical force, a law, or the like. More specifically, the physics engine 28 executes a process of executing the movements intended by each agent (including the robot) from the robot simulator 24 and the human simulator 26 in the virtual simulation space 22 (FIG. 4). At this time, if there is no interference with other obstacles, structures or other agents, the intended movement is generated. However, if there is interference, it processes the actual movement of each agent according to the laws of physics (laws of mechanics). Then, the integrated simulation data is output from the physics engine 28.

The simulation device 16 is provided with a display 105, and the display 105 displays, for example, an image of a virtual simulation space as shown in FIG. 4 together with a simulated robot or a human agent existing therein. .. For example, a human agent is represented as a three-dimensional object, placed in a virtual simulation space 22, and its walking motion is animated using an animation engine. The developer of the robot application program 18 creates and modifies the action determination program (application program) of the robot 12 while checking the image displayed on the display 105.

The human simulator 26 will be described in detail with reference to FIG. Like the robot simulator 24, the human simulator 26 is a function realized by a computer or a processor of the simulation device 16, and includes a human generation module 110. As described above, the human generation module 110 represents a new pedestrian (human agent) according to the appearance rate data 108 included in the environmental data 20 in the virtual simulation space 22 shown in FIG. 4 represented by the map data 106. , Generate periodically.

The appearance rate data 108 is the shape of a person, that is, the shape (pattern) of a person such as an adult, a child, a man, or a woman, and the appearance rate of each person for the person agent to be input to the virtual simulation space 22 of the simulation device 16. Is a preset database, and a human agent with a set shape is generated with a set probability.

The appearance rate data 108 may hold different data sets for each day of the week or time so that the simulation can be performed by designating the day of the week or time. Then, you can simulate various things such as what happens when you move the robot at a certain time.

However, if the appearance rate data 108 indicates that the generated human agents form a group, the human generation module 110 is a group having a member configuration according to the group member data 112 set as a simulation parameter. Generate a human agent.

The movement module 114 moves the human agent generated by the human generation module 110 in the virtual simulation space 22 (FIG. 4) based on the objective model 116. Here, the objective is the way in which the human agent reacts when the robot agent (not shown) enters the field of view of the human agent (not shown) in the virtual simulation space 22, that is, the person It is a set of instructions that defines the behavior patterns for robots that can actually be taken in real space. In this embodiment, four typical reaction modes (objectives) described later are set as a model, and when the human agent is moved, the movement module 114 is moved according to the objective indicated by the model. That is, the objective 116 functions as a first instruction set that causes a human agent to perform a reaction action to the existence of a robot in a virtual simulation space. These objectives are stored in the first memory (not shown) of the simulation system 10.

The available objectives preset in the objective model 116 are known to the developer who intends to debug, for example, by reading from the first memory and displaying a list thereof on a display 105 or the like. Can be done. The developer can select any objective among them and change (update) the objective with a script (122) as described later. At this time, the developer can change the part requiring change by using an input device (not shown) such as a keyboard provided in the simulation system 10. Then, the changed (updated) objective is also stored in the first memory. That is, since the objective model 116 is designed to have extensibility, it is effective that the objective updated by someone else can be known as list information.

For example, as an action pattern (objective) that a person takes when a robot enters his / her field of view, there are a pattern of interacting with the robot and a pattern of observing the robot, as shown in FIGS. 6 and 7.

FIG. 6 illustrates an objective in which when a person (indicated by “i”) finds a robot 12 in his / her field of view, he / she stops around the robot 12 at a stop distance Dstop from the robot 12 and interacts with the robot 12. doing.

In FIG. 7, when a person (indicated by “i”) finds the robot 12 in his / her field of view, the robot stops around the robot 12 with an observation distance Dobserve slightly larger than the stop distance Dstop. The objective of observing 12 is illustrated.

Other objectives include an objective that passes by the robot and an objective that slows down by the robot and passes by while looking at it. In the movement module 114, the human agent moves according to one of the registered objectives. To simulate.

However, the movement module 114 simulates the movement of the human agent in consideration of the model data from the Social Force Model 118 and the Interaction with robot model 120 included in the simulation parameters. .. The social force model is a model showing how pedestrians affect each other and change their behavior, as shown in FIG. 8 described later. For example, it is known as repulsive force or reaction force.

In the virtual simulation space 22, the behavior of a human agent changes under the influence of other agents (for example, avoiding people from colliding with each other, approaching a robot when they see it, etc.).

For example, as shown in FIG. 8, two people i and j may influence each other to determine the movement position in the next step, because the person agent i originally moves in the direction of di, j. However, since the human agent j is trying to move in the direction of the angle θi, j with respect to the human agent i at a velocity vi, j, the human agent i will eventually move in the direction of d'i, j. ..

The interaction model 120 is a model in which a pattern of interaction with a human robot is preset.

FIG. 9 schematically shows a part of the virtual simulation environment (space) shown in FIG. 4, and the virtual simulation space 22 assumes a closed space partitioned by an object 124 such as a wall provided with an entrance / exit 22a. There is. However, all the objects 124 are written in the map data 106 as obstacles.

The robot simulator 24 simulates the behavior of the robot in the next step according to the robot application program 18 based on the data such as the position of the robot (robot agent) included in the environment data 20. Here, the robot application program 18 is a program that controls each motor, that is, an actuator, and determines the behavior of the robot according to the sensor output input to the sensor input / output board 88 of FIG.

In the example of FIG. 9, the robot application program 18 is set to move according to the passage 22b from one entrance / exit 22a until the robot 12 enters the virtual simulation space 22 and exits from the other entrance / exit 22a, for example. Has been done. In FIG. 9, reference numeral 126 indicates a human agent put into the virtual simulation space 22 for simulation.

The human simulator 26 further performs the simulation according to the script 122 instead of the objective 116, if necessary. As described above, the objective 116 is a preset fixed robot reaction pattern, but the script 122 is an instruction to cause the simulation device 16, that is, the human simulator 26 to perform an atypical operation. Here, the script is a simple program (instruction set) that can be easily interpreted and executed by omitting the conversion work into a language that can be understood by a computer. When giving instructions to a computer, such as in software design, it is usually necessary to convert from a language that humans can normally read and write to a language that can be understood by a computer, the so-called machine language. It is a simple program that can be easily interpreted and executed by omitting the conversion work. These scripts correspond to the second instruction set, but they are stored in the second memory (not shown) of the simulation system 10. However, the first memory and the second memory may be different storage locations of the same memory.

It should be noted that the script 122 exists as a superstructure of the objective 116, and the objective 116 can be easily accessed from the script 122 side. Therefore, the objective 116 gives instructions only to the mobile module 114, while the script 122 gives instructions to the entire human simulator 26. Therefore, the human generation module 110 can also execute the instruction described in the script 122 in the same manner as the mobile module 114.

As shown in FIG. 1, incorporating a human simulator into the robot development environment is relatively new. In particular, the typical reaction behavior of a human agent when the robot comes into view is defined in advance. The idea of Objective 116 to do one stochastically is new. On top of that, it is a feature of this embodiment that the script 122 is placed above the objective 116.

The idea is to design the objective 116 and the script 122 independently, but if the objective 116 and the script 122 are prepared as independent ones, inconvenience will occur.

The developer trying to debug does not necessarily know the inside information of the simulator device 16 in detail, and in some cases, the developer himself may describe the person agent behavior similar to the objective 116 of the embodiment ad hoc. .. In such a case, it takes a long time to write the script (test code), and as a result, debugging does not proceed smoothly.

On the other hand, by making the objective 116 easily accessible from the script 122 as in the embodiment, the objective 116 can be changed by the script 122 and used as a test code.

An example of such a script is shown in FIG. In this embodiment, one script describing eight instructions is typically shown, but other scripts can of course be used.

Such a script 122 is a test code for executing the test mode in the simulation device 16, for example, when the test mode button (not shown) of the simulation device 16 is operated to call the test mode, or to set the test mode. When the test mode is called at startup in the file, a list of scripts 122 as shown in FIG. 11 is displayed, which the developer selects and causes the simulation device 16, especially the human simulator 26, to execute it. Details will be described later with reference to FIG.

Here, first, with reference to FIG. 10, an example of the operation in the simulation device 16 shown in FIG. 1 according to the objective model 116 will be described.

In the first step S1, the robot application program 18 set to move on the passage 22b shown in FIG. 4 (and FIG. 9) is read as an example, and the environment data 20 is read. That is, a parameter (appearance frequency) related to the generation of a human agent to be input to the virtual simulation space 22 indicated by the map data 106 and the appearance rate data 108 of the virtual simulation space 22 as shown in FIG. 4 (and FIG. 9) that simulates the behavior of the robot 12. , Movement route, number of groups, ratio of adults and children and men and women, height, etc.) and position data of each agent (including robot 12).

In the next step S3, one or two in the virtual simulation space 22 according to the above parameters and data of the person pattern from the group regarding the generation of the person agent presented by the environment data 20 by the person generation module 110 of the person simulator 26. Create and deploy the above person agents.

In step S5, the human simulator 26 performs the behavior of the human agent in the next step based on the current positions of the human agent and the robot 12 shown by the environmental data 20 and based on the anti-robot behavior pattern shown by the objective model 116. decide. The basic list of the above-mentioned four objectives is provided in a form that can be freely accessed by the developer (operator of the operating device 14). In addition, other objective models can be adopted.

In step S7, the robot simulator 24 determines the action of the robot 12 in the next step according to the robot application program 18 based on the current positions of the human agent and the robot 12 indicated by the environmental data 20.

Specifically, in step S7, when the robot 12 is a robot whose main purpose is to go to the destination, such as a transfer robot or a boarding robot, it is possible to consider another person from the current position and movement speed of other people. Performs a process of calculating a movement route that does not interfere with the movement of the robot. Further, when the robot 12 is a robot whose purpose is to provide services to people, for example, a person who is stopped near the robot in a simple case in order to approach a person who may be interested in the robot. Choose to go towards that person, choose a place where people tend to gather, wait for people to come in such a place, or get too close to a place where people pass too much to avoid congestion. It performs processing such as preventing it from occurring.

In the case of this step S7, since the action in the next step of the human agent in step S5 is determined, the above-mentioned process is executed in consideration of it. Therefore, it is possible to judge in the virtual simulation space whether or not the robot application program that operates in the real space coexisting with humans is appropriate.

After that, in step S9, the action of each agent is realized in the physics engine 28. More specifically, in step S9, as described above, the process of executing the movement intended by each agent (including the robot) in the virtual simulation space 22 is performed, and other obstacles, structures and other agents are performed. If there is no interference with, each agent works as intended. For example, if the agent tries to take one step forward, the agent takes one step forward as intended.

However, when there is interference, the actual movement of each agent is processed according to the laws of physics (law of mechanics). In a simple case, for example, if you try to move toward a wall, it will collide with the wall, you will not be able to move forward any further, and you will receive a reaction force according to a constant coefficient of restitution. In the case of a collision between agents, the forward movement and the movement due to the reaction force overlap, for example, considering a simple example, if a collision occurs from the side while moving forward, it will actually move diagonally forward. ..

In the next step S11, the environment data, the sensor data, and the like are updated.

Then, it is determined whether or not to end the simulation in step S13, and if the simulation is continued, the process returns to the previous step S3 again. For example, if the action determination program is terminated, "YES" is determined and the simulation is terminated.

In this way, the robot behavior can be simulated by performing simulation in the simulation device 16 so as to reproduce the behavior of people around the robot according to the objective 116 and confirming the state on the display 105 (FIG. 1). It is possible to efficiently develop a robot application program 18 that operates in a real space that coexists with humans without actually moving the robot in the real space.

In the simulation as described above, since the movement of the human agent is simulated according to the preset environment data 20 and the objective model 116, it is difficult to reproduce a special situation when debugging the robot application program. For example, such a situation occurs only in a specific time zone or when there are a large number of people, and it is often difficult to reproduce it only with the standard objective model 116. Therefore, sufficient debugging cannot be performed, and eventually it becomes necessary to check in real space.

Therefore, in the simulation device 16 of this embodiment, in addition to the standard objective, a test code (a script for reproducing (testing) a specific situation) that reproduces the behavior and mechanism of a human agent can be described and executed. Propose a new framework.

In the example shown in FIG. 11, eight commands are described in a script that describes the test content that "a child in a distant place moves a little and then interacts with the robot (comes forward)". The Japanese line indicates the meaning of the test code, and the next alphanumeric line indicates the actual test code.

The instruction 1 has the content of "continuing to execute for 5 seconds in the simulation time".

Instruction 2 has the content of Boy 02, "to generate a child". The appearance rate data 108 shown in FIG. 5 is a means for causing the human generation module 110 to generate a human agent according to a predetermined appearance rate, but this instruction 2 of the script is performed at an arbitrary timing regardless of the appearance rate. Any kind of human agent can be generated from the human generation module 110. As a result, it is possible to put a large number of human agents into the virtual simulation space 22 at a time for debugging to reproduce a specific situation.

The instruction 3 is to set the position of the human agent.

Command 4 is to move the human agent to a designated place.

The instruction 5 is to wait until the person agent arrives at the designated destination.

Command 6 is to move the human agent to a new destination.

Order 7 is to change the objective. However, it is the mobile module 114 that changes the objective. This is because only the mobile module 114 can access the objective model 116. Also, changing the front part of the objective is contrary to the purpose of trying to reproduce a specific situation for debugging using the objective (it takes too much time), so changing the objective by script is done in the first instruction set. Change some commands of an objective.

Instruction 8 has the content of "continuing to execute for 5 minutes in the simulation time". Since it is set that the simulation will end when the last line of the script is reached, the developer will watch "a person approaching the robot" for 5 minutes and write this instruction 8 for the purpose of verifying the operation. doing.

When the script is read, the modules 110 and 114 of the human simulator 26 shift to a special instruction mode that does not perform "stochastic transition". Therefore, it is necessary to set all the generation of human agents, the setting of destinations, and the determination of objectives (and their timing) with scripts. Therefore, the script 122 can issue instructions directly to the modules 110 and 114 of 2.

Then, when the test mode is called as described above, the processor (not shown) of the simulation device 16 puts the test code of the script 122 on one line (alphanumeric characters in FIG. 11) in the first step S21 of FIG. Line) Read.

Based on the read test code, the content of the test code is determined in steps S23, S29, S33, S37 or S41 (S43), and depending on the result, required in steps S25, S31, S35, S39 or S45. The processing is executed by one or more processors of the simulation device 16. Then, in step S27, the result is displayed on the display 105 (FIG. 1) and output from the network (not shown) to the operating device 14. Therefore, the developer, that is, the operator, can confirm the generation and behavior of the human agent according to the test code.

More specifically, in step S23, it is determined whether or not the test code read in step S21 is a human agent generation instruction. If "YES", in step S25, the human generation module 110 generates a human agent to the initial value. However, the human agent is generated for each step of the human simulator 26. Then, in step S27, the execution result of the test code in step S25 is output to the display 105 and the network.

In step S27, it is determined whether or not the test code read in step S21 is a position change command of the human agent. If "YES", in step S29, the movement module 114 causes the current position of the designated person agent to be changed. Then, in step S27, the execution result of the test code in step S31 is output to the display 105 and the network.

In step S31, it is determined whether or not the test code read in step S21 is a movement command of a human agent. If "YES", in step S33, the movement module 114 starts the movement of the designated person agent. Then, in step S27, the execution result of the test code in step S35 is output to the display 105 and the network.

In step S35, it is determined whether or not the test code read in step S21 is an objective setting instruction. If “YES”, in step S37, the processor of the simulation apparatus 16 changes the corresponding objective in the objective model 116. Then, in step S27, the execution result of the test code in step S39 is output to the display 105 and the network. As a result, the developer can visually confirm the simulation according to the script set by himself / herself. However, this objective change is made by the mobile module 114. Specifically, as described above, the objective can be changed by reading the objective set in the memory from the memory and changing the source code of the corresponding portion by using a keyboard or the like.

In step S39, it is determined whether or not the test code read in step S21 is a standby instruction. If "YES", it is determined in the next step S43 whether or not the waiting condition (for example, the elapse of the set time in the test code 1 or 8) is further satisfied. If "YES", in step S45, the execution result of the test code is output to the display 105 and the network in the same manner as in step S27.

As shown in FIGS. 10 and 12, the reason why the script 122 and the objective 116 can be executed separately is that the purpose (and the degree of abstraction) of the script and the objective 116 are different.

That is, the script 122 has the purpose of "recreating a specific situation (including the generation of a plurality of human agents and behavioral changes)", and the (plural) people for reproducing a specific situation in the script 122. It describes the timing of agent generation, movement timing, and objective change timing, and cannot be used in normal simulation (without using scripts).

On the other hand, the objective 116 is to describe and use "a reaction action performed by one human agent in response to the existence of a robot". Therefore, the objective 116 can be easily used in a normal simulation (without using a script) once the transition probability is defined.

In addition, by classifying in this way, the objective can be seen as a lower mutable library from the script. In other words, by using objectives with scripts, you can easily describe the situation you want to reproduce.

The robot capable of simulating the behavior of the simulation system 10 is not limited to the robot 12 shown in FIGS. 3 and 4 described in the embodiment. This simulation device can also be applied to communication robots of other types and structures.

10… Simulation system 12… Robot 14… Operating device 16… Simulation device 18… Robot application program 22… Virtual simulation space 24… Robot simulator 26… Human simulator 28… Physics engine 116… Objective model 122… Script

Claims (4)

A simulation system that simulates the behavior of a human agent put into a virtual simulation space.
A human generation module that inputs a human agent into the virtual simulation space,
A mobile module that causes the human agent to react to the presence of a robot according to the first instruction set, and a specific situation is reproduced in the virtual simulation space by partially modifying the first instruction set by the second instruction set. A simulation system equipped with means for reproducing the situation. A simulation system that simulates the behavior of a human agent put into a virtual simulation space.
A human generation module that inputs a human agent into the virtual simulation space,
A first memory that sets a first instruction set that causes the human agent to perform a reaction action to the presence of a robot,
A simulation system including a mobile module that controls the movement of the human agent according to the first instruction set of the first memory, and a display that displays the virtual simulation space including the human agent moved by the movement module. Further provided with a second memory for setting a second instruction set that is higher than the first instruction set and is capable of partially modifying the first instruction set.
The simulation system according to claim 2, wherein at least one of the man-generated module and the mobile module is controlled by the second instruction set. Further provided with means for causing a human agent to appear in the virtual simulation space by the human generation module according to the appearance rate.
The simulation system according to claim 3 , wherein the second instruction set causes a human agent to appear in the virtual simulation space by the human generation module regardless of the appearance rate.

Images