JP7335084B2 - Simulation device and program - Google Patents

Description

This invention is a simulation device and program.
In particular, for example, it relates to a simulation device and program with a debugging function.

Conventional simulation devices of this type are disclosed in Non-Patent Documents 1 and 2. With the simulation devices of these non-patent documents 1 and 2, by simulating and visualizing the sensors and actuators, it is possible to more easily simulate the adaptation of the robot in the coexistence environment with humans.

In Patent Literature 1, the inventors of the present invention further developed a simulation system that can efficiently develop a robot action determination program (robot application program) by simulating changes in the environment (such as changes in pedestrians and objects). is proposing.

JP 2017-170584 (B25J 5/00, B25J 13/00)

https://www.openrobots.org/morse/doc/stable/morse.html http://gazebosim.org/

Programs for service robots that operate on street corners and the like are complicated, and bugs that occur only at specific moments (for example, when the robot and pedestrians pass each other under specific conditions) can exist. It is difficult to clarify the cause of such a bug that occurs at a specific moment only by simulation, and generally log files are used. Since log data lacks anything other than sensing information, developers are required to imagine real-world situations and solve problems based on only the large amount of sensor data left in log data. . Especially for bugs that occur at specific moments, it is necessary to clarify the cause of the problem by imagining the correspondence between changes in the internal state of the program and changes in the real world only from sensor data. This task is difficult and takes a lot of time to debug.

SUMMARY OF THE INVENTION Therefore, a primary object of the present invention is to provide a novel simulation apparatus and program .

Another object of the present invention is to provide a simulation apparatus and program that facilitates investigation of the cause of bugs and enables efficient execution of debugging.

In order to solve the above problems, the present invention employs the following configurations. It should be noted that reference numerals in parentheses, supplementary explanations, etc. indicate the correspondence with the embodiments described later in order to facilitate understanding of the present invention, and do not limit the present invention in any way.

A first invention has a user interface capable of manually setting a desired execution speed for one step of a simulation, simulates the actions of a robot agent and a human agent put into a virtual simulation space, and displays the virtual simulation space on a display. A simulation apparatus to be displayed, comprising desired execution speed acquisition means for acquiring a desired execution speed set by a user interface, basic execution time acquisition means for acquiring a basic execution time based on simulation start time and end time, and desired execution speed Desired execution cycle calculation means for calculating a desired execution cycle according to the desired execution cycle, determination means for comparing the basic execution time and the desired execution cycle to determine whether or not the simulation can be executed according to the desired execution cycle, and the determination means determines that the simulation is executable. The simulation apparatus includes sleep means for causing sleep for a sleep time based on a basic execution time and a desired execution cycle when the simulation apparatus is executed.

In the first invention, a simulation device (16) constituting a simulation system (10: reference numerals illustrating corresponding parts in the embodiments; the same shall apply hereinafter) includes a display (105) and one step of simulation. Equipped with a user interface (109) capable of manually setting the execution speed, a robot (12) and a human agent are put into a virtual simulation space (22). For example, the motion of the robot (12) is simulated by the robot simulator (24) and the human agent is simulated by the human simulator (26) respectively. A desired execution speed acquisition means (S1, S21) acquires a desired execution speed (s') set by a user interface. Basic execution time acquisition means (S25) acquires basic execution time (t) based on simulation start time (time_s) and simulation end time (time_e). Desired execution cycle calculation means (S27) calculates a desired execution cycle (p) according to the desired execution speed, and determination means (S29) compares the basic execution time (t) with the desired execution cycle (p). , it is determined whether the simulation can be executed according to the desired execution cycle. Then, when the judging means judges that the execution is possible, the sleeping means (S17) causes the sleep for the sleep time based on the basic execution time and the desired execution cycle.

According to the first invention, by manually setting the desired execution speed, the execution time of one step of the simulation can be changed accordingly , making it easier to investigate the cause of instantaneously occurring bugs and efficiently debugging. can run to Furthermore, since the simulation is put to sleep for the necessary time according to the desired execution speed, the status of bugs can be easily visually recognized.

A second invention is a simulation apparatus according to the first invention, further comprising sleep time calculation means for calculating a sleep time according to a difference between the desired execution period and the basic execution time, wherein the sleep time calculation means Sleep for the calculated sleep time .

In the second invention, the sleep time calculation means (S31) calculates the sleep time according to the difference (pt) between the desired execution period and the basic execution time.

According to the second invention, it is possible to sleep for the necessary sleep time.

A third invention is a simulation apparatus according to the first or second invention, wherein when the determining means determines that the simulation is not executable, parameters including a desired execution cycle are changed to executable parameters. Further provide means .

In the third invention, the parameter changing means (S33) changes the parameter including the desired execution cycle to an executable parameter.

According to the third invention, when the execution speed is increased (when the speed is increased too much), it can be dealt with.
A fourth invention has a user interface capable of manually setting a desired execution speed for one step of the simulation, simulates the actions of a robot agent and a human agent put into a virtual simulation space, and displays the virtual simulation space on a display. A program for a simulation device to display, the program comprises a computer of the simulation device, a desired execution speed acquisition means for acquiring a desired execution speed set by a user interface, and a basic execution time based on the start time and end time of the simulation. a desired execution cycle calculating means for calculating a desired execution cycle according to a desired execution speed; and a judgment for comparing the basic execution time and the desired execution cycle to determine whether or not the simulation can be executed with the desired execution cycle. It is a simulation program that functions as a sleep means for sleeping for a sleep time based on a basic execution time and a desired execution cycle when the means and the judgment means judge that the execution is possible.
Also in the fourth invention, the same effects as in the first invention can be expected.

According to this invention, the time for executing one step in the simulation can be lengthened, so it becomes easy to investigate the cause of a bug that occurs at a specific moment. Therefore, robot application programs can be debugged efficiently.

The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of the embodiments with reference to the drawings.

FIG. 1 is a functional block diagram showing a simulation system of one embodiment of the invention. FIG. 2 is an illustrative view showing one example of a robot to which the present invention is applied. FIG. 3 is a block diagram showing the electrical configuration of the robot shown in FIG. 2. As shown in FIG. 4 is an illustrative view showing one example of the virtual simulation space of FIG. 1. FIG. FIG. 5 is a functional block diagram showing details of the human simulator of FIG. FIG. 6 is an illustrative view showing one example of a basic objective model of a human agent. FIG. 7 is an illustrative view showing another example of the basic objective model of the human agent. FIG. 8 is an illustrative view showing another example of the human agent objective model. 9 is an illustrative view showing an example of movement of a human agent in a part of the virtual simulation space of FIG. 4; FIG. FIG. 11 is a flowchart showing an example of the operation according to the objective of the simulation system of FIG. 1 embodiment. FIG. 11 is a flow chart showing the operation of the FIG. 1 embodiment. FIG. 12 is an illustrative view showing one example of a user interface for inputting a deceleration factor.

Referring to FIG. 1, a simulation system 10 of this embodiment comprises a robot 12, an operating device 14 for remotely controlling the robot 12, a robot agent simulating the robot 12, and a human agent simulating a person (pedestrian). It includes a simulation device 16 for simulating. The simulation system 10 is implemented by one or more computers (processors), except for the robot 12 .

The robot 12 of this embodiment is a robot that works in a coexisting environment with humans, like the communication robot shown in FIG. 2, for example. Since such a robot 12 is affected by the behavior of a person existing in the environment, in the simulation device 16 of this embodiment, when simulating the behavior of the robot 12 according to a robot application program (robot behavior determination program), By also simulating human behavior, the development of safe robot application programs for the robot 12 is aided.

Here, with reference to FIGS. 2 and 3, the configuration of the robot 12 will be described to the extent necessary for understanding the present invention. The robot 12 includes a carriage 30 with two wheels 32 and one follower wheel 34 provided on the underside of the carriage 30 for moving the robot 12 . The two wheels 32 are independently driven by wheel motors 36 (see FIG. 3) to move the cart 30, that is, the robot 12, in any direction, forward, backward, leftward, or rightward.

A cylindrical sensor mounting panel 38 is provided on the carriage 30, and a large number of distance sensors 40 are mounted on the sensor mounting panel 38. As shown in FIG. These distance sensors 40 measure the distance to objects (people, obstacles, etc.) around the robot 12 using, for example, infrared rays or ultrasonic waves.

A body 42 is provided upright on the sensor mounting panel 38 . Further, the above-described distance sensor 40 is further provided in the front central upper part of the body 42 (position corresponding to the chest of the person), and measures the distance in front of the robot 12, mainly to the person. Further, the body 42 is provided with a support 44 extending from substantially the center of the side upper end portion thereof, and an omnidirectional camera 46 is provided above the support 44 . The omnidirectional camera 46 captures the surroundings of the robot 12 and is distinguished from the eye camera 70 described later. As the omnidirectional camera 46, for example, a camera using a solid-state imaging device such as CCD or CMOS can be adopted.

An upper arm 50R and an upper arm 50L are provided on both side upper ends (positions corresponding to human shoulders) of the torso 42 by shoulder joints 48R and 48L, respectively. Although not shown, the shoulder joint 48R and the shoulder joint 48L each have degrees of freedom of three orthogonal axes. That is, the shoulder joint 48R can control the angle of the upper arm 50R around each of the three orthogonal axes. One axis (yaw axis) of the shoulder joint 48R is an axis parallel to the longitudinal direction (or axis) of the upper arm 50R, and the other two axes (pitch axis and roll axis) are perpendicular to that axis from different directions. It is the axis that Similarly, the shoulder joint 48L can control the angle of the upper arm 50L around each of the three orthogonal axes. One axis (yaw axis) of the shoulder joint 48L is an axis parallel to the longitudinal direction (or axis) of the upper arm 50L, and the other two axes (pitch axis and roll axis) are perpendicular to that axis from different directions. It is the axis that

Further, elbow joints 52R and 52L are provided at the distal ends of the upper arms 50R and 50L, respectively. Although not shown, the elbow joint 52R and the elbow joint 52L each have a degree of freedom of one axis, and can control the angles of the forearms 54R and 54L around this axis (pitch axis).

Hands 56R and 56L, which correspond to human hands, are provided at the tips of the forearms 54R and 54L, respectively. Although detailed illustration is omitted, these hands 56R and 56L are configured to be openable and closable, whereby the robot 12 can grip or pinch an object using the hands 56R and 56L. However, the shape of the hands 56R and 56L is not limited to the shape of the embodiment, and may be made to have a shape and functions very similar to those of human hands.

In addition, although not shown, the front surface of the carriage 30, the portion corresponding to the shoulder including the shoulder joint 48R and the shoulder joint 48L, the upper arm 50R, the upper arm 50L, the forearm 54R, the forearm 54L, the hand 56R and the hand 56L are provided with , a contact sensor 58 (generically shown in FIG. 3) is provided. A contact sensor 58 on the front of the truck 30 detects contact of a person or other obstacle with the truck 30 . Therefore, if the robot 12 comes into contact with an obstacle during its own movement, it can detect it and immediately stop driving the wheels 32 to abruptly stop the movement of the robot 12 . Further, other contact sensors 58 detect whether or not each part is touched.

A neck joint 60 is provided in the upper central portion of the body 42 (a position corresponding to the human neck), and a head 62 is provided thereon. Although not shown, the neck joint 60 has three degrees of freedom and can be angularly controlled around each of the three axes. One axis (yaw axis) is an axis directed directly above the robot 12 (vertically upward), and the other two axes (pitch axis and roll axis) are orthogonal axes in different directions.

A speaker 64 is provided on the head 62 at a position corresponding to a person's mouth. Speakers 64 are used by robot 12 to communicate audibly to people in its vicinity. A microphone 66R and a microphone 66L are provided at positions corresponding to human ears. Hereinafter, the right microphone 66R and the left microphone 66L may be collectively referred to as the microphone 66. FIG. A microphone 66 captures ambient sounds, especially the human voice with which the communication is being carried out.

Further, a right eyeball portion 68R and a left eyeball portion 68L are provided at positions corresponding to human eyes. Right eyeball portion 68R and left eyeball portion 68L include right eye camera 70R and left eye camera 70L, respectively. Hereinafter, the right eyeball portion 68R and the left eyeball portion 68L may be collectively referred to as the eyeball portion 68. As shown in FIG. Also, the right eye camera 70R and the left eye camera 70L may be collectively referred to as an eye camera 70. FIG.

The eye camera 70 photographs the face of a person approaching the robot 12, other parts or objects, and captures the corresponding video signal. In this embodiment, the robot 12 detects the line-of-sight directions (vectors) of each of the left and right eyes of a person from the video signal from the eye camera 70 .

Also, the eye camera 70 can be the same camera as the omnidirectional camera 46 described above. For example, the eye camera 70 is fixed within the eyeball portion 68, and the eyeball portion 68 is attached at a predetermined position within the head 62 via an eyeball support (not shown). Although illustration is omitted, the eyeball supporting portion has two degrees of freedom and can be angularly controlled around each of these axes. For example, one of the two axes is the upward direction axis (yaw axis) of the head 62, and the other is perpendicular to one axis and to the direction in which the front side (face) of the head 62 faces. It is the axis (pitch axis) in the direction to By rotating the eyeball supporting portion around each of these two axes, the eyeball portion 68 or the tip (front) side of the eye camera 70 is displaced, and the camera axis, that is, the line of sight direction is moved. The installation positions of the speaker 64, the microphone 66, and the eye camera 70 described above are not limited to these parts, and may be provided at appropriate positions.

As described above, the robot 12 of this embodiment has independent two-axis drive of the wheels 32, three degrees of freedom of the shoulder joints 48 (six degrees of freedom on the left and right), one degree of freedom on the elbow joints 52 (two degrees of freedom on the left and right), It has a total of 17 degrees of freedom, including 3 degrees of freedom of the neck joint 60 and 2 degrees of freedom of the eyeball supporting portion (4 degrees of freedom in left and right).

FIG. 3 is a block diagram showing the electrical configuration of the robot 12. As shown in FIG. Referring to FIG. 3, robot 12 includes one or more processors 80 . Processor 80 is connected via bus 82 to memory 84 , motor control board 86 , sensor input/output board 88 and audio input/output board 90 .

Although not shown, the memory 84 includes ROM, HDD and RAM. Various programs are stored in advance in the ROM and HDD.

The motor control board 86 is composed of, for example, a DSP, and controls the driving of each shaft motor of each arm, neck joint 60, eyeball portion 68, and the like. That is, the motor control board 86 receives control data from the processor 80 and controls two motors (in FIG. 3, collectively referred to as "right eyeball motor 92") for controlling the respective angles of the two axes of the right eyeball part 68R. to control the rotation angle of the Similarly, the motor control board 86 receives control data from the processor 80 and controls two motors (collectively referred to as "left eye motor 94" in FIG. 3) for controlling the respective angles of the two axes of the left eyeball 68L. (shown) to control the angle of rotation.

Also, the motor control board 86 receives control data from the processor 80, and controls three motors for controlling the angles of the three orthogonal axes of the shoulder joint 48R and one motor for controlling the angle of the elbow joint 52R. It controls the rotation angles of four motors (shown collectively as "right arm motor 96" in FIG. 3). Similarly, the motor control board 86 receives control data from the processor 80, three motors controlling the angles of the three orthogonal axes of the shoulder joint 48L and one motor controlling the angle of the elbow joint 52L. (collectively referred to as "left arm motor 98" in FIG. 3).

Furthermore, the motor control board 86 receives control data from the processor 80 and controls three motors (collectively referred to as "head motors 100" in FIG. ) to control the rotation angle. The motor control board 86 receives control data from the processor 80 and controls the rotation angles of the two motors that drive the wheels 32 (collectively referred to as "wheel motors 36" in FIG. 3).

A hand actuator 101 is further coupled to the motor control board 86, and the motor control board 86 receives control data from the processor 80 and controls opening and closing of the hands 56R and 56L.

The sensor input/output board 88, like the motor control board 86, is composed of a DSP, takes in signals from each sensor, and supplies them to the processor 80. FIG. That is, data relating to the reflection time from each of the range sensors 40 are input to the processor 80 through this sensor input/output board 88 . Also, the video signal from the omnidirectional camera 46 is input to the processor 80 after undergoing predetermined processing by the sensor input/output board 88 as necessary. A video signal from the eye camera 70 is similarly input to the processor 80 . Signals from the plurality of contact sensors 58 (collectively referred to as “contact sensors 58 ” in FIG. 3 ) are provided to processor 80 via sensor input/output board 88 . The voice input/output board 90 is also composed of a DSP in the same way, and the speaker 64 outputs voice according to voice synthesis data given from the processor 80 . Audio input from microphone 66 is also provided to processor 80 via audio input/output board 90 .

Processor 80 is also connected to communication LAN board 102 via bus 82 . Communication LAN board 102 is composed of a DSP, for example, and provides transmission data given from processor 80 to wireless communication module 104, and wireless communication module 104 transmits the transmission data to a server (not shown) or the like via a network. do. Communication LAN board 102 also receives data via wireless communication module 104 and provides the received data to processor 80 .

The robot to which the simulation system 10 of the embodiment can be applied is not limited to the robot 12 shown in FIGS. 2 and 3, and this simulation apparatus can also be applied to communication robots of other types and structures. .

Returning to FIG. 1, the operation device 14 is basically composed of one or more computers or processors, and a robot application program 18 is set in the memory (not shown) of the processor. A program is given to the robot 12 or simulation device 16 from a computer, for example wirelessly. The robot application program 18 is an application program for the robot 12. For example, when the robot 12 is operated as a guide robot, the robot 12 executes operations for that purpose.

The memory of the operating device 14 further stores environment data 20 in advance. The environment data 20 mainly includes map data of the virtual simulation space 22 shown in FIG. 4 and appearance rate data of human agents within the virtual simulation space 22 . The environment and objects within the environment are represented as 3D model data and visualized as a 3D virtual space.

A virtual simulation space 22 shown in FIG. 4 is assumed to be a three-dimensional closed space having an entrance 22a indicated by a dotted-line rectangle in the figure. In this virtual simulation space 22, a passage 22b indicated by a thick black line is set, and the robot 12 and the human agent pass through or exist in the passage 22b , the square, or the like. These passages 22b and plazas are partitioned by objects such as buildings and walls, and diverge or merge at places indicated by circles in the figure.

The simulation device 16 is basically composed of one or more processors or computers, and includes a robot simulator 24 that simulates the robot 12 and a human simulator 26 that simulates a human agent such as a pedestrian. Simulation results are input to the physics engine 28 .

The robot simulator 24 is, for example, the MORSE simulator exemplified in Non-Patent Document 1 above, and this MORSE simulator provides many sensors, actuators and robot models similar to the robot 12, and a model for such a model. An API (Application Programming Interface) is also available.

In sensor simulation, for example, in the case of a camera image, optical calculation is performed from the viewpoint of the camera to generate image information for each viewpoint of the camera. In the case of a laser range finder, the distance is measured from the sensor in each direction by simulating laser scanning, and the results are stored as sensor values.

When developers prepare robot application programs for their robots, those programs access these APIs to read sensor data (e.g. reading distances from a laser rangefinder) and to send commands to actuators (e.g. movement speed).

The people simulator 26 periodically creates new pedestrians, updates their positions, and removes them when they leave the simulated environment (virtual simulation space 22). A pedestrian or person is represented as a three-dimensional object, placed in the virtual simulator space 22, and its walking motion is animated using an animation engine.

However, since the human simulator 26 is provided with environmental data 20 as shown in FIG. 1, the human simulator 26 periodically generates new pedestrians with a predefined probability called the appearance rate contained in the environmental data. do. The pedestrian is then assigned to one of the doorways 22a shown in FIG. 4 and set to go to the other one.

Note that pedestrians are often generated in groups (like families or couples) and when they are displayed in groups, the size is defined by the distribution of group members, assuming that all members share the same purpose.

The physics engine 28 calculates what kind of movement path the robot or human agent will follow based on physical forces and laws. More specifically, the physics engine 28 performs processing that causes each agent (including robots) from the robot simulator 24 and the human simulator 26 to perform intended movements in the virtual simulation space 22 (FIG. 4). At this time, if there is no interference with other obstacles, structures, or other agents, it will generate the intended movement. However, in the presence of interference, the actual movement of each agent is processed according to the laws of physics (laws of mechanics). Then, the physics engine 28 outputs integrated simulation data.

The simulation device 16 is provided with a display 105, which displays an image of a virtual simulation space, such as that shown in FIG. 4, together with simulated robots and human agents present therein. be. For example, a human agent is represented as a three-dimensional object, placed in the virtual simulation space 22, and its walking motion is animated using an animation engine. A developer of the robot application program 18 creates and modifies an action determination program (application program) for the robot 12 while checking the image displayed on the display 105 .

Furthermore, the simulation device 16 is provided with an input device 107 . This input device 107 may be, for example, a keyboard or a touch panel. In the case of a touch panel, it may be incorporated in the display 105 and the display 105 may be configured as a touch display. The input device 107 can be used when the operator sets a desired execution speed s', which will be described later.

The person simulator 26 will be described in detail with reference to FIG. Human simulator 26 , like robot simulator 24 , is a function implemented by the computer or processor of simulation device 16 and includes human generation module 110 . This human generation module 110 creates new pedestrians (human agents) according to the occurrence rate data 108 contained in the environment data 20, as described above, in the virtual simulation space 22 represented by the map data 106, an example of which is shown in FIG. , is generated periodically.

Appearance rate data 108 is the shape of a person, that is, the shape (pattern) of an adult, a child, a man, a woman, etc., and the appearance rate of each person for the human agent to be injected into the virtual simulation space 22 of the simulation device 16. is a preset database, and a human agent with a set shape is generated with a set probability.

It should be noted that the appearance rate data 108 may hold different data sets for each day of the week and time so that the simulation can be performed by designating the day of the week and time. By doing so, various simulations can be performed, such as what happens when the robot is moved at a certain time.

However, if the appearance rate data 108 indicates that the human agents to be generated form a group, the human generation module 110 generates a plurality of human agents in a group having a member configuration according to the group member data 112 set as a simulation parameter. to generate human agents.

Movement module 114 moves the human agent generated by human generation module 110 within virtual simulation space 22 ( FIG. 4 ) based on objective model 116 . Here, the objective is the way a human agent (not shown) reacts when a robot agent (not shown) enters the field of view of the human agent (not shown) in the virtual simulation space 22, that is, the human agent reacts. It is an instruction set that defines the action patterns against robots that can actually be taken in real space. In this embodiment, four typical reaction modes (objectives) described later are set as models, and the movement module 114 moves the human agent according to the objectives indicated by the models. In other words, the objective model 116 functions as a first instruction set that causes the human agent to react to the presence of the robot in the virtual simulation space. These objectives are stored in a first memory (not shown) of simulation system 10 .

The available objectives preset in the objective model 116 can be read out from the first memory and displayed as a list on the display 105, for example, so that the developer who intends to debug can know them. can be done. The developer can select any objective in it and change (update) that objective in the script (122) as described below. At this time, the developer can use an input device (not shown) such as a keyboard provided in the simulation system 10 to change the portion to be changed. The modified (updated) objective is then also stored in the first memory. In other words, since the objective model 116 is designed to have extensibility, it is effective to be able to know objectives updated by someone else as list information.

For example, as behavior patterns (objectives) that a person takes when a robot comes into his/her field of view, there are a pattern of interacting with the robot and a pattern of observing the robot, as shown in FIGS.

FIG. 6 illustrates an objective that stops around robot 12 a stopping distance Dstop from robot 12 and interacts with robot 12 when a person (indicated by "i") discovers robot 12 within his/her field of view. are doing.

FIG. 7 shows that when a person (indicated by "i") discovers the robot 12 in his field of view, he stops around the robot 12 with an observation distance Dobserve slightly larger than the stopping distance Dstop from the robot 12, and the robot 12 illustrates an objective observing 12;

Objectives include other objectives that pass by the robot, and objectives that slow down and pass by the robot while looking at it. In the movement module 114, the human agent moves according to any registered objective. simulate as

However, the locomotion module 114 simulates locomotion of the human agent taking into account the respective model data from the Social Force Model 118 and the Interaction with robot model 120 included in the simulation parameters. . The social force model is a model that expresses how pedestrians affect each other and behavior changes, as shown in FIG. 8, which will be described later. For example, it is known as repulsion or reaction force.

In the virtual simulation space 22, the human agent's behavior changes under the influence of other agents ( for example, people avoid each other, approach robots when they see them, etc.).

For example, as shown in FIG. 8, two people i and j may influence each other to determine their movement position in the next step, since human agent i originally moves in the direction of di,j. However, since the human agent j is about to move in the direction of the angle θi,j with the speed vi,j with respect to the human agent i, the human agent i ends up moving in the direction of d'i,j. .

The interaction model 120 is a model in which patterns of human interaction with the robot are set in advance.

FIG. 9 schematically shows a part of the virtual simulation environment (space) shown in FIG. there is However, all objects 124 are written in the map data 106 as obstacles.

The robot simulator 24 simulates the action of the robot in the next step according to the robot application program 18 based on data such as the position of the robot (robot agent) included in the environment data 20 . Here, the robot application program 18 is a program that controls each motor, that is, actuators, and determines the action of the robot according to the sensor output input to the sensor input/output board 88 of FIG.

In the example of FIG. 9, the robot application program 18 is set so that the robot 12 enters the virtual simulation space 22 through one doorway 22a and moves along the path 22b until it exits through the other doorway 22a. It is Incidentally, reference numeral 126 in FIG. 9 indicates a human agent put into this virtual simulation space 22 for simulation.

Human simulator 26 also performs simulations according to script 122 rather than objective model 116 when necessary. The objective model 116 is, as described above, a preset, fixed response pattern to the robot, while the script 122 is instructions for causing the simulation device 16, ie, the human simulator 26, to perform non-fixed actions. Here, a script is a simple program (instruction set) that can be easily interpreted and executed without conversion into a computer-readable language. When giving instructions to a computer in software design, etc., it is usually necessary to convert a language that humans can normally read and write into a language that computers can understand, so-called machine language. It is a simple program that can be easily interpreted and executed by omitting the conversion work. Although these scripts fall under the second set of instructions, they are stored in a second memory (not shown) of simulation system 10 . However, the first memory and the second memory may be different memory locations in the same memory.

Here, first, with reference to FIG. 10, an example of the operation of the simulation device 16 shown in FIG. 1 according to the objective model 116 will be described. However, in the simulation shown in FIG. 10, a plurality of steps (S3-S17) are executed as one step. In practical operation, one step of the simulation can be executed in units of 0.1 milliseconds at the fastest, but here it is possible to set the unit to 10 milliseconds and execute the robot application program 18 in units of 100 milliseconds. It is possible. In this example, the robot application program 18 is executed once every 10 steps executed by the simulation device 16 . In other words, in this embodiment, it is assumed that the period of the robot application program is longer than the execution period of one step of the simulation.

In the first step S1, the computer of the simulation device 16 (hereinafter sometimes simply referred to as "computer") is set to move on the passage 22b shown in FIG. 4 (and FIG. 9) as an example. Along with reading the robot application program 18, the environment data 20 is read. That is, parameters (appearance frequency , movement route, number of people in the group, adult/child/male/female ratio, height, etc.) and location data of each agent (including the robot 12). This virtual simulation space 22 is displayed on the display 105 together with the robot 12 and the human agent.

Furthermore, in this step S1, the simulation is initialized. That is, the execution speed s is set to 1.0, and the number of basic execution steps fps is set to fps=30.0. However, fps is a numerical value indicating how many steps of simulation are executed per second, and in the initial setting of this embodiment, it is set so that 30 steps of simulation are executed per second. In this case, the basic execution period p is p=1.0×1/fps=1/30 (≈0.033) seconds.

In the next step S3, the computer stores the current system time as the execution start time time_s in the memory (not shown) of the simulation device 16, for example, in order to measure the execution cycle of one simulation step.

In the next step S5, the human generation module 110 of the human simulator 26 generates 1 or 2 human agents in the virtual simulation space 22 according to the above-mentioned parameters for human agent generation presented by the environment data 20 and the human pattern data from the group. Generate and place more human agents.

In step S7, the computer or human simulator 26 determines the next step of the human agent based on the current positions of the human agent and the robot 12 indicated by the environment data 20, and on the behavior pattern for the robot indicated by the objective model 116. Decide on action. A list of the four basic objectives described above is provided in a form that can be freely accessed by the developer (operator of the operation device 14). Additionally, other objective models can be employed.

At step S9, the computer, that is, the robot simulator 24 receives a one-step action instruction from the robot application program 18 and executes it. However, if the period of the robot application program is longer than the execution period of one step of the simulation, the robot simulator 24 may not do anything in step S9.

As for the behavior of the robot, for example, if the robot 12 is a robot whose main purpose is to go to a destination, such as a carrier robot or a boarding robot, the current positions and movement speeds of other people can be used as much as possible. Perform processing to calculate a movement route that does not interfere with movement. In addition, when the robot 12 is a robot whose purpose is to provide services to people, for example, in order to approach a person who is likely to be interested in the robot, in a simple case, a person standing near the robot choose a place where people tend to gather and wait for the person who comes, and do not approach a place where people pass excessively so as not to cause congestion. We also take steps to prevent it from happening.

However, since the action of the robot determines the action of the human agent in the next step in step S7, the above process is executed in consideration of that. Therefore, it is possible to judge the suitability of a robot application program that operates in a real space coexisting with people in the virtual simulation space.

Then, in step S11, the computer, that is, the physics engine 28 implements the actions of each agent. More specifically, in step S11, as described above, each agent (including robots) performs the intended movement within the virtual simulation space 22. Each agent behaves as intended if there is no interference with For example, if the agent tries to take one step forward, the agent takes one step forward as intended.

However, in the presence of interference, the actual movement of each agent is processed according to the laws of physics (laws of mechanics). In a simple case, for example, if you try to move towards a wall, you will hit the wall, you will not be able to move further, and you will receive a reaction force according to a certain coefficient of restitution. In the case of a collision between agents, the forward movement and the movement due to the reaction force overlap, and for example, if we consider a simple example, if the agent collides from the side while moving forward, it will actually move diagonally forward. .

In the next step S13, environmental data, sensor data, etc. are updated.

In step S15, the current system time is stored, for example, in the memory (not shown) of the simulation device 16 as the execution completion time time_e in order to measure the execution period of the simulation 1 step.

Furthermore, in step S17, the simulation device 16 is put to sleep according to the execution speed. However, the sleep time will be described later in detail with reference to FIG.

Then, in step S19, it is determined whether or not to end the simulation, and if the simulation is to be continued, the process returns to the previous step S3.

In this way, the simulator 16 performs a simulation to reproduce the behavior of people around the robot according to the objective model 116, and by confirming the state on the display 105 (FIG. 1), the behavior of the robot can be simulated. Therefore, it is possible to efficiently develop a robot application program 18 that works in a real space coexisting with people without actually moving the robot in the real space.

In the simulation operation described above, the execution speed s for executing the one-step simulation of the robot application program 18 is constant and high, making it difficult to reproduce and verify the instantaneous bugs described above. Therefore, in this embodiment, by making the execution speed s changeable, for example, the simulation is executed at a low speed only in situations where bugs are likely to occur, so that the program and the simulated situation can be viewed at the same time. Facilitate problem understanding and enable efficient debugging.

FIG. 11 is a flow chart showing the operation of such an embodiment, and in the first step S21 of FIG. 11, the computer reads the desired execution speed s' set (input) by the operator.

For example, a user interface 109 having an input section 111 as shown in FIG. 12 is displayed on the display 105 . The operator inputs the desired execution speed s′ to the input unit 111 . For example, a pull-down menu should be set in the input section 111 so that any numerical value such as "2", "3", "4", "5", . . . can be set as the desired execution speed s'. Just do it. Alternatively, numerical values may be directly input to the input unit 111 using the input device 107 .

The user interface 109 also displays the current execution speed s and an apply button 113 . When the apply button 113 is operated after inputting the desired execution speed s' into the input unit 111, the desired execution speed s' can be read in step S21.

Then, in step S23, the desired execution speed s' read in step S21 is set as the execution speed s.

In step S25, the computer calculates the basic execution time t (=time_e-time_s) based on the execution start time time_s stored in the memory of the computer when executing FIG. 10 and the execution completion time time_e stored in step S15. get.

At the next step S27, the computer calculates the desired execution period p (p=s.times.1/fps). As described above, the ideal execution step number fps is set to "30.0" in the initial setting in step S21. The desired execution period p is 3/30.0 (=0.1) seconds. In other words, the execution cycle is tripled compared to the default basic execution cycle of 0.033 seconds. Triple the execution period means that the execution time per step is reduced to 1/3. In this sense, the desired execution speed s' can also be called a deceleration factor.

At step S29, the computer determines whether or not the simulation can be executed with the desired execution period p. Specifically, it is determined whether or not the desired execution cycle p is greater than the basic execution time t seconds (step S25) (t<p?).

If "YES", ie, t<p, the computer calculates the sleep time (p−t) seconds in step S31 and returns. Based on the sleep time, the computer puts the simulation to sleep in step S17 of FIG. That is, the simulation of the step is stopped for the sleep time (pt), resulting in a reduction in the execution speed of the simulation. Therefore, by performing such a slowdown (slow) simulation only in bug-prone situations, viewing the program and the simulated situation simultaneously on the display 105 facilitates understanding of the problem and leads to efficient execution. Debugging can be realized.

In the example given above, if the base execution time t was 0.01 seconds, the sleep time would be 0.1 - 0.01 = 0.09 seconds until the next step of the simulation is executed. , is slept for 0.09 seconds.

However, when "NO" is determined in step S29, in step S33, the execution speed s and the execution period p are set as executable values, s=fps*t and p=t, respectively.

In this embodiment, in addition to slowing down the execution speed, it is also assumed to speed up the execution speed, and step S33 means processing assuming that the execution speed is increased (too fast).

When the position of the three-dimensional model is changed or the sensor process is simulated in the simulation device 16, the maximum speed of the simulation cannot be increased to infinity, and there is a limit to the maximum speed. That is, in step S33 of FIG. 11, in order to limit the execution time at the maximum speed to the basic execution time t, by substituting t for the period p, it is set so as not to sleep. Therefore, the execution speed is set to s=fps×t, and the cycle calculated in step S27 is set to t. Therefore, the sleep time calculated in step S31 is "0" (zero), and sleep is not performed in step S17 (FIG. 10) in this case.

Thus, in this embodiment, steps that do not require slow execution are simulated with a basic execution cycle.

It should be noted that the specific numerical values described in the examples are merely examples, and are not limited to these, and can be changed as appropriate.

10 ... simulation system 12 ... robot 14 ... operation device 16 ... simulation device 18 ... robot application program 22 ... virtual simulation space 24 ... robot simulator 26 ... human simulator 28 ... physics engine 105 ... display 107 ... input device 109 ... user interface 111 ... Input section

Claims (4)

A simulation system that has a user interface that can manually set a desired execution speed for one step of simulation, simulates the behavior of a robot agent and a human agent that are put into a virtual simulation space, and displays the virtual simulation space on a display. hand,
Desired execution speed acquisition means for acquiring a desired execution speed set by the user interface;
basic execution time acquisition means for acquiring basic execution time based on simulation start time and end time;
Desired execution cycle calculation means for calculating a desired execution cycle according to the desired execution speed;
determining means for comparing the basic execution time and the desired execution cycle to determine whether or not the simulation can be executed with the desired execution cycle;
A simulation apparatus comprising sleep means for causing sleep for a sleep time based on the basic execution time and the desired execution cycle when the judgment means judges that the simulation is executable. further comprising sleep time calculation means for calculating the sleep time according to the difference between the desired execution cycle and the basic execution time;
2. The simulation apparatus according to claim 1 , wherein said sleep means causes sleep for said sleep time calculated by said sleep time calculation means . 3. The simulation apparatus according to claim 1, further comprising parameter changing means for changing parameters including said desired execution period to executable parameters when said determining means determines that said execution is not executable. A program for a simulation device that has a user interface that can manually set the desired execution speed for one step of simulation, simulates the actions of a robot agent and a human agent put into a virtual simulation space, and displays the virtual simulation space on a display. and
The program causes the computer of the simulation device to
Desired execution speed acquisition means for acquiring a desired execution speed set by the user interface;
basic execution time acquisition means for acquiring basic execution time based on simulation start time and end time;
Desired execution cycle calculation means for calculating a desired execution cycle according to the desired execution speed;
determining means for comparing the basic execution time and the desired execution cycle to determine whether or not the simulation can be executed with the desired execution cycle;
A simulation program that functions as a sleep means that sleeps for a sleep time based on the basic execution time and the desired execution cycle when the determination means determines that the execution is possible.

Images