JP7627006B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

This disclosure relates to an information processing device, an information processing method, and an information processing program.

Conventionally, there have been devices that perform traffic simulations according to road conditions. The simulation devices, for example, reproduce vehicle operations on a computer to perform simulations of traffic flow and congestion.
Patent Document 1 describes a simulation device that virtually drives a vehicle model down a road, and when the vehicle model recognizes an object of attention that it should pay attention to, derives a selectable action from among a number of predetermined action rules, and operates the vehicle model based on that action.

JP 2008-171358 A

Not only vehicles but also pedestrians exist on roads. In recent years, it has become desirable to simulate the behavior of pedestrians moving on roads and predict road conditions. When multiple vehicles are moving on a road, a simulation is performed of the behavior of pedestrians depending on the movement conditions of each of the multiple vehicles. However, it is desirable to perform a simulation that is closer to that of real people by taking into account whether or not pedestrians can see vehicles, and taking into account the impact that vehicles that pedestrians cannot see have on pedestrians.

The present disclosure provides an information processing device, an information processing method, and an information processing program that are used to estimate the behavior of traffic participants on a road.

An information processing device of one embodiment includes an identification unit that identifies one moving body that is visually recognized by the subject of a simulation among multiple moving bodies that are the focus of the subject's attention, a first individual estimation unit that estimates a behavior that the subject will take in response to the moving body identified by the identification unit, a second individual estimation unit that estimates a behavior that the subject will take in response to other moving bodies different from the moving body identified by the identification unit, and an integrated estimation unit that estimates a behavior that the subject will take in response to all of the multiple moving bodies based on the behavior of the subject estimated by the first individual estimation unit and the behavior of the subject estimated by the second individual estimation unit.

According to one aspect, by taking into account whether or not the subject of the simulation sees a moving object, it is possible to consider the effect on the subject of a moving object that is not seen by the subject, and perform a simulation closer to that of a real person.

FIG. 1 is a diagram illustrating an information processing device according to an embodiment. FIG. 1 is a block diagram illustrating an information processing device according to an embodiment. FIG. 13 is a diagram for explaining an example of a forgetting function. FIG. 13 is a diagram for explaining an example of a simulation situation. FIG. 11 is a diagram for explaining an example of a judgment model. 11 is a diagram for explaining an example of coefficients used in an integrated estimation unit (control unit). FIG. 1 is a flowchart illustrating an information processing method according to an embodiment.

One embodiment is described below.

[Overview of information processing device 100]
First, an overview of an information processing device 100 according to an embodiment will be described.
FIG. 1 is a diagram illustrating an information processing device 100 according to an embodiment.

The information processing device 100 may be configured, for example, as a judgment estimation device that estimates the judgment of a pedestrian walking on a road. The information processing device 100 may also be configured, for example, as a judgment estimation model providing device that provides a judgment estimation model of the pedestrian.

In general, pedestrians traveling on a road may make decisions about passing depending on the situation of moving objects such as cars and motorcycles. For example, when a pedestrian crosses a road, the pedestrian often makes a decision about whether to cross the road or not depending on the position and traveling speed of the moving object. In this case, when multiple moving objects are traveling, it may be difficult for the pedestrian to visually recognize two or more vehicles at the same time.
As illustrated in FIG. 1, when moving bodies 302 approach pedestrian 301 from both the right and left directions, it is difficult for pedestrian 301 to see both bodies at the same time, and it is considered that pedestrian 301 will alternate between seeing one moving body 302 and the other moving body 302 to confirm that it is safe before crossing the road.

The information processing device 100, for example, identifies one of the multiple moving bodies 302 that are visually recognized by the pedestrian 301 who is the subject of the simulation. The information processing device 100 estimates the behavior of the pedestrian 301 in response to the identified moving body 302 (for example, behavior of whether or not to cross the road, etc.) (first individual estimation). In addition, the information processing device 100 estimates the behavior of the pedestrian 301 in response to another moving body 302 different from the identified moving body 302 as described above (for example, behavior of whether or not to cross the road, etc.) (second individual estimation). The information processing device 100 integrates the first individual estimation and the second individual estimation to estimate the behavior of the subject in response to all of the multiple moving bodies 302 (for example, behavior of whether or not to cross the road, etc.) (integrated estimation). The information processing device 100 prepares multiple types of models according to the relationship between the pedestrian 301 and one moving body 302 as the integrated estimation, and can respond to complex situations by combining them.
In this case, the information processing device 100 may estimate the above-mentioned behavior by estimating the decision of the pedestrian 301 (subject person) as to whether or not to perform the behavior.
The information processing device 100 may be a computer such as a server, a desktop, a laptop, or a tablet.

[Details of information processing device 100]
Next, details of the information processing device 100 according to an embodiment will be described.
FIG. 2 is a block diagram for explaining the information processing device 100 according to an embodiment.

The information processing device 100 includes a communication unit 121, a memory unit 122, a display unit 123, and a control unit 110. The communication unit 121, the memory unit 122, and the display unit 123 may be an embodiment of an "output unit". The control unit 110 includes, for example, an identification unit 111, a first individual estimation unit 112, a second individual estimation unit 113, an integrated estimation unit 114, and an output control unit 115. The control unit 110 may be configured, for example, by an arithmetic processing device of the information processing device 100. The control unit 110 (for example, an arithmetic processing device, etc.) may realize the functions of each unit (for example, the identification unit 111, the first individual estimation unit 112, the second individual estimation unit 113, the integrated estimation unit 114, and the output control unit 115, etc.) by, for example, appropriately reading and executing various programs, etc. stored in the memory unit 122, etc. Here, the first individual estimation unit 112 and the second individual estimation unit 113 may be realized as different functions, or may be realized as a single function, i.e., an "individual estimation unit" that combines the first individual estimation unit 112 and the second individual estimation unit 113.

The communication unit 121 can transmit and receive various information to and from, for example, a device (external device) outside the information processing device 100. The external device may be, for example, a server 200 (see FIG. 1) or a user terminal (not shown). The user terminal may be a terminal used by a user of the information processing device 100.

The storage unit 122 may store, for example, various information and programs. Examples of the storage unit 122 may be a memory, a solid state drive, a hard disk drive, etc.

The display unit 123 is capable of displaying, for example, various characters, symbols, images, etc.

The identification unit 111 identifies one moving body that is visually recognized by the subject of the simulation among multiple moving bodies that are the focus of the subject. The identification unit 111 may identify a vehicle as a moving body that is a focus of attention when estimating the subject's behavior of whether or not to cross a road when the vehicle is traveling on a road. The identification unit 111 identifies one moving body in the situation in which the simulation is performed, that is, the subject (e.g., a pedestrian, etc.) and the number and traveling conditions (e.g., position, traveling speed, acceleration, etc. relative to the subject) of moving bodies (e.g., vehicles and motorcycles, etc.) traveling around the subject. In this case, when there are multiple moving bodies (moving bodies of interest), the identification unit 111 identifies one of the multiple moving bodies. The one moving body identified by the identification unit 111 is identified as a moving body that is visually recognized by the subject.

The first individual estimation unit 112 estimates the behavior of the subject according to the moving object identified by the identification unit 111. That is, the first individual estimation unit 112 estimates the behavior (judgment) of the subject (pedestrian) as to whether or not to cross the road according to the running conditions (movement conditions) of the moving object identified by the identification unit 111. When the subject visually recognizes a moving object, the subject judges whether or not to cross the road according to the position of the moving object (e.g., its position relative to the subject), the running speed of the moving object, the acceleration of the moving object, etc., and is considered to take an action according to the result of the judgment.

In this case, the first individual estimation unit 112 may estimate the behavior of the subject (the subject's judgment) based on the learned model and the running conditions (moving state) of the moving body identified by the identification unit 111. The above-mentioned learned model is generated, for example, by learning the running conditions (moving state) of the moving body and the behavior of the subject (the subject's judgment) taken by the subject according to the moving state of the moving body. The learned model may be generated, for example, by the control unit 110 of the information processing device 100 (for example, a learning unit, etc. (not shown)), or may be generated by a learning device (for example, the server 200) external to the information processing device 100.

For example, when the learned model is generated outside the information processing device 100, the first individual estimation unit 112 may acquire the learned model via the communication unit 121, or may acquire the learned model using an external memory or the like (not shown). The control unit 110 (for example, a learning unit, etc.) and the learning device may, for example, reproduce the vehicle movement conditions (driving conditions) in a vehicle simulation, and perform learning based on acquiring pending judgments on whether or not a person will cross a road and whether or not to cross depending on the vehicle movement conditions (driving conditions), thereby generating a learned model.

That is, the first individual estimation unit 112 may use a learned model generated by learning the vehicle position and driving speed as the moving state (driving situation) of a single moving body, and the behavior (judgment) of whether the subject crosses the road, does not cross the road, or is undecided whether to cross the road, depending on the moving state (driving state) of the vehicle, as the behavior taken by the subject. The first individual estimation unit 112 may also use multiple learned models generated for the relationship between one subject (pedestrian) and one moving body. Therefore, the first individual estimation unit 112 may calculate, for example, the probability that the subject crosses the road, does not cross the road, and is undecided whether to cross the road, as the behavior (judgment) of the subject.

The second individual estimation unit 113 estimates the behavior of the subject in response to other moving bodies different from the moving body identified by the identification unit 111. When there are multiple other moving bodies, the second individual estimation unit 113 may estimate the behavior of the subject in response to each of the multiple other moving bodies. The other moving bodies are, for example, moving bodies that are not currently visible to the subject (pedestrian). The second individual estimation unit 113 estimates the behavior (judgment) of the subject (pedestrian) as to whether or not to cross a road in response to the running conditions (movement conditions) of the other moving bodies not identified by the identification unit 111. The second individual estimation unit 113 may estimate the behavior (judgment) of the subject (pedestrian) in response to the other moving bodies by using a learned model, similar to the above-mentioned first individual estimation unit 112.

That is, the second individual estimation unit 113 may estimate the behavior of the subject (the subject's judgment) based on the learned model and the moving state (running state) of other moving bodies not identified by the identification unit 111. The learned model may be similar to the learned model used by the first individual estimation unit 112, and is generated by learning the moving state (running state) of the moving body (other moving body) and the behavior (judgment) of the subject according to the moving state (running state) of the moving body (other moving body). The learned model may be generated, for example, by the control unit 110 (for example, a learning unit, etc. (not shown)) and a learning device, etc. (for example, the server 200, etc.).

In other words, the second individual estimation unit 113 may use a learned model generated by learning the vehicle position and driving speed as the moving state (driving state) of the moving body (other moving body) and the behavior of the subject, which is whether the subject crosses the road, does not cross the road, or is undecided whether to cross or not, depending on the moving state of the vehicle, as the behavior taken by the subject. The second individual estimation unit 113 may also use multiple learned models generated in the relationship between one subject (pedestrian) and one moving body. Therefore, the second individual estimation unit 113 may calculate, for example, the probability that the subject crosses the road, that the subject does not cross the road, and that the subject is undecided whether to cross the road, as the behavior (judgment) of the subject.

In the above-mentioned case, the second individual estimation unit 113 may estimate the behavior of the subject in response to the other moving object by using the time when the subject last paid attention to the other moving object as a reference, decreasing the probability of the behavior of the subject in response to the forgetting function that indicates that the subject forgets to pay attention to the other moving object, and increasing the probability that the behavior of the subject is undecided. The forgetting function is, for example, a function that reduces the weighting (reducing the probability) of the behavior of the subject (the subject's judgment) in response to the elapsed time since the subject last viewed the moving object, and increases the weighting (increasing the probability) of the behavior of the subject being undecided. In other words, the forgetting function is, for example, a function that takes over the judgment when the subject last viewed the moving object, and decreases the probability of crossing the road and the probability of not crossing the road as time passes, and increases the probability of undecidedness of whether to cross the road or not, as time passes. The second individual estimation unit 113 estimates the action that the subject will take, for example, by continuing the judgment made when the subject last saw the vehicle, reducing the probability of the judgment made when the subject last saw the vehicle based on a forgetting function, and increasing the probability that the action that the subject will take is undecided.

Figure 3 is a diagram to explain an example of a forgetting function.

As shown in FIG. 3 as an example, the forgetting function may be a function that maintains the weighting (probability) (i.e., the probability of judgment) for a certain time (for example, 1 second, etc.) from when the subject last saw the moving object, and then decreases the weighting (probability) (i.e., the probability of judgment). The above-mentioned certain time is not limited to 1 second, and may be various arbitrary times such as 0.5 seconds, 1.5 seconds, 2 seconds, 3 seconds, 4 seconds, 5 seconds, 8 seconds, and 10 seconds. The rate of decrease of the weighting (probability) may be, for example, 5% every 0.1 seconds, or may be a certain rate such as 1%, 2%, 3%, 8%, or 10% every 0.1 seconds. The rate of decrease is not limited to the above example, and various rates can be set. The rate of decrease of the weighting (probability) is not limited to a monotonous decrease, but may be gradually decreased in multiple steps, may be decreased exponentially, or may be decreased inversely proportional to time.
Although not shown, the forgetting function also includes the above-mentioned case of increasing the undetermined probability in addition to the example of decrease shown in FIG.
Note that the second individual estimation unit 113 may not set a forgetting function depending on the embodiment, and is not limited to the above-mentioned example. That is, the second individual estimation unit 113 may maintain the weighting (probability) of the last time the subject (pedestrian) saw the moving object while the subject does not visually recognize the moving object.

The integrated estimation unit 114 estimates the behavior of the subject in response to all of the multiple moving objects, based on the behavior (judgment) of the subject estimated by the first individual estimation unit 112 and the behavior of the subject estimated by the second individual estimation unit 113. As an example, the integrated estimation unit 114 may multiply the probability of the behavior (judgment) of the subject estimated by the first individual estimation unit 112 and the probability of the behavior (judgment) of the subject estimated by the second individual estimation unit 113, and estimate the behavior (judgment) of the subject with the highest probability as the behavior (judgment) of the subject in response to all of the multiple moving objects.

In this case, the integrated estimation unit 114 may multiply a value (probability) indicating the subject's behavior estimated by the first individual estimation unit 112 and the second individual estimation unit 113 by a coefficient for integrated processing that is set in advance according to the subject's behavior, and integrate the multiplication results to estimate the behavior taken by the subject according to all of the multiple moving objects.
As an example, the coefficient may be a value such that, for example, when the highest probability estimated by the first individual estimation unit 112 is “crossing the road” and the highest probability estimated by the second individual estimation unit 113 is “crossing the road,” the integrated result (accumulated result) is the highest value for “crossing the road.”
Similarly, the coefficient may be a value such that, for example, when the highest probability estimated by the first individual estimation unit 112 is "not crossing the road" (or "undetermined") and the highest probability estimated by the second individual estimation unit 113 is "not crossing the road" (or "undetermined"), the integrated result (accumulated result) has the highest value for "not crossing the road" (or "undetermined").
The coefficient may be a value that makes the overall estimation result estimated by the integrated estimation unit 114 similar to the intention observed (observation based on observation information) by using the result of the determination of whether or not a pedestrian will cross the road according to the vehicle driving situation reproduced in the vehicle simulation (observation information). This coefficient (value) may be set based on the results of experiments or the like conducted in advance.

Here, a specific example will be described.
FIG. 4 is a diagram for explaining an example of a simulation situation.
FIG. 5 is a diagram for explaining an example of a determination model.
FIG. 6 is a diagram for explaining an example of coefficients used in the integrated estimation unit 114 (control unit 110).

As an example shown in Figure 4, we will explain the case where a subject (pedestrian) crosses an intersection at a crossroads without traffic lights, where a straight-moving vehicle S is traveling from the right side of Figure 4 and a right-turning vehicle R is traveling from the bottom of Figure 4.

The information processing device 100 (control unit 110) assumes that pedestrians cannot observe all vehicles at once, and generates a model (gaze switching model) in which the pedestrian alternates his/her gaze to view vehicles, and determines whether or not to cross the road or whether the decision is undecided based on the result. The gaze switching model can be realized, for example, by a dynamic Bayesian network or the like. The dynamic Bayesian network may be, for example, a model used to describe the time evolution of probabilities such as a Bayesian network.

The information processing device 100 (control unit 110) constructs a model for pedestrian crossing judgment according to a straight-moving vehicle S coming from the left side of the pedestrian and a right-turning vehicle R coming from in front of the pedestrian as the pedestrian's judgment target. In this case, the information processing device 100 (control unit 110) breaks down the overall judgment (judgment on two vehicles S and R) into individual judgments by identifying the vehicle that the pedestrian pays attention to. For example, the information processing device 100 (control unit 110) estimates, as individual judgments, the probability of G (Go) crossing the road, W (Wait) not crossing the road, and U (Undecided) whether to cross the road. The information processing device 100 (control unit 110) may use, for example, an individual judgment model using a logistic regression model, or an individual judgment model using another model.

Here, if two vehicles shown in FIG. 4 as an example are S and R, the label of the vehicle of interest is expressed by the following formula (1), the vehicle of interest is defined by the following formula (2), the relative state quantity with respect to vehicle i is expressed by _Xt ⁱ , the individual intention of the pedestrian toward vehicle i is expressed by the following formula (3), and the intention of the pedestrian toward the whole is expressed by the following formula (4), a judgment model as shown in FIG. 5 as an example is obtained, and the probability of each of the pedestrian's overall intentions (G/W/U) to be estimated is calculated by the following formula (5). Here, G is the probability that the pedestrian will cross the road, W is the probability that the pedestrian will not cross the road, and U is the probability that the pedestrian has not yet decided whether to cross the road or not. The judgment model shown in FIG. 5 as an example estimates an individual judgment by an individual judgment model while the pedestrian is looking at the vehicle, and estimates an individual judgment by continuing the judgment when the pedestrian last saw the vehicle while the pedestrian is not looking at the vehicle (in this case, a forgetting function is considered), and then estimates the whole judgment by integrating the two individual judgments.

…（１）

…(1)

…（２）

…(2)

…（３）

…(3)

…（４）

…(4)

…（５）

…(5)

The first term on the right side of equation (5) represents the integration of individual judgments, the second term on the right side represents an individual judgment for a straight-moving vehicle S, and the third term on the right side represents an individual judgment for a right-turning vehicle R.
Here, the overall probability of judgment at time t is calculated from the following formula (6) based on formula (5).

…（６）

…(6)

Here, the second term on the right-hand side of equation (6) indicates the probability of whether or not a pedestrian will cross at t-1, and the first term on the right-hand side is an update equation for the probability of switching between models like the one below (see, for example, equations (7)-(9)) based on whether the pedestrian can see vehicle i.

In the first term on the right side of formula (6), the first individual judgment when the pedestrian is visually recognizing the vehicle may be constructed by a logistic regression model. In this case, the first individual judgment is estimated based on a one-to-one observation experiment between the pedestrian and a straight-moving vehicle S and a one-to-one observation experiment between the pedestrian and a right-turning vehicle R.
When a pedestrian is visually recognizing a vehicle i, the first term on the right side of equation (6) becomes the following equation (7).

…（７）

…(7)

In equation (7), the probability G that a pedestrian will cross the road and the probability W that a pedestrian will not cross the road are expressed as equation (8), and the probability U that it is undecided whether a pedestrian will cross the road or not (other probabilities) is expressed as equation (9). Here, i is expressed in equation (1) above, φ _t is an explanatory variable, and η _s is a regression coefficient.

…（８）

…(8)

…（９）

…(9)

On the other hand, the first term on the right side of equation (6) uses a forgetting function, an example of which is shown in Figure 3, and as the time passes when vehicle i is not seen, the probability of the last time it was seen (the probability calculated using equations (7)-(9) above) decreases according to the forgetting function. Of course, the forgetting function also includes the case where the undetermined probability increases.

When estimating the behavior (pedestrian's judgment) of the pedestrian in response to the straight-moving vehicle S and the right-turning vehicle R by integrating the individual judgments, the information processing device 100 (control unit 110) multiplies the coefficients shown in FIG. 6 as an example. The coefficients may be weighting coefficients for the probabilities of each state according to the state in which the probability of each individual judgment (first individual judgment and second individual judgment) is highest. The information processing device 100 (control unit 110) may multiply the probabilities of G crossing the road, W not crossing the road, and U undecided whether to cross the road, obtained in each individual judgment, by a weighting coefficient for integration as shown in FIG. 6 as an example, and may use the highest probability as a result of the multiplication as the behavior (pedestrian's overall judgment) of the pedestrian in response to the straight-moving vehicle S and the right-turning vehicle R.
The coefficients (weighting coefficients) are not limited to the values exemplified in FIG. 6, but may be various values that are appropriately set.

The output control unit 115 may provide the results of estimation by the integrated estimation unit 114 (the behavior taken by the subject) to the simulation device.
Furthermore, the output control unit 115 may control the output unit to output the result (the action taken by the subject) estimated by the integrated estimation unit 114. The output unit may be, for example, the communication unit 121, the storage unit 122, the display unit 123, etc., as described above.
That is, the output control unit 115 may control the communication unit 121 to transmit the result (information on the behavior of the subject) estimated by the integrated estimation unit 114 to the outside (external device). The external device may be, for example, the server 200 and a user terminal. The user terminal may be, for example, a terminal used by a user of the information processing device 100.
The output control unit 115 may, for example, control the storage unit 122 to store the results of estimation by the integrated estimation unit 114 (information regarding the behavior of the subject).
The output control unit 115 may, for example, control the display unit 123 to display the results estimated by the integrated estimation unit 114 (the behavior taken by the subject).

[Information processing method]
An information processing method according to an embodiment will be described.
FIG. 7 is a flowchart illustrating an information processing method according to an embodiment.

In step ST101, the identification unit 111 acquires target information. The target information may be, for example, information about the situation of the simulation. That is, as an example, the target information may be information about the situation of the road, the position of pedestrians on the road, and the movement situation (driving situation) of a moving body (vehicle) traveling on the road. The identification unit 111 may acquire the target information, for example, from a simulation device that performs the simulation, or may acquire the target information based on an input operation of the target information by a user to the information processing device 100.

In step ST102, the identification unit 111 identifies one of the multiple moving bodies that the subject of the simulation sees based on the target information of step ST101. The identification unit 111 may identify, as the moving body, a vehicle as a target of attention when estimating the behavior of the subject as to whether or not the subject will cross a road when the vehicle is traveling on a road. In this case, the identification unit 111 may further identify the position of the identified moving body (vehicle) relative to the subject.

In step ST103, the first individual estimation unit 112 estimates the behavior of the subject according to the moving body (vehicle) identified in step ST102. In this case, the first individual estimation unit 112 may use an individual judgment model for the moving body (vehicle) identified in step ST102. That is, the first individual estimation unit 112 may acquire a learned model corresponding to the moving body (vehicle) identified in step ST102, and estimate the behavior of the subject based on the learned model and the moving state (running state) of the moving body (vehicle) identified in step ST102.
In the example shown in Figure 4, a learned model corresponding to a straight-moving vehicle S and a learned model corresponding to a right-turning vehicle R are prepared in advance, and for example, when a subject (pedestrian) is viewing a straight-moving vehicle S, the first individual estimation unit 112 may estimate whether the subject (pedestrian) will cross the road or not, or whether it is undecided, in accordance with the straight-moving vehicle S, based on the learned model corresponding to the straight-moving vehicle S and the driving conditions of the straight-moving vehicle S.

In step ST104, the second individual estimation unit 113 estimates the behavior of the subject according to the other moving body (other vehicle) not identified in step ST103. In this case, the second individual estimation unit 113 may use an individual judgment model for the other moving body (other vehicle) not identified in step ST102, similar to the above-mentioned first individual estimation unit 112. That is, the second individual estimation unit 113 may acquire a learned model corresponding to the other moving body (other vehicle) not identified in step ST102, and estimate the behavior of the subject based on the learned model and the moving state (running state) of the other moving body (other vehicle) not identified in step ST102.
In the example of step ST103 using Figure 4, for example, when a subject (pedestrian) does not see a right-turning vehicle R, the second individual estimation unit 113 may estimate whether the subject (pedestrian) will cross the road or not, or whether it is undecided, based on a learned model corresponding to the right-turning vehicle R and the driving conditions of the right-turning vehicle R.

In this case, the second individual estimation unit 113 may estimate the behavior that the subject will take in response to other moving bodies (other vehicles) by reducing the probability that the behavior will be taken by the subject in response to the other moving bodies (other vehicles) in response to the forgetting function, and by increasing the probability that the behavior that the subject will take is undetermined.
That is, in the above-mentioned example using Figure 4, for example, by reducing a value corresponding to a forgetting function from the probability when the subject (pedestrian) last saw the right-turning vehicle R, and increasing the probability that the action to be taken by the subject is undecided, it is possible to estimate whether the subject (pedestrian) will cross the road or not, and whether the action is undecided, in response to the right-turning vehicle R.

The first individual estimation unit 112 and the second individual estimation unit 113 may alternate between the target moving objects, i.e., the moving objects identified in step ST102 (as an example, by alternating between them at predetermined time intervals), depending on the situation in which the subject (pedestrian) alternately views multiple moving objects, and estimate the behavior of the subject (pedestrian) in response to each moving object.

In step ST105, the integrated estimation unit 114 estimates the behavior of the subject in response to all of the multiple moving objects, based on the behavior (judgment) of the subject estimated in step ST103 and the behavior of the subject estimated in step ST104. In this case, the integrated estimation unit 114 may multiply (accumulate) the probability calculated in step ST103, the probability calculated in step ST104, and a weighting coefficient used when integrating the first individual judgment in step ST103 and the second individual judgment in step ST104, and estimate the state with the highest probability as the behavior (overall behavior) taken by the subject.

Each unit of the information processing device 100 described above may be realized as a function of a computer's arithmetic processing device, etc. That is, the identification unit 111, the first individual estimation unit 112, the second individual estimation unit 113 (individual estimation units), the integrated estimation unit 114, and the output control unit 115 (control unit 110) of the information processing device 100 may be realized as a identification function, a first individual estimation function, a second individual estimation function (individual estimation function), an integrated estimation function, and an output control function (control function), respectively, by a computer's arithmetic processing device, etc.
The information processing program can cause a computer to realize each of the above-mentioned functions. The information processing program may be recorded in a non-transitory computer-readable recording medium, such as a memory, a solid-state drive, a hard disk drive, or an optical disk.
As described above, each unit of the information processing device 100 may be realized by a computer arithmetic processing device or the like. The arithmetic processing device or the like is configured by, for example, an integrated circuit or the like. Therefore, each unit of the information processing device 100 may be realized as a circuit constituting the arithmetic processing device or the like. That is, the identification unit 111, the first individual estimation unit 112, the second individual estimation unit 113 (individual estimation unit), the integrated estimation unit 114, and the output control unit 115 (control unit 110) of the information processing device 100 may be realized as a identification circuit, a first individual estimation circuit, a second individual estimation circuit (individual estimation circuit), an integrated estimation circuit, and an output control circuit (control circuit) constituting the arithmetic processing device or the like of a computer.
The communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as a communication function, a storage function, and a display function (output function) including the functions of an arithmetic processing device, etc. The communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as a communication circuit, a storage circuit, and a display circuit (output circuit) by being configured, for example, by an integrated circuit, etc. The communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as a communication device, a storage device, and a display device (output device) by being configured, for example, by being configured by a plurality of devices.

The information processing device 100 can combine one or any two or more of the above-mentioned units.
In this disclosure, the term "information" is used, but the term "information" can be replaced with "data" and the term "data" can be replaced with "information."

[Aspects and Effects of the Present Embodiment]
Next, one aspect of this embodiment and the effects of each aspect will be described. Note that this embodiment is not limited to each aspect described below, and may be realized by appropriately combining each of the above-mentioned parts. Also, the effects described below are examples, and the effects of each aspect are not limited to those described below.

(Aspect 1)
An information processing device of one embodiment includes an identification unit that identifies one moving body that is visually recognized by the subject of a simulation among multiple moving bodies that are the focus of the subject's attention, a first individual estimation unit that estimates a behavior that the subject will take in response to the moving body identified by the identification unit, a second individual estimation unit that estimates a behavior that the subject will take in response to other moving bodies different from the moving body identified by the identification unit, and an integrated estimation unit that estimates a behavior that the subject will take in response to all of the multiple moving bodies based on the behavior of the subject estimated by the first individual estimation unit and the behavior of the subject estimated by the second individual estimation unit.
As a result, the information processing device estimates the behavior of the subject based on a one-to-one relationship between the subject and the moving body in each of the first individual estimation unit and the second individual estimation unit, so that the judgment model for individual estimation can be simplified compared to a judgment model set according to the overall situation of the simulation. In other words, by integrating multiple individual judgment models throughout the entire simulation, the information processing device can simplify the simulation compared to a case where one judgment model is used throughout the entire simulation. Furthermore, by integrating multiple individual judgment models, the information processing device can respond to more simulation situations.
The information processing device separately uses moving objects that the subject visually recognizes and moving objects that the subject does not visually recognize, and therefore can perform a simulation that corresponds to the actual situation of traffic participants with high accuracy. In addition, the information processing device performs individual judgments on moving objects that the subject recognizes but does not visually recognize, and therefore can perform a simulation that takes into account the impact of moving objects that are not visually recognized on the subject. In other words, the information processing device can construct a judgment model that is closer to the judgment of an actual human being by considering the presence or absence of visual recognition according to the visual behavior model of the subject, and can perform a simulation that corresponds to reality.

(Aspect 2)
In one embodiment of the information processing device, the second individual estimation unit may estimate an action that the subject will take in response to another moving body by using the time that the subject last paid attention to the other moving body as a reference, reducing the probability that the action that the subject will take is in response to a forgetting function that represents the subject forgetting to pay attention to the other moving body, and increasing the probability that the action that the subject will take is undecided.
This allows the information processing device to perform a simulation that matches the actual situation of the traffic participants, even when there are multiple moving objects of interest and one of them is not visible to the traffic participant. That is, the information processing device performs a judgment for each moving object as seen from the traffic participant, while using a forgetting function to represent the fading of information within the traffic participant about moving objects that are not visible to the traffic participant, thereby obtaining a realistic judgment result as an integration result.

(Aspect 3)
In one embodiment of an information processing device, each of the first individual estimation unit and the second individual estimation unit may estimate the behavior taken by the subject based on a learned model generated by learning the movement state of the moving body and the behavior taken by the subject in accordance with the movement state of the moving body, and the movement state of the moving body identified by the identification unit.
This allows the information processing device to estimate the behavior of the subject based on a one-to-one relationship between the subject and the moving object. The information processing device can also make judgments according to multiple individual situations by using a trained model that is appropriately selected according to the overall situation of the simulation. In other words, the information processing device can break down the overall situation of the simulation into multiple individual judgment models and estimate the behavior of the subject in each judgment model.

(Aspect 4)
In one aspect of the information processing device, the identification unit may identify a vehicle as a moving object, as an object of interest when estimating the behavior of a subject as to whether or not to cross a road, when the vehicle is traveling on a road.
This enables the information processing device to estimate whether the subject person will cross the road or not, depending on the driving conditions of the vehicle.

(Aspect 5)
In one aspect of the information processing device, each of the first individual estimation unit and the second individual estimation unit may utilize a learned model generated by learning the vehicle position and driving speed as the movement state of the moving body, and the behavior taken by the subject, such as whether the subject crosses the road, whether or not to cross the road, and whether or not to cross the road is undecided, depending on the movement state of the vehicle.
This enables the information processing device to calculate the probability of an individual judgment as to whether or not the subject person will cross the road, and whether or not it is undecided, depending on the driving conditions of the vehicle.

(Aspect 6)
In one embodiment of the information processing device, the integrated estimation unit may multiply a value indicating the subject's behavior estimated by the first individual estimation unit and the second individual estimation unit by a coefficient for integrated processing that is set in advance according to the subject's behavior, and integrate the results of the multiplication to estimate the behavior taken by the subject according to all of the multiple moving objects.
This allows the information processing device to integrate multiple individual judgments and estimate an overall judgment of the subject person according to multiple moving objects.

(Aspect 7)
In one aspect of the information processing method, a computer executes the following steps: an identification step of identifying one moving body that is visually recognized by the subject of a simulation among multiple moving bodies that are the focus of the subject's attention; a first individual estimation step of estimating a behavior that the subject will take in response to the moving body identified in the identification step; a second individual estimation step of estimating a behavior that the subject will take in response to other moving bodies different from the moving body identified in the identification step; and an integrated estimation step of estimating a behavior that the subject will take in response to all of the multiple moving bodies based on the behavior of the subject estimated in the first individual estimation step and the behavior of the subject estimated in the second individual estimation step.
As a result, the information processing method can achieve the same effects as the information processing device according to the above-described aspect.

(Aspect 8)
An information processing program of one embodiment causes a computer to realize a identification function that identifies one moving body that is visually recognized by the subject of a simulation among multiple moving bodies that are the focus of the subject's attention, a first individual estimation function that estimates a behavior that the subject will take in response to the moving body identified by the identification function, a second individual estimation function that estimates a behavior that the subject will take in response to other moving bodies different from the moving body identified by the identification function, and an integrated estimation function that estimates a behavior that the subject will take in response to all of the multiple moving bodies based on the behavior of the subject estimated by the first individual estimation function and the behavior of the subject estimated by the second individual estimation function.
As a result, the information processing program can achieve the same effects as the information processing device according to the above-described aspect.

REFERENCE SIGNS LIST 100 Information processing device 110 Control unit 111 Identification unit 112 First individual estimation unit 113 Second individual estimation unit 114 Integrated estimation unit 115 Output control unit 121 Communication unit 122 Storage unit 123 Display unit

Claims (8)

The information processing device has position information of the vehicle relative to the pedestrian ,
An identification unit that identifies one vehicle that is visually recognized by a pedestrian among a plurality of vehicles that are targets of attention of a pedestrian in a simulation;
a first individual estimation unit that estimates an action to be taken by the pedestrian according to a moving state including at least a position of the vehicle with respect to the pedestrian identified by the identification unit and information regarding a speed based on an amount of change in the position, the action being one of an action of the pedestrian crossing a road, an action of the pedestrian not crossing the road, and an action of the pedestrian not deciding whether to cross the road or not ;
a second individual estimation unit that estimates a behavior of the pedestrian based on a moving state including information on at least a position of another vehicle other than the vehicle identified by the identification unit with respect to the pedestrian and a speed based on an amount of change in the position;
an integrated estimation unit that estimates a behavior of a pedestrian in response to all of a plurality of vehicles , based on the behavior of the pedestrian estimated by the first individual estimation unit and the behavior of the pedestrian estimated by the second individual estimation unit;
An information processing device comprising: 2. The information processing device according to claim 1, wherein the second individual estimation unit estimates the action that the pedestrian will take in response to another vehicle by using the time that the pedestrian last paid attention to another vehicle as a reference, reducing the probability that the judgment is based on the time that the pedestrian last paid attention to another vehicle in accordance with a forgetting function that represents the pedestrian 's forgetting to pay attention to other vehicles , and increasing the probability that the judgment is that the action that the pedestrian will take is undecided. The information processing device according to claim 1 or 2, wherein the first individual estimation unit and the second individual estimation unit each estimate the behavior of a pedestrian based on a learned model generated by learning at least the position of a vehicle relative to a pedestrian , a moving state including information regarding a speed based on an amount of change in position, and the behavior of a pedestrian based on the moving state of the vehicle , and the moving state of a vehicle identified by the identification unit, and the second individual estimation unit performs estimation for another vehicle other than the vehicle identified by the identification unit. 4. The information processing device according to claim 1 , wherein the identification unit is configured to identify a vehicle as an object of interest when estimating a pedestrian's behavior as to whether or not the pedestrian will cross a road when the vehicle is traveling on the road. 5. The information processing device according to claim 4, wherein each of the first individual estimation unit and the second individual estimation unit utilizes a learned model generated by learning a vehicle 's position and traveling speed as a vehicle's movement state, and a decision as to whether a pedestrian will cross a road, not cross the road, or is undecided as to whether to cross the road, depending on the vehicle's movement state, as a pedestrian's behavior. The information processing device according to any one of claims 1 to 5, wherein the integrated estimation unit multiplies a value indicating the pedestrian 's behavior estimated by the first individual estimation unit and the second individual estimation unit by a coefficient for integration processing that is set in advance according to the pedestrian 's behavior, and integrates the multiplication result to estimate the behavior of the pedestrian in accordance with all of a plurality of vehicles . A computer that holds vehicle position information relative to pedestrians
A step of identifying one vehicle that is visible to a pedestrian among a plurality of vehicles that are the focus of attention of the pedestrian in the simulation;
a first individual estimation step of estimating an action to be taken by the pedestrian according to a moving state including information on at least a position of the vehicle relative to the pedestrian identified by the identification step and a speed based on an amount of change in the position, the action being one of an action of the pedestrian crossing a road, an action of the pedestrian not crossing the road, and an action of the pedestrian not deciding whether to cross the road or not ;
a second individual estimation step of estimating a behavior of the pedestrian based on a moving state including information on at least a position of another vehicle other than the vehicle identified by the identification step with respect to the pedestrian and a speed based on an amount of change in the position;
an integrated estimation step of estimating a behavior of a pedestrian in response to all of a plurality of vehicles , based on the behavior of the pedestrian estimated in the first individual estimation step and the behavior of the pedestrian estimated in the second individual estimation step;
An information processing method for performing the above. A computer that holds vehicle position information on pedestrians ,
An identification function that identifies one vehicle that is visible to a pedestrian among a plurality of vehicles that are the focus of attention of the pedestrian in the simulation;
a first individual estimation function for estimating an action to be taken by the pedestrian according to a moving state including information on at least a position of the vehicle relative to the pedestrian identified by the identification function and a speed based on an amount of change in the position , the action being one of the actions of the pedestrian crossing a road, the action of the pedestrian not crossing the road, and the action of the pedestrian not deciding whether to cross the road or not ;
a second individual estimation function that estimates a behavior of the pedestrian based on a moving state including information on a position of another vehicle other than the vehicle identified by the identification function with respect to the pedestrian and a speed based on an amount of change in the position;
an integrated estimation function that estimates a behavior of a pedestrian in response to all of a plurality of vehicles , based on the behavior of the pedestrian estimated by the first individual estimation function and the behavior of the pedestrian estimated by the second individual estimation function;
An information processing program that realizes this.

Images