JP7814632B2 - Motion evaluation device, motion evaluation method, and program - Google Patents

Description

The disclosed technology relates to a motion evaluation technology that evaluates the motion of a passenger driving a vehicle.

In recent years, technological developments have been made to encourage drivers of mobile vehicles to drive in an exemplary manner.
Patent Document 1 discloses a technique for evaluating the driving operation of a vehicle driver.
The technology described in Patent Document 1 uses information about devices obtained by sensors to create a database of time-series data on the operation of devices related to driving operations of a vehicle for each location, assuming that exemplary driving operations have been performed, and uses the database to evaluate driving operations based on the difference between operation data related to actual driving operations and operation data related to exemplary driving operations (paragraphs [0016], [0020], [0021], [0024], etc. of Patent Document 1).

JP 2009-294250 A

However, depending on the environment of the road on which the vehicle is traveling (road environment), it may be preferable for the system to operate before the occupant actually performs driving operations, for example, when the occupant is observing the surrounding situation in order to collect information necessary for driving operations.
The technology described in Patent Document 1 has a problem in that it is not possible to perform evaluation based on actions other than driving operations estimated from the state of devices related to driving operations, and therefore it is not possible to perform a more favorable driving evaluation.

The present disclosure aims to solve the above problem and enable more favorable driving evaluations than have been possible in the past.

The action evaluation device of the present disclosure includes:
a motion acquisition unit that acquires a motion of a driver of the vehicle;
a position acquisition unit that acquires the position of the moving object;
a road environment acquisition unit that refers to a database that stores in advance position information and road environment information in association with each other, and acquires the road environment information corresponding to the position acquired by the position acquisition unit;
a model acquisition unit that refers to a database that associates the road environment information with an exemplary behavior model that shows exemplary behavior according to the road environment and stores the association, and acquires an exemplary behavior model according to the road environment information acquired by the road environment acquisition unit at the position of the moving object ;
an evaluation value calculation unit that calculates an evaluation value of the action based on the action acquired by the action acquisition unit and the exemplary behavior model acquired by the model acquisition unit;
It is equipped with the following.
Alternatively, the action evaluation device of the present disclosure is
a motion acquisition unit that acquires a motion of a driver of the vehicle;
a position acquisition unit that acquires the position of the moving object;
a model acquisition unit that acquires an exemplary behavior model corresponding to the road environment at the position of the mobile object using a database that associates and pre-stores location information with an exemplary behavior model that shows exemplary behavior corresponding to the road environment;
an evaluation value calculation unit that calculates an evaluation value of the action based on the action acquired by the action acquisition unit and the exemplary behavior model acquired by the model acquisition unit;
A motion evaluation device comprising:
an accumulated inattentive state determination unit that obtains an accumulated value related to intermittent inattentive state by the occupant and performs an accumulated inattentive state determination by using the accumulated value and a predetermined threshold value;
an accumulated inattentive point acquisition unit that acquires accumulated inattentive points based on the position of the moving body when the accumulated inattentive state is determined to be present by the accumulated inattentive state determination unit;
an observation direction acquisition unit that acquires an observation direction that is a direction in which the occupant of the moving body is likely to be observing;
an accumulative inattentiveness frequent occurrence point acquisition unit that acquires accumulative inattentiveness frequent occurrence points, which are points where accumulative inattentiveness by occupants of each of a plurality of moving bodies frequently occurs;
an allowable observation direction acquisition unit that acquires an allowable observation direction, which is a direction in which observation should be allowed for each of the cumulative inattentive looking points;
Furthermore,
The model acquisition unit
When the cumulative inattentive driving point and the cumulative inattentive driving frequent point overlap,
and,
When the observation direction acquired by the observation direction acquisition unit overlaps with the allowable observation direction,
Observing the permissible observation direction at the position of the cumulative inattentive viewing point is acquired as the exemplary behavior model.
It is something.

This disclosure has the effect of enabling more favorable driving evaluations than conventional methods.

FIG. 1 is a diagram illustrating an example of the configuration of an action evaluation device according to a first embodiment of the present disclosure. FIG. 2 is a diagram illustrating a configuration example of an action evaluation system including an action evaluation device according to the first embodiment of the present disclosure. FIG. 3 is a flowchart illustrating an example of processing performed by the action evaluating device according to the first embodiment of the present disclosure. FIG. 4 is a diagram illustrating a configuration example of an action evaluation system including an action evaluation device according to the second embodiment of the present disclosure. FIG. 5 is a flowchart illustrating an example of processing performed by the action evaluating device according to the second embodiment of the present disclosure. FIG. 6 is a diagram illustrating a configuration example of an action evaluation system including an action evaluation device according to the third embodiment of the present disclosure. FIG. 7 is a flowchart illustrating an example of processing performed by the action evaluating device according to the third embodiment of the present disclosure. FIG. 8 is a diagram illustrating a configuration example of an action evaluation system including an action evaluation device according to the fourth embodiment of the present disclosure. FIG. 9 is a first diagram for explaining an example of accumulated inattentive behavior handled by the action evaluation device according to the fourth embodiment. FIG. 10 is a second diagram for explaining an example of accumulated inattentive behavior handled by the action evaluation device according to the fourth embodiment. FIG. 11 is a flowchart illustrating an example of processing performed by the action evaluating device according to the fourth embodiment of the present disclosure. FIG. 12 is a flowchart illustrating an example of processing by the model acquisition unit in the action evaluation device according to the fourth embodiment of the present disclosure. FIG. 13 is a diagram illustrating the relationship between the action evaluating device according to the fifth embodiment of the present disclosure and a server device that generates a database used in the action evaluating device. FIG. 14 is a diagram illustrating a first configuration example of an action evaluation system E including an action evaluation device according to the fifth embodiment of the present disclosure. FIG. 15 is a diagram illustrating a second configuration example of an action evaluation system including an action evaluation device according to the fifth embodiment of the present disclosure. FIG. 16 is a diagram illustrating a third configuration example of an action evaluation system including an action evaluation device according to the fifth embodiment of the present disclosure. FIG. 17 is a flowchart illustrating an example of processing by the action information providing unit in the action evaluating device according to the fifth embodiment of the present disclosure. FIG. 18 is a flowchart illustrating an example of processing performed by the server device when action information is provided from the action evaluating device according to the fifth embodiment of the present disclosure. FIG. 19 is a diagram illustrating a first example of a hardware configuration for realizing the functions according to the configuration of the present disclosure. FIG. 20 is a diagram illustrating a second example of a hardware configuration for realizing the functions according to the configuration of the present disclosure.

To explain the present disclosure in more detail, an embodiment of the present disclosure will be described below with reference to the accompanying drawings.

Embodiment 1.
In the first embodiment, a basic form of the present disclosure will be described.

An example of the configuration of the action evaluation device will be described.
FIG. 1 is a diagram illustrating an example of a configuration of an action evaluating apparatus 100 according to a first embodiment of the present disclosure.
The action evaluation device 100 evaluates the action of a passenger on board a moving body, such as a vehicle.
The action evaluation device 100 evaluates whether the occupant's action deviates from a model action stored in advance.
The action evaluation device 100 shown in FIG. 1 includes an action acquisition unit 111 , a position acquisition unit 112 , a model acquisition unit 113 , and an evaluation value calculation unit 114 .

The motion acquisition unit 111 acquires the motion of a driver driving a vehicle.
The action acquisition unit 111 acquires the action itself performed by the occupant.
The movement acquired by the movement acquisition unit 111 is, for example, the movement of the occupant determined using a video image of the interior of the vehicle.
This makes it possible to obtain both the behavior before and the behavior after the driving operation, compared to conventional driving operations that are inferred from the state of the equipment.
In this case, the actions that can be acquired by the action acquisition unit 111 are shown in the form of, for example, the direction in which the occupant's face is facing, the direction in which the occupant is looking, the position of the occupant's hands, change values indicating the extent to which each of these has changed over time, and the time indicating how long each of these remains constant.

The position acquisition unit 112 acquires the position of the mobile object.
The position acquisition unit 112 acquires positions in chronological order from at least one of a GPS (Global Positioning System) device mounted on a mobile object and a car navigation device, for example.
The position acquisition unit 112 can output motion information, which is a set of information indicating the acquired position and information indicating the motion acquired by the motion acquisition unit 111 .

The model acquisition unit 113 acquires an exemplary behavior model that indicates an exemplary behavior according to the road environment at the position acquired by the position acquisition unit 112 .
For example, the model acquisition unit 113 acquires an exemplary behavior model that corresponds to the road environment at the location of the mobile body using a database that pre-stores location information and an exemplary behavior model that shows exemplary behavior according to the road environment, in association with the location information.
The "road environment" refers to the environment around the road in which an occupant must take an action before actually operating the vehicle.
When the moving body is a vehicle, the road environment includes at least one of a motorway, a merging area of a motorway, near an intersection, near a crosswalk, the presence or absence of a traffic light, the shape of the road, the width of the road, the degree of vehicle congestion, or the degree of pedestrian congestion.
An example of a motorway is an expressway.
A merging area of an expressway is, for example, an area including an acceleration lane located at a merging point between a main lane of an expressway and a merging lane that merges onto the main lane.
The vicinity of an intersection indicates, for example, an environment in which an intersection exists in the direction of travel of the vehicle based on the actual position of the vehicle.
The vicinity of a pedestrian crossing indicates, for example, an environment in which a pedestrian crossing exists in the direction of travel of the vehicle based on the actual position of the vehicle.
The presence or absence of a traffic light indicates, for example, the environment, such as whether or not there is a traffic light in the direction of travel of the vehicle based on the actual position of the vehicle.
The shape of the road includes, for example, an environment in which the road in the direction of travel of the vehicle is curved with respect to the actual position of the vehicle.
The width of the road indicates the environment, for example, the width of the road in the direction of travel of the vehicle based on the actual position of the vehicle.
The degree of vehicle congestion indicates the number of vehicles traveling on the road in the direction of travel of the vehicle, based on the actual position of the vehicle. When there are many vehicles traveling on the road (for example, in the case of a traffic jam), it is possible to evaluate the behavior of the occupant, for example, the driver, based on the idea that it is better to prepare to press the brake pedal than when there are few vehicles traveling on the road.
The degree of pedestrian congestion indicates the number of people walking around the roadway in the direction of travel of the vehicle, based on the actual position of the vehicle. When there are many people walking around the roadway and the roadway is narrow, it is possible to evaluate the behavior of the occupant, for example, the driver, based on the idea that it would be better for the occupant to observe the sidewalk.
The "model behavior model" is information indicating model behavior according to the road environment.
The model behavior model includes at least one of a viewing direction, cumulative inattentiveness, or a position of a body part.
The "viewing direction" is the direction from which the occupant is likely to be looking.
The "observation direction" includes at least one of the direction in which the occupant's face is facing (face orientation) and the occupant's line of sight. By including this behavior in the model behavior model, it becomes possible to evaluate, for example, whether the occupant is observing the direction of the sidewalk when the mobile object is located in a road environment such as near a crosswalk. Also, for example, it becomes possible to evaluate, for example, whether the occupant is observing the direction of the road sign when the mobile object is located in a road environment where a road sign is located diagonally above and in front of the mobile object.
"Cumulative inattentiveness" refers to an inattentive behavior based on a cumulative value of intermittent inattentiveness by a vehicle occupant, and is a behavior that occurs when, for example, the cumulative value reaches or exceeds a threshold value over a predetermined unit of time. By including this behavior in the model behavior model, it becomes possible to evaluate, for example, when the vehicle is in a merging area of a highway, whether the occupant is sufficiently observing alternately the area in front of the vehicle and the area behind the road where the vehicle is merging before merging.
The "position of body parts" includes, for example, the angle of the hand position relative to the steering wheel. By including this movement in the model behavior model, it becomes possible to evaluate, for example, whether the occupant is ready to operate the turn signal before entering an intersection when the vehicle is turning right or left at an intersection.
The exemplary behavior model may include driving operations performed on the moving object, such as accelerating or decelerating the vehicle's speed, turning on or off a turn signal, and the like.

The evaluation value calculation unit 114 calculates an evaluation value of the action performed by the occupant.
The evaluation value calculation unit 114 calculates an evaluation value of the movement based on the movement acquired by the movement acquisition unit 111 and the exemplary behavior model acquired by the model acquisition unit 113 .
The evaluation value can be calculated using a known technique.
The evaluation value calculation unit 114, for example, compares the behavior acquired by the behavior acquisition unit 111 with the exemplary behavior model acquired by the model acquisition unit 113 to calculate a difference value, and derives an evaluation value indicating the degree of deviation of the actual behavior from the exemplary behavior model using the difference value. The evaluation value in this case can also be expressed as the degree of deviation or as a driving degree.

In addition to the above components, the action evaluation device 100 also includes a control unit (not shown), a storage unit (not shown), and a communication unit (not shown).
A control unit (not shown) controls the entire action evaluation device 100 and each of its components. The control unit (not shown) starts up the action evaluation device 100 in accordance with, for example, an external command. The control unit (not shown) also controls the state of the action evaluation device 100 (operation state = state such as startup, shutdown, or sleep).
A storage unit (not shown) stores each piece of data used in the action evaluation device 100. The storage unit (not shown), for example, stores the output (output data) from each component in the action evaluation device 100, and outputs data requested by each component to the requesting component.
The communication unit (not shown) communicates with external devices. For example, communication is performed between the action evaluation device 100 (100A) and peripheral devices (an occupant monitoring device 300A and a server device 600E, which will be described later). For example, if the action evaluation device 100 and the occupant monitoring device are not connected by wire, the communication unit (not shown) has a function of communicating between the action evaluation device 100 and the occupant monitoring device 300A. The communication unit (not shown) also has a function of communicating with the server device 600E, which is an external device.
The control unit (not shown), the storage unit (not shown), and the communication unit (not shown) are the same in the embodiments described below.

Next, a configuration example of an action evaluation system including an action evaluation device will be described.
FIG. 2 is a diagram illustrating an example of the configuration of an action evaluation system 1A including an action evaluation apparatus 100A according to the first embodiment of the present disclosure.
The action evaluation system 1A shown in FIG. 2 includes an action evaluation device 100A, a model behavior database 200A (database), an occupant monitoring device 300A, and an output device 400A.
The passenger monitoring device 300A, the model behavior database 200A (database), the action evaluation device 100A, and the output device 400A will be described in detail in this order.

The occupant monitoring device 300A is a device that monitors the state of an occupant, and is a device used in, for example, a DMS (Driver Monitoring System).
The occupant monitoring device 300A includes an image acquisition unit 301A and a motion determination unit 302A.

The video acquisition unit 301A acquires a video of the interior of a moving object.
The image acquisition unit 301A acquires an image from, for example, an imaging device that captures an image of the interior of a mobile object.

The movement determining unit 302A determines the movement of the occupant captured in the video acquired by the video acquiring unit 301A.
The movement determining unit 302A obtains the image from the image obtaining unit 301A, and determines the movement of the occupant using the obtained image.
The technique for determining the movement of the occupant can be realized using known techniques.

The occupant monitoring device 300A outputs information indicating the occupant's movements determined by the movement determination unit 302A to the movement evaluation device 100A.

The model behavior database 200A (database) is a database that stores in advance location information and model behavior models in association with each other.
The model behavior database 200A (database) may be a database that stores location information and model behavior models in association with each other, or may be a database that stores location information, road environment information indicating the road environment, and model behavior models in association with each other.
The location information is information in the form of, for example, coordinate values of map data.
The location information is not limited to a single point, but may be information indicating an area.
The road environment information is information that indicates the road environment so that each of the above-mentioned types of road environment can be distinguished.

The action evaluation device 100A shown in FIG. 2 is communicably connected to an occupant monitoring device 300A, an exemplary behavior database 200A (database), and an output device 400A.
The action evaluation device 100A includes an action acquisition unit 111A, a position acquisition unit 112A, a model acquisition unit 113A, and an evaluation value calculation unit 114A.

The motion acquisition unit 111A has the same functions as the motion acquisition unit 111 already described, and a detailed description thereof will be omitted.
The action acquisition unit 111A acquires the occupant's actions output by the occupant monitoring device 300A and uses them for processing.

The location acquisition unit 112A has the same functions as the location acquisition unit 112 already described, and detailed explanation will be omitted.

The model acquisition unit 113A has the same functions as the model acquisition unit 113 already described, and a detailed description thereof will be omitted.
The model acquisition unit 113A refers to the model behavior database 200A to acquire a model behavior model that corresponds to the road environment at the location of the mobile object.

The evaluation value calculation unit 114A has the same function as the evaluation value calculation unit 114 already described, and a detailed description thereof will be omitted.
The evaluation value calculation unit 114A outputs the calculated evaluation value to the output device 400A.

The output device 400A has at least one of a display function, a sound output function, and a vibration output function.
The output device 400A shown in FIG. 2 outputs information based on the information received from the action evaluation device 100A.
For example, the output device 400A receives an instruction to display an image including an evaluation value from the action evaluation device 100A, and displays the image including the evaluation value.

An example of processing by the action evaluation device will be described.
Since the processing of the action evaluating apparatus 100 and the processing of the action evaluating apparatus 100A are similar, an example of the processing of the action evaluating apparatus 100A will be described here as a representative.
FIG. 3 is a flowchart showing an example of processing by the action evaluating apparatus 100A according to the first embodiment of the present disclosure.
Action evaluation device 100A starts the processing shown in Fig. 3 upon receiving a command from a control unit (not shown) or an external command. Action evaluation device 100A starts the processing, for example, when operation of a mobile object is started.

The action evaluation apparatus 100A first executes an action acquisition process (step ST1110).
In the action acquisition process, the action acquisition unit 111A of the action evaluation device 100A acquires the action of the occupant driving the moving body.
The action acquisition unit 111A acquires information relating to the action output from the occupant monitoring device 300A and outputs the information to the position acquisition unit 112A.

The action evaluation device 100A then executes a position acquisition process (step ST1120).
In the position acquisition process, the position acquisition unit 112A of the action evaluation device 100A acquires the position of the moving object.
Upon acquiring the information relating to the motion output by the motion acquisition unit 111A, the position acquisition unit 112A acquires position information indicating the position of the moving object. The position acquisition unit 112A outputs motion information, which is a set of the position information and the information relating to the motion, to the model acquisition unit 113A.

The action evaluation apparatus 100A then executes a model acquisition process (step ST1130).
In the model acquisition process, the model acquisition unit 113A of the action evaluation device 100A acquires an exemplary behavior model that indicates an exemplary action according to the road environment at the position acquired by the position acquisition unit 112A.
When the model acquisition unit 113A acquires the behavior information output by the position acquisition unit 112A, it uses the position information included in the behavior information to refer to the model behavior database 200A and acquires an model behavior model that corresponds to the road environment at the location of the mobile body.
The model acquisition unit 113A outputs the motion information and the exemplary behavior model to the evaluation value calculation unit 114A.

The action evaluation apparatus 100A then executes an evaluation value calculation process (step ST1140).
In the evaluation value calculation process, the evaluation value calculation unit 114A of the action evaluation device 100A calculates an evaluation value of the action based on the action acquired by the action acquisition unit 111 and the exemplary behavior model acquired by the model acquisition unit 113A.
Specifically, for example, when the evaluation value calculation unit 114A acquires the motion information and the exemplary behavior model output by the model acquisition unit 113A, the evaluation value calculation unit 114A compares the information related to the motion included in the motion information with the exemplary behavior model to calculate a difference value. The evaluation value calculation unit 114A uses the difference value to calculate an evaluation value (deviation degree) indicating the degree of deviation of the actual motion from the exemplary behavior model.
The evaluation value calculation unit 114A outputs the evaluation value as the calculation result to the output device 400A.
As a result, the output device 400A displays the evaluation value.

The action evaluation apparatus 100A then executes an end determination process (step ST1150).
In the termination determination process, a control unit (not shown) of action evaluation apparatus 100A determines whether to terminate the processing of action evaluation apparatus 100A. The control unit (not shown) determines whether to terminate the processing of action evaluation apparatus 100A, for example, in accordance with an external termination command or an execution program.
If the control unit (not shown) determines not to end the processing of the action evaluation device 100A, the process proceeds to step ST1110, and the processing is repeated from step ST1110.
When the control unit (not shown) determines that the processing of the action evaluation device 100A should be terminated, the action evaluation device 100A terminates the processing.

The above-described disclosure allows for a more favorable driving evaluation than in the past.
For example, before the occupant actually performs the driving operation of the vehicle, it becomes possible to evaluate the actions that the occupant should perform.
Furthermore, for example, the evaluation of the occupant's actions can be changed depending on the road environment at the location of the vehicle.

The action evaluation device of the present disclosure is configured, for example, as follows.

a motion acquisition unit that acquires a motion of a driver of the vehicle;
a position acquisition unit that acquires the position of the moving object;
a model acquisition unit that acquires an exemplary behavior model corresponding to the road environment at the position of the mobile object using a database that associates and pre-stores location information with an exemplary behavior model that shows exemplary behavior corresponding to the road environment;
an evaluation value calculation unit that calculates an evaluation value of the action based on the action acquired by the action acquisition unit and the exemplary behavior model acquired by the model acquisition unit;
A motion evaluation device comprising:

As a result, the present disclosure has an effect of providing an action evaluation device that performs a more favorable driving evaluation than conventional devices.

The action evaluation method of the present disclosure is configured, for example, as follows.

a motion acquisition step in which a motion acquisition unit of the motion evaluation device acquires a motion of a occupant driving the moving object;
a position acquisition step in which a position acquisition unit of the action evaluation device acquires a position of the moving object;
a model acquisition step in which a model acquisition unit of the behavior evaluation device acquires an exemplary behavior model corresponding to the road environment at the position of the moving object using a database in which position information and an exemplary behavior model showing exemplary behavior corresponding to the road environment are associated and stored in advance;
an evaluation value calculation step in which an evaluation value calculation unit of the action evaluation device calculates an evaluation value of the action based on the action acquired by the action acquisition unit and the exemplary behavior model;
A motion evaluation method comprising:

As a result, the present disclosure has the effect of providing an action evaluation method that can perform a more preferable driving evaluation than conventional methods.

The action evaluation device of the present disclosure is further configured, for example, as follows.

The movement acquired by the movement acquisition unit is a movement of the occupant determined using an image of an interior of the vehicle.
A behavior evaluation device characterized by:

As a result, the present disclosure has the advantage of being able to evaluate various occupant actions that are not directly reflected in driving operations, as compared to conventional methods.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

The action evaluation device of the present disclosure is further configured, for example, as follows.

The road environment includes at least one of a motorway, a merging area of a motorway, a vicinity of an intersection, a vicinity of a pedestrian crossing, the presence or absence of a traffic light, the shape of a road, the width of a road, the degree of vehicle congestion, or the degree of pedestrian congestion.
A behavior evaluation device characterized by:

This provides an advantage that the present disclosure makes it possible to evaluate the occupant's behavior at a point where desirable behavior is likely to occur before the actual driving operation.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

The action evaluation device of the present disclosure is further configured, for example, as follows.

The exemplary behavior model includes at least one of a viewing direction, a cumulative inattentiveness, or a position of a body part.
A behavior evaluation device characterized by:

This provides an advantage, for example, in that it is possible to evaluate whether the occupant's observation direction is facing in a direction that allows them to collect information necessary for driving, or to evaluate whether the occupant's body parts are in a position that allows them to prepare for driving.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

The action evaluation device of the present disclosure is further configured, for example, as follows.

The observation direction includes at least one of a face direction of the occupant or a line of sight direction of the occupant.
A behavior evaluation device characterized by:

As a result, the present disclosure provides an effect of further improving the accuracy of evaluating the observation direction.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

Embodiment 2.
In the second embodiment, a configuration will be described in which a component for performing action judgment is provided inside the action evaluation device.
In addition, the second embodiment will explain a configuration in which the action evaluation device includes a component that determines whether to issue a warning to the occupant.
In the second embodiment, among the components according to the second embodiment, those components that are the same as the components according to the first embodiment that have already been described will not be described again.

A configuration example of an action evaluation device and an action evaluation system including the device according to a second embodiment of the present disclosure will be described.
FIG. 4 is a diagram illustrating an example of the configuration of an action evaluation system 1B including an action evaluation apparatus 100B according to the second embodiment of the present disclosure.
The movement evaluation system 1B includes a movement evaluation device 100B, a model behavior database 200B (database), an output device 400B, and an imaging device 500B.
The model behavior database 200B (database) is the same as the model behavior database 200A already described, and the output device 400B is the same as the output device 400A already described, so that redundant description of each will be omitted.
The imaging device 500B and the action evaluation device 100B will be described in detail in that order.

Imaging device 500B captures an image of the interior of a moving object and outputs the captured image.
When the moving body is a vehicle, the imaging device 500B is an in-vehicle camera for capturing images of the interior of the vehicle.

The action evaluating device 100B includes the same components as the action evaluating devices 100 and 100A already described.
In addition, the action evaluation device 100B has a component for determining an action, and a component for determining whether to issue a warning to the occupant.
The action evaluation device 100B shown in Figure 4 is configured to include an image acquisition unit 121B, an action determination unit 122B, an action acquisition unit 111B, a position acquisition unit 112B, a model acquisition unit 113B, an evaluation value calculation unit 114B, and a warning unit 123B.
The position acquisition unit 112B is similar to the position acquisition units 112 and 112A already described, and the model acquisition unit 113B is similar to the model acquisition units 113 and 113A already described, so that the description of each will be omitted to avoid duplication.

The video acquisition unit 121B acquires a video of the interior of a moving object.
Specifically, the image acquisition unit 121B acquires an image from an imaging device 500B that captures an image of the interior of a moving object.

The movement determining unit 122B determines the movement of the occupant captured in the video acquired by the video acquiring unit 121B.
Specifically, the movement determining unit 122B acquires an image from the image acquiring unit 121B, and determines the movement of the occupant using the acquired image.
The technique for determining the movement of the occupant can be realized using known techniques.

The motion acquisition unit 111B has the same functions as the motion acquisition unit 111 already described.
The movement acquisition unit 111B acquires information relating to the movement of the occupant output by the movement determination unit 122B and uses the information for processing.

The evaluation value calculation unit 114B has a function of calculating an evaluation value similar to the evaluation value calculation unit 114 already described.
The evaluation value calculation unit 114B outputs the calculated evaluation value to the warning unit 123B.

The warning unit 123B issues a warning to the occupant using the evaluation value.
Specifically, the warning unit 123B compares the evaluation value with a pre-stored evaluation threshold and determines whether the evaluation value is equal to or greater than the evaluation threshold. For example, if the evaluation value is a deviation and the evaluation threshold is the deviation threshold, the warning unit 123B determines whether the deviation is equal to or greater than the deviation threshold.
The warning unit 123B instructs the output device 400B to issue a warning when the determination result indicates that the occupant's behavior deviates from the model behavior indicated in the model behavior model according to the road environment.
The warning is given by, for example, at least one of displaying an image, outputting a sound, or outputting a vibration.

A processing example of the action evaluation device according to the second embodiment of the present disclosure will be described.
FIG. 5 is a flowchart showing an example of processing by the action evaluating apparatus 100B according to the second embodiment of the present disclosure.
Action evaluation device 100B starts the processing shown in Fig. 5 upon receiving a command from a control unit (not shown) or an external command. Action evaluation device 100B starts the processing, for example, when operation of a mobile object is started.

First, the action evaluation apparatus 100B executes a video acquisition process (step ST2110).
In the video acquisition process, video acquisition section 121B of action evaluation device 100B acquires video from imaging device 500 that captures images of the interior of a moving object. Video acquisition section 121B outputs the acquired video to action determination section 122B.

The action evaluation device 100B then executes a action determination process (step ST2120).
In the movement determination process, the movement determination unit 122B of the movement evaluation device 100B determines the movement of the occupant captured in the video acquired by the video acquisition unit 121B. The movement determination unit 122B outputs information related to the movement of the occupant, which is the determination result, to the movement acquisition unit 111B.

The action evaluation apparatus 100B then executes an action acquisition process (step ST2130).
In the action acquisition process, the action acquisition unit 111B of the action evaluation device 100B acquires the action of the occupant driving the moving body.
The motion acquisition unit 111B acquires information relating to the motion output from the motion determination unit 122B and outputs the information to the position acquisition unit 112B.

The action evaluation device 100B then executes a position acquisition process (step ST2140).
In the position acquisition process, the position acquisition unit 112B of the action evaluation device 100B acquires the position of the moving object.
Upon acquiring the information relating to the motion output by the motion acquisition unit 111B, the position acquisition unit 112B acquires position information indicating the position of the moving object. The position acquisition unit 112B outputs motion information, which is a set of the position information and the information relating to the motion, to the model acquisition unit 113B.

The action evaluation apparatus 100B then executes a model acquisition process (step ST2150).
In the model acquisition process, the model acquisition unit 113B of the action evaluation device 100B acquires an exemplary behavior model that indicates an exemplary action according to the road environment at the position acquired by the position acquisition unit 112A.
When the model acquisition unit 113B acquires the behavior information output by the position acquisition unit 112B, it uses the position information included in the behavior information to refer to the model behavior database 200B and acquires a model behavior model that corresponds to the road environment at the location of the moving body.
The model acquisition unit 113B outputs the motion information and the exemplary behavior model to the evaluation value calculation unit 114B.

The action evaluation device 100B then executes an evaluation value calculation process (step ST2160).
In the evaluation value calculation process, the evaluation value calculation unit 114B of the action evaluation device 100B calculates an evaluation value of the action based on the action acquired by the action acquisition unit 111 and the exemplary behavior model acquired by the model acquisition unit 113B.
Specifically, for example, when the evaluation value calculation unit 114B acquires the movement information and the exemplary behavior model output by the model acquisition unit 113B, the evaluation value calculation unit 114B compares the information related to the movement included in the movement information with the exemplary behavior model to calculate a difference value. The evaluation value calculation unit 114B uses the difference value to calculate an evaluation value (deviation degree) indicating the degree of deviation of the actual movement from the exemplary behavior model.
The evaluation value calculation unit 114B outputs the evaluation value as the calculation result to the warning unit 123B.

The action evaluation device 100B then executes an evaluation value determination process ("evaluation value≧evaluation threshold", step ST2170).
In the evaluation value determination process, the warning unit 123B of the action evaluation device 100B compares the evaluation value with an evaluation threshold value.
The warning unit 123B determines whether the evaluation value is equal to or greater than the evaluation threshold value.
Specifically, when the evaluation value is a deviation degree and the evaluation threshold is a deviation threshold, the warning unit 123B determines whether the deviation degree is equal to or greater than the deviation threshold.

When it is determined that the evaluation value is equal to or greater than the evaluation threshold value ("YES" in step ST2170), the warning unit 123B executes a warning process (step ST2180).
In the warning process, the warning unit 123B instructs the output device 400 to issue a warning indicating that the evaluation is low.
When issuing a warning command, the warning unit 123B may issue a command to present information such as the evaluation value itself and the degree of lowness of the evaluation.
After executing the warning process, the action evaluation device 100B then proceeds to the end determination process (step ST2190).

If the warning unit 123B determines that the evaluation value is not greater than or equal to the evaluation threshold, i.e., that the evaluation value is less than the evaluation threshold (step ST2170 "NO"), the action evaluation device 100B then proceeds to the termination determination process (step ST2190).

The action evaluation apparatus 100B executes an end determination process (step ST2190).
In the termination determination process, a control unit (not shown) of the action evaluation device 100B determines whether to terminate the processing of the action evaluation device 100B. The control unit (not shown) determines whether to terminate the processing of the action evaluation device 100B, for example, in accordance with an external termination command or an execution program.
If the control section (not shown) determines not to end the processing of the action evaluation device 100B, the process proceeds to step ST2110, and the process is repeated from step ST2110.
When the control unit (not shown) determines that the processing of the action evaluation device 100B should be terminated, the action evaluation device 100B terminates the processing.

The action evaluation device of the present disclosure is further configured, for example, as follows.

an image acquisition unit that acquires an image from an imaging device that captures an image of the interior of the moving body;
a motion determination unit that determines a motion of the occupant captured in the video;
Equipped with
A behavior evaluation device characterized by:

As a result, the present disclosure has a configuration for determining the occupant's movements within the movement evaluation device, and therefore has the effect of being able to evaluate the occupant's movements even if the device for determining the occupant's movements is not installed on the moving body.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

The action evaluation device of the present disclosure is further configured, for example, as follows.

a warning unit that issues a warning to the occupant using the evaluation value;
Equipped with
A behavior evaluation device characterized by:

As a result, the present disclosure has an effect of being able to warn the occupant in accordance with the evaluation value.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

Embodiment 3.
In the third embodiment, an exemplary behavior model adapted to a road environment can be shared for each similar road environment.
In the third embodiment, among the components of the third embodiment, those components that are similar to those of the first or second embodiment already described will be omitted from the description as appropriate.

A configuration example of an action evaluation device according to a third embodiment of the present disclosure and an action evaluation system including the device will be described.
FIG. 6 is a diagram illustrating a configuration example of an action evaluation system 1C including an action evaluation apparatus 100C according to the third embodiment of the present disclosure.
The movement evaluation system 1C includes a movement evaluation device 100C, a model behavior database 200C (database), a road environment database 210C (database), an output device 400C, and an imaging device 500C.
The output device 400C is similar to the output devices 400A and 400B already described, and the image capture device 500C is similar to the image capture device 500B already described, so that redundant description of each will be omitted.
The road environment database 210C (database), the model behavior database 200C (database), and the action evaluation device 100C will be described in detail in that order.

The road environment database 210C (database) is a database that pre-stores the location information and the road environment information in association with each other.

The model behavior database 200C (database) is a database that pre-stores the road environment information and the model behavior model in association with each other.

The road environment database 210C and the model behavior database 200C constitute the databases of the present disclosure.
The road environment database 210C and the model behavior database 200C may be in the form of an integrated database.

The action evaluation device 100C shown in Figure 6 is configured to include an image acquisition unit 121C, an action judgment unit 122C, an action acquisition unit 111C, a position acquisition unit 112C, a road environment acquisition unit 131C, a model acquisition unit 113C, an evaluation value calculation unit 114C, and a warning unit 123C.

The image acquisition unit 121C is configured in the same manner as the image acquisition unit 121B already described, the motion determination unit 122C is configured in the same manner as the motion determination unit 122B already described, the motion acquisition unit 111C is configured in the same manner as the motion acquisition units 111A and 111B already described, the position acquisition unit 112C is configured in the same manner as the position acquisition units 112A and 112B already described, the evaluation value calculation unit 114C is configured in the same manner as the A and 114B already described, and the warning unit 123C is configured in the same manner as the warning unit 123B already described, and therefore descriptions of each unit will be omitted to avoid duplication.
The road environment acquisition unit 131C and the model acquisition unit 113C will be described.

The road environment acquisition unit 131C acquires road environment information according to the position of the mobile object.
The road environment acquisition unit 131C refers to a database (road environment database 210C) that pre-stores location information and road environment information in association with each other, and acquires road environment information corresponding to the location acquired by the location acquisition unit 112C.

The model acquisition unit 113C acquires an exemplary behavior model according to road environment information.
The model acquisition unit 113C refers to a database (model behavior database 200C) that pre-stores road environment information and model behavior models in association with each other, and acquires a model behavior model that corresponds to the road environment information acquired by the road environment acquisition unit 131C.

A processing example of the action evaluation device according to the third embodiment of the present disclosure will be described.
FIG. 7 is a flowchart showing an example of processing by the action evaluating apparatus 100C according to the third embodiment of the present disclosure.
Upon receiving a command from a control unit (not shown) or an external command, action evaluation device 100C starts the processing shown in Fig. 5. Action evaluation device 100C starts the processing, for example, when operation of a mobile object is started.

The action evaluation device 100C first performs video acquisition processing (step ST3110) in the same manner as the video acquisition processing (step ST2110) already described.

The action evaluation device 100C then performs an action judgment process (step ST3120) similar to the action judgment process (step ST2120) already described.

The action evaluation device 100C then performs an action acquisition process (step ST3130) similar to the action acquisition process (step ST2130) already described.

The action evaluation device 100C then performs a position acquisition process (step ST3140) similar to the position acquisition process (step ST2140) already described.

Next, the action evaluation device 100C executes a road environment acquisition process (step ST3150).
In the road environment acquisition process, the road environment acquisition unit 131C of the action evaluation device 100C acquires road environment information according to the position of the mobile object.
Specifically, the road environment acquisition unit 131C refers to a database (road environment database 210C) that pre-stores location information and road environment information in association with each other, and acquires road environment information corresponding to the location acquired by the location acquisition unit 112C.

The action evaluation apparatus 100C then executes a model acquisition process (step ST3160).
In the model acquisition process, the model acquisition unit 113C of the action evaluation device 100C acquires an exemplary behavior model according to the road environment information.
Specifically, the model acquisition unit 113C refers to a database (model behavior database 200C) that pre-stores road environment information in association with model behavior models, and acquires a model behavior model corresponding to the road environment information acquired by the road environment acquisition unit 131C.

The action evaluation device 100C then performs an evaluation value calculation process (step ST3170) similar to the evaluation value calculation process (step ST2160) already described.

The action evaluation device 100C then performs an evaluation value determination process ("evaluation value ≧ evaluation threshold", step ST3180) in the same manner as the evaluation value determination process already described ("evaluation value ≧ evaluation threshold", step ST2170).

If the warning unit 123C of the action evaluation device 100C determines that the evaluation value is greater than or equal to the evaluation threshold (step ST3180 "YES"), it executes a warning process (step ST3190) similar to the warning process (step ST2180) already described.
After executing the warning process, the action evaluation device 100C then proceeds to the termination determination process (step ST3200).

If the warning unit 123C determines that the evaluation value is not greater than or equal to the evaluation threshold, i.e., that the evaluation value is less than the evaluation threshold (step ST3180 "NO"), the action evaluation device 100C then proceeds to the termination determination process (step ST3200).

The action evaluation device 100C executes the termination determination process (step ST3200) in the same manner as the termination determination process (step ST2190) already described.

The above-described configuration allows for the establishment of a shareable model of behavior for each road environment, thereby enabling the creation of a more efficient system for behavior evaluation.

The action evaluation device of the present disclosure is further configured, for example, as follows.

a road environment acquisition unit that refers to a database that stores the location information and the road environment information in advance in association with each other, and acquires road environment information corresponding to the location acquired by the location acquisition unit;
Furthermore,
The model acquisition unit
referencing a database in which road environment information and model behavior models are stored in advance in association with each other, and acquiring a model behavior model corresponding to the road environment information acquired by the road environment acquisition unit;
A behavior evaluation device characterized by:

As a result, the present disclosure has the effect of making it possible to set a common model of exemplary behavior for each road environment, thereby enabling the construction of a more efficient system for behavior evaluation.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

Embodiment 4.
In the fourth embodiment, a model behavior is described in which, among the occupant's behaviors such as cumulative inattentiveness, which is determined by accumulating intermittent inattentiveness by the occupant, observing a direction that is acceptable depending on the road environment is used as a model behavior.
In embodiment 4, among the components of embodiment 4, duplicate explanations will be omitted as appropriate for components that are similar to the components of embodiment 1, embodiment 2, or embodiment 3 already described.

A configuration example of an action evaluation device and an action evaluation system including the device according to a fourth embodiment of the present disclosure will be described.
FIG. 8 is a diagram showing a configuration example of an action evaluation system 1D including an action evaluation device 100D according to the fourth embodiment of the present disclosure.
FIG. 9 is a first diagram for explaining an example of accumulated inattentive behavior handled by the action evaluation device 100D according to the fourth embodiment.
FIG. 10 is a second diagram for explaining an example of accumulated inattentive behavior handled by the action evaluation device 100D according to the fourth embodiment.
The movement evaluation system 1D includes a movement evaluation device 100D, a model behavior database 200D (database), an output device 400D, and an imaging device 500D.
The model behavior database 200D (database) is similar to the model behavior databases 200A and 200B already described, the output device 400D is similar to the output devices 400A, 400B, and 400C already described, and the imaging device 500D is similar to the imaging devices 500B and 500C already described, and therefore descriptions of each will be omitted to avoid duplication.
The action evaluation device 100D will now be described in detail.

The action evaluation device 100D is configured to include a cumulative inattentive behavior judgment unit 141D, a cumulative inattentive behavior point acquisition unit 142D, an observation direction acquisition unit 143D, a cumulative inattentive behavior frequent point acquisition unit 144D, an allowable observation direction acquisition unit 145D, a model acquisition unit 113D, an evaluation value calculation unit 114D, and a warning unit 123D.

The evaluation value calculation unit 114D is configured in the same way as the already described A, 114B, and 114C, and the warning unit 123D is configured in the same way as the already described warning units 123B and 123C, so descriptions of each will be omitted to avoid duplication.

The cumulative inattentive driving determination unit 141D determines whether the occupant's behavior is cumulative inattentive driving.
The cumulative inattentive driving judgment unit 141D acquires a cumulative value relating to intermittent inattentive driving by the occupant, and performs a cumulative inattentive driving judgment, which is a judgment of whether or not there is a cumulative inattentive driving state, using the cumulative value and a predetermined threshold value (accumulation value threshold value).
The cumulative inattentive driving determination unit 141D, for example, analyzes video acquired from the imaging device 500D to acquire a cumulative value related to intermittent inattentive driving by the occupant, and each time the cumulative value is acquired, compares the cumulative value with a threshold value to determine whether the cumulative value is equal to or greater than the threshold value. If the cumulative value is equal to or greater than the threshold value, the cumulative inattentive driving determination unit 141D determines that the occupant is in an accumulated inattentive driving state. Alternatively, the cumulative inattentive driving determination unit 141D may be configured to determine that the occupant is in an accumulated inattentive driving state using, for example, a known technology for determining cumulative inattentive driving in addition to the above-described determination method.
The cumulative inattentive driving determination unit 141D outputs a result of the cumulative inattentive driving determination, for example, a result indicating that the cumulative value has reached a threshold value or more.
The cumulative inattentive driving determination unit 141D may be configured to acquire a cumulative value relating to intermittent inattentive driving by an occupant from an occupant monitoring device.

Here, we will explain the cumulative inattentive driving. Figure 9 shows a case where the mobile body 1000 is a vehicle and the cumulative inattentive driving frequency point is a point included in a merging area (merging road 2000, merged road 3000) on a highway.
When a moving object 1000 is traveling from a merging road 2000 to a merged road 3000 in a traveling direction 1100 (or a predicted traveling direction), if the observation direction 1200 of the occupant of the moving object 1000 is within the allowable observation range α (assuming the allowable observation range α is set in advance), it is normally determined that the occupant is not looking inattentively. However, in a situation such as that shown in Figure 9, it is expected that the occupant will alternate between an observation direction 1200 that checks the road ahead of the merging road 3000 and an observation direction 1300 that checks the road behind the merging road 3000. In this case, the observation directions in the observation range β between the observation direction 1200 and the observation direction 1300 are accumulated, and if the accumulated value is mainly used to determine the accumulated inattentiveness, excessive accumulated inattentiveness will be detected. In the present disclosure, for example, as shown in Figure 10, performance evaluation can be performed using a range of acceptable observation directions, using an acceptable observation direction 4010 at a cumulative inattentiveness frequent point, which is a point where cumulative inattentive behavior by occupants of each of the multiple moving bodies occurs frequently, in comparison with a normal acceptable observation direction 4000 (corresponding to an acceptable observation range α).

The cumulative inattentive point acquisition unit 142D acquires cumulative inattentive points, which are points at which it has been determined that the occupant has engaged in the cumulative inattentive behavior.
The cumulative inattentive state acquisition unit 142D acquires cumulative inattentive state points based on the position of the moving body when the cumulative inattentive state determination unit 141D determines that the moving body is in an accumulated inattentive state.
The cumulative inattentive point is expressed as information such as the position at the moment when it is determined that the driver is in an inattentive state, or a representative value or average value of the positions during the time when the inattentive state values (cumulative values) are accumulated.
The cumulative inattentive point acquisition unit 142D may acquire information (location information) relating to the locations of the cumulative inattentive points from, for example, a mobile-mounted device mounted on the mobile body. Alternatively, the cumulative inattentive point acquisition unit 142D may be configured to calculate information (location information) relating to the locations of the cumulative inattentive points using information acquired from, for example, a mobile-mounted device mounted on the mobile body.

The observation direction acquisition unit 143D acquires the observation direction due to the accumulated inattentive driving.
The observation direction acquisition unit 143D acquires the observation direction, which is the direction from which the occupant of the moving body is likely to be observing.
The observation direction, like the "observation direction" already described, is represented by the facial direction of the occupant, the line of sight of the occupant, or information calculated from the facial direction of the occupant and the line of sight of the occupant. The observation direction may be represented by information indicating an area consisting of a plurality of observation directions. The observation direction may be represented by a representative value or an average value of the area consisting of a plurality of observation directions.
The observation direction acquisition unit 143D acquires the observation direction by analyzing, for example, the image acquired from the image capture device 500D. Alternatively, the observation direction acquisition unit 143D may be configured to acquire the occupant's observation direction from an occupant monitoring device mounted on the vehicle. Alternatively, the observation direction acquisition unit 143D may be configured to acquire the observation direction by calculating it using information acquired from the occupant monitoring device mounted on the vehicle.

The cumulative inattentive driving frequent spot acquisition unit 144D acquires cumulative inattentive driving frequent spots, which are spots where cumulative inattentive driving by occupants of each of a plurality of mobile bodies frequently occurs.
The cumulative inattentive driving points are location information that indicates points where cumulative inattentive driving is occurring frequently, and is generated based on information collected from multiple mobile bodies.
The cumulative inattentive driving frequent points include those represented by information indicating an area consisting of a plurality of cumulative inattentive driving frequent points.
The cumulative inattentive driving points may be expressed as information such as a representative value or average value of an area consisting of a plurality of cumulative inattentive driving points.
The cumulative inattentive driving spot acquisition unit 144D acquires information related to cumulative inattentive driving spot locations output from a server device (a server device not shown, or a server device 600E related to embodiment 5 described below) at a pre-stored fixed time or at predetermined intervals, for example.
Alternatively, for example, when the cumulative inattentive state is determined to be present by the cumulative inattentive state determination unit 141D, the cumulative inattentive state acquisition unit 144D uses the cumulative inattentive state points acquired by the cumulative inattentive state acquisition unit 142D to acquire information relating to the cumulative inattentive state points included in the area based on the cumulative inattentive state points from the server device.
The information relating to the cumulative inattentive driving high incidence points includes location information indicating the locations of the cumulative inattentive driving high incidence points.
The cumulative inattentive driving frequent point acquisition unit 144D extracts and outputs location information indicating the locations of the cumulative inattentive driving frequent points included in the information relating to the cumulative inattentive driving frequent points.
The information relating to the cumulative high incidence points of inattentive driving also includes permissible observation direction information indicating the permissible observation direction at the cumulative high incidence points.

The allowable observation direction acquisition unit 145D acquires the allowable observation direction, which is the direction in which observation should be allowed for each cumulative inattentive driving frequency point.
The allowable observation direction is information generated based on information collected from multiple moving bodies, and is represented as information indicating the direction that observation should be allowed at each point where cumulative inattentive driving occurs frequently.
For example, if the moving body is a vehicle and the cumulative frequent inattentive driving point is a point included in a merging area on a motorway, the allowable observation direction is a direction based on the position of the occupant, heading toward the road where the vehicle will merge (the merging road).
Also, for example, if the moving body is a vehicle and the cumulative frequent inattentive driving point is a point included in the merging area from a branch road to a main road, the allowable observation direction is a direction based on the position of the occupant, heading toward the road where the vehicle is expected to merge (the merging road).
Also, for example, if the moving body is a vehicle and the cumulative frequent inattentive driving point is a point included in the surrounding area of an intersection, the allowable observation direction is a direction based on the position of the occupant, and is the direction toward the road connected to the intersection (the merging road).
The allowable viewing direction may be expressed by information indicating a region consisting of a plurality of allowable viewing directions, or may be expressed by a representative value or an average value of the region consisting of a plurality of allowable viewing directions.
The allowable observation direction acquisition unit 145D acquires allowable observation direction information included in the information related to the accumulated inattentive driving frequent points acquired by the accumulated inattentive driving frequent points acquisition unit 144D.

The model acquisition unit 113D acquires an exemplary behavior model using the cumulative inattentive driving points, cumulative inattentive driving frequent points, observation direction, and allowable observation direction.
When the cumulative inattentive driving point and the cumulative inattentive driving frequent point overlap, and when the observation direction acquired by the observation direction acquisition unit 143D overlaps with the permissible observation direction acquired by the permissible observation direction acquisition unit 145D, the model acquisition unit 113D acquires as an exemplary behavior model the observation of the permissible observation direction at the position of the cumulative inattentive driving point.
Specifically, model acquisition unit 113D determines whether the cumulative inattentive driving points acquired by cumulative inattentive driving point acquisition unit 142D and the cumulative inattentive driving frequent points acquired by cumulative inattentive driving frequent point acquisition unit 144D overlap. This determination includes determining whether the cumulative inattentive driving points and the cumulative inattentive driving frequent points match, or determining whether an area based on the cumulative inattentive driving points and an area based on the cumulative inattentive driving frequent points overlap at least partially.
The model acquisition unit 113D also determines whether the observation direction acquired by the observation direction acquisition unit 143D overlaps with the allowable observation direction acquired by the allowable observation direction acquisition unit 145D. This determination includes determining whether the observation direction and the allowable observation direction match, or whether a directional region based on the observation direction overlaps at least partially with a directional region based on the cumulative inattentive driving frequency points.
When the cumulative inattentiveness points acquired by the cumulative inattentiveness point acquisition unit 142D and the cumulative inattentiveness frequent points acquired by the cumulative inattentiveness frequent point acquisition unit 144D overlap, and when the observation direction acquired by the observation direction acquisition unit 143D and the permissible observation direction acquired by the permissible observation direction acquisition unit 145D overlap, the model acquisition unit 113D acquires as an exemplary behavior model the observation of the permissible observation direction at the location of the cumulative inattentiveness point. This is because the permissible observation direction at the cumulative inattentiveness frequent point can be said to be the direction in which an occupant, such as a driver, should observe at the location of the moving body. Specifically, for example, when the cumulative inattentiveness frequent points include a point included in a merging area on a motorway, the permissible observation direction is a direction based on the position of the occupant, heading toward the road where the moving body will merge. By using such an allowable observation direction as a model behavior, the behavior evaluation device of the present disclosure can evaluate the behavior of a occupant driving a moving body by modeling behavior such as cumulative inattentive driving, with the observation direction being the direction toward the merging road in a merging area on a motorway.

A processing example of the action evaluation device according to the fourth embodiment of the present disclosure will be described.
FIG. 11 is a flowchart showing an example of processing by the action evaluating device 100D according to the fourth embodiment of the present disclosure.

The process shown in FIG. 11 is a motion evaluation method using the motion evaluation device 100D.
For example, when the action evaluation device 100D acquires an image from the imaging device 500D, the action evaluation device 100D starts the processing shown in FIG.
11 when it acquires the cumulative value related to the cumulative inattentive behavior. In this case, the action evaluation device 100D may acquire the cumulative value related to the cumulative inattentive behavior from, for example, an occupant monitoring device (for example, the occupant monitoring device 300A already described).

The action evaluation device 100D first executes the cumulative inattentive behavior determination process (step ST4110).
In the cumulative inattentive driving judgment process, the cumulative inattentive driving judgment unit 141D of the action evaluation device 100D acquires a cumulative value related to intermittent inattentive driving by the occupant of the moving body, and performs a cumulative inattentive driving judgment, which is a judgment of whether or not there is a cumulative inattentive driving state, using the cumulative value and a predetermined threshold value.
The cumulative inattentive behavior determining unit 141D may, for example, first analyze the video and accumulate values related to the cumulative inattentive behavior to obtain a cumulative value, or may obtain a cumulative value output from an occupant monitoring device.
The cumulative inattentive driving determination unit 141D then compares the cumulative value relating to intermittent inattentive driving with a pre-stored threshold value.
The cumulative inattentive driving determination unit 141D then determines that the driver is in an accumulated inattentive driving state if the cumulative value relating to intermittent inattentive driving exceeds a pre-stored threshold value.
The cumulative inattentive driving determination unit 141D then notifies the cumulative inattentive driving point acquisition unit 142D, the observation direction acquisition unit 143D, the cumulative inattentive driving frequent point acquisition unit 144D, and the allowable observation direction acquisition unit 145D that it has been determined that the driver is in a cumulative inattentive driving state.

If the action evaluation device 100D determines that the accumulated inattentive state is not present (step ST4111 "NO"), it returns to processing of step ST4110.

When the action evaluation device 100D determines that the driver has accumulated inattentive driving ("YES" in step ST4111), it then executes an accumulated inattentive driving point acquisition process (step ST4120).
In the cumulative inattentive point acquisition process, the cumulative inattentive point acquisition unit 142D of the action evaluation device 100D acquires cumulative inattentive points based on the position of the moving body when the cumulative inattentive state is determined to be present by the cumulative inattentive state determination unit 141D.
When the cumulative inattentive state acquisition unit 142D receives a notification from the cumulative inattentive state determination unit 141D indicating that a cumulative inattentive state has been determined, the cumulative inattentive state acquisition unit 142D acquires cumulative inattentive state information indicating the cumulative inattentive state. The cumulative inattentive state information is represented by positions or areas where cumulative inattentive states have occurred, based on the position of the mobile body acquired via a communication unit (not shown). The cumulative inattentive state acquisition unit 142D outputs the cumulative inattentive state information to the model acquisition unit 113D.

Next, the action evaluation device 100D executes an observation direction acquisition process (step ST4130).
In the observation direction acquisition process, the observation direction acquisition unit 143D of the action evaluation device 100D acquires the observation direction that is the direction from which the occupant of the moving object is likely to be observing.
When the observation direction acquisition unit 143D receives a notification from the cumulative inattentive state determination unit 141D indicating that the cumulative inattentive state has been determined, the observation direction acquisition unit 143D acquires observation direction information indicating the observation direction. The observation direction information indicates, for example, the facial orientation of the occupant, the line of sight of the occupant, or the direction from which the occupant is likely to be observing based on the facial orientation and line of sight of the occupant acquired using video of the interior of the vehicle. The observation direction acquisition unit 143D outputs the observation direction information to the model acquisition unit 113D.

Next, the action evaluation device 100D executes a process of acquiring accumulated inattentive driving frequent points (step ST4140).
In the cumulative inattentive spot acquisition process, the cumulative inattentive spot acquisition unit 144D of the action evaluation device 100D acquires cumulative inattentive spot locations, which are locations where cumulative inattentive behavior by the occupants of each of the multiple mobile bodies frequently occurs.
When the cumulative inattentiveness frequent point acquisition unit 144D receives a notification from the cumulative inattentiveness determination unit 141D indicating that a cumulative inattentiveness state has been determined, it acquires cumulative inattentiveness frequent point information indicating cumulative inattentiveness frequent points. The cumulative inattentiveness frequent point information indicates the locations or areas of cumulative inattentiveness frequent points, which are points where cumulative inattentiveness by the occupants of each of the multiple mobile bodies frequently occurs. For example, the cumulative inattentiveness frequent point acquisition unit 144D uses the cumulative inattentiveness points acquired by the cumulative inattentiveness point acquisition unit 142D to acquire information related to cumulative inattentiveness frequent points included in an area based on the cumulative inattentiveness points from, for example, a server device. The cumulative inattentiveness frequent point acquisition unit 144D outputs the cumulative inattentiveness frequent point information included in the acquired information related to cumulative inattentiveness frequent points to the model acquisition unit 113D.

Next, action evaluation device 100D executes an allowable observation direction acquisition process (step ST4150).
In the allowable observation direction acquisition process, the allowable observation direction acquisition unit 145D of the action evaluation device 100D acquires the allowable observation direction, which is the direction in which observation should be allowed for each of the cumulative inattentive looking frequent points.
When the allowable observation direction acquisition unit 145D receives a notification from the cumulative inattentiveness determination unit 141D indicating that a cumulative inattentiveness state has been determined, it acquires allowable observation direction information indicating the allowable observation direction included in the information related to the cumulative inattentiveness frequent points acquired from the server device by the cumulative inattentiveness frequent point acquisition unit 144D. The allowable observation direction information indicates a direction or a directional area consisting of multiple directions that is allowable for observation at the cumulative inattentiveness frequent points, which are points where cumulative inattentiveness by occupants of each of multiple mobile bodies frequently occurs. The allowable observation direction acquisition unit 145D outputs the acquired allowable observation direction information to the model acquisition unit 113D.

Next, the action evaluation device 100D executes an allowable cumulative inattentive driving determination process (step ST4160).
In the allowable cumulative inattentive behavior determination process, the model acquisition unit 113D of the action evaluation device 100D determines whether the observation direction at the position of the accumulated inattentive behavior point is a direction in which observation should be allowed (allowable observation direction).
When the cumulative inattentive driving points and the cumulative inattentive driving frequent points overlap, and when the observation direction acquired by the observation direction acquisition unit 143D overlaps with the allowable observation direction acquired by the allowable observation direction acquisition unit 145D, the model acquisition unit 113D acquires as an exemplary behavior model the observation of the allowable observation direction at the position of the cumulative inattentive driving point. Note that the state in which the cumulative inattentive driving points and the cumulative inattentive driving frequent points overlap, and the state in which the observation direction acquired by the observation direction acquisition unit 143D overlaps with the allowable observation direction acquired by the allowable observation direction acquisition unit 145D overlap, are not limited to states in which they strictly match, but also include, for example, states in which the difference value is equal to or less than a predetermined threshold.

If the model acquisition unit 113D determines that the observation direction and the allowable observation direction do not overlap (step ST4160 "NO"), the action evaluation device 100D then proceeds to the evaluation value calculation process (step ST4180).

If the model acquisition unit 113D determines that the observation direction overlaps with the permissible observation direction ("YES" in step ST4160), the action evaluation device 100D then executes a permissible accumulated inattentive behavior information output process (step ST4170).
In the allowable cumulative inattentive driving information output process, the model acquisition unit 113D of the action evaluation device 100D acquires allowable cumulative inattentive driving information including location information indicating the accumulated inattentive driving points and allowable observation directions indicating the allowable observation directions as an exemplary behavior model.
The position information indicating the cumulative inattentive points and the permissible observation direction indicating the permissible observation direction correspond to an exemplary behavior model that indicates the behavior of observing the permissible observation direction at the position of the cumulative inattentive points.

Here, a detailed example of the processing of the model acquisition unit 113D will be described.
FIG. 12 is a flowchart showing an example of processing by the model acquiring unit 113D in the action evaluating apparatus 100D according to the fourth embodiment of the present disclosure.
The model acquisition unit 113D first compares the accumulated inattentive driving points indicated by the accumulated inattentive driving point information with the accumulated inattentive driving frequent points indicated by the accumulated inattentive driving frequent point information (step ST4161).
The model acquisition unit 113D then determines whether the cumulative inattentive driving points and the cumulative inattentive driving frequent points overlap (step ST4162).
If the cumulative inattentive driving point and the cumulative inattentive driving frequent point overlap ("YES" in step ST4162), the model acquisition unit 113D then compares the observation direction with the allowable observation direction (step ST4163).
The model acquisition unit 113D then determines whether the observation direction overlaps with the permissible observation direction (step ST4164).

If the observation direction overlaps with the permissible observation direction ("YES" in step ST4164), the model acquisition unit 113D then executes permissible accumulated inattentive driving information output processing (step ST4170).
In the permissible accumulated inattentive information output process, the model acquisition unit 113D acquires permissible accumulated inattentive information including position information indicating the cumulative inattentive points and permissible observation directions indicating the permissible observation directions as an exemplary behavior model.

If the cumulative inattentive driving points and cumulative inattentive driving frequent points do not overlap (step ST4162 "NO"), or if the observation direction and the allowable observation direction do not overlap (step ST4164 "NO"), the model acquisition unit 113D executes a database reference process (step ST4165).
In the database reference process, the model acquisition unit 113D references the model behavior database 200, searches for a model behavior model at the cumulative inattentiveness points, and outputs the model behavior model according to the search results. If a model behavior model is found, the model acquisition unit 113D acquires the model behavior model and outputs the model behavior model to the evaluation value calculation unit 114D, but if no model behavior model is found, the process ends.
When the model acquiring section 113D finishes the process, the action evaluating apparatus 100D then proceeds to the evaluation value calculation process (step ST4180).

In addition, the processes in the allowable cumulative inattentive driving judgment process (step ST4160), particularly including the database reference process (step ST4165) and the allowable cumulative inattentive driving information output process (step ST4170), correspond to the model acquisition process of the present disclosure.

Returning to the description of FIG.
The action evaluating device 100D then executes an evaluation value calculation process (step ST4180) similar to the evaluation value calculation process (for example, step ST2160) already described.
The evaluation value calculation unit 114D outputs the evaluation value as the calculation result to the warning unit 123D.

The action evaluation device 100D then performs an evaluation value determination process ("evaluation value ≧ evaluation threshold", step ST4190) similar to the evaluation value determination process already described (e.g., step ST2170).

When it is determined that the evaluation value is equal to or greater than the evaluation threshold value ("YES" in step ST4190), the warning unit 123D executes a warning process (step ST4200).
In the warning process, the warning unit 123D instructs the output device 400D to issue a warning indicating that the evaluation is low.
When issuing a warning command, the warning unit 123D may issue a command to present information such as the evaluation value itself and the degree of lowness of the evaluation.
After the warning unit 123D has executed the process, the action evaluation apparatus 100D then proceeds to an end determination process (step ST4210).

If the warning unit 123D determines that the evaluation value is not greater than or equal to the evaluation threshold, i.e., that the evaluation value is less than the evaluation threshold (step ST4190 "NO"), the action evaluation device 100D then proceeds to the termination determination process (step ST4210).

The action evaluation device 100D then executes an end determination process (step ST4210).
In the termination determination process, a control unit (not shown) of the action evaluation device 100D determines whether to terminate the processing of the action evaluation device 100D. The control unit (not shown) determines whether to terminate the processing of the action evaluation device 100D, for example, in accordance with an external termination command or an execution program.
If the control unit (not shown) determines not to end the processing of the action evaluation device 100D, the process proceeds to step ST4110, and the process is repeated from step ST4110.
When the control unit (not shown) determines that the processing of the action evaluation device 100D should be ended, the action evaluation device 100D ends the processing.

The action evaluation device of the present disclosure is further configured, for example, as follows.

an accumulated inattentive state determination unit that acquires an accumulated value related to intermittent inattentive state by the occupant and performs an accumulated inattentive state determination by using the accumulated value and a predetermined threshold value;
an accumulated inattentive point acquisition unit that acquires accumulated inattentive points based on the position of the moving body when the accumulated inattentive state is determined to be present by the accumulated inattentive state determination unit;
an observation direction acquisition unit that acquires an observation direction that is a direction in which the occupant of the moving body is likely to be observing;
an accumulative inattentiveness frequent occurrence point acquisition unit that acquires accumulative inattentiveness frequent occurrence points, which are points where accumulative inattentiveness by occupants of each of a plurality of moving bodies frequently occurs;
an allowable observation direction acquisition unit that acquires an allowable observation direction, which is a direction in which observation should be allowed for each of the cumulative inattentive looking points;
Furthermore,
The model acquisition unit
When the cumulative inattentive driving point and the cumulative inattentive driving frequent point overlap,
and,
When the observation direction acquired by the observation direction acquisition unit overlaps with the allowable observation direction,
Observing the permissible observation direction at the position of the cumulative inattentive viewing point is acquired as the exemplary behavior model.
A behavior evaluation device characterized by:

As a result, the present disclosure has the effect of enabling more accurate behavior evaluation of occupant behavior such as cumulative inattentive behavior.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

The action evaluation device of the present disclosure is further configured, for example, as follows.

When the cumulative inattentiveness frequent point includes a point included in a merging area on a motorway,
the allowable observation direction is a direction based on the position of the occupant and is a direction toward a merging road where the moving object will merge;
A behavior evaluation device characterized by:

As a result, the present disclosure has the effect of allowing occupant behavior such as cumulative inattentiveness in a direction where it is preferable to observe at points included in merging areas on expressways, with regard to occupant behavior such as cumulative inattentiveness.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

Embodiment 5.
In the fifth embodiment, a form in which the action evaluation device in each moving object and the server device cooperate to generate a database or update the database will be described.
In embodiment 5, among the components of embodiment 5, those components that are similar to the components of embodiment 1, embodiment 2, embodiment 3, or embodiment 4 already described will be omitted from the redundant description as appropriate.

A description will be given of an action evaluation device according to a fifth embodiment of the present disclosure.
FIG. 13 is a diagram showing the relationship between an action evaluating apparatus 100E according to the fifth embodiment of the present disclosure and a server apparatus 600E that generates a database used in the action evaluating apparatus 100E.
The action evaluation device 100E (100E-1, 100E-2, 100E-3, ..., 100E-n (n is an integer of 2 or more)) is mounted on a mobile body 700E (700E-1, 700E-2, 700E-3, ..., 700E-n (n is an integer of 2 or more)) and is connected to the server device 600E via a network.
The server device 600E acquires operation information from a plurality of moving objects 700E (700E-1, 700E-2, 700E-3, . . . , 700E-n) via the network.
The motion evaluation devices 100E (100E-1, 100E-2, 100E-3, ..., 100E-n) mounted on the respective moving bodies 700E (700E-1, 700E-2, 700E-3, ..., 700E-n) provide motion information to the server device 600E.

A configuration example of an action evaluation device and an action evaluation system including the device according to a fifth embodiment of the present disclosure will be described.
FIG. 14 is a diagram illustrating a first configuration example of an action evaluation system 1E including an action evaluation apparatus 100E according to the fifth embodiment of the present disclosure.
FIG. 15 is a diagram illustrating a second configuration example of an action evaluation system 1E′ including an action evaluation apparatus 100E′ according to the fifth embodiment of the present disclosure.
FIG. 16 is a diagram illustrating a third configuration example of an action evaluation system 1E″ including an action evaluation apparatus 100E″ according to the fifth embodiment of the present disclosure.
The action evaluation system 1E (1E', 1E'') includes an action evaluation device 100E (100E', 100E''), an output device 400E, an imaging device 500E, and a server device 600E.
The output device 400E is similar to the output devices 400A, 400B, 400C, and 400D already described, and the imaging device 500E is similar to the imaging devices 500B, 500C, and 500D already described, so that redundant descriptions of each will be omitted.
The action evaluation devices (100E, 100E', 100E'') and the server device 600E will be described in detail in that order.

The action evaluating apparatus according to the fifth embodiment is configured to include the same components as any of the action evaluating apparatuses 100A, 100B, 100C, and 100D already described in the respective embodiments.
For example, an action evaluating apparatus 100E shown in FIG. 14 includes the same components as those of the action evaluating apparatus 100B already described in the second embodiment.
For example, an action evaluating apparatus 100E' shown in FIG. 15 includes the same components as those of the action evaluating apparatus 100C already described in the third embodiment.
For example, an action evaluating device 100E'' shown in FIG. 16 includes the same components as those of the action evaluating device 100D already described in the fourth embodiment.
In addition to this configuration, the action evaluating apparatus according to the fifth embodiment further includes a action information providing unit 151E.
For example, each of the action evaluation devices 100E, 100E', and 100E'' shown in the respective drawings further includes a action information providing unit 151E.
Of the internal configurations of the action evaluation devices 100E, 100E', and 100E'' according to embodiment 5, the components that are similar to those of the action evaluation devices 100A, 100B, 100C, and 100D already described in the respective embodiments have already been described in the respective embodiments, and therefore will not be described here.
The operation information providing unit 151E will be described.

The motion information providing unit 151E provides the motion information to the server device 600E that generates the database of the present disclosure.
Specifically, the action information providing unit 151E in the action evaluation device 100E, 100E' outputs action information, which is information combining information related to the action acquired by the action acquisition unit 111E and location information acquired by the location acquisition unit 112E, to the server device 600E.
Specifically, the action information providing unit 151E in the action evaluation device 100E'' uses the accumulated inattentive point acquisition unit 142E as location information and the observation direction acquired in chronological order by the observation direction acquisition unit as information related to the action, and outputs action information, which is information combining location information and information related to the action, to the server device 600E.

Server device 600E generates the database of the present disclosure.
Server device 600E determines an exemplary behavior model according to the road environment for each position based on the operation information, and generates a database that associates the positions, road environments, and exemplary behavior models.
Furthermore, if a database has already been generated, server device 600E updates the generated database using the determined exemplary behavior model.
The database generated in server device 600E is the database already described in each embodiment, specifically, at least one of model behavior databases 200A, 200B, 200C, 200D, 200E (databases), or road environment database 210C (database).
The server device 600E provides the generated or updated database to the action evaluation devices 100E, 100E', and 100E'' of the respective moving bodies 700E (700E-1, 700E-2, 700E-3, . . . , 700E-n).
Alternatively, the server device 600E may be configured to use the generated or updated database to cooperate with the action evaluation devices 100E, 100E', 100E" of each moving body 700E (700E-1, 700E-2, 700E-3, ..., 700E-n) and provide information in response to a request from the action evaluation devices 100E, 100E', 100E". The information provided by the server device 600E to the action evaluation devices 100E, 100E', 100E" may be a model behavior model or road environment information.

An example of the configuration of a server device according to the fifth embodiment will be described.
The server devices 600E, 600E', and 600E'' shown in FIGS. 14, 15, and 16, respectively, are configured to include a motion information collecting unit 601E and a model behavior database 200E (database).

The motion information collection unit 601E collects motion information from a plurality of moving bodies 700E (700E-1, 700E-2, 700E-3, ..., 700E-n), associates location information with an exemplary behavior model according to the road environment based on the collected motion information, and stores the information in a database (at least one of the exemplary behavior databases 200A, 200B, 200C, 200D, 200E (databases), or the road environment database 210C (database)).
That is, the location information and exemplary behavior model associated with the database according to the fifth embodiment of the present disclosure are each generated based on the location information collected from a plurality of moving bodies 700E (700E-1, 700E-2, 700E-3, ..., 700E-n) and the movement information including the movements of the occupants.

A processing example of the action evaluation device according to the fifth embodiment of the present disclosure will be described.
FIG. 17 is a flowchart showing an example of processing by the action information providing unit 151 in the action evaluating apparatus 100E, 100E', or 100E'' according to the fifth embodiment of the present disclosure.
For example, when the motion information providing unit 151E receives information from an external source, the motion information providing unit 151E starts the process shown in Fig. 17. Alternatively, the motion information providing unit 151E starts the process shown in Fig. 17 based on, for example, an instruction from a control unit (not shown).

The motion information providing unit 151E executes motion information acquisition determination processing (step ST5110).
The motion information providing unit 151E determines whether motion information has been acquired in the motion information acquisition determination process.
When the motion information providing unit 151E determines that the motion information has not been acquired ("NO" in step ST5110), it continues the process of step ST5110 until the motion information is acquired.

If the motion information provider 151E determines in the motion information acquisition determination process that it has acquired the motion information ("YES" in step ST5110), it then executes the motion information provision process (step ST5120).
In the motion information providing process, the motion information providing unit 151E outputs motion information, which is information combining position information and information related to the motion of the occupant, to the server device 600E.
Specifically, the action information providing unit 151E in the action evaluation device 100E, 100E' outputs action information, which is information combining information related to the action acquired by the action acquisition unit 111E and location information acquired by the location acquisition unit 112E, to the server device 600E.
Specifically, the action information providing unit 151E in the action evaluation device 100E'' uses the accumulated inattentive point acquisition unit 142E as location information and the observation direction acquired in chronological order by the observation direction acquisition unit as information related to the action, and outputs action information, which is information combining location information and information related to the action, to the server device 600E.

After executing the motion information provision process, the motion information providing unit 151E terminates the process shown in Figure 17.

A processing example of the server device according to the fifth embodiment of the present disclosure will be described.
FIG. 18 is a flowchart illustrating an example of processing performed by the server device 600E when action information is provided from the action evaluation devices 100E, 100E', and 100E'' according to the fifth embodiment of the present disclosure.
For example, when server device 600E receives information from an external source, it starts the process shown in FIG.
Alternatively, server device 600E starts the process shown in FIG. 18 based on a command from a control unit (not shown) in server device 600E.
Server device 600E executes a motion information acquisition determination process (step ST5210).
In the motion information acquisition determination process, motion information collection unit 601E of server device 600E determines whether motion information has been acquired.

When the motion information collecting section 601E determines that the motion information has not been acquired ("NO" in step ST5210), it continues the process of step ST5210 until the motion information is acquired.
If the action information collecting section 601E determines that the action information has been acquired ("YES" in step ST5210), the server device 600E then executes an exemplary behavior model storage process (step ST5220).
In the model behavior model storage process, the behavior information collection unit 601E of the server device 600E collects behavior information from multiple moving bodies 700E (700E-1, 700E-2, 700E-3, ..., 700E-n), associates location information with a model behavior model corresponding to the road environment based on the collected behavior information, and stores the associated information as a database (at least one of model behavior databases 200A, 200B, 200C, 200D, 200E (databases), or road environment database 210C (database)).
Specifically, the motion information collecting unit 601E acquires motion information from the motion evaluation devices 100E, 100E', and 100E" of each moving body 700E (700E-1, 700E-2, 700E-3, ..., 700E-n) and stores the motion information. The motion information collecting unit 601E aggregates the position information and motion-related information included in the stored motion information to determine an exemplary behavior model for each position. For example, when the number of identical motions for each position reaches or exceeds a predetermined threshold (motion count threshold), the motion information collecting unit 601E determines the motion as an exemplary behavior model that shows an exemplary behavior according to the road environment. The motion information collecting unit 601E associates the determined exemplary behavior model with the position information and stores the association data in a database.

After executing the model behavior model storage process, the behavior information collection unit 601E terminates the process shown in Figure 18.

The action evaluation device of the present disclosure is further configured, for example, as follows.

the location information and the exemplary behavior model associated with the database are generated based on location information collected from a plurality of moving bodies and movement information including the movement of the occupant, respectively;
A behavior evaluation device characterized by:

As a result, the present disclosure has the advantage of being able to use an exemplary behavior model based on a large amount of actual data showing actual behavior at each location, thereby improving the accuracy of driving evaluation or behavior evaluation.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

The action evaluation device of the present disclosure is further configured, for example, as follows.

an operation information providing unit that outputs the operation information to a server device that generates the database;
Furthermore,
A behavior evaluation device characterized by:

As a result, the present disclosure has the advantage of being able to use an exemplary behavior model based on a large amount of actual data showing actual behavior at each location, thereby improving the accuracy of driving evaluation or behavior evaluation.
Furthermore, the present disclosure achieves the same effects as those described above by applying the above configuration to a motion evaluation system or the above motion evaluation method.

Here, a hardware configuration for realizing the functions of the present disclosure will be described.
FIG. 19 is a diagram illustrating a first example of a hardware configuration for realizing the functions according to the configuration of the present disclosure.
FIG. 20 is a diagram illustrating a second example of a hardware configuration for realizing the functions according to the configuration of the present disclosure.
The action evaluation devices 100, 100A, 100B, 100C, 100D, 100E (100E-1, 100E-2, 100E-3, ..., 100E-n (n is an integer of 2 or more)), 100E'(100E'-1,100E'-2,100E'-3, ..., 100E'-n (n is an integer of 2 or more)), 100E''(100E''-1,100E''-2,100E''-3, ..., 100E''-n (n is an integer of 2 or more)), occupant monitoring device 300A, and server device 600E of the present disclosure are each realized by hardware such as shown in FIG. 19 or FIG. 20 .

As shown in FIG. 19, each of the action evaluation devices 100, 100A, 100B, 100C, 100D, 100E, 100E', and 100E'' is configured with, for example, a processor 10001, a memory 10002, an input/output interface 10003, and a communication circuit 10004.
The processor 10001 and the memory 10002 are, for example, installed in a computer.
The memory 10002 stores the computer, which includes: motion acquisition units 111, 111A, 111B, 111C, 111E, and 111E'; position acquisition units 112, 112A, 112B, 112C, 112E, and 112E'; model acquisition units 113, 113A, 113B, 113C, 113D, 113E, 113E', and 113E''; evaluation value calculation units 114, 114A, 114B, 114C, 114D, and 114E; image acquisition units 121B, 121C, 121E, 121E', operation judgment units 122B, 122C, 122E, 122E', warning units 123B, 123C, 123D, 123E, road environment acquisition unit 131C, cumulative inattentive driving judgment unit 141D, cumulative inattentive driving point acquisition unit 142D, observation direction acquisition unit 143D, cumulative inattentive driving frequent point acquisition unit 144D, allowable observation direction acquisition unit 145D, operation information provision unit 151E, and programs for functioning as a control unit not shown are stored. The processor 10001 reads out and executes the program stored in the memory 10002, whereby the motion acquisition units 111, 111A, 111B, 111C, 111E, and 111E', the position acquisition units 112, 112A, 112B, 112C, 112E, and 112E', the model acquisition units 113, 113A, 113B, 113C, 113D, 113E, 113E', and 113E'', the evaluation value calculation units 114, 114A, 114B, 114C, 114D, and 114E, and the image acquisition unit 12 The functions of 1B, 121C, 121E, 121E', operation judgment units 122B, 122C, 122E, 122E', warning units 123B, 123C, 123D, 123E, road environment acquisition unit 131C, cumulative inattentive driving judgment units 141D, 141E, cumulative inattentive driving point acquisition units 142D, 142E, observation direction acquisition units 143D, 143E, cumulative inattentive driving frequent point acquisition units 144D, 144E, allowable observation direction acquisition units 145D, 145E, operation information provision unit 151E, and a control unit not shown are realized.
Furthermore, the memory 10002 or another memory not shown implements a storage unit not shown.
Furthermore, the communication circuit 10004 realizes a communication unit (not shown).

Similarly, the occupant monitoring device 300A is composed of, for example, a processor 10001, a memory 10002, an input/output interface 10003, and a communication circuit 10004.
The processor 10001 and the memory 10002 are, for example, installed in a computer.
The memory 10002 stores programs for causing the computer to function as the image acquisition unit 301A, the operation determination unit 302A, and a control unit (not shown). The processor 10001 reads and executes the programs stored in the memory 10002, thereby realizing the functions of the image acquisition unit 301A, the operation determination unit 302A, and the control unit (not shown).
Furthermore, the memory 10002 or another memory (not shown) implements a storage unit (not shown) in the occupant monitoring device 300A.
Furthermore, the communication circuit 10004 realizes a communication unit (not shown) in the occupant monitoring device 300A.

Similarly, the server device 600E is configured by, for example, a processor 10001, a memory 10002, an input/output interface 10003, and a communication circuit 10004.
The processor 10001 and the memory 10002 are, for example, installed in a computer.
The memory 10002 stores a program for causing the computer to function as the motion information collecting circuitry 601E and a control unit (not shown). The processor 10001 reads and executes the program stored in the memory 10002, thereby realizing the functions of the motion information collecting circuitry 601E and the control unit (not shown).
Furthermore, memory 10002 or another memory not shown implements a storage unit not shown in server device 600E.
Furthermore, communication circuit 10004 realizes a communication unit (not shown) in server device 600E.

The processor 10001 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcontroller, or a DSP (Digital Signal Processor).
The memory 10002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory) or flash memory, or may be a magnetic disk such as a hard disk or flexible disk, or may be an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or may be a magneto-optical disk.
The processor 10001 and the memory 10002 or the communication circuit 10004 are connected in a state where they can transmit data to each other. The processor 10001, the memory 10002, and the communication circuit 10004 are also connected in a state where they can transmit data to other hardware via the input/output interface 10003.

Or, in the action evaluation device 100, 100A, 100B, 100C, 100D, 100E, 100E', 100E'', the action acquisition unit 111, 111A, 111B, 111C, 111E, 111E', the position acquisition unit 112, 112A, 112B, 112C, 112E, 112E', the model acquisition unit 113, 113A, 113B, 113C, 113D, 113E, 113E', 113E'', the evaluation value calculation unit 114, 114A, 114B, 114C, 114D, 114E, the image acquisition unit 121B, 121C, 121E, The functions of 121E', operation judgment units 122B, 122C, 122E, 122E', warning units 123B, 123C, 123D, 123E, road environment acquisition unit 131C, cumulative inattentive driving judgment units 141D, 141E, cumulative inattentive driving point acquisition units 142D, 142E, observation direction acquisition units 143D, 143E, cumulative inattentive driving frequent point acquisition units 144D, 144E, allowable observation direction acquisition units 145D, 145E, operation information provision unit 151E, and a control unit not shown may be realized by a dedicated processing circuit 20001, as shown in FIG. 20.

Similarly, the functions of the image acquisition unit 301A, operation determination unit 302A, and a control unit (not shown) in the occupant monitoring device 300A may be realized by a dedicated processing circuit 20001.

Similarly, the functions of the operation information collection unit 601E and the control unit (not shown) in the server device 600E may be realized by a dedicated processing circuit 20001.

The processing circuit 20001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), a SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration).
Furthermore, the memory 20002 or another memory not shown implements a storage unit not shown.
The memory 20002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory) or flash memory, or may be a magnetic disk such as a hard disk or flexible disk, or may be an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or may be a magneto-optical disk.
Furthermore, the communication circuit 20004 realizes a communication unit (not shown).
The processing circuit 20001 and the memory 20002 or the communication circuit 20004 are connected in a state where they can transmit data to each other. In addition, the processing circuit 20001, the memory 20002, and the communication circuit 20004 are connected in a state where they can transmit data to other hardware via the input/output interface 20003.
In the action evaluation devices 100, 100A, 100B, 100C, 100D, 100E, 100E', and 100E'', the action acquisition units 111, 111A, 111B, 111C, 111E, and 111E', the position acquisition units 112, 112A, 112B, 112C, 112E, and 112E', the model acquisition units 113, 113A, 113B, 113C, 113D, 113E, 113E', and 113E'', the evaluation value calculation units 114, 114A, 114B, 114C, 114D, and 114E, the image acquisition units 121B, 121C, and 121D The functions of the operation determination units 122B, 122C, 122E, 122E', the warning units 123B, 123C, 123D, 123E, the road environment acquisition unit 131C, the cumulative inattentive driving determination units 141D, 141E, the cumulative inattentive driving point acquisition units 142D, 142E, the observation direction acquisition units 143D, 143E, the cumulative inattentive driving frequent point acquisition units 144D, 144E, the allowable observation direction acquisition units 145D, 145E, the operation information provision unit 151E, and the control unit (not shown) may be realized by separate processing circuits, or may be realized together by a processing circuit.
Similarly, in the occupant monitoring device 300A, the functions of the image acquisition unit 301A, the operation determination unit 302A, and the control unit (not shown) may be realized by separate processing circuits, or may be realized collectively by a processing circuit.
Similarly, the functions of the operation information collecting unit 601E and the control unit (not shown) in the server device 600E may be realized by separate processing circuits, or may be realized together by a processing circuit.

Or, in the action evaluation devices 100, 100A, 100B, 100C, 100D, 100E, 100E', 100E'', the action acquisition units 111, 111A, 111B, 111C, 111E, 111E', the position acquisition units 112, 112A, 112B, 112C, 112E, 112E', the model acquisition units 113, 113A, 113B, 113C, 113D, 113E, 113E', 113E'', the evaluation value calculation units 114, 114A, 114B, 114C, 114D, 114E, the image acquisition units 121B, 121C, 121E, 121E', and the action determination unit 122B , 122C, 122E, 122E', warning unit 123B, 123C, 123D, 123E, road environment acquisition unit 131C, cumulative inattentive driving judgment unit 141D, 141E, cumulative inattentive driving point acquisition unit 142D, 142E, observation direction acquisition unit 143D, 143E, cumulative inattentive driving frequent point acquisition unit 144D, 144E, allowable observation direction acquisition unit 145D, 145E, operation information provision unit 151E, and some of the functions of a control unit not shown may be realized by processor 10001 and memory 10002, and the remaining functions may be realized by processing circuit 20001.
Similarly, in the occupant monitoring device 300A, some of the functions of the image acquisition unit 301A, the operation determination unit 302A, and a control unit not shown may be realized by the processor 10001 and the memory 10002, and the remaining functions may be realized by the processing circuit 20001.
Similarly, in the server device 600E, some of the functions of the operation information collection unit 601E and the control unit (not shown) may be realized by the processor 10001 and memory 10002, and the remaining functions may be realized by the processing circuit 20001.

Note that within the scope of this disclosure, the embodiments may be freely combined, any components of the embodiments may be modified, or any components of the embodiments may be omitted. For example, the components of the action evaluation device according to embodiment 4 may be combined with components of other embodiments.

This disclosure enables more favorable driving evaluation than conventional methods, and is therefore suitable for use in behavior evaluation devices that evaluate the behavior of occupants driving a moving object such as a vehicle.

1A, 1B, 1C, 1D, 1E Movement evaluation system, 100, 100A, 100B, 100C, 100D, 100E (100E-1, 100E-2, 100E-3, ..., 100E-n (n is an integer of 2 or more)), 100E' (100E'-1, 100E'-2, 100E'-3, ..., 100E'-n (n is an integer of 2 or more)), 100E'' (100E''-1, 100E''-2, 100E''-3, ..., 100E''-n (n is an integer of 2 or more)) Movement evaluation device, 111, 111A, 111B, 111C, 111E, 111E' Action acquisition unit, 112, 112A, 112B, 112C, 112E, 112E′ Position acquisition unit, 113, 113A, 113B, 113C, 113D, 113E, 113E′, 113E″ Model acquisition unit, 114, 114A, 114B, 114C, 114D, 114E, 114E′, 114E″ Evaluation value calculation unit, 121B, 121C, 121E, 121E′ Video acquisition unit, 122B, 122C, 122E, 122E′ Action determination unit, 123B, 123C, 123D, 123E Warning unit, 131C Road environment acquisition unit, 141D, 141E Accumulative inattentive behavior determination unit, 142D, 142E Accumulative inattentive point acquisition unit, 143D, 143E Observation direction acquisition unit, 144D, 144E Accumulative inattentive frequent point acquisition unit, 145D, 145E Allowable observation direction acquisition unit (corresponding to "model acquisition unit"), 151E Action information provision unit, 200A, 200B, 200C, 200D, 200E Model behavior database (database), 210C Road environment database (database), 300A Occupant monitoring device, 301A Video acquisition unit, 302A Action determination unit, 400A, 400B, 400C, 400D, 400E Output device, 500B, 500C, 500D, 500E Imaging device, 600E Server device, 601E Operation information collection unit, 700E (700E-1, 700E-2, 700E-3, ..., 700E-n (n is an integer of 2 or more)) Mobile object, 1000 Vehicle (mobile object), 1100 Direction of travel (predicted direction of travel), 1200 Normal allowable observation direction, 1300 Allowable observation direction at cumulative inattentiveness frequent occurrence point, 2000 Merging lane, 3000 Merged lane, 4000 Normal allowable observation direction, 4010 Allowable observation direction at cumulative inattentiveness frequent occurrence point, 10001 Processor, 10002 Memory, 10003 Input/output interface, 10004 Communication circuit, 20001 Processing circuit, 20002 Memory, 20003 Input/output interface, 20004 Communication circuit.

Claims (15)

a motion acquisition unit that acquires a motion of a driver of the vehicle;
a position acquisition unit that acquires the position of the moving object;
a road environment acquisition unit that refers to a database that stores in advance position information and road environment information in association with each other, and acquires road environment information corresponding to the position acquired by the position acquisition unit;
a model acquisition unit that refers to a database that associates road environment information with an exemplary behavior model that shows exemplary behavior according to the road environment and stores the information in advance, and acquires an exemplary behavior model according to the road environment information acquired by the road environment acquisition unit at the position of the moving object;
an evaluation value calculation unit that calculates an evaluation value of the action based on the action acquired by the action acquisition unit and the exemplary behavior model acquired by the model acquisition unit;
A motion evaluation device comprising: a motion acquisition unit that acquires a motion of a driver of the vehicle;
a position acquisition unit that acquires the position of the moving object;
a model acquisition unit that acquires an exemplary behavior model corresponding to the road environment at the position of the mobile object using a database that associates and pre-stores location information with an exemplary behavior model that shows exemplary behavior corresponding to the road environment;
an evaluation value calculation unit that calculates an evaluation value of the action based on the action acquired by the action acquisition unit and the exemplary behavior model acquired by the model acquisition unit;
Equipped with
an accumulated inattentive state determination unit that obtains an accumulated value related to intermittent inattentive state by the occupant and performs an accumulated inattentive state determination by using the accumulated value and a predetermined threshold value;
an accumulated inattentive point acquisition unit that acquires accumulated inattentive points based on the position of the moving body when the accumulated inattentive state is determined to be present by the accumulated inattentive state determination unit;
an observation direction acquisition unit that acquires an observation direction that is a direction in which the occupant of the moving body is likely to be observing;
an accumulative inattentiveness frequent occurrence point acquisition unit that acquires accumulative inattentiveness frequent occurrence points, which are points where accumulative inattentiveness by occupants of each of a plurality of moving bodies frequently occurs;
an allowable observation direction acquisition unit that acquires an allowable observation direction, which is a direction in which observation should be allowed for each of the cumulative inattentive looking points;
Furthermore,
The model acquisition unit
When the cumulative inattentive driving point and the cumulative inattentive driving frequent point overlap,
and,
When the observation direction acquired by the observation direction acquisition unit overlaps with the allowable observation direction,
Observing the permissible observation direction at the position of the cumulative inattentive viewing point is acquired as the exemplary behavior model.
A behavior evaluation device characterized by: When the cumulative inattentiveness frequent point includes a point included in a merging area on a motorway,
the allowable observation direction is a direction based on the position of the occupant and is a direction toward a merging road where the moving object will merge;
3. The action evaluation device according to claim 2. The movement acquired by the movement acquisition unit is a movement of the occupant determined using an image of an interior of the vehicle.
4. The action evaluation device according to claim 1, wherein the action evaluation device is a motion evaluation device. The road environment includes at least one of a motorway, a merging area of a motorway, a vicinity of an intersection, a vicinity of a pedestrian crossing, the presence or absence of a traffic light, the shape of a road, the width of a road, the degree of vehicle congestion, or the degree of pedestrian congestion.
4. The action evaluation device according to claim 1, wherein the action evaluation device is a motion evaluation device. The exemplary behavior model includes at least one of a viewing direction, a cumulative inattentiveness, or a position of a body part.
4. The action evaluation device according to claim 1, wherein the action evaluation device is a motion evaluation device. The observation direction includes at least one of a face direction of the occupant or a line of sight direction of the occupant.
7. The action evaluation device according to claim 6. an image acquisition unit that acquires an image from an imaging device that captures an image of the interior of the moving body;
a motion determination unit that determines a motion of the occupant captured in the video;
Equipped with
The action evaluation device according to any one of claims 1 to 3. the location information and the exemplary behavior model associated with the database are generated based on location information collected from a plurality of moving bodies and movement information including the movement of the occupant, respectively;
4. The action evaluation device according to claim 1, wherein the action evaluation device is a motion evaluation device. an operation information providing unit that outputs the operation information to a server device that generates the database;
Furthermore,
The action evaluation device according to claim 9 . a warning unit that issues a warning to the occupant using the evaluation value;
Equipped with
The action evaluation device according to any one of claims 1 to 3. a motion acquisition step in which a motion acquisition unit of the motion evaluation device acquires a motion of a occupant driving the moving object;
a position acquisition step in which a position acquisition unit of the action evaluation device acquires a position of the moving object;
a road environment acquisition step in which a road environment acquisition unit of the action evaluation device refers to a database in which position information and road environment information are associated and stored in advance, and acquires the road environment information corresponding to the position acquired by the position acquisition unit;
a model acquisition step in which a model acquisition unit of the behavior evaluation device refers to a database in which the road environment information and an exemplary behavior model showing exemplary behavior according to the road environment are associated and stored in advance, and acquires an exemplary behavior model according to the road environment information acquired by the road environment acquisition unit at the position of the moving object;
an evaluation value calculation step in which an evaluation value calculation unit of the action evaluation device calculates an evaluation value of the action based on the action acquired by the action acquisition unit and the exemplary behavior model;
A motion evaluation method comprising: a motion acquisition step in which a motion acquisition unit of the motion evaluation device acquires a motion of a occupant driving the moving object;
a position acquisition step in which a position acquisition unit of the action evaluation device acquires a position of the moving object;
a model acquisition step in which a model acquisition unit of the action evaluation device acquires an exemplary behavior model corresponding to the road environment at the position of the moving object using a database in which position information and an exemplary behavior model showing exemplary behavior corresponding to the road environment are associated and stored in advance;
an evaluation value calculation step in which an evaluation value calculation unit of the action evaluation device calculates an evaluation value of the action based on the action acquired by the action acquisition unit and the exemplary behavior model;
Equipped with
a cumulative inattentive driving determination step in which a cumulative inattentive driving determination unit of the action evaluation device acquires a cumulative value related to intermittent inattentive driving by the occupant and performs a cumulative inattentive driving determination, which is a determination of whether the occupant is in a cumulative inattentive driving state, using the cumulative value and a predetermined threshold value;
a cumulative inattentive state determination unit step of determining an inattentive state based on the position of the moving object when the cumulative inattentive state determination unit determines that the moving object is in an inattentive state;
an observation direction acquisition step in which an observation direction acquisition unit of the action evaluation device acquires an observation direction that is a direction in which the occupant of the moving body is likely to be observing;
a cumulative inattentive spot acquisition step in which a cumulative inattentive spot acquisition unit of the action evaluation device acquires cumulative inattentive spot locations, which are locations where cumulative inattentive behavior by occupants of each of a plurality of moving bodies frequently occurs;
an allowable observation direction acquisition step in which an allowable observation direction acquisition unit of the action evaluation device acquires an allowable observation direction, which is a direction in which observation should be allowed for each of the cumulative inattentive driving points;
Furthermore,
In the model acquisition step, the model acquisition unit
When the cumulative inattentive driving point and the cumulative inattentive driving frequent point overlap,
and,
When the observation direction acquired by the observation direction acquisition unit overlaps with the allowable observation direction,
Observing the permissible observation direction at the position of the cumulative inattentive viewing point is acquired as the exemplary behavior model.
A motion evaluation method comprising: a motion acquisition unit that acquires motions of a driver of a vehicle;
a position acquisition unit that acquires the position of the moving object;
a road environment acquisition unit that refers to a database that stores in advance position information and road environment information in association with each other, and acquires road environment information corresponding to the position acquired by the position acquisition unit;
a model acquisition unit that refers to a database that associates road environment information with an exemplary behavior model that shows exemplary behavior according to the road environment and stores the information in advance, and acquires an exemplary behavior model according to the road environment information acquired by the road environment acquisition unit at the position of the moving object;
an evaluation value calculation unit that calculates an evaluation value of the action based on the action acquired by the action acquisition unit and the exemplary behavior model acquired by the model acquisition unit;
A motion evaluation device comprising:
A program to run as. a motion acquisition unit that acquires motions of a driver of a vehicle;
a position acquisition unit that acquires the position of the moving object;
a model acquisition unit that acquires an exemplary behavior model corresponding to the road environment at the position of the mobile object using a database that associates and pre-stores location information with an exemplary behavior model that shows exemplary behavior corresponding to the road environment;
an evaluation value calculation unit that calculates an evaluation value of the action based on the action acquired by the action acquisition unit and the exemplary behavior model acquired by the model acquisition unit;
Equipped with
an accumulated inattentive state determination unit that obtains an accumulated value related to intermittent inattentive state by the occupant and performs an accumulated inattentive state determination by using the accumulated value and a predetermined threshold value;
an accumulated inattentive point acquisition unit that acquires accumulated inattentive points based on the position of the moving body when the accumulated inattentive state is determined to be present by the accumulated inattentive state determination unit;
an observation direction acquisition unit that acquires an observation direction that is a direction in which the occupant of the moving body is likely to be observing;
an accumulative inattentiveness frequent occurrence point acquisition unit that acquires accumulative inattentiveness frequent occurrence points, which are points where accumulative inattentiveness by occupants of each of a plurality of moving bodies frequently occurs;
an allowable observation direction acquisition unit that acquires an allowable observation direction, which is a direction in which observation should be allowed for each of the cumulative inattentive looking points;
Furthermore,
The model acquisition unit
When the cumulative inattentive driving point and the cumulative inattentive driving frequent point overlap,
and,
When the observation direction acquired by the observation direction acquisition unit overlaps with the allowable observation direction,
Observing the permissible observation direction at the position of the cumulative inattentive viewing point is acquired as the exemplary behavior model.
A behavior evaluation device characterized by:
A program to run as.