JP7710162B2 - Information output method, information output device, and program - Google Patents

Description

The present invention relates to an information output method, an information output device, and a program.

Previously, a method for predicting events that may occur at road intersections has been disclosed (see Patent Document 1).

Japanese Patent Application Publication No. 9-251596

However, there is a problem in that events may not be predicted properly.

The present invention provides an information output method that appropriately identifies and outputs events that may occur in a moving object.

The information output method according to one aspect of the present invention is an information output method that acquires a model that outputs first event information indicating an event that occurred in a first moving body based on first moving body information, which is information about a first moving body input to the model, acquires second moving body information, which is information about a second moving body, generates third moving body information by modifying the acquired second moving body information, acquires information indicating an event that occurred in the second moving body as second event information that is output by inputting the acquired second moving body information into the model, acquires information indicating an event that is output by inputting the generated third moving body information into the model as third event information, and generates and outputs difference information indicating a difference between the second event information and the third event information.

These comprehensive or specific aspects may be realized as a system, device, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, device, integrated circuit, computer program, and recording medium.

The information output device of the present invention can appropriately identify and output events that may occur to a moving object.

FIG. 1 is an explanatory diagram illustrating a schematic configuration of a detection system according to an embodiment. FIG. 2 is an explanatory diagram that illustrates a schematic functional configuration of the vehicle according to the embodiment. FIG. 3 is an explanatory diagram illustrating a schematic functional configuration of the detection device according to the embodiment. FIG. 4 is a flow diagram showing the process of the detection system according to the embodiment. FIG. 5 is an explanatory diagram illustrating a schematic configuration of a specific system according to an embodiment. FIG. 6 is an explanatory diagram illustrating a schematic functional configuration of a specific device according to the embodiment. FIG. 7 is an explanatory diagram illustrating an example of vehicle information and event information acquired by a vehicle in the embodiment. FIG. 8 is an explanatory diagram illustrating an example of vehicle information and event information changed by the change unit in the embodiment. FIG. 9 is an explanatory diagram illustrating an example of integrated event information according to the embodiment. FIG. 10 is an explanatory diagram illustrating an example of difference information according to the embodiment. FIG. 11 is an explanatory diagram illustrating a first example of output information according to the embodiment. FIG. 12 is an explanatory diagram illustrating a second example of the output information according to the embodiment. FIG. 13 is a flow diagram showing the process of the specific system in the embodiment.

(Findings on which the present invention is based)
The present inventors have found that the following problems arise with the technology for predicting events described in the "Background Art" section.

Based on information acquired by a sensor, a moving body (such as a vehicle) can detect events occurring on or around the moving body, and predict events that are likely to occur on or around the moving body. When an event is detected or predicted, it is expected that the moving body will take measures to avoid the event, or to improve the moving body or its travel route in advance to prevent the event from occurring.

The above-mentioned mobile objects include not only mobile objects operated by an on-board driver, but also mobile objects that run autonomously without a driver on-board, or mobile objects that run controlled via a communication line by a driver not inside the vehicle but in a remote location.

However, the information acquired by the sensor may contain errors or may be missing. In this way, when the information acquired by the sensor contains errors or missing information, there is a problem that an event cannot be predicted appropriately, in other words, the prediction of an event may fail.

The present invention provides an information output method that can appropriately predict events that may occur in a moving body. More specifically, the present invention provides an information output method that identifies in advance, among events that may occur in a moving body, events that may fail to be detected during actual movement, and outputs the information.

In order to solve such problems, an information output method according to one aspect of the present invention is an information output method that acquires a model that outputs first event information indicating an event that occurred to a first moving body based on first moving body information, which is information about a first moving body input to the model, acquires second moving body information, which is information about a second moving body, generates third moving body information by modifying the acquired second moving body information, acquires information indicating an event that occurred to the second moving body as second event information that is output by inputting the acquired second moving body information into the model, acquires information indicating an event that is output by inputting the generated third moving body information into the model as third event information, and generates and outputs difference information indicating the difference between the second event information and the third event information.

According to the above aspect, it is possible to output differential information indicating events that may occur in a moving body and may fail to be detected during actual movement. The output differential information indicates a change in event information caused by making a change to the second moving body information. Furthermore, the change made to the second moving body information may correspond to an error or deficiency contained in information acquired by a sensor that may occur during actual movement. Therefore, it can be said that the output differential information indicates an event that may not be detected due to an error or deficiency in information that can be acquired by a sensor during actual movement. In this way, the above information output method can appropriately identify and output an event that may occur in a moving body by outputting differential information indicating an event that may fail to be detected during actual movement among events that occur in a moving body.

For example, the second mobile body information may include information generated from a sensor value generated by sensing with a sensor equipped on the second mobile body when the second mobile body moves, and when generating the third mobile body information, the third mobile body information may be generated by making the change to the second mobile body information corresponding to a failure of sensing by the sensor equipped on the second mobile body.

According to the above aspect, since the change made to the second moving body information corresponds to a failure of sensing by the sensor equipped in the second moving body, the output difference information is difference information indicating an event that may fail to be detected during actual movement. Therefore, the above information output method can more appropriately identify and output an event that may occur in the moving body.

For example, the frequency with which the sensing by the sensor equipped in the second moving body fails may be measured, and when generating the third moving body information, the third moving body information may be generated by making the changes based on the measured frequency.

According to the above aspect, the second moving body information is changed based on the frequency at which sensing by the sensor equipped in the second moving body fails, so the output difference information takes into account the actual frequency of failures and becomes difference information indicating events that may fail to be detected during actual movement. Therefore, the above information output method can more appropriately identify and output events that may occur in the moving body.

For example, when generating the difference information, the difference information may be generated to indicate events indicated in the second event information excluding the events indicated in the third event information.

According to the above aspect, by removing the events indicated in the third event information from the events indicated in the second event information, difference information can be easily generated. Therefore, the above information output method can more easily appropriately identify and output events that may occur in a moving body.

For example, the first moving body information may further include information generated from a sensor value generated by sensing with a sensor installed on the moving path of the first moving body, and the second moving body information may further include information generated from a sensor value generated by sensing with a sensor installed on the moving path of the second moving body.

According to the above aspect, since mobile body information is generated using sensor values generated by sensing sensors installed on the moving path in addition to the sensors equipped on the mobile body, the generated mobile body information becomes more appropriate. This also improves the appropriateness of the event information and difference information. Therefore, the above information output method can more appropriately identify and output events that may occur in the mobile body.

For example, the first moving body information may include position information or speed information of the first moving body, the presence or absence of obstacles around the first moving body, position information or speed information of obstacles around the first moving body, the number of passengers on the first moving body, the amount of people flowing around the first moving body, or the amount of vehicle flowing around the first moving body, and the second moving body information may include position information or speed information of the second moving body, the presence or absence of obstacles around the second moving body, position information or speed information of obstacles around the second moving body, the number of passengers on the second moving body, the amount of people flowing around the second moving body, or the amount of vehicle flowing around the second moving body.

According to the above aspect, the difference information is generated using, as the moving body information, the position information or speed information of the moving body, the presence or absence of an obstacle around the moving body, or the position information or speed information of an obstacle existing around the moving body. Therefore, the above information output method can more easily appropriately identify and output an event that may occur to the moving body.

For example, the event may include a person or object jumping out in the direction of travel of a moving body that is the first moving body or the second moving body, a person or object being caught up in the moving body when changing direction, a collision when the moving body merges with another moving body, another moving body cutting in on the moving body in the moving direction, deviation of the moving body's path of travel, a collision caused by a vehicle and a pedestrian starting to move simultaneously in a crossing area without traffic lights, or intentional obstruction around the vehicle.

According to the above aspect, a model is used based on the moving body information to detect events such as a person or object jumping out in the moving body's direction of travel, a person or object being caught up in the moving body when changing direction, a collision when the moving body merges with another moving body, another moving body cutting in on the moving body's direction of travel, or a deviation in the moving body's path of travel. Therefore, the above information output method can more easily appropriately identify and output events that may occur to a moving body.

For example, the difference information may include an image showing on a map the location where the event indicated by the difference information occurred.

According to the above aspect, the location where an event that may not be detectable during actual movement occurs is shown and presented on a map. Therefore, a person viewing the map can easily grasp the event on the map, and it is expected that measures will be taken to make the event less likely to actually occur. Therefore, the above information output method can appropriately identify and output an event that may occur to a moving body, while also leading to measures to reduce the detection of the event.

For example, the difference information may include the degree to which each of multiple factors of the event indicated by the difference information influenced the occurrence of the event.

According to the above aspect, the degree to which each of the multiple factors of the event influenced the occurrence of the event is presented. Therefore, it is expected that a person who visually recognizes the above degree will take measures to make the event less likely to actually occur, or to make it less likely that the event will fail to be detected. Therefore, the above information output method can appropriately identify and output an event that may occur in a moving object, while also leading to measures to reduce the detection of the event.

For example, the model may be a neural network model.

According to the above aspect, it is possible to more easily and appropriately predict events occurring in a moving object using a neural network model.

In addition, an information output device according to one embodiment of the present invention is an information output device that includes an acquisition unit, a modification unit, a detection unit, and a generation unit, wherein the detection unit acquires a model that outputs first event information indicating an event that occurred to a first moving body based on first moving body information, which is information regarding a first moving body input to the model, the acquisition unit acquires second moving body information, which is information regarding a second moving body, the modification unit generates third moving body information by modifying the second moving body information acquired by the acquisition unit, the detection unit acquires information indicating an event that occurred to the second moving body as second event information, which is output by inputting the second moving body information acquired by the acquisition unit into the model, the detection unit acquires information indicating an event that occurred to the second moving body as third event information, which is output by inputting the third moving body information generated by the modification unit into the model, and the generation unit generates and outputs difference information indicating the difference between the second event information and the third event information.

The above aspect provides the same effect as the above information output method.

A program according to one aspect of the present invention is a program that causes a computer to execute the above-mentioned information output method.

The above aspect provides the same effect as the above information output method.

These comprehensive or specific aspects may be realized as a system, device, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or as any combination of a system, device, integrated circuit, computer program, or recording medium.

The following describes the embodiment in detail with reference to the drawings.

The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in an independent claim that indicates a superordinate concept are described as optional components.

(Embodiment)
In the present embodiment, an identification method and an identification device that appropriately identify and output an event that may occur in a moving object will be described. The identification method and the identification device are also referred to as an information output method and an information output device, respectively.

Below, we will explain (1) a detection system that detects events that occur in a vehicle, and (2) an identification system that identifies in advance events that the detection system may fail to detect when the vehicle is moving.

The term "event" refers to various events related to the vehicle, and more specifically, may refer to events that impede the vehicle's progress or the safe progress of the vehicle.

1 is an explanatory diagram that illustrates a schematic configuration of a detection system 5 according to the present embodiment. The detection system 5 is a system that detects an event that occurs in a vehicle 7.

As shown in FIG. 1, the detection system 5 includes a presentation device 30 and a detection device 50, which are communicatively connected via a network N. The presentation device 30 and the detection device 50 are communicatively connected to a vehicle 7 via the network N. The detection system 5 may further include a vehicle 7.

The vehicle 7 is an example of a moving body that is a target for which an event is detected by the detection system 5. The vehicle 7 generates information about the vehicle 7 (also called vehicle information) from sensor values generated by sensing using the sensors, and transmits the generated vehicle information to the detection device 50. The vehicle information is an example of moving body information.

The detection device 50 is a device that detects an event occurring in the vehicle 7.
The vehicle information acquisition unit 30 acquires vehicle information from the vehicle 7, detects an event occurring in the vehicle 7 using a detection model, and transmits event information indicating the event to the presentation device 30.

The presentation device 30 is a device that presents events occurring in the vehicle 7. The presentation device 30 acquires event information transmitted by the detection device 50 and presents the acquired event information. The event information is presented by displaying an image showing the presented information on a display screen, or by outputting audio showing the presented information through a speaker. The presented event information is expected to be viewed or heard by the monitor P. The monitor P who has viewed or visually recognized the event information is expected to control the vehicle 7 according to the content of the event shown in the event information. In particular, if the event is an event that hinders the safe progress of the vehicle 7, the monitor P is expected to control the vehicle 7 to avoid the event.

Figure 2 is an explanatory diagram that shows a schematic diagram of the functional configuration of vehicle 7 in this embodiment.

2, vehicle 7 includes, as functional units, sensors 51A and 51B, a processing unit 52, and a providing unit 53. The functional units of vehicle 7 can be realized by a processor (e.g., a CPU (Central Processing Unit), not shown) included in vehicle 7 executing a predetermined program using a memory (not shown).

The sensor 51A is a sensor that performs sensing of the vehicle 7 or the surroundings of the vehicle 7. When the sensor 51A performs sensing, it generates a sensor value indicating the result of the sensing and provides the sensor value to the processing unit 52. The sensor 51A may be a camera, a LiDAR (Light Detection and Ranging), an IMU (Inertial Measurement Unit), etc.
The sensor 51A includes an IMU (Inertial Measurement Unit), or a GPS receiver. The sensor values include at least an image captured by a camera (e.g., an RGB image), a point cloud acquired by a LiDAR, acceleration information and angular velocity information acquired by an IMU, or GPS data acquired by a GPS receiver. The sensor 51A repeatedly acquires the sensor values at appropriate intervals (e.g., about 0.1 seconds to several seconds). The interval at which the sensor 51A acquires the sensor values may be determined according to the contents of sensing. The interval at which the sensor 51A acquires the sensor values may be fixed or may vary.

Sensor 51B is a sensor similar to sensor 51A and operates independently of sensor 51A. Sensor 51B is expected to be a sensor that senses information different from the information sensed by sensor 51A, but may also be a sensor that senses the same information as the information sensed by sensor 51A.

For example, sensor 51A is a GPS receiver and sensor 51B is a camera.

Note that in FIG. 2, two sensors, 51A and 51B, are shown, but any number of sensors may be used as long as they are one or more.

The processing unit 52 is a functional unit that acquires information about the vehicle 7 (vehicle information) by processing the sensor values provided by the sensors 51A and 51B. The vehicle information includes, for example, position information or speed information about the vehicle 7, the presence or absence of obstacles around the vehicle 7, position information or speed information about obstacles around the vehicle 7, the number of passengers in the vehicle 7, the amount of people flowing around the vehicle 7 (also called people flow rate), or the amount of vehicles flowing around the vehicle 7 (also called vehicle flow rate). The processing unit 52 may acquire vehicle information by processing not only the sensor values provided by the sensors 51A etc., but also information acquired from an external device.

The method by which the processing unit 52 calculates the vehicle information based on the sensor value can be achieved by known technology. For example, if the sensor 51A is a GPS receiver, the processing unit 52 processes a signal (corresponding to the sensor value) received by the GPS receiver from a GPS satellite to obtain location information indicating the location of the GPS receiver, in other words, the vehicle 7. Also, for example, when a rental contract is made to ride the vehicle 7, the processing unit 52 obtains the number of passengers in the vehicle 7 by referring to the contents of the rental contract obtained from an external device.

The providing unit 53 is a functional unit that provides the vehicle information calculated by the processing unit 52 to the detection device 50. The providing unit 53 transmits the vehicle information to the detection device 50 through a communication interface (not shown) connected to the network N.

Figure 3 is an explanatory diagram that shows a schematic diagram of the functional configuration of the detection device 50 in this embodiment.

As shown in FIG. 3, the detection device 50 includes, as functional units, an acquisition unit 501, a detection unit 502, and an output unit 503. The functional units of the detection device 50 can be realized by a processor (e.g., a CPU, not shown) included in the detection device 50 executing a predetermined program using a memory (not shown).

The acquisition unit 501 is a functional unit that acquires vehicle information from the vehicle 7. The acquisition unit 501 receives vehicle information transmitted by the vehicle 7 (provision unit 53) through a communication interface (not shown) connected to the network N. The acquisition unit 501 provides the acquired vehicle information to the detection unit 502.

The acquisition unit 501 may acquire the vehicle information of the vehicle 7 from equipment other than the vehicle 7. Specifically, the acquisition unit 501 may acquire vehicle information generated from a sensor value acquired by a sensor (not shown) installed on the travel path of the vehicle 7, sensing the vehicle 7 or the surroundings of the vehicle 7. The sensor installed on the travel path of the vehicle 7 is, for example, a camera installed on a traffic light or a street light connected to the network N. In this case, the acquisition unit 501 acquires the image generated by the camera through the network N.

The detection unit 502 is a functional unit that acquires event information indicating an event that has occurred in the vehicle 7 based on the vehicle information acquired by the acquisition unit 501. The detection unit 502 has a model 505 that outputs event information based on the vehicle information, and acquires the event information using the model 505.

The detection unit 502 provides the acquired event information to the presentation device 30. Specifically, the detection unit 502 transmits the acquired event information to the presentation device 30 through a communication interface (not shown) connected to the network N. Here, the event that occurred to the vehicle 7 includes, for example, at least one of a person or object jumping out in the direction of travel of the vehicle 7, a person or object being caught when the vehicle 7 changes its direction of travel, a collision when the vehicle 7 merges with another vehicle, another moving object cutting in on the direction of travel of the vehicle 7, deviation of the travel path of the vehicle 7, a collision between the vehicle 7 and a pedestrian due to the vehicle 7 starting to travel simultaneously in a crossing area without traffic lights, and intentional obstruction around the vehicle 7.

The model 505 is a detection model that detects an event that has occurred in a vehicle. Specifically, when vehicle information is input, the model 505 outputs event information indicating an event that has occurred in a vehicle related to the vehicle information based on the input vehicle information. The model 505 is, for example, a learning model constructed in advance by machine learning. In advance machine learning, for example, a sufficient number (tens of thousands to hundreds of thousands) of pairs of vehicle information when various events have occurred in the vehicle and event information indicating the events are input to a learning machine as training data. Then, learning progresses by adjusting internal parameters included in the model so that the event information included in the training data can be output based on the characteristics of the vehicle information included in the training data. The model is, for example, a neural network model.

The output unit 503 is a functional unit that provides the event information acquired by the detection unit 502 to the presentation device 30. The output unit 503 transmits the event information to the presentation device 30 via a communication interface (not shown) connected to the network N.

Figure 4 is a flow diagram showing the processing of the detection system 5 in this embodiment.

In step S101, the sensors 51A and the like of the vehicle 7 acquire sensor values by sensing the vehicle 7 or the surroundings of the vehicle 7.

In step S102, the processing unit 52 of the vehicle 7 acquires vehicle information by processing the sensor values acquired in step S101.

In step S103, the detection unit 502 of the detection device 50 acquires event information using the vehicle information acquired in step S102 and the model 505.

In step S104, the output unit 503 outputs the event information acquired in step S103 to the presentation device 30. It is expected that the output presentation information will be visually recognized by the observer P in an appropriate format.

In this way, the detection system 5 detects an event occurring in the vehicle 7. It is assumed that the observer P becomes aware of the event by visually checking the information detected and presented by the detection system 5.

(2) Identification System Fig. 5 is an explanatory diagram that shows a schematic configuration of the identification system 1 according to the present embodiment. The identification system 1 is a system that identifies in advance an event that may fail to be detected when the vehicle 7 is moving.

As shown in FIG. 5, the identification system 1 includes a presentation device 20 and a specific device 10, which are communicatively connected via a network N. The presentation device 20 and the specific device 10 are communicatively connected to a vehicle 3 via the network N. The identification system 1 may further include a vehicle 3.

Vehicle 3 is an example of a moving body used to identify an event that detection system 5 may fail to detect when vehicle 7 is moving. The functions provided by vehicle 3 are the same as those of vehicle 7 in detection system 5. Note that vehicle 3 may be the same vehicle as vehicle 7, or may be a different vehicle.

Vehicle 3 generates information about vehicle 3 (also called vehicle information) from the sensor values generated by sensing using the sensors, and transmits the generated vehicle information to the detection device 50.

The identification device 10 is a device that identifies in advance events that the detection system 5 may fail to detect when the vehicle 7 is moving. The identification device 10 acquires vehicle information from the vehicle 3, and uses a detection model from the acquired vehicle information and vehicle information obtained by modifying the acquired vehicle information to identify events that the detection system 5 may fail to detect when the vehicle 7 is moving. The identification device 10 also transmits identification information that identifies the above events to the presentation device 20.

The presentation device 20 is a device that presents events that may fail to be detected when the vehicle 7 is moving. The presentation device 20 acquires event information transmitted by the specific device 10 and presents the acquired event information. The event information is presented by displaying an image showing the presented information on a display screen, or by outputting audio showing the presented information through a speaker. It is assumed that the presented event information will be viewed or heard by an observer Q. It is assumed that the observer Q who has viewed or visually recognized the event information will take measures to avoid the event or measures to prevent the event from occurring.

For example, if the event is a person running out into the direction of travel of a vehicle, measures to make the event less likely to actually occur include installing equipment to prevent people from entering the road (such as guardrails or signs displaying warning information). Furthermore, measures to make it less likely that the event will be detected unsuccessfully include installing more sensors or adjusting their installation positions.

Figure 6 is an explanatory diagram that shows a schematic diagram of the functional configuration of the specific device 10 in this embodiment.

As shown in FIG. 6, the specific device 10 includes, as functional units, an acquisition unit 101, a change unit 102, a detection unit 103, and a generation unit 104. The functional units of the specific device 10 can be realized by a processor (e.g., a CPU (Central Processing Unit), not shown) included in the specific device 10 executing a predetermined program using a memory (not shown).

The acquisition unit 101 is a functional unit that acquires vehicle information. Specifically, the acquisition unit 101 acquires vehicle information (corresponding to second moving body information) related to the movement of the vehicle 3. The acquisition unit 101 provides the acquired vehicle information to the change unit 102 and the detection unit 103. The vehicle information provided in this manner is also referred to as vehicle information 111. The vehicle information 111 is information generated by the processing unit 52 from sensor values generated by sensing using the sensor 51A etc. equipped in the vehicle 3, and is information related to the vehicle 3.

The acquisition unit 101 may acquire the vehicle information 111 of the vehicle 3 from equipment other than the vehicle 3. Specifically, the acquisition unit 101 may acquire the vehicle information 111 generated from a sensor value acquired by a sensor (not shown) installed on the travel path of the vehicle 3, by sensing the vehicle 3 or the surroundings of the vehicle 3.

The modification unit 102 is a functional unit that modifies the vehicle information 111. Specifically, the modification unit 102 acquires the vehicle information 111 from the acquisition unit 101, and generates vehicle information 112 (corresponding to third mobile object information) by modifying the acquired vehicle information 111. The modification unit 102 provides the generated vehicle information 112 to the detection unit 103. Specifically, the modification unit 102 modifies the latitude or longitude included in the position information as the vehicle information 111, or modifies the time corresponding to the vehicle information 111.

When generating the vehicle information 112, the modification unit 102 applies modifications to the vehicle information 111 to accommodate the failure of sensing by the sensor 51A or the like provided on the vehicle 3, thereby generating the vehicle information 112.

When generating the vehicle information 112, the modification unit 102 may generate the vehicle information 112 by making the above modification based on the frequency at which sensing by the sensor 51A, etc. equipped in the vehicle 3 actually fails. The frequency at which sensing by the sensor 51A, etc. equipped in the vehicle 3 actually fails is derived, for example, as follows.

That is, the modification unit 102 acquires vehicle information 111 (also referred to as vehicle information A) of the vehicle 3 generated using the results of sensing by a sufficient number of sensors that are not expected to fail sensing and are installed in appropriate positions where sensing is not expected to fail.

The change unit 102 also acquires vehicle information 111 (also referred to as vehicle information B) of the vehicle 3 that is generated using the results of sensing by the sensor 51A and the like equipped in the vehicle 3. Since the vehicle information B is acquired from sensor values acquired by fewer sensors than the sensors involved in acquiring the vehicle information A, it is expected that the number of entries included in the vehicle information B will be smaller than the number of entries included in the vehicle information A.

Then, the change unit 102 compares the vehicle information A with the vehicle information B to identify the difference between the vehicle information B and the vehicle information A. The frequency with which the difference between the vehicle information B and the vehicle information A thus identified occurs corresponds to the frequency with which sensing by the sensor 51A etc. equipped in the vehicle 3 actually fails. The above sensors in a sufficient number that are considered not to fail sensing may include, in addition to the sensor 51A equipped in the vehicle 3, sensors installed in other vehicles or on the travel route.

The detection unit 103 is a functional unit that uses the model 106 to output event information based on the input vehicle information 111. The detection unit 103 acquires the model 106 in advance. The model 106 is a model that outputs event information (corresponding to first event information) indicating an event that has occurred in the vehicle based on the vehicle information 111 (corresponding to first moving body information) of the vehicle input to the model 106, and is the same model as the model 505.

The detection unit 103 acquires the vehicle information 111 provided by the acquisition unit 101, and acquires event information 121 (corresponding to second event information) that is output by inputting the acquired vehicle information 111 into the model 106. The detection unit 103 provides the acquired event information 121 to the generation unit 104.

When the detection unit 103 acquires the vehicle information 112 provided by the change unit 102, the detection unit 103 acquires event information 122 (corresponding to third event information) that is output by inputting the acquired vehicle information 112 into the model 106. The detection unit 103 provides the acquired event information 122 to the generation unit 104.

The generation unit 104 is a functional unit that generates and outputs difference information indicating the difference between the event information 121 and the event information 122. The generation unit 104 transmits the difference information to the presentation device 20 through a communication interface (not shown) connected to the network N.

When generating difference information, the generating unit 104 first integrates the event information 121 and the event information 122. When integrating, if the event information 121 and the event information 122 each contain the same event that occurred at the same time and the same place, the integrated event information is generated so that these events are treated as one event. At this time, the "same time" is not limited to being the exact same time, and two times having a predetermined time difference (for example, a time difference within a few seconds) may be treated as the same time. In other words, two times having the predetermined time difference may be regarded as the same. Similarly, the "same place" is not limited to being the exact same place, and two points having a predetermined distance (for example, a distance within a few meters) may be treated as the same place. In other words, two points having the predetermined distance may be regarded as the same. Note that the latitude corresponding to 1 meter is, for example, about 10 ^-5 degrees near the equator, and the latitude corresponding to 1 meter is, for example, about 10 ^-5 degrees.

Next, the generating unit 104 generates information indicating events, excluding the events indicated in the event information 122, among the events indicated in the integrated event information, as differential information. The differential information can be obtained by subtracting the events indicated in the event information 122 from the events indicated in the integrated event information. The differential information can also be obtained as information indicating events, excluding the events indicated in the event information 122, among the events indicated in the event information 121, while equating the two times having the above-mentioned predetermined time difference and equating the two points having the above-mentioned predetermined distance. In other words, the differential information is information indicating events that have occurred in the vehicle 3 and that are no longer detected by the model 106 due to a change being made to the vehicle information 111 of the vehicle 3. It can be said that the differential information is information indicating events that may fail to be detected when the vehicle 7 moves.

The difference information may be expressed in various formats. For example, the difference information may include an image showing on a map the location where the event indicated by the difference information occurred. The difference information may also include, for example, the degree to which each of multiple factors of the event indicated by the difference information influenced the event.

The processing of the specific device 10 will be explained in more detail below.

Figure 7 is an explanatory diagram showing an example of vehicle information 111 and event information 121 acquired by vehicle 3 in this embodiment.

The vehicle information 111 shown in FIG. 7 is location information including latitude and longitude indicating the position of the vehicle 3 at each time. The vehicle information 111 shown in FIG. 7 is location information generated by processing, by the processing unit 52, GPS data acquired by the GPS receiver, which is the sensor 51A equipped in the vehicle 3, for example. Note that the vehicle information 111 shown in FIG. 7 is vehicle information extracted when an event occurs in the vehicle 3 from the vehicle information repeatedly acquired by the sensor 51A equipped in the vehicle 3.

The event information 121 of vehicle 3 shown in FIG. 7 is information indicating the content of an event that occurred in vehicle 3. The event information 121 of vehicle 3 shown in FIG. 7 is event information 121 that is output by inputting the vehicle information 111 shown in FIG. 7 to the detection unit 103 (model 106).

For example, the vehicle information in the first line shown in FIG. 7 (also referred to as vehicle information #1) indicates that vehicle 3 is located at "latitude 34.838968 degrees, longitude 135.673072 degrees" at the time "March 30, 2020, 4:53:23:061953."

The event information in the first row shown in FIG. 7 (also referred to as event information #1) indicates that at the time indicated in vehicle information #1, a "running out" event occurred in vehicle 3, more specifically, a person running out in the direction of travel of vehicle 3.

Figure 8 is an explanatory diagram showing an example of vehicle information 112 and event information 122 that have been modified by the modification unit 102 in this embodiment.

The vehicle information 112 shown in FIG. 8 is the vehicle information 112 in which the modification unit 102 has made modifications to the vehicle information 111 shown in FIG. 7 (i.e., the latitude and longitude indicating the position of the vehicle 3 at each time).

Specifically, vehicle information 112 shown in FIG. 8 is vehicle information 111 shown in FIG. 7 that has been modified such that (a) vehicle information #2, #4, and #6 are deleted, (b) time information of vehicle information #1, #3, #5, and #7 is changed, (c) latitude of vehicle information #5 and #7 is changed, and (d) longitude of vehicle information #5 is changed.

Furthermore, the event information 122 of vehicle 3 shown in FIG. 8 is the event information 122 output by inputting the vehicle information 112 shown in FIG. 8 to the detection unit 103 (model 106).

For example, vehicle information #1 shown in FIG. 8 indicates that vehicle 3 is located at latitude 34.838968 degrees, longitude 135.673072 degrees at time "March 30, 2020, 4:53:24:061953."

Furthermore, event information #1 shown in FIG. 8 indicates that a "jump-out" event occurred in vehicle 3, in other words, a person jumping out in the direction of travel of vehicle 3.

Figure 9 is an explanatory diagram showing an example of integrated event information in this embodiment.

The integrated event information shown in FIG. 9 is generated by integrating the event information shown in FIG. 7 and the event information shown in FIG. 8 by the generation unit 104.

When integrating, the generation unit 104 leaves as is the event information that exists only before the change, and sets the "Detection after change" column to "No."

On the other hand, when integrating, the generation unit 104 leaves the changed values for the vehicle information and event information that exist before and after the change, and sets the "detection after change" column to "Yes."

The integrated event information can be said to be information indicating whether the event information detected from the vehicle information before the change is detected from the vehicle information after the change.

Note that when integrating, event information that exists only after a change is not included in the integrated event information. This is because event information that exists only after a change has no particular meaning in terms of detection by the detection system 5.

Figure 10 is an explanatory diagram showing an example of difference information in an embodiment.

The difference information shown in FIG. 10 indicates event information that exists before the change but does not exist after the change at the time of integration (i.e., event information for which the "Detected after change" column in FIG. 9 is "No"). Specifically, difference information #1, #2, and #3 in FIG. 10 are extracted from integrated event information #2, #4, and #6 in FIG. 9, respectively.

The difference information shown in FIG. 10 corresponds to difference information indicating the difference between the event information indicating the event that occurred in vehicle 3 and the event information after the change by the change unit 102, and more specifically, it can be said to indicate the events indicated in the event information indicating the event that occurred in vehicle 3, excluding the events indicated in the event information after the change by the change unit 102.

Figure 11 is an explanatory diagram showing a first example of output information in this embodiment.

The output information shown in FIG. 11 is an example of an image (image 61) that shows on a map the location where the event indicated by the difference information occurred.

Image 61 shows a figure 62 indicating the location where the events indicated by the difference information occurred. The figure 62 is displayed with a darker color the more events occurred at the location.

Image 61 is generated by identification device 10, transmitted to presentation device 20, and displayed by presentation device 20. By visually checking image 61, observer Q can grasp the locations where events frequently occur as locations on a map.

Having identified the locations where such events frequently occur, the monitor Q is expected to take measures to prevent the events indicated in the event information from actually occurring. Specifically, the monitor Q is expected to take measures to install guardrails to prevent people or objects from running out onto the road in locations where there are many cases of people or objects running out onto the road.

Figure 12 is an explanatory diagram showing a second example of output information in this embodiment.

The output information shown in FIG. 12 includes the degree to which each of the multiple factors of the event indicated by the difference information influenced the occurrence of the event.

Specifically, the output information shown in Figure 12 indicates the degree to which each of the factors that caused the event, "number of passengers," "passenger flow rate," and "vehicle flow rate," influenced the occurrence of the event, expressed as a numerical value with the sum of these being 1.

The degree to which each factor influenced the occurrence of an event can be calculated, for example, by correlating factors that may be related to whether or not an event occurs with whether or not the event occurs. In addition, the degree to which each factor influenced the occurrence of an event can be calculated, for example, as the standard partial regression coefficient of the results of a regression analysis using factors that may be related to whether or not an event occurs.

For example, Figure 12 shows an example in which the degree to which the factor "number of passengers" influenced the occurrence of the event was 0.4, the degree to which the factor "people flow rate" influenced the occurrence of the event was 0.3, and the degree to which the factor "vehicle flow rate" influenced the occurrence of the event was 0.3.

By visually determining the degree to which each factor influenced the occurrence of an event in this way, someone can take measures to make the event less likely to actually occur.

The processing of the specific system 1 configured as described above will now be explained.

Figure 13 is a flow diagram showing the processing of a specific system 1 in an embodiment.

In step S201, the sensors 51A and the like of the vehicle 3 acquire sensor values by sensing the vehicle 3 or the surroundings of the vehicle 3.

In step S202, the processing unit 52 of the vehicle 3 processes the sensor value acquired in step S201 to acquire the vehicle information 111. The acquisition unit 101 of the identification device 10 acquires the vehicle information 111 acquired by the processing unit 52.

In step S203, the change unit 102 of the specific device 10 generates vehicle information 112 (see FIG. 6) by making changes to the vehicle information (corresponding to vehicle information 111, see FIG. 6) acquired by the acquisition unit 101 in step S202.

In step S204, the detection unit 103 of the identification device 10 inputs the vehicle information 111 acquired by the acquisition unit 101 in step S202 into the model 106 to acquire event information 121 (see FIG. 6).

In step S205, the detection unit 103 of the identification device 10 inputs the vehicle information 112 (see FIG. 6) modified by the modification unit 102 in step S203 into the model 106 to obtain event information 122 (see FIG. 6).

In step S206, the generation unit 104 of the identification device 10 generates difference information indicating the difference between the event information 121 acquired by the detection unit 103 in step S204 and the event information 122 acquired by the detection unit 103 in step S205.

In step S207, the generation unit 104 of the specific device 10 outputs the difference information generated in step S206.

In this way, the identification system 1 can appropriately identify and output events that may occur to a moving object.

As described above, according to the identification method (i.e., the information output method) of this embodiment, it is possible to output differential information indicating events that may occur in a moving body and that may fail to be detected during actual movement. The output differential information indicates a change in the event information caused by making a change to the second moving body information. Furthermore, the change made to the second moving body information may correspond to an error or deficiency contained in the information acquired by the sensor that may occur during actual movement. Therefore, it can be said that the output differential information indicates an event that may not be detectable due to an error or deficiency in the information that can be acquired by the sensor during actual movement. In this way, the above-mentioned information output method can appropriately identify and output an event that may occur in a moving body by outputting differential information indicating events that may fail to be detected during actual movement among events that occur in a moving body.

In addition, since the change made to the second moving body information corresponds to a failure of sensing by the sensor equipped in the second moving body, the output difference information is difference information indicating an event that may fail to be detected during actual movement. Therefore, the above information output method can more appropriately identify and output an event that may occur in the moving body.

In addition, because the second moving body information is modified based on the frequency with which sensing by the sensor equipped in the second moving body fails, the output differential information takes into account the actual frequency of failures and becomes differential information indicating events that may fail to be detected during actual movement. Therefore, the above-mentioned information output method can more appropriately identify and output events that may occur in the moving body.

In addition, by excluding the events indicated in the third event information from the events indicated in the second event information, difference information can be easily generated. Therefore, the above information output method can more easily appropriately identify and output events that may occur in a moving body.

In addition, since the mobile body information is generated using sensor values generated by sensors installed along the moving path in addition to the sensors equipped on the mobile body, the generated mobile body information becomes more appropriate. This also improves the appropriateness of the event information and difference information. Therefore, the above information output method can more appropriately identify and output events that may occur to the mobile body.

In addition, difference information is generated using the moving body information, such as position information or speed information of the moving body, the presence or absence of obstacles around the moving body, or position information or speed information of obstacles around the moving body. Therefore, the above information output method can more easily appropriately identify and output events that may occur to the moving body.

In addition, based on the moving body information, the model detects events such as a person or object jumping out in the direction of movement of the moving body, a person or object being caught when the moving body changes direction, a collision when the moving body merges with another moving body, another moving body cutting in on the moving body's direction of movement, or a deviation of the moving body's path of movement. Therefore, the above information output method can more easily appropriately identify and output events that may occur to a moving body.

In addition, the location where an event occurs that may not be detectable during actual movement is shown on a map. Therefore, a person viewing the map can easily grasp the event on the map, and it is expected that measures will be taken to make the event less likely to actually occur. Therefore, the information output method appropriately identifies and outputs events that may occur to a moving body, while also leading to measures to reduce the detection of the event.

In addition, the degree to which each of the multiple factors of the event influenced the occurrence of the event is presented. Therefore, it is expected that a person who visually recognizes the above-mentioned degrees will take measures to make the event less likely to actually occur, or to make it less likely that the event will be detected. Therefore, the above-mentioned information output method can appropriately identify and output an event that may occur in a mobile object, while also leading to measures to reduce the detection of the event.

In addition, by using neural network models, it is easier to more accurately predict events that occur in moving objects.

In the above embodiment, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that realizes the specific device (i.e., information output device) of the above embodiment is a program such as the following.

That is, this program causes a computer to execute an information output method that acquires a model that outputs first event information indicating an event that occurred in a first moving body based on first moving body information, which is information about a first moving body input to the model, acquires second moving body information, which is information about a second moving body, generates third moving body information by modifying the acquired second moving body information, acquires information indicating an event that occurred in the second moving body as second event information that is output by inputting the acquired second moving body information into the model, acquires information indicating an event that is output by inputting the generated third moving body information into the model as third event information, and generates and outputs difference information indicating the difference between the second event information and the third event information.

Although the identification method and the like relating to one or more aspects have been described based on the embodiment, the present invention is not limited to this embodiment. As long as it does not deviate from the spirit of the present invention, various modifications conceived by a person skilled in the art to this embodiment and forms constructed by combining components in different embodiments may also be included within the scope of one or more aspects.

The present invention can be used in a prediction device that predicts events that may occur in a vehicle.

REFERENCE SIGNS LIST 1 Identification system 3, 7 Vehicle 5 Detection system 10 Identification device 20, 30 Presentation device 50 Detection device 51A, 51B Sensor 52 Processing unit 53 Providing unit 61 Image 62 Figure 101, 501 Acquisition unit 102 Change unit 103, 502 Detection unit 104 Generation unit 106, 505 Model 111, 112 Vehicle information 121, 122 Event information 503 Output unit N Network P, Q Observer

Claims (11)

A model is obtained that outputs first event information indicating an event that has occurred on a first moving object based on first moving object information that is information about the first moving object input to the model;
Acquire second mobile object information, which is information about the second mobile object;
generating third mobile object information by modifying the acquired second mobile object information;
acquiring, as second event information, information indicating an event that has occurred in the second moving object, which is output by inputting the acquired second moving object information into the model;
acquiring, as third event information, information indicating an event that is output by inputting the generated third moving object information into the model;
generating and outputting difference information indicating a difference between the second event information and the third event information ;
The second moving object information includes information generated from a sensor value generated by sensing a sensor provided in the second moving object during a movement of the second moving object,
When generating the third mobile object information,
The change corresponding to the failure of sensing by the sensor of the second moving body is applied to the second moving body information, thereby generating the third moving body information.
Information output method. Furthermore, a frequency of failure of the sensing by the sensor of the second moving object is measured;
When generating the third mobile object information,
The information output method according to claim 1 , further comprising the step of: generating the third mobile object information by making the change based on the measured frequency. When generating the difference information,
The information output method according to claim 1 , further comprising generating the difference information indicating events indicated in the second event information excluding the events indicated in the third event information. The first moving object information further includes information generated from a sensor value generated by sensing a sensor installed on a moving path of the first moving object,
The information output method according to claim 1 , wherein the second moving object information further includes information generated from a sensor value generated by sensing using a sensor installed on a moving path of the second moving object. The first mobile information is
position information or speed information of the first moving body, the presence or absence of obstacles around the first moving body, position information or speed information of obstacles around the first moving body, the number of passengers on the first moving body, the amount of people flowing around the first moving body, or the amount of vehicle flowing around the first moving body,
The second mobile information is
The information output method according to any one of claims 1 to 4, including position information or speed information of the second moving body, the presence or absence of obstacles around the second moving body, position information or speed information of obstacles around the second moving body, the number of passengers on the second moving body, the amount of people flowing around the second moving body, or the amount of vehicle flowing around the second moving body. The information output method according to any one of claims 1 to 5, wherein the event includes a person or object jumping out in the direction of movement of a moving body that is the first moving body or the second moving body, a person or object being caught up in the moving body when changing its direction of travel, a collision when the moving body merges with another moving body, another moving body cutting in on the moving body in the direction of travel, deviation of the moving body's path of travel, a collision caused by a vehicle and a pedestrian starting to move simultaneously in a crossing area without traffic lights , or intentional obstruction around the vehicle. The information output method according to claim 1 , wherein the difference information includes an image showing, on a map, a position where an event indicated by the difference information occurred. The information output method according to claim 1 , wherein the difference information includes a degree to which each of a plurality of factors of the event indicated by the difference information influenced the occurrence of the event. The information output method according to any one of claims 1 to 8 , wherein the model is a neural network model. The apparatus includes an acquisition unit, a change unit, a detection unit, and a generation unit,
The detection unit acquires a model that outputs first event information indicating an event that has occurred to the first moving object based on first moving object information that is information about the first moving object input to the model;
The acquisition unit acquires second mobile object information which is information related to a second mobile object;
The change unit generates third mobile object information by modifying the second mobile object information acquired by the acquisition unit,
The detection unit acquires, as second event information, information indicating an event that has occurred in the second moving object, which is output by inputting the second moving object information acquired by the acquisition unit into the model;
The detection unit acquires, as third event information, information indicating an event that is output by inputting the third moving object information generated by the change unit into the model;
the generation unit generates and outputs difference information indicating a difference between the second event information and the third event information ;
The second moving object information includes information generated from a sensor value generated by sensing a sensor provided in the second moving object during a movement of the second moving object,
When generating the third mobile object information, the change unit
The change corresponding to the failure of sensing by the sensor of the second moving body is applied to the second moving body information, thereby generating the third moving body information.
Information output device. A program for causing a computer to execute the information output method according to any one of claims 1 to 9 .