JP7812697B2 - Accident analysis device, accident analysis method, and program - Google Patents

Description

The present invention relates to an accident analysis device, an accident analysis method, and a program.

An accident analysis device has been proposed that acquires vehicle data measured by sensors installed in the accident vehicle and video data captured by a camera installed in the accident vehicle, and analyzes the circumstances of the accident caused by the accident vehicle based on the acquired vehicle data and video data (for example, Patent Document 1).

Patent No. 6679152

In accident analysis devices such as those described in Patent Document 1, it is desirable to improve the accuracy of analysis of accident situations.

The present invention therefore aims to provide an accident analysis device that improves the accuracy of accident situation analysis.

In order to achieve the above object, the accident analysis device of the present invention comprises:
an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle;
an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and the acquired video data,
The analysis unit
If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs;
A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of the occurrence of an accident is determined based on the audio data from which the frequency corresponding to a human scream has been removed .

If there are multiple times in the audio data included in the video data when the volume is higher than the determination value, the analysis unit may determine that the time when the volume is greatest is the time when the accident occurred.

If there are multiple times when the volume of the audio data included in the video data is higher than the threshold value, the analysis unit may determine that the first time the volume exceeds the threshold value is the time when the accident occurred.

The analysis unit may determine the severity of the accident based on audio data included in the video data.

Another accident analysis device of the present invention includes:
an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle;
an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and the acquired video data,
The analysis unit
If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs;
A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of the occurrence of an accident is determined based on the audio data from which the frequency corresponding to a human scream has been removed.

The accident analysis method of the present invention also includes:
An accident analysis method using an accident analysis device,
an acquisition step of acquiring vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle;
an analysis step of analyzing the circumstances of the accident caused by the vehicle based on the acquired vehicle data and the acquired video data,
In the analyzing step,
If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs;
If there is no timing when the volume of the audio data included in the video data is higher than the determination value, the timing when the volume is at its maximum is determined to be the timing when the accident occurred .
Another self-analysis method of the present invention includes:
An accident analysis method using an accident analysis device,
an acquisition step of acquiring vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle;
an analysis step of analyzing the circumstances of the accident caused by the vehicle based on the acquired vehicle data and the acquired video data,
In the analyzing step,
If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs;
A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of the occurrence of an accident is determined based on the audio data from which the frequency corresponding to a human scream has been removed.

The program of the present invention also includes:
Computer,
an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle;
an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and video data;
The analysis unit
If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs;
If there is no timing when the volume of the audio data included in the video data is higher than the determination value, the timing when the volume is at its maximum is determined to be the timing when the accident occurred .
Another program of the present invention is
Computer,
an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle;
an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and video data;
The analysis unit
If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs;
A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of the occurrence of an accident is determined based on the audio data from which the frequency corresponding to a human scream has been removed.

This invention can improve the accuracy of accident situation analysis.

1 is a diagram illustrating an example of an accident analysis system according to an embodiment of the present invention. FIG. 2 is a schematic diagram showing the state of the present embodiment from inside the vehicle facing forward. FIG. 1 is a functional block diagram of a drive recorder according to an embodiment of the present invention. 1 is a diagram illustrating an example of a hardware configuration of an accident analysis device according to an embodiment of the present invention. 1 is a functional block diagram of an accident analysis device according to an embodiment of the present invention. 10 is a flowchart illustrating an example of an accident situation analysis process. 10 is a flowchart illustrating an example of a collision timing identification process. 10 is a flowchart illustrating an example of a collision object identification process. 10A and 10B are diagrams illustrating a method for determining whether a vehicle has rolled over. FIG. 10 is a diagram illustrating an example of identifying a collision target. FIG. 10 is a diagram illustrating an example of identifying a collision target. FIG. 10 is a diagram illustrating an example of information indicating the status of an accident output by the accident analysis device. 10 is a diagram for explaining a processing procedure when the accident analysis device detects surrounding objects and identifies their coordinates. FIG. 4 is a diagram for explaining a processing procedure when the accident analysis device estimates the relative positional relationship between a vehicle and a surrounding object. FIG. 4 is a diagram for explaining a processing procedure when the accident analysis device estimates the absolute positions of a vehicle and a surrounding object. FIG. FIG. 10 is a diagram for explaining a process for estimating the absolute position of a vehicle. FIG. 10 is a diagram illustrating an example of an image generated by the accident analysis device. FIG. 10 is a diagram for explaining a collision direction. FIG. 10 is a diagram illustrating a process for determining a contact object. FIG. 10 is a diagram for explaining a contact portion. 10A and 10B are diagrams for explaining a process for identifying the positional relationship between a pedestrian and a crosswalk/safety zone. FIG. 2 is a diagram for explaining the traveling directions of a vehicle and other vehicles. FIG. 2 is a diagram illustrating an example of an accident case DB.

Embodiments of the present invention will be described below with reference to the drawings. Note that identical or corresponding parts in the drawings are designated by the same reference numerals.

Figure 1 is a diagram showing an example of an accident analysis system according to this embodiment. In the accident analysis system 1 according to this embodiment, a drive recorder 1100 (see Figure 2) is installed in the policyholder's vehicle 1000, and the drive recorder 1100 is connected to a cloud storage server (not shown) via a network (including a public line network) via wireless communication, and data acquired by the drive recorder 1100 (video data and vehicle data) is stored in the cloud storage server. The network is also connected to an accident analysis device 10 and a terminal 20. The accident analysis system 1 includes at least the drive recorder 1100 and the accident analysis device 10.

In this embodiment, an example is shown in which the drive recorder 1100 connects to a cloud storage server (not shown) via a network (including a public line network) via wireless communication, but for example, a dedicated mirror-type terminal device with similar functionality may be prepared and installed in place of the mirror 1300.

A data storage area for this customer service is secured on the cloud storage server. More specifically, a data storage area that can be accessed by the general center server 30 or the like is secured for each drive recorder 1100 installed in the vehicle 1000.

The accident analysis device 10 has the function of acquiring vehicle data measured by sensors equipped on the accident vehicle 1000 (hereinafter sometimes referred to as the "own vehicle") or the drive recorder 1100, and video data captured by a camera equipped on the drive recorder 1100 of the vehicle 1000, from a cloud storage server via a network, and analyzing the circumstances of the accident caused by the vehicle 1000 based on the acquired vehicle data and video data. Note that while this embodiment describes an example in which the vehicle data and video data are acquired from a cloud storage server, the vehicle data and video data may also be acquired directly from the vehicle.

The accident analysis device 10 also has a function to search for accident cases corresponding to the circumstances of the accident caused by the vehicle 1000 by comparing the circumstances of the accident obtained by the analysis with an accident case database that associates the circumstances of past accidents with information on past accident cases (hereinafter referred to as "accident cases"). The accident analysis device 10 also has a function to generate an image (reconstruction) showing the circumstances of the accident caused by the vehicle 1000 by mapping the position of the vehicle 1000 and the positions of other vehicles obtained by analyzing the circumstances of the accident on map data. The image showing the circumstances of the accident caused by the vehicle 1000 may be of any type, such as a bird's-eye view or a video.

The accident analysis device 10 may be configured using one or more physical information processing devices, or may be configured using a virtual information processing device running on a hypervisor, or may be configured using a cloud server.

The terminal 20 is operated by, for example, an insurance company operator, and displays the accident situation analyzed by the accident analysis device 10, accident cases corresponding to the accident conditions, and images generated by the accident analysis device 10. The terminal 20 can be any information processing device with a display, such as a personal computer (PC), notebook PC, tablet terminal, or smartphone.

The drive recorder 1100 may be connected via a network to a server at a general center that provides roadside assistance and emergency services. In this case, for example, the general center's server may be composed of one or more computers and connected to a terminal device (not shown) installed at the general center and operated by each operator at the general center. The general center's operator may be able to communicate with the drive recorder 1100 installed in the vehicle 1000, roadside assistance companies, and emergency services (fire departments and private emergency service companies) using, for example, a headset connected to the terminal device. The network may also be connected to systems for roadside assistance companies and emergency services, which may retrieve data from designated data storage areas in the cloud storage server in response to requests from the general center's server.

2 shows the state of the vehicle 1000 from inside the vehicle 1000, looking forward. The drive recorder 1100 includes a camera, which is attached to the front of the vehicle 1000 or to the windshield as shown in the figure, so as to capture at least video in the direction of travel of the vehicle 1000. The camera may also be capable of capturing video from the side and rear of the vehicle 1000. As shown in FIG. 2, the vehicle 1000 is equipped with an automatic diagnostic system 1400 (e.g., an OBD2 (On-Board Diagnostic system-II)) inside, and the automatic diagnostic system 1400 is connected to the drive recorder 1100. The automatic diagnostic system 1400 is connected to the control units (e.g., ECUs (Electronic Control Units)) of each part of the vehicle 1000 (engine, accelerator, brake, turn signal, etc.) via a CAN (Controller Area Network) or the like, and is able to obtain information on the status of each part (presence or absence of accelerator or brake operation, presence or absence of a malfunction, speed, engine speed, etc.).

Figure 3 shows a functional block diagram of the drive recorder 1100 according to this embodiment. The drive recorder 1100 installed in the vehicle 1000 has a first communication unit 1110, a second communication unit 1120, a positioning unit 1130, a video recording unit 1140, a sound recording unit 1150, an acceleration measurement unit 1160, an automatic diagnosis data acquisition unit 1170, a control unit 1180, a sensor unit 1190, and a data storage unit 1200.

The control unit 1180 is composed of processors such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), and causes each component to execute processing related to this customer service. The control unit 1180 transmits measurement data (vehicle data and video data) from each part of the drive recorder 1100 to a cloud storage server via the second communication unit 1120. As a result, the vehicle data and video data of the vehicle 1000 acquired by the drive recorder 110 are stored in the cloud storage server.

The first communication unit 1110 is a communication unit for communicating with a general center, for example, via VoIP (Voice over Internet Protocol). However, it may also be the communication function of a general mobile phone. The second communication unit 1120 has the function of transmitting data to a cloud storage server, for example, via wireless communication (public line network).

Furthermore, when instructed by the control unit 1180, the positioning unit 1130 acquires the absolute position (e.g., latitude and longitude) of the vehicle 1000 using, for example, the Global Positioning System (GPS), and stores it in the data storage unit 1200.

The recording unit 1140 stores video data (video data) captured by a camera mounted on the drive recorder 1100 in the data storage unit 1200. The sound recording unit 1150 stores audio data (audio data) input from a microphone in the data storage unit 1200. The sound recording unit 1150 may be integrated with the recording unit 1140. Furthermore, the recording unit 1140 continuously records and stores the data in the data storage unit 1200 when instructed to do so by the control unit 1180, for example. The control unit 1180 is capable of extracting both video data from a certain period before a specific time (such as the timing of an accident) and video data from a certain period after that specific time. The same applies to the sound recording unit 1150. The recording unit 1140 is capable of capturing video images of both the exterior and interior of the vehicle 1000. In this embodiment, the video data is described as including audio data, but video data may also refer to video data that does not include audio.

The acceleration measurement unit 1160 measures acceleration values using, for example, an acceleration sensor and outputs the values to the control unit 1180. When instructed by the control unit 1180, the automatic diagnosis data acquisition unit 1170 acquires automatic diagnosis data from the automatic diagnosis system 1400 installed inside the vehicle 1000. The automatic diagnosis data includes data indicating the presence or absence of damage to the engine, battery, fuel system, etc. The automatic diagnosis data also includes vehicle data used to recreate the accident situation, such as control information for each part of the vehicle 1000 (accelerator, brake, steering wheel, turn signal), and operation information (vehicle speed, throttle opening (accelerator opening), brake operation information, steering wheel operation information, and turn signal operation status, etc.). The vehicle data includes at least position information and acceleration information of the vehicle 1000. In this embodiment, the vehicle data also includes vehicle speed data acquired from the vehicle. The vehicle data also includes information related to the driver, such as brake operation information and steering wheel operation information.

The sensor unit 1190 includes a driver's gaze sensor composed of a camera or the like, and a breath sensor that detects the driver's breath. The gaze sensor is composed of a camera that captures an image of the driver and an image processing device or the like, and is capable of detecting the driver's gaze (gaze direction), outputting the detected driver's gaze (gaze direction) information to the control unit 1180. The breath sensor detects the alcohol concentration (ethanol concentration) in the driver's breath (or the air inside the vehicle), and outputs the detected alcohol concentration information to the control unit 1180. That is, in this embodiment, the vehicle data includes information about the driver, such as the driver's gaze (gaze direction) and information about the driver's breath (whether or not they have been drinking). In this embodiment, the drive recorder 1100 is configured to include a sensor unit 1190 that includes a gaze sensor, a breath sensor, etc. However, the drive recorder 1100 may also acquire vehicle data such as information regarding the driver's gaze (direction of gaze) and the driver's breath (whether or not they have been drinking) from sensors (such as a gaze sensor or a breath sensor) included in or installed in the vehicle 1000.

The sensor unit 1190 may include, for example, a vehicle speed sensor, a geomagnetic sensor, a throttle sensor, and/or a turn signal detection sensor. Furthermore, the vehicle data measured by the sensors provided in the vehicle 1000 may be, for example, the speed of the vehicle 1000, the direction in which the vehicle 1000 is facing (for example, the angle when north is 0 degrees), the throttle opening (accelerator opening), and the operation status of the turn signals. At least some of this data may be acquired by the automatic diagnosis data acquisition unit 1170 from the automatic diagnosis system 1400 (sensors provided in the vehicle 1000).

The data storage unit 1200 is composed of storage devices such as memory, HDD (Hard Disk Drive) and/or SSD (Solid State Drive), and stores in advance equipment data such as company name, organization name, vehicle registration number, driver identifier (ID), and telephone number, and also stores data acquired by each component in response to instructions from the control unit 1180.

Figure 4 is a diagram showing an example hardware configuration of the accident analysis device 10. The accident analysis device 10 has a processor 11 such as a CPU (Central Processing Unit) or GPU (Graphical Processing Unit), a storage device 12 such as memory, HDD (Hard Disk Drive) and/or SSD (Solid State Drive), a communication IF (Interface) 13 for wired or wireless communication, an input device 14 for accepting input operations, and an output device 15 for outputting information. The input device 14 is, for example, a keyboard, touch panel, mouse, and/or microphone. The output device 15 is, for example, a display and/or speaker.

Figure 5 is a functional block diagram of the accident analysis device 10 according to this embodiment. The accident analysis device 10 includes a memory unit 100, an acquisition unit 101, an analysis unit 102, a generation unit 103, a search unit 104, and an output unit 105. The memory unit 100 can be implemented using the memory unit 12 included in the accident analysis device 10. The acquisition unit 101, the analysis unit 102, the generation unit 103, the search unit 104, and the output unit 105 can be implemented by the processor 11 of the accident analysis device 10 executing a program stored in the memory unit 12. The program can be stored on a storage medium. The storage medium storing the program may be a non-transitory computer-readable medium. The non-transitory storage medium is not particularly limited, and may be, for example, a USB (Universal Serial Bus) memory, a CD-ROM, or other storage medium.

The memory unit 100 stores an accident case database (DB) that associates accident circumstances with accident case examples, and a map data DB. The accident case DB may also include information indicating the degree of fault. The accident case and degree of fault corresponding to the circumstances of an accident that occurred may be searchable using information indicating the accident circumstances as a key. The map data DB includes various data such as road data, road width, direction of travel, road type, traffic signs (stop sign, no entry, etc.), speed limits, traffic light locations, and the number of intersecting roads at intersections. The memory unit 100 may also be implemented as an external server that can communicate with the accident analysis device 10. Although not shown in the figure, the memory unit 100 also stores operator information for insurance company operators.

The acquisition unit 101 has the function of acquiring vehicle data and video data measured and captured by the vehicle 1000 (accident vehicle) from a cloud storage server.

The analysis unit 102 has the function of analyzing the circumstances of an accident caused by the vehicle 1000 based on the vehicle data and video data acquired by the acquisition unit 101.

The analysis unit 102 has the function of determining the timing of an accident based on the audio data contained in the video data.

The analysis unit 102 also has the function of estimating the collision direction of the vehicle in the event of an accident based on the acceleration data included in the vehicle data, and identifying, as the collision target, surrounding objects included in the video data within a predetermined time period before and after the determined timing of the accident, those that are in the collision direction and whose distance to the vehicle 1000 is decreasing.

In addition, the analysis unit 102 determines whether the vehicle overturned at the time of the accident based on the vertical acceleration data included in the vehicle data, and if it determines that the vehicle overturned, it does not use the acceleration data after the overturn in analyzing the circumstances of the accident.

The analysis unit 102 also has the function of analyzing the driver's condition at the time of the accident as the accident circumstances based on information about the driver contained in the vehicle data.

Furthermore, the vehicle data of vehicle 1000 includes at least information indicating the absolute position of vehicle 1000 measured by the GPS of positioning unit 1130, and analysis unit 102 may estimate the absolute position of the other vehicle based on the information indicating the absolute position of vehicle 1000 (position data acquired by positioning unit 1130) and information indicating the relative positional relationship between vehicle 1000 and the other vehicle, obtained by analyzing images of the other vehicle captured in the video data. Furthermore, analysis unit 102 may estimate the absolute positions of vehicle 1000 and the other vehicle in chronological order.

For example, the analysis unit 102 estimates the time series of the absolute position and orientation of the vehicle 1000 from information indicating the absolute position of the vehicle 1000 measured by GPS and the movement of feature points in each frame included in the video data. That is, the analysis unit 102 integrates information indicating the absolute position of the vehicle 1000 measured by GPS and information indicating the relative position obtained from the video data to estimate the time series of the absolute position and orientation of the vehicle 1000.

The analysis unit 102 also estimates, for example, (1) "depth of the target = depth of the target in the direction directly ahead of the camera" using the formula "depth of the other vehicle (target) = focal length of the camera × real-world height of the assumed other vehicle (target) / height of the other vehicle (target) in the image." (2) Based on the image coordinates (X, Y) of the target center, converts "depth" into "distance = straight-line distance between the optical center of the camera and the target." Specifically, the distance may be calculated by extending a vector from the optical center of the camera in the direction of the target until it reaches the depth calculated in (1) and calculating the length of that vector.

The analysis unit 102 may also calculate the angle between the direction directly ahead of the camera provided in the drive recorder 1100 and the direction from the optical center of the camera to the other vehicle (target), and transfer the vector in the camera coordinate system pointing from the camera to the other vehicle (target) to a world coordinate system (latitude and longitude) using GPS to analyze the circumstances of the accident.

The analysis unit 102 may also analyze the circumstances of the accident by estimating the absolute position of the vehicle 1000 by analyzing the video data, and averaging the absolute position of the accident vehicle estimated by analyzing the video data and the absolute position of the vehicle 1000 measured by the positioning unit 1130 based on a predetermined weight, and regarding this absolute position as the absolute position of the vehicle 1000.

The analysis unit 102 may also analyze the circumstances of the accident by comparing the image size of the portion of the video data in which the other vehicle is captured with data indicating the correspondence between image size and distance, and estimating the distance between the vehicle 1000 and the other vehicle, which is one piece of information indicating the relative positional relationship between the vehicle 1000 and the other vehicle.

The analysis unit 102 may also analyze the circumstances of the accident by comparing the difference between the coordinates of the point in the video data where the other vehicle is captured and the center coordinates of the video data with data indicating the correspondence between coordinates and angles, and estimating the angular difference between the direction of travel of the vehicle 1000 and the direction in which the other vehicle is located, which is one piece of information indicating the relative positional relationship between the vehicle 1000 and the other vehicle.

The accident circumstances analyzed by the analysis unit 102 may include at least the color of the traffic light when the vehicle 1000 passes through the intersection, the priority relationship when passing through the intersection, and whether the vehicle 1000's speed exceeds the speed limit.

The generation unit 103 has a function of generating an image (reconstruction) showing the situation of an accident caused by the vehicle 1000 by mapping the absolute position of the vehicle 1000 and the absolute position of the other vehicle onto map data. The image may include an overhead view or a video. For example, the generation unit 103 may generate a video in which images showing the situation of the accident are arranged in chronological order. Note that if there is no other vehicle involved in the accident, the generation unit 103 generates an image showing the situation of the accident caused by the vehicle 1000 by mapping at least the absolute position of the vehicle 1000 onto map data. In this case, surrounding objects of the collision target (people, utility poles, guardrails, etc.) may be mapped onto the map data.

The search unit 104 has the function of searching for accident cases corresponding to the circumstances of the accident caused by the vehicle 1000 by comparing the circumstances of the accident caused by the vehicle 1000 analyzed by the analysis unit 102 with an accident case database (accident case data) that associates the accident circumstances with past accident cases. The search unit 104 may also search for the fault ratio of the vehicle 1000 in the accident caused by the vehicle 1000 as an accident case corresponding to the circumstances of the accident caused by the vehicle 1000.

Furthermore, an accident case may include one or more correction ratios (correction information) for correcting the fault ratio. If one or more correction ratios include a correction ratio that corresponds to the circumstances of the accident caused by the accident vehicle, the search unit 104 may correct the fault ratio of the accident vehicle in accordance with that correction ratio.

The output unit 105 has the function of outputting to the terminal 20 information indicating the accident situation analyzed by the analysis unit 102, accident cases corresponding to the accident conditions searched for by the search unit 104, and images indicating the accident situation generated by the generation unit 103.

Note that the functions of the accident analysis device 10 may be realized in cooperation with the terminal 20, and therefore some functions may be provided on the terminal 20 side. Furthermore, the accident analysis device 10 and the terminal 20 may be integrated into a single information processing device.

Figure 6 is a flowchart showing an example of an accident situation analysis process in which the accident analysis device 10 analyzes the circumstances of an accident. First, the accident analysis device 10 acquires vehicle data and video data from the vehicle 1000 involved in the accident at the time of the accident (a predetermined period before and after the accident) (S10, S11). Specifically, the accident analysis device 10 acquires the vehicle data and video data from a cloud storage server via a network. Alternatively, for example, the vehicle data and video data may be transmitted to the accident analysis device 10 via a wireless signal using a communication function provided in the vehicle 1000 or the drive recorder 1100. Furthermore, the vehicle data and video data may be imported into the accident analysis device 10 by connecting a storage medium on which the vehicle data and video data are stored to the accident analysis device 10.

Next, the accident analysis device 10 performs image analysis on the video data at the time of the accident for each frame to identify one or more surrounding objects (other vehicles, people, signs, road structures, etc.) that are present around the vehicle 1000 that are captured in each frame of the image, and further identifies the coordinates (coordinates in the image) that indicate the position of the identified surrounding objects in the image (S12). Next, the accident analysis device 10 estimates the relative positional relationship between the vehicle 1000 and the surrounding objects present around the vehicle 1000 for each surrounding object based on the coordinates (depth of the surrounding object) of the identified surrounding objects (S13). Note that the terms "identified" and "estimated" are used here, and while "identified" may be used when there is a higher probability, it does not necessarily mean that the object can be "identified," and there is no significant difference in meaning from "estimated." This is the same for this embodiment.

Next, the accident analysis device 10 estimates the absolute position of the vehicle 1000 using the position information of the vehicle 1000 acquired by the positioning unit 1130 and/or video data at the time of the accident. The accident analysis device 10 also estimates the absolute positions of surrounding objects around the vehicle 1000 based on the estimated absolute position of the vehicle 1000 and the relative positional relationship between the vehicle 1000 and surrounding objects around the vehicle 1000 estimated in the processing procedure of step S13 (S14).

Steps S10 to S14 will be described in detail below.

Next, the accident analysis device 10 executes a collision timing identification process to identify (determine) the collision timing between the vehicle 1000 and the collision object (S15).

Figure 7 is a flowchart showing an example of the collision timing identification process. In the collision timing identification process, the accident analysis device 10 first acquires audio data at the time of the accident (S101). For example, audio data at the time of the accident (a predetermined period before and after the accident) can be extracted from video data already acquired from a cloud storage server.

Next, the accident analysis device 10 applies a high-frequency exclusion filter to the acquired audio data to remove high-frequency (predetermined frequency) sounds equivalent to screams from the audio data (S102). Therefore, the high-frequency exclusion filter needs to be capable of removing sounds with frequencies equivalent to human screams (e.g., 4 kHz or higher). It is also possible to set a filter that removes sounds of a predetermined frequency equivalent to braking sounds before a collision or other noises. In this way, by removing sounds of a predetermined frequency from the audio data, it is possible to prevent human screams and the like from being detected as the sound of an accident collision, and it is possible to appropriately analyze and recreate the accident situation.

Next, the accident analysis device 10 determines whether there is a timing when the volume of the audio data, from which high frequencies have been removed, is higher than the accident determination value (S103). The accident determination value may be determined in advance based on statistical data on the volume of sounds generated during accidents, etc.

If there is a timing when the volume is higher than the accident determination value (S103; Yes), that timing is identified as the collision timing, i.e., the timing when the accident occurred (S104). If there is no timing when the volume is higher than the accident determination value (S103; No), the timing when the volume is at its maximum is identified as the collision timing (S105). Note that if there are multiple timings when the volume is higher than the accident determination value, the timing when the volume is at its maximum may be identified as the collision timing, or the first timing when the volume exceeds the accident determination value may be identified as the collision timing. Thereafter, the collision timing identification process is terminated, and the process returns to FIG. 6.

In this embodiment, the timing of the collision (timing of the accident) is determined based on the audio data (volume), but the type and severity of the accident may also be determined based on the audio data (volume). For example, if the volume exceeds the major accident determination value, it may be determined to be a major accident. Also, if there is no time when the volume is higher than the accident determination value, it may be determined to be a minor accident.

In this way, the accident analysis device 10 of this embodiment identifies the collision timing based on audio data, thereby improving the accuracy of identifying the collision timing. In particular, the collision timing is identified using audio data from which audio of a specific frequency has been excluded, thereby improving the accuracy of identifying the collision timing. Note that in this embodiment, the collision timing is identified based on the volume, but the collision timing may also be identified based on the quality, type, or frequency of the sound.

In cases where it is uncertain whether or not the timing of the collision can be determined, such as when audio data cannot be acquired normally or when there is no time when the volume is higher than the accident determination threshold, the timing of the collision may be determined based on acceleration data included in the vehicle data. Alternatively, the timing of the collision may be determined based on both audio data and acceleration data.

After executing the collision timing identification process, the accident analysis device 10 determines whether the vehicle 1000 has rolled over based on the vertical acceleration data included in the vehicle data (S16).

Figure 9 shows a method for determining whether the vehicle 1000 has rolled over in this embodiment. Figure 9(A) shows time-series vertical acceleration data for the vehicle 1000 before and after an accident. Figure 9(A) shows two peaks where the acceleration is positive (upward). The second peak (MAX) has a higher acceleration than the first peak (2nd).

Figure 9(B) shows the vertical acceleration integral value of vehicle 1000 in a time series before and after an accident. In Figure 9(B), the acceleration integral value increases from the second acceleration point in Figure 9(A), and then suddenly increases and swings out of control at the maximum acceleration point. In this embodiment, if the acceleration integral value exceeds a predetermined rollover determination value, it is determined that vehicle 1000 has rolled over at that point. In Figure 9(B), the acceleration integral value exceeds the rollover determination value at the maximum acceleration point, so it is determined that vehicle 1000 has rolled over at the maximum acceleration point.

In this way, rollover detection is based on the vertical acceleration (integral value) of the vehicle 1000, improving the accuracy of rollover detection. It is also possible to detect rollover based on the vertical acceleration value instead of the integrated value.

If it is determined that the vehicle 1000 has overturned because the vertical acceleration exceeds the overturn determination value (S17; Yes), the acceleration data from the time of the overturn is filtered so that it is not used in other processing or analysis (S18).

This prevents the accident situation from being analyzed and recreated based on acceleration data after the fall, thereby improving the accuracy of the accident situation reconstruction. For example, when estimating the trajectory of the vehicle 1000 based on acceleration information to create a reconstruction of the accident situation, it is possible to prevent the vehicle's trajectory from being drawn based on the acceleration after the fall, resulting in an inappropriate reconstruction diagram or trajectory being drawn.

If it is determined that the vehicle 1000 has not overturned (S17; No), step S18 is skipped.

Next, the accident analysis device 10 estimates the direction in which the vehicle 1000 collided with another vehicle, obstacle, etc. (e.g., a head-on collision) based on the acceleration data in the forward/backward and left/right directions at the collision timing identified in step S15 (S19). The acceleration data in the forward/backward and left/right directions is included in the vehicle data. Step S19 will be described in detail later.

Next, the accident analysis device 10 executes a collision object identification process to identify the collision object from the detected surrounding objects (S20).

Figure 8 is a flowchart showing an example of the collision target identification process. In the collision target identification process, the accident analysis device 10 first extracts surrounding objects detected N seconds (e.g., 2 seconds; N can be any number) before and after the collision timing identified in the process of step S15 (S201). The surrounding objects extracted here become collision target candidates.

The accident analysis device 10 then excludes from the extracted surrounding objects any surrounding objects that are not in the collision direction estimated in step S19 (S202). Here, for example, it determines that any surrounding object that is not within an extension of X degrees (e.g., 120 degrees; X can be any value) of the collision direction estimated in step S19 cannot be a collision target, and excludes that surrounding object from the list of potential collision targets.

Next, the accident analysis device 10 determines that surrounding objects whose distance from the vehicle (vehicle 1000) is increasing during the N seconds before and after the collision timing are unlikely to be collision targets, and excludes those surrounding objects from the list of collision target candidates (S203).

Next, the accident analysis device 10 determines that surrounding objects that are at a distance of D or more (e.g., 10 meters or more) from the vehicle (vehicle 1000) at the time of collision cannot be collision targets, and excludes those surrounding objects from the list of collision target candidates (S204).

After narrowing down the potential collision targets in this way, the accident analysis device 10 determines whether there are multiple objects surrounding the potential collision targets (S205).

If there are not multiple potential collision target neighboring objects (S205; No), the remaining potential collision target neighboring objects are identified as the collision target (S206). If there are multiple potential collision target neighboring objects (S205; Yes), the remaining potential collision target neighboring object with the shortest distance from the vehicle is identified as the collision target (S207). Then, the collision target identification process ends, and the process returns to FIG. 6.

In this way, the accident analysis device 10 of this embodiment identifies the object of collision based on the collision direction of the vehicle and the distance and direction of surrounding objects from the vehicle, thereby improving the accuracy of identifying the object of collision.

Note that the method used in this embodiment is not limited to this, and any method may be used as long as it can appropriately identify the object of collision.

Figures 10 and 11 are diagrams showing examples of identifying a collision target. As shown in Figure 10, if the collision direction of the host vehicle is approximately 7 o'clock (rear left), surrounding objects to the right and to the front, which are not in the collision direction, are excluded from the list of collision target candidates as they are unlikely to collide. The direction of collision is the opposite direction of the detected acceleration (impact direction). Also, even if a surrounding object is in the impact direction, if its distance from the host vehicle is increasing (the vehicle at the bottom right in the figure), it is excluded from the list of collision target candidates as it is unlikely to collide. As a result, vehicles in the vicinity of the collision direction (rear left) are identified as collision targets.

As shown in Figure 11, if the collision direction of the host vehicle is approximately 3 o'clock (to the right), surrounding objects that are in the direction of impact but are increasing in distance from the host vehicle (the vehicle in the upper right corner of the figure) are excluded from the list of potential collision targets as they are unlikely to collide. As a result, vehicles in the vicinity of the collision direction are identified as potential collision targets.

Next, the accident analysis device 10 generates an image showing the accident situation (a reconstruction of the accident situation, including an overhead view and video) by mapping the calculated absolute positions of the vehicle 1000 and the surrounding objects around the vehicle 1000 onto map data (S21).

The accident analysis device 10 then outputs information indicating the circumstances of the accident caused by the vehicle 1000 using at least the vehicle data of the vehicle 1000, the video data at the time of the accident, the absolute positions of the vehicle 1000 and surrounding objects estimated in step S14, the collision timing identified in step S15, whether the vehicle 1000 rolled over determined in step S16, the collision direction of the vehicle 1000 estimated in step S19, the object of collision identified in step S20, and map data (S22). The information indicating the circumstances of the accident also includes the information shown in FIG. 12. The information indicating the circumstances of the accident includes, for example, multiple items (which may be referred to as tags), and the circumstances of the accident are identified by combining these items.

Next, the accident analysis device 10 searches the accident case database using each item as a key to obtain accident cases corresponding to the circumstances of the accident caused by the vehicle 1000 (S23). This completes the accident situation analysis process.

Steps S21 to S23 will be described in detail below.

The order of the processing steps described above can be rearranged as desired as long as no inconsistencies occur in the processing. For example, the processing steps S12 to S20 can be rearranged as appropriate. In addition, some processing (such as steps S21 and S23) may be executed as separate processes.

Figure 12 is a diagram showing example items included in information indicating the accident situation. Information indicating the accident situation includes multiple items, as shown in Figure 12. The "Details" shown in Figure 12 indicate specific content for each item. "A: Contact object (automobile, obstacle, etc.)" shown in Figure 12 is information indicating the object with which vehicle 1000 made contact (collided). "B: Contact location" indicates the location of vehicle 1000 in contact (collided) with the object (e.g., the front, right side, left front bumper, etc.). "C: Road type" indicates the type of road (straight, curved, intersection, T-junction, expressway, etc.) on which vehicle 1000 was traveling at the time of the accident. "D: Signal color of own vehicle and other vehicle" indicates the signal color of vehicle 1000 and the signal color of the other vehicle in the case of an accident at an intersection. "E: Positional relationship between pedestrian and crosswalk/safety zone" indicates whether the pedestrian was involved in the accident on a crosswalk or safety zone, or whether the pedestrian was involved in the accident at a location other than a crosswalk or safety zone.

"F: Direction of travel of own vehicle and other vehicle at intersection" indicates whether vehicle 1000 and other vehicle were traveling straight, changing lanes, turning left, turning right, or making a quick right turn at the time of the accident. "G: Lane position on the expressway" indicates the lane (main lane, passing lane) in which vehicle 1000 was traveling when the accident occurred on the expressway. "H: Priority relationship at intersection" indicates which of vehicle 1000 and other vehicle should have had priority to pass the intersection if an accident occurred at an intersection. "I: Whether or not vehicle 1000 and other vehicle violated a stop sign or red light" indicates whether vehicle 1000 and other vehicle violated a stop sign or red light. "J: Whether or not or not speed limit was violated" indicates whether vehicle 1000 and other vehicle were observing the speed limit immediately before the accident. "K: Speed of own vehicle and other vehicle before collision" indicates the speed of vehicle 1000 and other vehicle immediately before the accident. "L: Presence or absence of obstacle on road" indicates whether there was an obstacle on the road on which vehicle 1000 was traveling at the time of the collision. "M: Door of collided vehicle open/closed" indicates whether the door of the other vehicle was open at the time of the collision. "N: Presence or absence of turn signal before collision" indicates whether vehicle 1000 had its turn signal on before the collision.

"Whether or not attention to the road ahead was detected" indicates whether or not the driver of vehicle 1000 was looking ahead at the time of the collision. "Whether or not drinking alcohol was detected" indicates whether or not drinking alcohol was detected in the driver of vehicle 1000 at the time of the collision. "Braking timing" indicates whether the brakes were applied early, normal, or late before the collision. "Steering operation at the time of impact" indicates whether or not there was steering operation before the collision, whether or not there was sudden steering, etc. This information is determined and analyzed based on information about the driver contained in the vehicle data. Note that in this embodiment, analysis of information indicating the accident situation is based on information about the driver, including brake operation information, steering operation information, information about the driver's line of sight (gaze direction), and information about the driver's breath (whether or not they had been drinking), but analysis of information indicating the accident situation may also be based on information about other drivers.

Next, the processing steps (steps S10 to S14, S19, S21 to S23 in FIG. 6) performed by the accident analysis device 10 when analyzing the circumstances of an accident caused by the vehicle 1000 will be described in detail. In the following description, it is assumed that the accident analysis device 10 has already acquired vehicle data and video data from the vehicle 1000 involved in the accident. It is also assumed that the vehicle data and video data each contain time information or synchronization information. In other words, in this embodiment, when analyzing video data at a certain point in time, it is possible to perform the analysis using vehicle data corresponding to that point in time. Conversely, when analyzing vehicle data at a certain point in time, it is possible to perform the analysis using video data corresponding to that point in time.

(Detection of surrounding objects and identification of coordinates)
13 is a diagram for explaining the processing procedure when the accident analysis device 10 detects surrounding objects and identifies their coordinates. This processing procedure corresponds to the processing procedure of step S12 in FIG.

The analysis unit 102 analyzes each image obtained by breaking down the video data into frames, thereby identifying one or more surrounding objects that appear in the image, including vehicles, people, bicycles, traffic signs, traffic lights (including traffic light colors and arrow signals), structures around the road (electric poles, street lights, guardrails, etc.), fallen objects, road markings, crosswalks, and lanes. Furthermore, for each of the one or more identified surrounding objects, the analysis unit 102 identifies the coordinates of the area in the image in which the surrounding object appears.

The example in Figure 13 shows an example of the Xth frame image obtained by breaking down video data into frames. The X axis indicates the number of pixels in the left-right direction (X coordinate), with 0 at the left end, and the Y axis indicates the number of pixels in the up-down direction (Y coordinate), with 0 at the bottom end. In the example in Figure 13, a truck is visible in the left lane behind the lane in which vehicle 1000 is traveling (the second lane from the left), and a passenger car is visible in the lane to the right of the lane in which vehicle 1000 is traveling. The analysis unit 102 is equipped with a trained model that has learned the ability to recognize recognition objects (e.g., other vehicles, people, bicycles, road signs, traffic lights, structures around the road (electric poles, streetlights, guardrails, etc.), fallen objects, lanes, etc.), and by inputting an image into the trained model, the type and area of one or more surrounding objects appearing in the image may be identified. This type of processing can be achieved using existing technologies such as YOLO (Your Only Look Once) and Mask R-CNN (Mask Regional Convolutional Neural Network).

In the example of A in Figure 13, the analysis unit 102 has determined that a truck is captured in area C1 (a rectangular area with X = 700 and Y = 300 at the top left and X = 900 and Y = 300 at the bottom right), and that a passenger car is captured in area C2 (a rectangular area with X = 1400 and Y = 250 at the top left and X = 1750 and Y = 50 at the bottom right). Similarly, it has also determined that utility poles are captured in areas C3 to C6 (coordinates not shown).

The example in Figure 13B shows an example of the results of the analysis unit 102 identifying surrounding objects for each frame.

The analysis unit 102 may also compare the analysis results of the images for each frame to determine whether the vehicle's turn signals are flashing, whether the vehicle doors have opened or closed, and whether a person is standing, sitting, or lying down. For example, the analysis unit 102 may determine whether the turn signals are flashing by determining whether the color of the turn signal portion of the vehicle recognized in each image changes periodically.

The analysis unit 102 may also determine whether a vehicle door has opened or closed by judging changes in the vehicle door portion recognized in each image. The analysis unit 102 may also determine whether a person is standing, sitting, or lying down based on the ratio of the vertical length to the horizontal length of the area in which the person is captured. Note that a person's pose can be determined using conventional technology, such as Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields.

(Estimation of the relative positional relationship between the vehicle and surrounding objects)
14 is a diagram for explaining the processing procedure when the accident analysis device 10 estimates the relative positional relationship between the vehicle 1000 and surrounding objects. This processing corresponds to the processing procedure of step S13 in FIG.

The analysis unit 102 estimates the relative positional relationship between the vehicle 1000 and the surrounding objects around the vehicle 1000 for each surrounding object based on the area in which the one or more identified surrounding objects are captured in the image. More specifically, the analysis unit 102 estimates the distance from the vehicle 1000 to the surrounding object (d1 and d2 in B of FIG. 14) based on the size of the area in which the surrounding object is captured. Furthermore, the analysis unit 102 estimates the left-right angle (θ1 and θ2 in B of FIG. 14) at which the surrounding object is located, with the traveling direction of the vehicle 1000 (toward the center of the image) set as the reference (0 degrees), based on the difference in left-right between the center coordinates of the area in which the surrounding object is captured and the center coordinates of the image.

[Example of calculating distance to surrounding objects]
A specific example is shown using A in FIG. 14 . First, the analysis unit 102 is assumed to have stored data indicating how many pixels an object 1 meter in length occupies in an image captured by a camera included in the drive recorder 1100, placed vertically (or horizontally) 1 meter away. Because this data varies depending on the camera's angle of view (lens angle), it may be stored in association with each model of camera included in the drive recorder 1100. Furthermore, depending on the model of camera included in the drive recorder 1100, a wide-angle lens or a fisheye lens is often used. Therefore, the analysis unit 102 may perform distortion correction on the video data captured by the camera included in the drive recorder 1100 in accordance with the lens characteristics of the camera included in the drive recorder 1100, and then calculate the distance to the surrounding object and the horizontal angle at which the surrounding object is located, as described below. Distortion correction can be achieved using conventional techniques such as cubic interpolation.

In the example of A in Figure 14, if a 1m long object is placed vertically 1m away and photographed, it is assumed to occupy 100 pixels vertically. Similarly, if a 1m long object is placed vertically (or horizontally) 10m away and photographed, it is determined in advance how many pixels it will occupy in the image. In the example of Figure 14, if a 1m long object is placed vertically 10m away and photographed, it is assumed to occupy 10 pixels vertically.

The vertical (or horizontal) size of each surrounding object is also determined in advance. For example, the vertical length (height) of a truck may be set to 2 m, and that of a passenger car may be set to 1.5 m.

As mentioned above, if a 1m long object located 1m away has 100 pixels in the vertical direction, and a 1m long object located 10m away has 10 pixels in the vertical direction, then if a 2m high track is located 1m away, the number of pixels in the vertical direction can be calculated to be 200 pixels, and if a 2m high track is located 10m away, the number of pixels in the vertical direction can be calculated to be 20 pixels. More specifically, if the number of pixels is X and the distance is Y, the following equation holds:
Y = 200 ÷ X (Equation 1)

Next, the analysis unit 102 calculates the distance between the vehicle 1000 and the truck using Equation 1. In the example of Figure 14A, the vertical length of area C1 is 50 ((400 - 300) ÷ 2) = 50 pixels. Therefore, according to Equation 1, it can be calculated that Y = 200 ÷ 50 = 4 m.

Note that if the height (height from the ground) at which the camera equipped in the drive recorder 1100 is installed on the vehicle 1000 and the direction in which the camera equipped in the drive recorder 1100 should be pointed are fixed with high precision, it is possible to determine the distance to a surrounding object using only the Y-axis value. However, according to the processing procedure described above, it becomes possible to determine the distance between the vehicle 1000 and a surrounding object even if the installation position of the camera equipped in the drive recorder 1100 varies depending on the driver, such as when a drive recorder is rented.

[Example of calculation of the angle in the left and right direction when surrounding objects are present]
A specific example will be shown using A in Fig. 14. First, it is assumed that the analysis unit 102 stores data indicating the angle of view (lens angle) of the camera equipped in the drive recorder 1100. Since this data differs depending on the model of the camera equipped in the drive recorder 1100, the data may be stored in association with each model of the camera equipped in the drive recorder 1100.

Here, dividing the angle of view of the camera equipped in the drive recorder 1100 by the number of pixels in the horizontal direction of the screen determines how many degrees of the angle of view one pixel corresponds to. For example, if the total number of pixels in the horizontal direction of the screen is 2000 pixels and the angle of view of the camera equipped in the drive recorder 1100 is 80 degrees, 200 pixels corresponds to 8 degrees. In other words, by calculating the difference in the number of horizontal pixels between the center position of the area in the image in which the surrounding object is captured and the center position of the image, it is possible to calculate the left-right angle (angle in the horizontal plane) at which the surrounding object exists, assuming that the direction of travel of the vehicle 1000 is 0 degrees. Note that if the angle of view corresponding to one pixel increases or decreases depending on the position on the screen, the angle per pixel may be corrected based on the image coordinates.

For example, in Figure 14A, the center position in the X direction of the area showing the truck is X = 800. Furthermore, the angle of view of the camera equipped in the drive recorder 1100 is 80 degrees, and the total number of pixels in the horizontal direction of the screen is 2000 pixels. Furthermore, the center position of the image is X = 1000. Therefore, the center position of the area showing the truck and the center position of the image are separated by 200 pixels (1000 - 800). As mentioned above, 200 pixels corresponds to 8 degrees, so the horizontal angle between the vehicle 1000 and the truck (θ1 in Figure 14B) can be calculated to be 8 degrees.

Note that with the calculation method described above, it becomes impossible to calculate the distance and angle if a surrounding object comes extremely close to the vehicle 1000 and part of the surrounding object is outside the image. However, if part of the surrounding object is outside the image, it is possible to estimate that the distance between the surrounding object and the vehicle 1000 is within a certain distance. It is also possible to estimate the position of a surrounding object that has disappeared from the image based on changes in the position of the surrounding object in images of previous and subsequent frames.

For example, suppose that in an image of a certain frame X, part of a surrounding object is outside the image, but in the image of the previous frame, frame X-1, the entire surrounding object is captured, and the distance between the vehicle 1000 and the surrounding object is 10 m. Furthermore, suppose that in the image of frame X-2, which is one frame before that, the distance between the vehicle 1000 and the surrounding object is 11 m. In this case, the distance between the vehicle 1000 and the surrounding object in the image of frame X can be estimated to be 9 m. Furthermore, when the surrounding object is a vehicle, distance estimation can be performed by image recognition of the entire vehicle, but image recognition can be limited to a portion of the vehicle (such as the license plate). This makes it possible to estimate the distance and angle to the vehicle 1000, even if part of the vehicle is outside the image, as long as the license plate is captured in the image.

(Estimates the absolute position of the vehicle and surrounding objects)
15 is a diagram for explaining the processing procedure when the accident analysis device 10 estimates the absolute positions of the vehicle 1000 and surrounding objects. This processing corresponds to the processing procedure of step S14 in FIG.

First, the analysis unit 102 estimates the absolute position of the vehicle 1000. The analysis unit 102 may estimate the absolute position of the vehicle 1000 using position information of the vehicle 1000 acquired by the positioning unit 1130 (GPS) provided in the drive recorder 1100, which is included in the vehicle data of the vehicle 1000. Alternatively, the analysis unit 102 may more accurately estimate the absolute position of the vehicle 1000 by combining the absolute position of the vehicle 1000 obtained by analyzing video data at the time of the accident using, for example, a conventional technology called Structure From Motion or Simultaneous Localization and Mapping (hereinafter referred to as "SFM" for convenience) with the absolute position of the vehicle 1000 estimated by the positioning unit 1130.

[SFM-based absolute position estimation of the vehicle 1000]
By using SFM, feature points included in the image of each frame of video data can be extracted, corresponding points (corresponding feature points) in the image of each frame can be identified from the extracted feature points, and the movement of the identified corresponding points can be tracked to reproduce the camera movement. The extraction of feature points can be achieved, for example, by using a conventional technology called SIFT features. Furthermore, the identification of corresponding points can be achieved, for example, by using a conventional technology called FLANN (Fast Library for Approximate Nearest Neighbors).

Here, since the camera that captured the video data is a camera equipped in the drive recorder 1100 mounted on the vehicle 1000, the reproduced camera movement can be considered to be the movement of the vehicle 1000. In addition, since the vehicle data for the vehicle 1000 includes vehicle speed data, the distance traveled by the vehicle 1000 between each frame can be estimated by comparing the vehicle speed data with the frame rate of the video data. For example, if the vehicle 1000 is traveling at a speed of 54 km/h and the frame rate of the video data is 15 frames per second, the distance traveled by the vehicle 1000 per frame will be approximately 1 meter.

In order to determine the movement of the vehicle 1000 relative to the surrounding environment, rather than other vehicles traveling alongside it, the analysis unit 102 excludes feature points of moving surrounding objects when extracting feature points, and extracts feature points of fixedly installed surrounding objects. For example, the analysis unit 102 may extract feature points of traffic signs, traffic lights, and structures around the road (electric poles, streetlights, guardrails, etc.). The type of surrounding object can be identified by having the trained model described above identify it.

For example, Figure 15A shows image data for frame N, and Figure 15B shows image data for frame N+1. In Figures 15A and 15B, areas C3 to C6 corresponding to fixedly installed surrounding objects move backward as vehicle 1000 moves, but the position of area C1, which is the moving truck, remains almost unchanged, while the position of area C2, which is the passenger car traveling in the oncoming lane toward vehicle 1000, changes significantly.

By using SFM, the analysis unit 102 can calculate information indicating the relative position of the vehicle 1000 with respect to the first frame of the video data as a reference, such as, for example, that the position of the vehicle 1000 in the second frame has moved 2 m forward and 1 m to the left compared to the first frame, and that the position of the vehicle 1000 in the third frame has moved 2.5 m forward and 0.5 m to the left compared to the second frame.

Next, the analysis unit 102 selects the location information corresponding to the time the first frame of the video data was captured from the location information indicating the absolute position of the vehicle 1000 acquired by the positioning unit 1130. The location information acquired by the positioning unit 1130 includes time information, and the video data in this embodiment also includes time information indicating the time it was recorded. Therefore, the analysis unit 102 can select the GPS location information corresponding to the first frame of the video data by comparing the time included in the video data with the time included in the location information.

Next, the analysis unit 102 uses the GPS position information corresponding to the first frame of the video data to determine the absolute position of the vehicle 1000 in the second and subsequent frames of the video data. As described above, the analysis unit 102 calculates information indicating relative movement between frames. Furthermore, the vehicle data in this embodiment includes orientation data indicating the direction in which the front of the video is facing, and the orientation data makes it possible to determine which direction the front is pointing. Therefore, by using the selected GPS position information as the absolute position of the vehicle 1000 in the first frame, the analysis unit 102 can calculate the absolute position (latitude and longitude) of the vehicle 1000 corresponding to the relative position of the vehicle 1000 in the second frame (the relative position determined by SFM). For example, if the latitude and longitude of the first frame acquired by the positioning unit 1130 are 134.45 degrees and 32.85 degrees, the latitude and longitude corresponding to a position 2 m forward north and 1 m west from these latitude and longitude as the starting point will be the absolute position of the vehicle 1000 in the second frame. The analysis unit 102 repeats this process for each frame, thereby calculating the absolute position (latitude and longitude) of the vehicle 1000 based on the SFM for all frames.

[Estimation of the absolute position of the vehicle 1000]
As shown in FIG. 16 , the analysis unit 102 estimates the absolute position of the vehicle 1000 for each frame using the absolute position of the vehicle 1000 based on the SFM and the absolute position of the vehicle 1000 based on the GPS. Points f11 to f16 shown on the left side of FIG. 16 are assumed to be the absolute positions of the vehicle 1000 obtained by analyzing frames 1 to 6 of the video data, respectively. Points f21 to f26 shown in the center of FIG. 16 are assumed to be the absolute positions of the vehicle 1000 at the times corresponding to frames 1 to 6 of the video data, among the absolute positions of the vehicle 1000 obtained by the GPS. Points f31 to f36 shown on the right side of FIG. 16 indicate the estimated absolute positions of the vehicle 1000. As mentioned above, point f11, which indicates the absolute position of the vehicle 1000 corresponding to one frame of the video data, is the same point as point f21.

The analysis unit 102 optimizes the absolute position of vehicle 1000 based on SFM (points f11 to f16) and the absolute position of vehicle 1000 based on GPS (points f21 to f26) so that there is no significant discrepancy. For example, the absolute position of vehicle 1000 may be estimated as an absolute position obtained by simply averaging the absolute position of vehicle 1000 based on SFM and the absolute position of vehicle 1000 based on GPS. For example, for speeds of 30 km/h or more, the sum of these values may be divided by 2 to determine the absolute position of vehicle 1000. If the vehicle speed of vehicle 1000 at points f13 and f23 is 30 km/h or more, the latitude of vehicle 1000's absolute position (point f33) can be calculated by (latitude of point f13 + latitude of point f23) ÷ 2. Similarly, the longitude of the absolute position of vehicle 1000 (point f33) can be calculated by (longitude of point f13 + longitude of point f23) / 2.

Furthermore, the analysis unit 102 may determine the absolute position of the vehicle 1000 by averaging the absolute position of the vehicle 1000 based on the SFM and the absolute position of the vehicle 1000 based on the GPS with a predetermined weight depending on the vehicle speed of the vehicle 1000. For example, for a speed of less than 30 km/h (e.g., 5 km/h), the absolute position of the vehicle 1000 based on the SFM may be considered to be more accurate, and the absolute position of the vehicle 1000 may be determined by averaging the absolute position of the vehicle 1000 based on the SFM and the absolute position of the vehicle 1000 based on the GPS in a ratio of, for example, 4:1. If the vehicle speed of the vehicle 1000 at points f16 and f26 is 5 km or faster, the latitude of the absolute position of the vehicle 1000 (point f36) can be calculated by (latitude of point f16 x 4 + latitude of point f26 x 1) ÷ 5. Similarly, the longitude of the absolute position of vehicle 1000 (point f36) can be calculated as (longitude of point f16 x 4 + longitude of point f26 x 1) ÷ 5.

The analysis unit 102 may estimate the absolute position of the vehicle 1000 for each frame using the absolute position of the vehicle 1000 based on the SFM that has been subjected to noise removal processing and the absolute position of the vehicle 1000 based on the GPS that has been subjected to noise removal processing. A Kalman filter, for example, may be used for noise removal. The Kalman filter used to remove noise from the absolute position of the vehicle 1000 based on the SFM and the Kalman filter used to remove noise from the absolute position of the vehicle 1000 based on the GPS may be different Kalman filters. This makes it possible to further improve the accuracy of the estimated absolute position of the vehicle 1000.

[Estimating the absolute position of surrounding objects]
The analysis unit 102 calculates the absolute positions of the peripheral objects for each frame based on the estimated absolute position of the vehicle 1000 for each frame and the relative positional relationship between the vehicle 1000 and the peripheral objects for each frame. As described above, the relative positional relationship between the vehicle 1000 and the peripheral objects is indicated by the distance between the vehicle 1000 and the peripheral objects and the angle in the left-right direction at which the peripheral objects are located, with the traveling direction of the vehicle 1000 (toward the center of the image) being set as the reference (0 degrees), as shown in B of FIG. 14. The analysis unit 102 determines the moving direction of the absolute position of the vehicle 1000 obtained by taking the difference between the absolute positions of the vehicle 1000 for each frame as the traveling direction of the vehicle 1000, and calculates the latitude and longitude corresponding to the relative position of the peripheral objects based on the estimated traveling direction, thereby obtaining the absolute positions (latitude, longitude) of the peripheral objects.

(Generation of images showing accident situations)
The generation unit 103 generates an image showing the accident situation by mapping the absolute position of the vehicle 1000 and the absolute positions of the surrounding objects for each frame, which are estimated by the processing procedure described above, onto a road map. This processing corresponds to the processing procedure of step S21 in Figure 6.

Figure 17 shows an example of an image generated by the accident analysis device 10. The Y and X axes in Figure 17 correspond to latitude and longitude, respectively. In the roads shown in Figures 17A to 17E, the center line represents the median strip. The two lanes on the right are lanes running from bottom to top, while the two lanes on the left are lanes running from top to bottom. It is assumed that Figures 17A to 17E correspond to frames X, X+1, X+2, X+3, and X+4, respectively, but this is merely an example and is not intended to be limiting. The accident situation can be reproduced by mapping the absolute position of vehicle 1000 and the absolute positions of surrounding objects for each frame on a road map. For example, at time A in Figure 17, vehicle V2 is approaching vehicle 1000 from the opposite lane, and at time B in Figure 17, vehicle 1000 and vehicle V2 are approaching each other, resulting in a collision. In addition, the display of whether or not there is a collision and the location of the collision may use the results estimated using the processing procedure for "estimating the collision location" described below.

Incidentally, passenger car V2 is not present in C to E of Figure 17. This is because passenger car V2 has moved behind vehicle 1000, and is no longer visible on the camera equipped with drive recorder 1100. If the camera equipped with drive recorder 1100 is capable of capturing images of the rear of the vehicle, it would be possible to estimate the movement of passenger car V2 after the collision by analyzing the video data captured behind vehicle 1000 and reflect this in the image showing the accident situation.

(Estimation of collision direction)
The analysis unit 102 estimates the direction in which the vehicle 1000 collides with another vehicle, an obstacle, etc., based on the acceleration generated in the vehicle 1000 at the moment of the collision. This process corresponds to the processing procedure of step S19 in FIG. 6.

More specifically, the analysis unit 102 determines the direction of the collision based on the acceleration pattern generated in the vehicle 1000 at the moment of the collision. When a collision occurs, the vehicle 1000 experiences negative acceleration in the direction of the object of collision, and also experiences acceleration in the opposite direction to the collision direction due to the reaction of the collision. In other words, the vehicle 1000 generally experiences acceleration that moves back and forth on an axis connecting the direction of the object of collision and a direction 180 degrees opposite to that direction.

Therefore, the analysis unit 102 sums the output values from the acceleration measurement unit 1160 for a specified period from the time of the collision by direction, and estimates the direction with the largest sum as the direction of collision. More specifically, as shown in FIG. 18A, assuming that the direction of acceleration is 0 degrees in the direction directly ahead of the vehicle 1000, the acceleration directions are summed for eight directions: forward (D1: 337.5 degrees to 22.5 degrees), right-front (D2: 22.5 degrees to 67.5 degrees), right (D3 direction: 67.5 degrees to 112.5 degrees), right-rear (D4: 112.5 degrees to 157.5 degrees), rear (D5: 157.5 degrees to 202.5 degrees), left-rear (D6: 202.5 degrees to 247.5 degrees), left (D7: 247.5 degrees to 292.5 degrees), and left-front (D8: 292.5 degrees to 337.5 degrees), and the direction with the largest total value is deemed to be the collision direction. The acceleration measured by the acceleration measurement unit 1160 is an acceleration that follows the law of inertia. Therefore, in the event of a collision, a positive acceleration is measured in the direction of the collision.

The analysis unit 102 may consider the time at which the acceleration measurement unit 1160 detects the greatest acceleration to be the time of the collision. If the camera provided in the drive recorder 1100 is a drive recorder, the analysis unit 102 may consider the time of the collision recorded in the drive recorder to be the time of the collision.

As shown in B of Figure 18, when vehicle 1000 comes into contact with vehicle V at its front right, it is assumed that the acceleration data acquired from the acceleration sensor is in chronological order, for example, at time 1 (45 degree direction, 3.0 G), time 2 (47 degree direction, 1.0 G), time 3 (225 degree direction, 2.5 G), time 4 (227 degree direction, 0.5 G), and time 5 (40 degree direction, 2.0 G). In other words, vehicle 1000 suddenly decelerates upon colliding with vehicle V at time 1, and then accelerates back and forth between the front right and rear left.

In this case, the total acceleration corresponding to the right front (D2) is 3.0G + 1.0G + 2.0G = 6.0G, and the total acceleration corresponding to the left rear (D6) is 2.5G + 0.5G = 3.0G. Therefore, the analysis unit 102 estimates that the right front (D2) direction is the collision direction.

The analysis unit 102 may also analyze the video data to estimate the direction in which the vehicle 1000 collided with the pedestrian. For example, in the process of detecting surrounding objects described above, if a pedestrian appears in an image that is larger than a predetermined size, it may be estimated that the vehicle 1000 and the pedestrian collided in the head-on (D1) direction. When the vehicle 1000 collides with a pedestrian, the acceleration detected by the acceleration sensor is smaller than when a collision occurs with another vehicle, and when the vehicle 1000 collides with a pedestrian, it is almost always a head-on collision. Therefore, when detecting a collision between the vehicle 1000 and a pedestrian, it is preferable to analyze the video data.

(Output of information showing the accident situation)
The analysis unit 102 outputs information indicating the circumstances of the accident caused by the vehicle 1000. This processing corresponds to the processing procedure of step S22 in FIG. 6. As described above, the analysis unit 102 outputs each item shown in FIG. 12 as information indicating the circumstances of the accident. The processing procedure for identifying each item shown in FIG. 12 will be described in detail below. In the following description, "another vehicle" refers to the vehicle with which the vehicle 1000 collided.

"A: Object of contact (car, obstacle, etc.)"
The analysis unit 102 identifies the object that has collided (come into contact) by the collision object identification process shown in FIG. 8 . A specific example will be described using FIG. 19 . In FIG. 19 , it is assumed that the vehicle 1000 has collided with a surrounding object at its right front. In the case of A in FIG. 19 , the object closest to the right front direction (collision direction) of the vehicle 1000 is the passenger car V1. Therefore, the analysis unit 102 identifies that the vehicle 1000 has collided with the passenger car V1. Similarly, in the case of B in FIG. 19 , the object closest to the right front direction (collision direction) of the vehicle 1000 is the passenger car V5. Therefore, the analysis unit 102 identifies that the vehicle 1000 has collided with the passenger car V5.

Note that if the collision direction of the vehicle 1000 is in a direction not captured by the camera equipped in the drive recorder 1100 (i.e., if only the front of the vehicle 1000 is captured), it is not possible to estimate the absolute value of surrounding objects in that direction, and therefore it is not possible to identify the object that has come into contact. In this case, the analysis unit 102 may determine that a vehicle of some kind has collided.

"B: Contact site"
The analysis unit 102 identifies the contact area according to the estimated collision direction of the vehicle 1000. Specifically, if the collision direction is the front (D1) in FIG. 18 , the contact area is the front bumper shown in FIG. 20 . If the collision direction is the right front (D2), the contact area is the right front bumper shown in FIG. 20 . If the collision direction is the right (D3), the contact area is the right side surface shown in FIG. 20 . If the collision direction is the right rear (D4), the contact area is the right rear bumper shown in FIG. 20 . If the collision direction is the rear (D5), the contact area is the rear bumper shown in FIG. 20 . If the collision direction is the left rear (D6), the contact area is the left rear bumper shown in FIG. 20 . If the collision direction is the left (D7), the contact area is the left side surface shown in FIG. 20 . If the collision direction is the left front (D8), the contact area is the left front bumper shown in FIG. 20 .

"C: Road type"
The analysis unit 102 identifies the road type of the road on which the vehicle 1000 was traveling at the time of the collision by comparing the absolute position of the vehicle 1000 at the time of the collision with the map data. The road type identified by the analysis unit 102 may include information such as a straight road, a curve, the inside of an intersection, or the inside of a T-junction, in addition to information such as an ordinary road or an expressway.

"D: Signal color of your vehicle and other vehicles"
The analysis unit 102 identifies the traffic light color using the traffic light color and arrow signal identified by image analysis of the video data for each frame. For example, if the road type of the road on which the vehicle 1000 was traveling at the time of the collision was an intersection, the analysis unit 102 identifies the traffic light color on the vehicle 1000's side (green, yellow, or red) as the color of the traffic light in the image of the frame closest to the time of the collision and identified before the vehicle 1000 entered the intersection, among the traffic light colors identified by analyzing the video data. Whether the vehicle 1000 has not yet entered the intersection can be determined by comparing the absolute position of the vehicle 1000 with map data.

It is assumed that the color of the traffic light on the intersecting side at an intersection is not visible in the video data, or is visible only for a very short time, making it difficult to determine. Therefore, the analysis unit 102 may determine that the color of the traffic light on the intersecting side is red if the color of the traffic light on the vehicle 1000 side is green. If the other vehicle that collided with vehicle 1000 was traveling on the road that intersects the intersection, the traffic light on the other vehicle's side will be determined to be red (i.e., the other vehicle entered the intersection on a red light). Similarly, the analysis unit 102 may determine that the color of the traffic light on the intersecting side is green if the color of the traffic light on the vehicle 1000 side is red. If the other vehicle that collided with vehicle 1000 was traveling on the road that intersects the intersection, the traffic light on the other vehicle's side will be determined to be green (i.e., the other vehicle entered the intersection on a green light).

"E: Positional relationship between pedestrians, crosswalks, and safety zones"
If the object that has come into contact with the vehicle 1000 is a pedestrian, the analysis unit 102 identifies the positional relationship between the pedestrian and the crosswalk/safety zone using the absolute position of the pedestrian and the absolute position of the crosswalk (or safety zone) identified by image analysis of the video data for each frame. Note that the analysis unit 102 may obtain the absolute position of the crosswalk from map data.

The analysis unit 102 calculates the point at which a line connecting the absolute position of the vehicle 1000 and the absolute position of the pedestrian collided with, obtained by analyzing an image frame taken before the time the vehicle 1000 collided with the pedestrian, collides with the side of the crosswalk or safety zone farthest from the vehicle 1000. Next, if the distance between the person's absolute position and this point is within a predetermined distance (e.g., 7 m), the analysis unit 102 determines that the pedestrian involved in the accident was on a crosswalk or safety zone. Furthermore, if the distance between the person's absolute position and this point exceeds the predetermined distance (e.g., 7 m), the analysis unit 102 determines that the pedestrian involved in the accident was not on a crosswalk or safety zone.

Figure 21 is a diagram illustrating the process of identifying the positional relationship between a pedestrian and a crosswalk/safety zone. For example, in Figure 21, the analysis unit 102 calculates point P2 where line L connecting absolute position P1 of vehicle 1000 (absolute position of the center of vehicle 1000) and absolute position P3 of the pedestrian meets the side of the crosswalk farthest from vehicle 1000, and if the distance between point P2 and absolute position P3 of the pedestrian is a predetermined distance, it determines that pedestrian R was on the crosswalk.

In the example of Figure 21A, the pedestrian is on a crosswalk, but in the example of Figure 21B, the pedestrian is not on a crosswalk. However, in both examples, the analysis unit 102 considers the pedestrian to be on a crosswalk if the distance between the pedestrian's absolute value P3 and point P2 is within a predetermined distance. This is because, in accident cases, it is not particularly important whether the pedestrian involved in the accident was definitely walking on a crosswalk, and it is often determined that the pedestrian was walking on a crosswalk even if they were slightly off the crosswalk.

"F: Trajectories of own and other vehicles at intersections"
If the accident occurs near an intersection (if the absolute position of vehicle 1000 at the time of collision is within a predetermined range from the center position of the intersection), analysis unit 102 identifies the travel trajectory of vehicle 1000 by arranging the absolute position of vehicle 1000 for each frame on map data of the intersection. Furthermore, analysis unit 102 identifies the travel trajectory of other vehicles by arranging the absolute positions of other vehicles for each frame on map data of the intersection. More specifically, analysis unit 102 identifies whether vehicle 1000 and other vehicles went straight through the intersection, turned right, turned left, or made a quick right turn, based on the magnitude and direction of the curvature of the travel trajectories.

Figure 22 is a diagram illustrating the traveling directions of vehicle 1000 and other vehicles. For example, if the curvature of the travel trajectory within a predetermined area including an intersection is less than a predetermined value, the analysis unit 102 may determine that vehicle 1000 or other vehicles have traveled straight through the intersection, and if the curvature of the travel trajectory within the predetermined area is equal to or greater than a predetermined value, the analysis unit 102 may determine that vehicle 1000 or other vehicles have turned left or right at the intersection. Furthermore, if the travel trajectories of the host vehicle and other vehicles making a right turn pass inside the center of the intersection, the analysis unit 102 may determine that the turn is a quick right turn. Furthermore, if the curvature reverses within a predetermined time (e.g., 2 seconds), the analysis unit 102 may determine that vehicle 1000 and other vehicles have changed lanes.

"G: Lane position on the highway"
If the accident occurs on a highway (if the absolute position of the vehicle 1000 at the time of the collision is on the highway), the analysis unit 102 identifies whether the vehicle 1000 was traveling on a main lane or an overtaking lane by analyzing the video data frame by frame or by comparing the absolute position of the vehicle 1000 with map data. For example, if the vehicle 1000 is traveling in the rightmost lane, it is identified as traveling in the overtaking lane, and if the vehicle is traveling in a lane other than the rightmost lane, it is identified as traveling on a main lane.

"H: Priority at intersections"
When an accident occurs near an intersection, the analysis unit 102 extracts from the map data the presence or absence of a stop sign and the road width at the intersection to determine whether the vehicle 1000 had priority to enter the intersection. More specifically, the analysis unit 102 identifies the vehicle traveling on the side of the roads intersecting the intersection that does not have a stop sign as having priority. Furthermore, if the difference in width between the roads intersecting the intersection is two times or more, the analysis unit 102 identifies the vehicle traveling on the wider road as having priority. Furthermore, if there is no stop sign and there is no difference in road width, the analysis unit 102 identifies the vehicle as not having priority.

"I: Stop sign, red light violation or not?"
The analysis unit 102 determines whether the vehicle 1000 and the other vehicle passed a stop sign without stopping by comparing the travel trajectory of the vehicle 1000, which is obtained by aligning the absolute positions of the vehicle 1000 and the other vehicle for each frame, with the map data. Furthermore, when the vehicle 1000 or the other vehicle passes a stop sign without stopping, the analysis unit 102 determines that the vehicle 1000 or the other vehicle has violated a stop sign if the vehicle speed of the vehicle 1000 or the other vehicle at the time of passing the stop sign is equal to or greater than a predetermined speed (e.g., 5 km) in a predetermined section (e.g., 3 m before and after) before and after the stop sign. On the other hand, the analysis unit 102 determines that the vehicle 1000 or the other vehicle has not violated a stop sign if the vehicle speed of the vehicle 1000 or the other vehicle at the time of passing is less than the predetermined speed.

The analysis unit 102 also detects whether the vehicle 1000 has passed through a traffic light by comparing the travel trajectory of the vehicle 1000, obtained by arranging the absolute position of the vehicle 1000 for each frame, with the map data. If the vehicle 1000 has passed through a traffic light, the analysis unit 102 obtains the color of the traffic light by analyzing the image of the frame before the time the vehicle 1000 passed through the traffic light and that is the last frame in which the traffic light is captured. If the color of the traffic light is red, the analysis unit 102 determines that the vehicle 1000 ran a red light when passing through the traffic light.

"J: Whether or not the speed limit has been violated"
The analysis unit 102 identifies the speed limit set for the road on which the vehicle 1000 traveled by comparing the travel trajectory of the vehicle 1000, obtained by arranging the absolute position of the vehicle 1000 for each frame, with the map data. The analysis unit 102 also compares the vehicle speed data included in the vehicle data of the vehicle 1000, which is the vehicle speed at a timing a predetermined time before (e.g., 5 seconds before) the time of the collision of the vehicle 1000, with the speed limit. If the vehicle speed exceeds the speed limit, the analysis unit 102 identifies that the vehicle 1000 was speeding. If the vehicle speed is equal to or less than the speed limit, the analysis unit 102 identifies that the vehicle 1000 was not speeding.

"K: Speed before collision between your vehicle and other vehicle"
The analysis unit 102 identifies the speed of the vehicle 1000 before the collision from the vehicle speed data included in the vehicle data of the vehicle 1000. The analysis unit 102 also calculates the vehicle speed of the other vehicle from the change in the absolute position of the other vehicle estimated for each frame of the video data. For example, if the distance traveled by the other vehicle per frame is 1 meter and the frame rate of the video data is 15 frames per second, it can be determined that the other vehicle was traveling at a speed of 54 km/h (15 x 60 x 60 frames per hour).

"L: Presence or absence of obstacles on the road"
The analysis unit 102 performs image analysis on the video data for each frame to identify the presence or absence of an obstacle on the road. The analysis unit 102 may determine that an object that cannot be determined as a road is present on the lane on which the vehicle 1000 is traveling, that the object is not a person, a car, a bicycle, or a motorcycle, and that the absolute position of the object does not change between frames, as an obstacle.

"M: Opening and closing the door of the other vehicle involved in the collision"
The analysis unit 102 performs image analysis on the video data for each frame to identify whether the door of the other vehicle was open or closed.

"N: Whether or not the turn signal was activated before the collision"
The analysis unit 102 determines whether or not the turn signal of the vehicle 1000 was operating before the collision based on the turn signal operation information included in the vehicle data of the vehicle 1000. The analysis unit 102 also determines whether or not the turn signal was operating before the collision by performing image analysis on the video data for each frame.

"Whether or not attention to the road ahead is detected"
The analysis unit 102 determines whether or not the driver was inattentive to the road ahead based on information about the driver's line of sight (gaze direction) included in the vehicle data of the vehicle 1000. For example, if the driver's line of sight was not directed forward from a predetermined time before the accident occurred until the time the accident occurred, it is determined that the driver was inattentive to the road ahead.

"Whether or not drinking was detected"
The analysis unit 102 determines whether drinking has been detected (drinking driving) based on information about the driver's breath (whether or not drinking has been performed) included in the vehicle data of the vehicle 1000. For example, if the alcohol concentration indicated by the information about the driver's breath exceeds a predetermined reference value, it is determined that drinking has been detected. Note that if there is a passenger, this item may be excluded because there is a possibility that the passenger's drinking may be detected. The presence or absence of a passenger may also be determined from video data inside the vehicle.

"Brake timing"
The analysis unit 102 determines whether the braking timing is early, normal, or late (sudden braking) based on the brake operation information included in the vehicle data of the vehicle 1000. For example, by comparing the brake operation timing from the time of the accident with a predetermined reference value, it can determine whether the braking timing is early, normal, or late (sudden braking).

"Steering operation during impact"
The analysis unit 102 determines whether or not the steering wheel has been operated or whether or not the steering wheel has been turned suddenly, based on steering wheel operation information included in the vehicle data of the vehicle 1000. For example, if the steering wheel operation speed immediately before the accident is faster than a predetermined reference value, it may be determined that the steering wheel has been turned suddenly.

The information items indicating the accident situation listed here are examples, and other information may also be included.

(Search for corresponding accident cases)
The analysis unit 102 searches for and outputs an accident case corresponding to the analyzed accident situation. This process corresponds to the processing procedure of step S23 in FIG. 6. The analysis unit 102 searches for an accident case and fault ratio corresponding to the situation of the accident caused by the vehicle 1000 by comparing it with an accident case DB (accident case data) that associates the accident situation with past accident cases. FIG. 23 is a diagram showing an example of the accident case DB. In step S23, a screen of the post-incident case DB as shown in FIG. 23 is output. As shown in FIG. 23, the accident case includes "search conditions" for searching for the accident case, "accident details" indicating the details of the accident, "fault ratio (basic)" indicating the basic fault ratio, and "correction factors" which are conditions that require correction of the fault ratio.

In the example of Figure 23, if the object that vehicle 1000 collided with was another car (corresponding to "passenger car or truck" in item A), the accident occurred in an intersection where vehicle 1000 collided with another car (corresponding to "in an intersection" in item C), the accident occurred between a vehicle going straight and a vehicle turning right (corresponding to vehicle 1000 going straight and another vehicle turning right in item F, or another vehicle going straight and vehicle 1000 turning right), and the light was red for the vehicle going straight and yellow for the vehicle turning right (corresponding to vehicle 1000 going red and another vehicle yellow, or vehicle 1000 yellow and another vehicle red in item D), then this corresponds to Accident Example 1 and the fault ratio between the vehicle going straight and the vehicle turning right is 70:30.

Furthermore, as a correction factor, when a vehicle going straight violates the speed limit by 15 km/h or more (speeding limit of 15 km/h or more in items J and K), the fault ratio between the vehicle going straight and the vehicle turning right is shown to be 75:25. Similarly, when a vehicle going straight violates the speed limit by 30 km/h or more (speeding limit of 30 km/h or more in items J and K), the fault ratio between the vehicle going straight and the vehicle turning right is shown to be 80:20. Furthermore, when a vehicle turning right turns right without using a turn signal (no turn signal in item N), the fault ratio between the vehicle going straight and the vehicle turning right is shown to be 60:40.

In addition, the correction factors may also include correction factors that are difficult for the accident analysis device 10 to determine, such as "other significant negligence" (e.g., inattention to the road ahead, drunk driving, sudden steering, sudden braking, etc.) among the percentage of fault.

The search unit 104 searches for accident cases that match the identified content by comparing the content identified for each item of information indicating the accident situation with the accident case database. The search unit 104 also obtains the fault ratio corresponding to the searched accident case from the accident case database. The search unit 104 also searches for the presence or absence of a corresponding correction element by comparing the correction element of the searched accident case with each item indicating the accident situation. If a corresponding correction element is present, the search unit 104 corrects the fault ratio according to the correction ratio of the corresponding correction element.

The output unit 105 also outputs the accident details and fault ratio of the accident cases searched by the search unit 104 to the terminal 20. Furthermore, if the correction elements include correction elements that are difficult to determine by the accident analysis device 10, the output unit 105 may output the accident case number, the determined fault ratio, and the wording of the correction elements that are difficult to determine to the terminal 20.

When searching the accident case database, the search unit 104 may search for accident cases that meet some of the search criteria in addition to accident cases that meet all of the search criteria. Furthermore, when searching for accident cases that meet some of the search criteria, the search unit 104 may rank the accident cases according to the number of mismatched items, and the output unit 105 may output the accident details in ranked order.

In addition, accident cases may be further ranked in descending order of frequency of occurrence, and the search unit 104 may search the accident case DB in descending order of rank.

The embodiments described above are intended to facilitate understanding of the present invention and are not intended to limit the scope of the present invention. The flowcharts, sequences, elements included in the embodiments, as well as their arrangement, materials, conditions, shapes, and sizes, are not limited to those illustrated and may be modified as appropriate. Furthermore, configurations shown in different embodiments may be partially substituted or combined.

1...accident analysis system, 10...accident analysis device, 11...processor, 12...storage device, 13...communication IF, 14...input device, 15...output device, 20...terminal, 100...storage unit, 101...acquisition unit, 102...analysis unit, 103...generation unit, 104...search unit, 105...output unit, 1000...vehicle, 1100...drive recorder, 1110...first communication unit, 1120...second communication unit, 1130...positioning unit, 1140...recording unit, 1150...sound recording unit, 1160...acceleration measurement unit, 1170...automatic diagnosis data acquisition unit, 1180...control unit, 1190...sensor unit, 1200...data storage unit, 1300...mirror, 1400...automatic diagnosis system

Claims (3)

an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle;
an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and the acquired video data,
The analysis unit
If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs;
An accident analysis device characterized by excluding frequencies corresponding to human screams from audio data included in the video data, and determining the timing of an accident based on the audio data from which the frequencies corresponding to human screams have been excluded. An accident analysis method using an accident analysis device,
an acquisition step of acquiring vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle;
an analysis step of analyzing the circumstances of the accident caused by the vehicle based on the acquired vehicle data and the acquired video data,
In the analyzing step,
If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs;
An accident analysis method comprising: excluding frequencies corresponding to human screams from audio data included in the video data; and determining the timing of an accident based on the audio data from which the frequencies corresponding to human screams have been excluded. Computer,
an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle;
an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and video data;
The analysis unit
If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs;
A program for excluding frequencies corresponding to human screams from audio data included in the video data, and determining the timing of an accident based on the audio data from which the frequencies corresponding to human screams have been excluded.