JP7618011B2 - Collision Avoidance Systems - Google Patents

Description

The present invention relates to a collision avoidance system for preventing collision accidents with various collision avoidance targets, including other vehicles and people, while driving various vehicles, including various heavy machinery.

Traditionally, various vehicles, including heavy machinery, are used at civil engineering and construction sites, and there is a need to prevent vehicle contact accidents to ensure the safety of workers operating the vehicles and workers working around the vehicles.
For example, Patent Document 1 discloses a vehicle collision prevention device that detects the illumination of the brake lights of a leading vehicle from an image captured in front of the vehicle, detects the deceleration state of the leading vehicle, and determines the possibility of a collision between the leading vehicle and the vehicle itself.
Furthermore, Patent Document 2 discloses a construction vehicle equipped with an obstacle detection means for detecting obstacles present behind the self-propelled vehicle body that can move forward and backward, and a collision prevention auxiliary device for executing collision prevention control based on detection information from the obstacle detection means when the vehicle body is moving backward.

Japanese Patent Application Publication No. 11-39597 Utility Model Registration No. 3219005

However, in Patent Document 1, since it is premised on detecting the illumination of the brake lights of the preceding vehicle, the collision avoidance target is limited to a vehicle having brake lights and traveling in front of the vehicle. Therefore, there is a problem that the collision prevention device does not function when only the front or side of the vehicle to be avoided is visible from the vehicle, or when the vehicle to be avoided is stopped. In addition, it is not possible to avoid collisions with various obstacles other than the vehicle, such as stationary objects such as materials placed at the work site, fixed structures such as fences and pillars installed at the work site, and workers, and there is a problem that it lacks versatility and functionality.
In Patent Document 2, an obstacle (collision avoidance target) present behind the vehicle is detected by an obstacle detection means, and collision prevention control is executed based on the detection information of the obstacle detection means when the vehicle is moving backward, so the obstacle detection means does not function when the vehicle is moving forward, resulting in a problem of lack of safety. Also, since the obstacle detection means only detects the presence or absence of an obstacle using a 2D laser sensor or the like and cannot identify the type of obstacle, unnecessary (unnecessary) deceleration or stopping is performed, which may reduce work efficiency and result in a problem of lack of practicality.

The present invention has been made in consideration of the above circumstances, and aims to provide a collision avoidance system that can detect and identify the type of collision avoidance target regardless of the posture or orientation (arrangement) of various collision avoidance targets present around the vehicle being driven, and that can prevent excessive detection and erroneous detection, thereby reliably and efficiently preventing contact between the vehicle and the collision avoidance target.

A collision avoidance system according to the present invention that meets the above-mentioned objective is a collision avoidance system that detects a collision avoidance target using image recognition by machine learning while a vehicle is being driven, and prevents a collision between the vehicle and the collision avoidance target,
the machine learning inference unit determines whether the collision avoidance target is present in the vicinity of the vehicle from the image acquired by the image acquisition unit and the machine learning model; a collision avoidance setting unit acquires position information of the collision avoidance target detected by the machine learning inference unit and determines the possibility of a collision between the vehicle and the collision avoidance target; an alarm output unit outputs an alarm to a driver of the vehicle based on the determination of the collision avoidance setting unit; and a display unit displays a real-time video created based on the image, wherein one or more horizontal lines displayed parallel to a width direction of the vehicle and indicating a distance from the vehicle are drawn on the real-time video, and when the machine learning inference unit determines that the collision avoidance target is present, a rectangular frame indicating the range of the collision avoidance target is drawn on the real-time video in addition to the horizontal line, and the collision avoidance setting unit determines the possibility of a collision between the vehicle and the collision avoidance target from the positional relationship between a bottom side of the rectangular frame and the horizontal line .

The collision avoidance system according to the present invention preferably has a vehicle state recognition unit that detects whether a reverse signal is being output from the vehicle and determines whether the vehicle is moving backwards, and the collision avoidance setting unit preferably disables the output of an alarm by the alarm output unit or switches the type of alarm output by the alarm output unit based on the determination result of the vehicle state recognition unit.

The collision avoidance system according to the present invention may have a vehicle state recognition unit that detects whether a reverse signal is being output from the vehicle and determines whether the vehicle is reversing or not, and the collision avoidance setting unit may be able to select whether to disable the output of an alarm by the alarm output unit or switch the type of alarm output by the alarm output unit based on the determination result of the vehicle state recognition unit, or to forcibly stop the operation of the vehicle state recognition unit and output an alarm by the alarm output unit regardless of whether the vehicle is reversing or not.

In the collision avoidance system of the present invention, it is preferable that the system has a display unit that displays a real-time video created based on the image, and the real-time video has one or more horizontal lines drawn parallel to the width direction of the vehicle and indicating the distance from the vehicle, and when the machine learning inference unit determines that a collision avoidance target exists, a rectangular frame indicating the range of the collision avoidance target is drawn on the real-time video in addition to the horizontal lines.

In the collision avoidance system of the present invention, it is preferable that the images and the real-time video are stored in the machine learning setting unit.

In the collision avoidance system according to the present invention, it is further preferable that the images are stored for a period going back a predetermined time from when the machine learning inference unit determined that the collision avoidance target exists.

In the collision avoidance system according to the present invention, it is further preferable that the collision avoidance setting unit adjusts a collision determination area in which it is determined that there is a possibility of a collision between the vehicle and the collision avoidance target by expanding or contracting the length of the horizontal line.

A collision avoidance method for detecting a collision avoidance target using image recognition by machine learning while a vehicle is being driven and preventing a collision between the vehicle and the collision avoidance target includes a machine learning process for performing machine learning based on a learning image including the collision avoidance target to be detected and constructing a machine learning model, an image acquisition process for acquiring images of the periphery of the vehicle while the vehicle is being driven, an object detection process for determining whether the collision avoidance target is present around the vehicle from the images acquired in the image acquisition process and the machine learning model, a collision determination process for acquiring position information of the collision avoidance target detected in the object detection process and determining the possibility of a collision between the vehicle and the collision avoidance target, and an alarm output process for outputting an alarm to the driver of the vehicle when it is determined in the collision determination process that there is a possibility of a collision between the vehicle and the collision avoidance target.

In the above collision avoidance method, it is preferable that in the collision determination step, it is determined whether the vehicle is reversing or not, and the output of the warning by the warning output step is disabled or the type of warning output by the warning output step is switched.

In the above collision avoidance method, the collision determination step may determine whether the vehicle is reversing or not, and disable the output of the alarm by the alarm output step, or switch the type of alarm output by the alarm output step, or select whether the alarm is output by the alarm output step regardless of whether the vehicle is reversing or not.

The above collision avoidance method preferably includes a video display step in which a real-time video created based on the image is displayed, and one or more horizontal lines are drawn on the real-time video, parallel to the width direction of the vehicle, and indicating the distance from the vehicle, and when it is determined in the object detection step that a collision avoidance object exists, a rectangular frame indicating the range of the collision avoidance object is drawn on the real-time video in addition to the horizontal lines.

In the above collision avoidance method, it is preferable that in the collision determination step, the possibility of a collision between the vehicle and the collision avoidance target is determined based on the positional relationship between the bottom side of the rectangular frame and the horizontal line.

In the above collision avoidance method, it is further preferable that in the collision determination step, the length of the horizontal line is expanded or contracted to adjust a collision determination area in which it is determined that there is a possibility of a collision between the vehicle and the collision avoidance target.

The collision avoidance system of the present invention detects collision avoidance targets using image recognition based on machine learning while the vehicle is being driven, acquires position information of the collision avoidance target, and when it is determined that there is a possibility of collision between the vehicle and the collision avoidance target, outputs an alarm to the vehicle driver. This prevents excessive detection and erroneous detection, and reliably and efficiently prevents contact with the collision avoidance target, resulting in excellent safety.

In the collision avoidance system of the present invention, when the image acquisition unit is mounted on a vehicle and has one or more cameras that capture images of at least the rear of the vehicle and the area that is in the driver's blind spot, it is possible to prevent collision avoidance targets from being missed, resulting in excellent reliability of collision avoidance.

In the collision avoidance system according to the present invention, when the output of an alarm is disabled or the type of alarm to be output is switched based on the result of determining whether the vehicle is reversing or not, it is possible to prevent unnecessary (excessive) alarms from being output and effectively avoid collisions.

In the collision avoidance system according to the present invention, if it is possible to select whether to disable the output of an alarm or switch the type of alarm to be output based on the result of determining whether the vehicle is reversing or not, or to output an alarm regardless of whether the vehicle is reversing or not, the conditions for outputting an alarm can be switched (selected) as needed, providing excellent versatility.

In the collision avoidance system of the present invention, a real-time video is created based on images acquired while the vehicle is being driven, and when it is determined that a collision avoidance target is present around the vehicle, a rectangular frame indicating the range of the collision avoidance target is drawn on the real-time video in addition to a horizontal line indicating the distance from the vehicle. This prevents the driver of the vehicle from overlooking the collision avoidance target, and allows the driver to accurately grasp the distance from the vehicle to the collision avoidance target, enabling them to take prompt action to avoid a collision.

In the collision avoidance system according to the present invention, when the images and real-time video acquired by the image acquisition unit are stored in the machine learning setting unit, machine learning is performed using those images and real-time video, and the accuracy of detecting collision avoidance targets is continuously improved.

In the collision avoidance system of the present invention, if images are saved for a period going back a predetermined time from when the machine learning inference unit determines that a collision avoidance target exists, there is a possibility that images will be acquired from a period (time zone) when a collision avoidance target was not detected, even though a collision avoidance target actually existed, and by using these images for machine learning, the detection accuracy of the collision avoidance target is further improved.

In the collision avoidance system of the present invention, the possibility of a collision between the vehicle and a collision avoidance object is determined from the positional relationship between the bottom edge of a rectangular frame and the horizontal line. Therefore , regardless of the posture or orientation of the collision avoidance object, the possibility of a collision between the collision avoidance object and the vehicle is determined with high accuracy according to the distance from the vehicle to the collision avoidance object, and a collision is reliably avoided.

In the collision avoidance system of the present invention, when the length of the horizontal line is expanded or contracted to adjust the collision determination area in which it is determined that there is a possibility of a collision between the vehicle and the object for collision avoidance, the possibility of a collision between the vehicle and the object for collision avoidance is effectively determined only in the intended area, excessive warnings are reduced, and the burden on the driver is also reduced.

1 is a block diagram showing a configuration of a collision avoidance system according to an embodiment of the present invention; This is an image of one frame in a real-time video displayed on the display unit of the collision avoidance system. 4 is an explanatory diagram of a collision determination process in the collision avoidance method executed in the collision avoidance system. FIG. 10 is a flowchart showing a process for saving real-time video in the collision avoidance method executed in the collision avoidance system. 4 is a flowchart showing a collision avoidance process in the collision avoidance method executed in the collision avoidance system.

Next, with reference to the attached drawings, an embodiment of the present invention will be described for understanding the present invention.
A collision avoidance system 10 and a collision avoidance method according to an embodiment of the present invention will be described below with reference to FIGS.
The collision avoidance system 10 and the collision avoidance method prevent contact accidents with various collision avoidance targets including other vehicles and people while driving various vehicles including various heavy machinery. Specifically, while driving the vehicle, a collision between the vehicle and the collision avoidance target can be prevented by detecting the collision avoidance target using image recognition by machine learning. Here, while driving the vehicle includes not only while the vehicle is moving forward or backward, but also while the vehicle is stopped.

As shown in FIG. 1, the collision avoidance system 10 includes a machine learning setting unit 11 that performs machine learning based on learning images that include collision avoidance objects to be detected (e.g., people, vehicles including heavy machinery, obstacles that impede passage, etc.) and constructs a machine learning model.
The collision avoidance system 10 also includes an image acquisition unit 12 that acquires images of the surroundings of the vehicle while the vehicle is being driven, and a display unit 13 that displays a real-time video created based on the images acquired by the image acquisition unit 12.
The collision avoidance system 10 is equipped with a machine learning inference unit 14 that determines whether a collision avoidance target is present around the vehicle based on the image acquired by the image acquisition unit 12 and a machine learning model, and a collision avoidance setting unit 15 that acquires position information of the collision avoidance target detected by the machine learning inference unit 14 and determines the possibility of a collision between the vehicle and the collision avoidance target.
The collision avoidance system 10 also includes an alarm output unit 16 that outputs an alarm to the driver of the vehicle based on the determination of the collision avoidance setting unit 15 .
Furthermore, the collision avoidance system 10 has a vehicle state recognition unit 17 that detects whether or not a reverse signal is being output from the vehicle to determine whether or not the vehicle is moving backward.
The collision avoidance system 10 can be easily retrofitted (installed) on existing vehicles, but can also be incorporated during the manufacturing stage of the vehicle.

Here, a part of the collision avoidance system 10 can be configured by a computer. That is, a program for executing the collision avoidance method is installed in the computer, and the CPU and GPU of the computer execute the program, thereby making the computer function as the above-mentioned machine learning setting unit 11, image acquisition unit 12, machine learning inference unit 14, collision avoidance setting unit 15, and vehicle state recognition unit 17. As the display unit 13, a display integrated with or connected to the computer is preferably used, but if the vehicle is equipped with a car navigation system or a television, the monitor of the display unit 13 may be used as the display unit 13, or the windshield of the vehicle having a display function may be used as the display unit 13.
The machine learning setting unit 11 constructs a machine learning model using a conventionally known algorithm such as a neural network.
The image acquisition unit 12 is mounted on the vehicle and has one or more cameras that capture at least the rear of the vehicle and the area that is a blind spot for the driver (for example, the rear parts on both the left and right sides of the vehicle). The image acquisition unit 12 may have cameras that capture the front and left and right sides of the vehicle, and the number and arrangement of the cameras are appropriately selected according to the capture range (viewing angle) of each camera, etc. When the image acquisition unit 12 has multiple cameras, the display unit 13 may display multiple real-time videos created by the image acquisition unit 12 based on images captured by each camera side by side, or one real-time video may be selected and displayed sequentially from the multiple real-time videos at a predetermined time interval. Alternatively, a real-time video in a direction determined by the collision avoidance setting unit 15 to be a collision possibility may be displayed preferentially.

As shown in FIG. 2, a plurality of (three in this example) horizontal lines 20 are drawn in the real-time video, which are displayed parallel to the width direction of the vehicle and indicate the distance from the vehicle. When the machine learning inference unit 14 determines that a collision avoidance target (a person in this example) 21 exists, a rectangular frame 22 indicating the range of the collision avoidance target 21 is drawn in addition to the horizontal lines 20 in the real-time video. The display color of each horizontal line is appropriately selected, but for example, the horizontal lines may be color-coded in order of proximity to the vehicle, such as red, yellow, and green, to indicate danger, or all may be the same color. In addition, the number of horizontal lines and the distance from the vehicle to each horizontal line (the arrangement interval of the horizontal lines) are appropriately selected. When the collision avoidance target 21 and the vehicle approach each other and the collision avoidance target 21 crosses any of the horizontal lines 20, an alarm can be issued.
In this embodiment, the distance from the vehicle to each horizontal line 20 is displayed, but this display may be omitted, or the driver may operate to switch between display and non-display. Furthermore, in this embodiment, auxiliary lines 23 are drawn alternately with the horizontal lines 20, allowing the driver to know the distance from the vehicle in detail, but the number, arrangement interval, length, display color, etc. of the auxiliary lines may be selected appropriately and may be omitted. In addition, the display color of the rectangular frame 22 is selected appropriately, but the display color of the rectangular frame 22 may be changed or the rectangular frame 22 may be made to flash when a warning is issued. In addition, in FIG. 2, reference numeral 24 denotes a road on which the vehicle is traveling.

Next, a method for determining the possibility of a collision between the vehicle and a collision avoidance target by the collision avoidance setting unit 15 will be specifically described with reference to FIG.
In FIG. 3, a backhoe is detected as a collision avoidance target 25, and three horizontal lines 20 and a rectangular frame 22 indicating the range of the collision avoidance target 25 are drawn, similarly to FIG.
Here, the coordinates of the left and right ends of the n-th horizontal line 20, counting from the side closest to the vehicle, are respectively (XLn, Yn) and (XRn, Yn). Also, the coordinates of the left and right ends of the bottom side of the rectangular frame 22 at a certain time are respectively (XtL, Yt) and (XtR, Yt). The length of the horizontal line 20 is selected appropriately, but it is preferably equal to the width of the vehicle.
At this time, if either or both of XLn≦XtL≦XRn and XLn≦XtR≦XRn are satisfied, and Yt≦Yn is satisfied, the collision avoidance target 25 is located closer to the vehicle than the position of the nth horizontal line 20, and at least a part of the collision avoidance target 25 overlaps with the nth horizontal line 20 (the width of the vehicle). Therefore, the collision avoidance setting unit 15 determines that there is a possibility of a collision with the collision avoidance target 25 if the vehicle continues to move forward, and a predetermined alarm is output by the alarm output unit 16.
Therefore, the collision avoidance setting unit 15 adjusts the collision determination area in which it is determined that there is a possibility of a collision between the vehicle and the collision avoidance object 25 by expanding or contracting the length of the horizontal line 20. In other words, if the length of the horizontal line 20 is long, the collision determination area is set wide, and if the length of the horizontal line 20 is short, the collision determination area is set narrow. It is possible to select which horizontal line 20 must satisfy the above condition to determine that there is a possibility of the vehicle colliding with the collision avoidance object 25.

The warning output unit 16 is required to be able to output a warning to the driver in the form of sound and/or light when the collision avoidance setting unit 15 determines that there is a possibility of a collision between the vehicle and a collision avoidance target 25 (when action is required to avoid a collision between the vehicle and the collision avoidance target 25).
The warning output unit 16 is preferably the above-mentioned display unit 13, a speaker built into or connected to a computer, or a warning light mounted on a vehicle and connected to a computer. For example, the warning can be issued by displaying characters, figures, or symbols (marks, signs) that indicate a danger on the display unit 13, making them emit light or blink, emitting a sound from a speaker, or turning on or blinking a warning light. The sound (warning sound) emitted when warning is issued includes not only a buzzer or chime, but also a voice (word) that indicates the danger of a collision.
The type of warning output from the warning output unit 16 can be appropriately selected (changed) according to the type of collision avoidance target and the distance from the vehicle to the collision avoidance target. For example, as the distance from the vehicle to the collision avoidance target decreases, the warning sound can be changed from a quiet (small) sound to a loud (large) sound, or the color of the warning light can be changed (e.g., green → yellow → red). In addition, when the distance from the vehicle to the collision avoidance target is far, only the display unit can emit light or blink, or the warning light can be turned on or blinked, and when the distance from the vehicle to the collision avoidance target is close, the warning sound can be used in addition. In either case, the driver can be effectively alerted.

The reverse signal detected by the vehicle state recognition unit 17 is an electrical signal output from the vehicle when the vehicle gear is in reverse, and this electrical signal is converted into a digital signal and input to the vehicle state recognition unit 17, thereby determining that the vehicle is moving backward. When no reverse signal is output from the vehicle, the vehicle is determined to be moving forward or stopped (including when the gear is in neutral or parking). Then, the collision avoidance setting unit 15 disables the output of an alarm by the alarm output unit 16 or switches the type of alarm output by the alarm output unit based on the determination result of the vehicle state recognition unit 17.
For example, as described above, even if the machine learning inference unit 14 determines that there is a collision avoidance target in front of the vehicle and the collision avoidance setting unit 15 determines that there is a possibility of a collision between the vehicle and the collision avoidance target based on the position information of the collision avoidance target (the distance from the vehicle to the collision avoidance target), when the vehicle state recognition unit 17 detects a reverse signal and determines that the vehicle is reversing, it is possible to configure so that no alarm is output (the alarm output is disabled) or a weak alarm is output (the type of alarm is switched), and when the reverse signal is not detected and it is determined that the vehicle is not reversing (the vehicle is moving forward or stopped), an alarm (strong alarm) is output from the alarm output unit 16 (the type of alarm is switched).

In addition, even if the machine learning inference unit 14 determines that there is a collision avoidance target behind the vehicle and the collision avoidance setting unit 15 determines that there is a possibility of a collision between the vehicle and the collision avoidance target based on the position information of the collision avoidance target, when the vehicle state recognition unit 17 determines that a reverse signal is not detected and the vehicle is not reversing, the alarm output unit 16 can be set to not output an alarm (disable the alarm output) or to output a weak alarm (switch the type of alarm), and when a reverse signal is detected and it is determined that the vehicle is reversing, an alarm (strong alarm) can be output (switch the type of alarm).
In other words, by disabling the output of an alarm or switching (selecting) the type (strength) of the alarm to be output depending on a combination of the position of the collision avoidance target (the direction in which the collision avoidance target is located as seen from the vehicle and the distance from the vehicle to the collision avoidance target) and the direction in which the vehicle is traveling (the vehicle is reversing, moving forward, or stopped), the driver can be effectively notified of the risk of collision.

In addition, as described above, the collision avoidance setting unit 15 can select whether to disable the output of the alarm by the alarm output unit 16 or to switch the type of alarm output by the alarm output unit 16 based on the determination result of the vehicle state recognition unit 17, or whether the operation of the vehicle state recognition unit 17 is forcibly stopped so that the alarm output by the alarm output unit 16 is output regardless of whether the vehicle is moving backwards or not. By allowing the driver to select either one of the options, versatility is improved. For example, if the operation of the vehicle state recognition unit 17 is forcibly stopped so that the alarm output by the alarm output unit 16 is output regardless of whether the vehicle is moving backwards or not, an alarm is output without fail when a collision avoidance target existing around the vehicle approaches within a certain distance, and the driver can be alerted.
In addition, the images acquired by the image acquisition unit 12 and the real-time video created based on the images are stored in the machine learning setting unit 11. These images and real-time videos are stored in the memory of the computer at the time of acquisition and creation, respectively, but the storage here means that the data is stored in various storage media (not shown) such as a hard disk built into or external to the computer, or a CD-ROM or USB memory connected to the computer, and is in a state where it can be read by the machine learning setting unit 11 when necessary. As a result, the stored images and real-time videos are used for machine learning in the machine learning setting unit 11, and the detection accuracy of the collision avoidance target is improved. Note that the images are stored for a period going back a predetermined time from when the machine learning inference unit 14 determined that a collision avoidance target exists around the vehicle. For example, images for the past 10 frames from the time when the machine learning inference unit 14 detected the collision avoidance target are stored. In addition, the images to be stored are images (raw images) acquired by the image acquisition unit 12 (camera), images drawn with a horizontal line superimposed on the raw image, and images drawn with a horizontal line and a rectangular frame superimposed on the raw image. The real-time video is preferably stored while the collision avoidance system 10 is active, regardless of whether a collision avoidance object is detected or not.

Next, a collision avoidance method will be described for detecting a collision avoidance target using image recognition based on machine learning while the vehicle is being driven, and preventing a collision between the vehicle and the collision avoidance target.
First, in the machine learning process, machine learning is performed based on learning images that include collision avoidance objects to be detected, and a machine learning model is constructed. This is a process for learning in advance various collision avoidance objects (people, various vehicles including various heavy machinery, other obstacles, etc.) that must be avoided while driving the vehicle. This makes it possible to detect whether or not a collision avoidance object exists while the vehicle is actually driving (at the time of recognition), and to identify the type of collision avoidance object.
Next, in an image acquisition process, images of the surroundings of the vehicle are acquired while the vehicle is being driven, and in an object detection process, it is determined whether a collision avoidance object is present around the vehicle based on the images acquired in the image acquisition process and a machine learning model.
Then, when it is determined in the object detection process that a collision avoidance object exists, in a collision determination process, position information of the collision avoidance object detected in the object detection process is acquired and the possibility of collision between the vehicle and the collision avoidance object is determined.

The warning output step outputs a warning to a driver of the vehicle based on the determination by the collision avoidance setting unit. That is, the warning is output when the collision determination step determines that there is a possibility of a collision between the vehicle and the collision avoidance target. The driver can avoid the collision by changing the course of the vehicle or stopping the vehicle.
In addition, in the collision determination step, it is determined whether the vehicle is moving backwards, and the output of the alarm by the alarm output step is invalidated or the type of alarm to be output by the alarm output step is switched. Therefore, depending on whether the vehicle is moving away from or approaching the collision avoidance target, the output of the alarm can be invalidated or the type of alarm to be output can be appropriately switched, thereby preventing excessive and unnecessary alarms from being output.

In addition, in the collision determination step, it may be possible to select whether to determine whether the vehicle is reversing or not and to disable the output of the alarm by the alarm output step or to switch the type of alarm to be output by the alarm output step, or to output the alarm by the alarm output step regardless of whether the vehicle is reversing or not. In this case, as described above, in addition to disabling the output of the alarm or switching the type of alarm to be output based on the positional relationship between the vehicle and the collision avoidance target, it is possible to forcibly output an alarm when the collision avoidance target is present within a certain distance from the vehicle, regardless of whether the vehicle is reversing or not (the tendency of the distance between the vehicle and the collision avoidance target to increase or decrease), and it is possible to use both of them as necessary (for example, depending on the type of the vehicle or the collision avoidance target, etc.).
During the driving of the vehicle, a video display process is executed, and a real-time video is created based on the images acquired in the image acquisition process, and is displayed on a display unit mounted on the vehicle. In the real-time video, one or more horizontal lines are drawn parallel to the width direction of the vehicle and indicate the distance from the vehicle. This allows the driver to check how far various objects around the vehicle are from the vehicle while driving the vehicle. Furthermore, when it is determined in the above-mentioned object detection process that a collision avoidance object exists, a rectangular frame indicating the range of the collision avoidance object is drawn in addition to the horizontal line in the real-time video. Therefore, the driver can easily grasp the distance from the vehicle to the collision avoidance object without missing the collision avoidance object.

Next, the process of saving real-time video in the collision avoidance method executed by the collision avoidance system according to the embodiment of the present invention will be described with reference to FIG.
When the collision avoidance system 10 is started, first, it is confirmed whether there is free space in the video storage folder in which real-time video is stored. Here, the collision avoidance system 10 is preferably started in conjunction with the start of the vehicle engine, but may be started by the driver separately pressing a button, etc. (S1: Step 1).
If it is determined in step 1 that there is no free space in the video storage folder, old real-time video files stored in the video storage folder are deleted. Here, the capacity per real-time video file is predetermined, and old real-time video files are repeatedly deleted until it is determined that there is free space in the video storage folder for at least one file. It is preferable that the files are automatically deleted in order of oldest saved date and time, but it is also possible to allow the driver to select the files to be deleted (S2: step 2).

If it is determined in step 1 that the video storage folder has free space, creation of a new real-time video is started (S3: step 3).
First, images of the outside of the vehicle are captured by cameras mounted on the vehicle, the number and arrangement of which are as described above (S4: step 4).
Then, based on the image acquired in step 4 and the machine learning model, it is determined whether a collision avoidance target is present around the vehicle (S5: step 5).
If it is determined in step 5 that no collision avoidance target exists, a drawn image is created in which a horizontal line 20 (see Figures 2 and 3) is drawn on the image captured by the camera (raw camera image) (S6: step 6).
If it is determined in step 5 that a collision avoidance object exists, a drawn image is created in which a horizontal line 20 is drawn on the image captured by the camera (raw camera image) as well as a rectangular frame 22 (see Figures 2 and 3) indicating the range of the collision avoidance object 21 or 25 (S7: step 7).

The drawn image created in step 6 or step 7 is acquired frame by frame (S8: step 8) and added to the real-time video being created. That is, a computer having a program for executing the collision avoidance method installed therein processes the image (creates the drawn image), and the processed images (drawn images) are connected in sequence frame by frame to create (update) the real-time video (S9: step 9).
Then, it is confirmed whether the capacity of the real-time video file after the new drawn image is added in step 9 has reached the upper limit of the capacity per file (S10: step 10).
If it is not determined in step 10 that the upper limit of the capacity per file has been reached, the process returns to step 4 to continue creating the real-time moving image, and the operations of steps 4 to 10 are repeated (S11: step 11).
If it is determined in step 10 that the upper limit of the capacity per file has been reached, the real-time video being created is saved in a file, and the process returns to step 1 to start creating a new real-time video, and the operations of steps 1 to 10 are repeated (S12: step 12).
While the collision avoidance system 10 is activated, the process of saving real-time video continues.

Next, the collision avoidance process that is performed in parallel with the real-time video saving process shown in FIG. 4 will be described with reference to FIG.
If it is determined in step 5 of FIG. 4 that a collision avoidance target exists, the collision avoidance method shown in FIG. 5 is performed in parallel while the processing from step 7 onward is being performed as the storage processing of the real-time video.
First, it is determined whether the setting (mode) of the collision avoidance system 10 is the digital signal input mode (S13: step 13).
If it is determined in step 13 that the digital signal input mode is selected, it is confirmed whether a reverse signal is present (whether a reverse signal is being output from the vehicle) (S14: step 14).
If it is determined in step 14 that a reverse signal is present, it is determined that the vehicle is moving backwards, and a first alarm is issued (output) (S15: step 15).
If it is determined in step 14 that there is no reverse signal, it is determined that the vehicle is moving forward or stopped, and a second alarm is issued (output) (S16: step 16).
If it is determined in step 13 that the digital signal input mode is not in effect, detection of the digital signal (reverse signal) is not performed, and the third alarm is issued (output) regardless of the vehicle state (whether the vehicle is reversing or not) (S17: step 17).

When any of the first to third alarms is issued, a check is made to see if there is available space in the image storage folder in which the images captured by the camera (raw images) and the processed images (drawn images) are stored (S18: step 18).
If it is determined in step 18 that there is no free space in the image storage folder, the files of old images (raw images and drawn images) stored in the image storage folder are deleted. Here, the deletion of old image files is repeated until it is determined that there is free space equal to or greater than the total capacity of the newly acquired images (raw images and drawn images). It is preferable that the files are automatically deleted in order of the oldest saved date and time, but it is also possible to allow the driver to select the files to be deleted (S19: step 19).
If it is determined in step 18 that the image folder has free space, the images (raw images and drawn images) at the time when the collision avoidance target was detected in step 5 (see FIG. 4) are saved. Here, images from a predetermined period going back from when the collision avoidance target was detected (e.g., 10 frames) are saved. In addition, text data of the type of collision avoidance target detected in step 5 and its location information (e.g., the distance from the vehicle to the collision avoidance target) are also saved (S20: step 20).
In this embodiment, the contents of the first warning and the third warning are the same, but the contents of the first to third warnings are appropriately selected, and may be the same in part or in whole, or may be different. Also, depending on conditions (for example, a combination of the position of a collision avoidance target and the traveling direction of the vehicle), any of the first to third warnings may be omitted (invalidated).

The above describes an embodiment of the present invention, but the present invention is not limited to the configuration described in the above embodiment, and also includes other embodiments and variations that are possible within the scope of the matters described in the claims.

10: Collision avoidance system, 11: Machine learning setting section, 12: Image acquisition section, 13: Display section, 14: Machine learning inference section, 15: Collision avoidance setting section, 16: Alarm output section, 17: Vehicle state recognition section, 20: Horizontal line, 21: Collision avoidance target, 22: Rectangular frame, 23: Auxiliary line, 24: Road, 25: Collision avoidance target

Claims (4)

A collision avoidance system that detects a collision avoidance target using image recognition by machine learning while a vehicle is being driven, and prevents a collision between the vehicle and the collision avoidance target,
a machine learning setting unit that performs machine learning based on learning images including the collision avoidance target to be detected and constructs a machine learning model; an image acquisition unit that acquires images of the periphery of the vehicle while the vehicle is being driven; a machine learning inference unit that determines whether the collision avoidance target is present around the vehicle based on the images acquired by the image acquisition unit and the machine learning model; a collision avoidance setting unit that acquires position information of the collision avoidance target detected by the machine learning inference unit and determines the possibility of a collision between the vehicle and the collision avoidance target; an alarm output unit that outputs an alarm to a driver of the vehicle based on the determination of the collision avoidance setting unit; and a real-time video created based on the images is displayed. a display unit, wherein one or more horizontal lines are drawn on the real-time video parallel to a width direction of the vehicle and indicating a distance from the vehicle, and when the machine learning inference unit determines that a collision avoidance target is present, a rectangular frame indicating the range of the collision avoidance target is drawn on the real-time video in addition to the horizontal line, and the collision avoidance setting unit adjusts a collision determination area in which it is determined that there is a possibility of collision between the vehicle and the collision avoidance target by expanding or contracting a length of the horizontal line, and the possibility of collision between the vehicle and the collision avoidance target is determined from the positional relationship between a bottom side of the rectangular frame and the horizontal line. The collision avoidance system according to claim 1, wherein the images and the real-time video are stored in the machine learning setting unit. The collision avoidance system according to claim 1, characterized in that the collision avoidance setting unit switches the type of warning output by the warning output unit depending on the type of the collision avoidance target and the distance from the vehicle to the collision avoidance target. 4. The collision avoidance system according to claim 3 , wherein the alarm output unit switches the type of alarm by a volume of sound and/or by turning on, blinking or changing the color of a light.

Images