JP4840438B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to a vehicle periphery monitoring device that detects and displays an obstacle present around a vehicle.

  2. Description of the Related Art Conventionally, a technique for notifying a user of the presence of an obstacle around a vehicle by displaying an image obtained by capturing a situation around the vehicle on a display unit is known.

In recent years, a technique has also been proposed in which obstacle detection information is superimposed and displayed on an image obtained by capturing a situation around a vehicle. For example, Patent Document 1 discloses a technique in which detection information detected by an obstacle detection sensor is converted into a symbol, and the symbol is superimposed on an image captured by a camera that captures the periphery of the vehicle and displayed on a monitor. .
Japanese Patent Laid-Open No. 2005-45602

  However, the technique disclosed in Patent Document 1 has a problem that it is difficult for the user to distinguish the moving direction of an obstacle. Specifically, in the technique disclosed in Patent Document 1, a symbol indicating the obstacle detection distance displayed on the monitor and a detection range indicating where the obstacle displayed on the monitor is within the detection range of the obstacle detection sensor is displayed on the monitor. Since it is only displayed superimposed on the obstacle inside, it is difficult for the user to recognize the moving direction such as whether the obstacle is approaching or moving away from the own vehicle. It was difficult to distinguish the moving direction of objects.

  The present invention has been made in view of the above-described conventional problems, and an object of the present invention is to distinguish the moving direction of an obstacle with respect to the host vehicle when detecting and displaying an obstacle existing around the vehicle. It is an object of the present invention to provide a vehicle periphery monitoring device that can be performed more easily.

In order to solve the above-described problem, the vehicle periphery monitoring device according to claim 1 is imaged by an imaging unit that images the periphery of the vehicle, an obstacle detection unit that detects an obstacle around the vehicle, and the imaging unit. And a display unit that displays an image. When the obstacle detection unit detects an obstacle, the vehicle periphery monitoring device displays a mark indicating the obstacle superimposed on the image. Once an obstacle is detected by the object detection unit, a detection data acquisition unit that acquires the detection data of the obstacle over time, and the detection data acquired over time by the detection data acquisition unit are accumulated and stored. And a plurality of new and old marks corresponding to the new and old detection data are superimposed on the image based on the accumulated storage unit and the new and old detection data stored in the cumulative storage unit. And display As old and new distinction serial marks is possible, it is characterized in that it comprises a display control unit for displaying by changing the size of the individual of the mark to become smaller as the older the mark.

According to this, with respect to the obstacle detected by the obstacle detection unit, the plurality of old and new marks indicating the obstacle are individually changed so that the old and new can be distinguished, and an image captured by the imaging unit (hereinafter referred to as an image) , Referred to as a captured image), and a mark that changes as the obstacle moves with time is displayed superimposed on the captured image. Therefore, the movement direction of the obstacle can be expressed by the change of these marks. Therefore, according to the above configuration, when an obstacle existing around the vehicle is detected and displayed, the user can more easily distinguish the moving direction of the obstacle with respect to the own vehicle.
In addition, according to this, when an obstacle is approaching the host vehicle, it can be expressed as if the obstacle is approaching by displaying the larger the mark on the host vehicle side. When the obstacle is moving away from the host vehicle, it can be expressed as if the obstacle is moving away by displaying a smaller mark on the host vehicle side.
In order to solve the above problem, the vehicle periphery monitoring device according to claim 2 is imaged by an imaging unit that images the periphery of the vehicle, an obstacle detection unit that detects an obstacle around the vehicle, and an imaging unit. And a display unit that displays an image. When the obstacle detection unit detects an obstacle, the vehicle periphery monitoring device displays a mark indicating the obstacle superimposed on the image. Once an obstacle is detected by the object detection unit, a detection data acquisition unit that acquires the detection data of the obstacle over time, and the detection data acquired over time by the detection data acquisition unit are accumulated and stored. And a plurality of new and old marks corresponding to the new and old detection data are superimposed on the image based on the accumulated storage unit and the new and old detection data stored in the cumulative storage unit. And display In order to make it possible to distinguish between new and old marks, a display control unit is provided that displays the marks by changing the transmittance of each mark so that the older the mark, the higher the transmittance. .
Therefore, the movement direction of the obstacle can be expressed by the change of these marks. Therefore, according to the above configuration, when an obstacle existing around the vehicle is detected and displayed, the user can more easily distinguish the moving direction of the obstacle with respect to the own vehicle.
In addition, according to this, when an obstacle is approaching the host vehicle, it can be expressed as if the obstacle is approaching by displaying a lower transmittance for the mark on the host vehicle side. When the obstacle is moving away from the host vehicle, the higher the transmittance of the mark on the host vehicle side, the higher the transmittance, so that it can be expressed as if the obstacle is moving away.
In order to solve the above-described problem, the vehicle periphery monitoring device according to claim 3 is imaged by an imaging unit that images the periphery of the vehicle, an obstacle detection unit that detects an obstacle around the vehicle, and the imaging unit. And a display unit that displays an image. When the obstacle detection unit detects an obstacle, the vehicle periphery monitoring device displays a mark indicating the obstacle superimposed on the image. Once an obstacle is detected by the object detection unit, a detection data acquisition unit that acquires the detection data of the obstacle over time, and the detection data acquired over time by the detection data acquisition unit are accumulated and stored. And a plurality of new and old marks corresponding to the new and old detection data are superimposed on the image based on the accumulated storage unit and the new and old detection data stored in the cumulative storage unit. And display So as to allow new and old distinction serial marks, it is characterized in that it comprises a hue of each of the marks, saturation, and a display control unit for displaying by changing at least one of lightness.
Therefore, the movement direction of the obstacle can be expressed by the change of these marks. Therefore, according to the above configuration, when an obstacle existing around the vehicle is detected and displayed, the user can more easily distinguish the moving direction of the obstacle with respect to the own vehicle.
Also, according to this, when the obstacle is approaching the host vehicle, for example, the mark on the host vehicle side is displayed with a hue that is more dangerous, higher saturation, or higher brightness. This makes it possible to express as if an obstacle is approaching. On the other hand, when an obstacle is moving away from the host vehicle, for example, the mark on the host vehicle side is displayed with a safer hue, lower saturation, or lower brightness. It is possible to express as if they are moving away.
In order to solve the above-described problem, the vehicle periphery monitoring apparatus according to claim 4 is imaged by an imaging unit that images both of the periphery, an obstacle detection unit that detects an obstacle around the vehicle, and an imaging unit. And a display unit that displays an image. When the obstacle detection unit detects an obstacle, the vehicle periphery monitoring device displays a mark indicating the obstacle superimposed on the image. Once an obstacle is detected by the object detection unit, a detection data acquisition unit that acquires the detection data of the obstacle over time, and the detection data acquired over time by the detection data acquisition unit are accumulated and stored. And a plurality of new and old marks corresponding to the new and old detection data are superimposed on the image based on the accumulated storage unit and the new and old detection data stored in the cumulative storage unit. And display In order to make it possible to distinguish between new and old marks, a display control unit that changes and displays at least one of hue, saturation, and brightness of each of the marks is provided, and the obstacle detection unit includes: A plurality of distance sensors each having a different detection area, and the display control unit displays the mark with a central axis of directivity of the distance sensor and a central axis of overlap of the detection areas of the distance sensor. As a detection area axis that is an area to be displayed, the mark is displayed at a location on the detection area axis according to the distance of the obstacle detected by the distance sensor, and any distance sensor of the plurality of distance sensors The detection area axis for displaying the mark is determined in accordance with whether an obstacle is detected.
Therefore, the movement direction of the obstacle can be expressed by the change of these marks. Therefore, according to the above configuration, when an obstacle existing around the vehicle is detected and displayed, the user can more easily distinguish the moving direction of the obstacle with respect to the own vehicle.
Further, according to this, at least one of the plurality of detection area axes respectively corresponding to the plurality of distance sensors, depending on which of the plurality of distance sensors has detected the obstacle. A mark indicating the obstacle is displayed at a location on the axis according to the distance of the obstacle detected by the distance sensor. Further, since the detection area axis is fixed according to the installation location and the installation direction of the distance sensor, the position in the captured image is determined in advance when the installation location and the imaging direction of the imaging unit are fixed. . Therefore, according to the above configuration, it is possible to indicate the obstacle in the captured image by this mark while limiting the place where the mark indicating the obstacle in the captured image is displayed to a partial area in advance. Become. As a result, compared to the process of accurately obtaining the position of the obstacle in the captured image and displaying the mark superimposed on the position corresponding to this position in the captured image, the obstacle is displayed in the captured image by a simpler calculation. It is possible to superimpose and display a mark indicating.
Furthermore, according to this, since it becomes possible to display a mark indicating an obstacle using the axis of the center of the detection areas of the plurality of distance sensors as the detection area axis, a location on the detection area axis can be displayed. It becomes possible to indicate the position of the obstacle more accurately by the mark displayed on the screen.

The vehicle periphery monitoring apparatus according to claim 5 is characterized in that it is possible to distinguish between old and new marks by changing and displaying the size of each mark so that the older the mark becomes smaller. .

  As a result, when the obstacle is approaching the host vehicle, it can be expressed as if the obstacle is approaching by displaying the larger the mark on the host vehicle side. When the vehicle is moving away from the host vehicle, it can be expressed as if the obstacle is moving away by displaying a smaller mark on the vehicle side.

Further, the vehicle periphery monitoring apparatus according to claim 6 is characterized in that display is performed by changing the transmittance of each of the marks so that the older the mark, the higher the transmittance.

  As a result, when the obstacle is approaching the host vehicle, it is possible to express as if the obstacle is approaching by displaying the transmittance as low as the mark on the host vehicle side, When the obstacle is moving away from the host vehicle, it is possible to express as if the obstacle is moving away by displaying a higher transmittance for the mark on the host vehicle side.

The vehicle periphery monitoring device according to claim 7 is characterized in that at least one of hue, saturation, and brightness of each of the marks is changed and displayed.

  According to this, when an obstacle is approaching the host vehicle, for example, the mark on the host vehicle side is displayed with a more dangerous hue, higher saturation, or higher brightness. It becomes possible to express as if an obstacle is approaching. On the other hand, when an obstacle is moving away from the host vehicle, for example, the mark on the host vehicle side is displayed with a safer hue, lower saturation, or lower brightness. It is possible to express as if they are moving away.

In the vehicle periphery monitoring apparatus according to claim 8, the obstacle detection unit is a plurality of distance sensors each having a different detection area, and the display control unit is a central axis of directivity of the distance sensor. As a detection area axis that is an area for displaying the mark, the mark is displayed at a location on the detection area axis according to the distance of the obstacle detected by the distance sensor, and the plurality of distance sensors A detection area axis for displaying the mark is determined according to which of the distance sensors detects an obstacle.

  According to this, on the axis of at least one of a plurality of detection area axes respectively corresponding to the plurality of distance sensors, depending on which of the plurality of distance sensors has detected an obstacle. The mark indicating the obstacle is displayed at the location according to the distance of the obstacle detected by the distance sensor. Further, since the detection area axis is fixed according to the installation location and the installation direction of the distance sensor, the position in the captured image is determined in advance when the installation location and the imaging direction of the imaging unit are fixed. . Therefore, according to the above configuration, it is possible to indicate the obstacle in the captured image by this mark while limiting the place where the mark indicating the obstacle in the captured image is displayed to a partial area in advance. Become. As a result, compared to the process of accurately obtaining the position of the obstacle in the captured image and displaying the mark superimposed on the position corresponding to this position in the captured image, the obstacle is displayed in the captured image by a simpler calculation. It is possible to superimpose and display a mark indicating.

The vehicle periphery monitoring device according to claim 9 is characterized in that the detection data is information on a distance and a position of an obstacle detected by the obstacle detection unit. As in the ninth aspect, the obstacle detection unit may detect the distance and position information of the obstacle, and the detected distance and position information may be used as detection data.

  According to this, since it is possible to display a mark indicating an obstacle using the axis of the overlap of the detection areas of a plurality of distance sensors as the detection area axis, it is displayed at a position on the detection area axis. The marked mark makes it possible to indicate the position of the obstacle more accurately.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the overall configuration of a vehicle periphery monitoring device 100 to which the present invention is applied. A vehicle periphery monitoring device 100 shown in FIG. 1 is mounted on a vehicle and includes an obstacle sensor 1, a camera 2, a display 3, a control device 4, and a storage device 5, and includes a CAN (Controller Area Network) and the like. Are connected to each other via an in-vehicle LAN conforming to the communication protocol. Hereinafter, a vehicle equipped with the vehicle periphery monitoring device 100 is referred to as a host vehicle.

  The obstacle sensor 1 detects an obstacle around the rear of the vehicle. Therefore, the obstacle sensor 1 functions as an obstacle detection unit. For example, a plurality of obstacle sensors 1 are installed on a rear bumper at the rear of the vehicle, and detect obstacles around the rear of the vehicle.

  In the present embodiment, an ultrasonic sensor is used as the obstacle sensor 1. Hereinafter, as shown in US1 to US4 in FIG. 2, a case where four ultrasonic sensors US1 to US4 are installed at the rear of the vehicle will be described as an example. Ultrasonic sensor US1-US4 is a distance sensor used in order to detect the distance with an obstacle by transmitting an ultrasonic wave and receiving the reflected wave. In addition, the ultrasonic sensors US1 to US4 include information on the transmitted AC pulse signal (hereinafter referred to as a transmitted pulse signal) and information on the received reflected AC pulse signal (hereinafter referred to as a received pulse signal). Is output to the control device 4. For example, the ultrasonic sensors US1 to US4 transmit ultrasonic waves in order at intervals of a predetermined time. For example, ultrasonic waves are transmitted in the order of the ultrasonic sensor US1, the ultrasonic sensor US2, the ultrasonic sensor US3, and the ultrasonic sensor US4 with an interval of 50 msec. As a result, it is possible to distinguish from which ultrasonic sensor the reflected wave of the ultrasonic wave transmitted is received. In addition, the timing of ultrasonic wave transmission from the ultrasonic sensor US1, the ultrasonic sensor US2, the ultrasonic sensor US3, and the ultrasonic sensor US4 is controlled by the control device 4. In the following, one set of ultrasonic waves is transmitted in the order of the ultrasonic sensor US1, the ultrasonic sensor US2, the ultrasonic sensor US3, and the ultrasonic sensor US4, and then returns to the ultrasonic sensor US1 and makes a round. To do.

  The camera 2 is installed in the rear part of the vehicle and images the situation around the rear part of the vehicle. Therefore, the camera 2 functions as an imaging unit. Further, the camera 2 outputs image data around the rear of the vehicle obtained by imaging to the control device 4. In addition, the camera 2 is installed so that each detection area of the ultrasonic sensors US1 to US4 is included in the imaging range, for example.

  The display 3 displays an image based on the image captured by the camera 2. Therefore, the display 3 functions as a display unit of claims. The display 3 also displays a superimposed image described later. Note that the display 3 may be configured to use a display of a car navigation device.

  The control device 4 is mainly composed of a microcomputer including a CPU, ROM, RAM, backup RAM, etc. (all not shown), and executes various processes by executing various control programs stored in the ROM. To do. The control device 4 obtains the distance to the obstacle and determines the position of the obstacle based on the information on the transmission pulse signal and the information on the reception pulse signal output from the ultrasonic sensors US1 to US4. The distance to the obstacle is obtained by calculating the time taken from the transmission of the ultrasonic wave to the reception of the reflected wave based on the information of the transmission pulse signal and the information of the reception pulse signal. Further, the position of the obstacle is determined according to which ultrasonic sensor has received the reflected wave of the ultrasonic wave transmitted from the ultrasonic sensor based on the information of the transmission pulse signal and the information of the reception pulse signal. . The position of the obstacle is determined by selecting any detection area axis from among a plurality of virtual detection area axes. Therefore, the information on the detection area axis corresponds to the information on the position of the obstacle.

  Here, the detection area axis and the process of determining the position of the obstacle will be described. First, the detection area axis is an axis that is virtually provided to indicate the approximate position of the obstacle in the captured image. In this embodiment, the detection axis is the central axis of the ultrasonic sensor and the detection. The central axis of each directivity overlap of a plurality of ultrasonic sensors having different areas is used as a detection area axis. The center here refers to the center in both the horizontal direction and the vertical direction.

  More specifically, as shown in FIG. 2, the central axis of the sensor detection area SA1 of the ultrasonic sensor US1 is the detection area axis A1, the central axis of the sensor detection area SA2 of the ultrasonic sensor US2 is the detection area axis A2, The central axis of the sensor detection area SA3 of the sonic sensor US3 is a detection area axis A3, and the central axis of the sensor detection area SA4 of the ultrasonic sensor US4 is a detection area axis A4. Further, as shown in FIG. 2, the axis of the center of the adjacent sensor detection area SCA1 that is an overlap of the sensor detection area SA1 of the ultrasonic sensor US1 and the sensor detection area SA2 of the ultrasonic sensor US2 is defined as the detection area axis CA1 and the ultrasonic wave. The central axis of the adjacent sensor detection area SCA2 that is the overlap of the sensor detection area SA2 of the sensor US2 and the sensor detection area SA3 of the ultrasonic sensor US3 is the detection area axis CA2, and the sensor detection area SA3 of the ultrasonic sensor US3 is super The axis of the center of the adjacent sensor detection area SCA3 that overlaps the sensor detection area SA4 of the sonic sensor US4 is set as a detection area axis CA3. The sensor detection areas SA1 to SA4 correspond to the directivities of the ultrasonic sensors US1 to US4, respectively.

  For determining the position of the obstacle, the reflected ultrasonic wave transmitted from the ultrasonic sensor US1 is received by the ultrasonic sensor US1, and the magnitude of the reflected wave is equal to or greater than a predetermined threshold. Then, the detection area axis A1 is selected. In addition, when the reflected ultrasonic wave transmitted from the ultrasonic sensor US1 is received by the ultrasonic sensor US2 and the magnitude of the reflected wave is equal to or greater than a predetermined threshold, the detection area axis CA1 is selected. . In addition, when the reflected ultrasonic wave transmitted from the ultrasonic sensor US2 is received by the ultrasonic sensor US1 and the magnitude of the reflected wave is equal to or greater than a predetermined threshold, the detection area axis CA1 is selected. . Further, when the reflected ultrasonic wave transmitted from the ultrasonic sensor US2 is received by the ultrasonic sensor US2 and the magnitude of the reflected wave is equal to or greater than a predetermined threshold, the detection area axis A2 is selected. . When the reflected ultrasonic wave transmitted from the ultrasonic sensor US2 is received by the ultrasonic sensor US3 and the magnitude of the reflected wave is equal to or greater than a predetermined threshold, the detection area axis CA2 is selected. . In addition, when the reflected ultrasonic wave transmitted from the ultrasonic sensor US3 is received by the ultrasonic sensor US2, and the magnitude of the reflected wave is equal to or greater than a predetermined threshold, the detection area axis CA2 is selected. . Furthermore, when the reflected ultrasonic wave transmitted from the ultrasonic sensor US3 is received by the ultrasonic sensor US3 and the magnitude of the reflected wave is equal to or greater than a predetermined threshold, the detection area axis A3 is selected. . Further, when the reflected ultrasonic wave transmitted from the ultrasonic sensor US3 is received by the ultrasonic sensor US4 and the magnitude of the reflected wave is equal to or greater than a predetermined threshold, the detection area axis CA3 is selected. . In addition, when the reflected ultrasonic wave transmitted from the ultrasonic sensor US4 is received by the ultrasonic sensor US3 and the magnitude of this reflected wave is equal to or greater than a predetermined threshold, the detection area axis CA3 is selected. . When the reflected ultrasonic wave transmitted from the ultrasonic sensor US4 is received by the ultrasonic sensor US4 and the magnitude of the reflected wave is equal to or greater than a predetermined threshold, the detection area axis A4 is selected. .

  Note that the magnitude of the reflected wave may be a configuration in which the peak of the received pulse signal is at a height, or may be a configuration in which the peak is a width at a predetermined height of the received pulse signal. In the following description, the case where the peak height of the received pulse signal is used as the magnitude of the reflected wave will be described as an example. The predetermined threshold here is a threshold for determining whether the peak of the received pulse signal is derived from an obstacle, and can be arbitrarily set according to each of the above cases. Is. For example, when an ultrasonic wave is transmitted and a reflected wave is received by the same ultrasonic sensor, an obstacle near the central axis of the sensor detection area of the ultrasonic sensor can be detected. It may be set to a value that does not detect an obstacle near the central axis of the adjacent sensor detection area between the ultrasonic sensor and the adjacent ultrasonic sensor. In addition, when ultrasonic waves are transmitted on one side of the ultrasonic sensors arranged next to each other and reflected waves are received on the other side, near the center axis of the adjacent sensor detection area of those ultrasonic sensors These obstacles may be detected, and may be set to a value that does not detect an obstacle near the center axis of the sensor detection area of the ultrasonic sensors.

  In addition, a process for detecting an obstacle and obtaining a distance to the obstacle (hereinafter referred to as an obstacle detection process) is a combination of transmitting an ultrasonic wave and receiving a reflected wave with the same ultrasonic sensor. , And one of the ultrasonic sensors arranged next to each other, the ultrasonic wave is transmitted, and the other is performed for each combination of receiving the reflected wave. Here, the flow of the obstacle detection process in the control device 4 will be described with reference to FIG. FIG. 3 is a flowchart showing a flow of obstacle detection processing in the control device 4. This flow is started, for example, when a shift position sensor (not shown) detects that the shift lever has entered the reverse position (R).

  First, in step S1, initialization is performed and the process proceeds to step S2. In step S2, an ultrasonic wave is transmitted from the ultrasonic sensor, and the process proceeds to step S3. In addition, the timing which transmits an ultrasonic wave from an ultrasonic sensor shall shift | deviate for every fixed time so that the ultrasonic wave transmission in each of ultrasonic sensor US1-US4 may not fog.

  In step S3, the reflected wave of the ultrasonic wave transmitted from the ultrasonic sensor is received, and the process proceeds to step S4. Note that this reception is performed by the same ultrasonic sensor as the ultrasonic sensor that transmits the ultrasonic wave, and when the reception is performed by the ultrasonic sensor that is adjacent to the ultrasonic sensor that transmits the ultrasonic wave. There is. For example, when an ultrasonic wave is transmitted from the ultrasonic sensor US1, there is a flow of reception by the ultrasonic sensor US1 and reception by the ultrasonic sensor US2.

  In step S4, a threshold value and a determination counter upper limit value (hereinafter referred to as determination counter upper limit value) are set, and the process proceeds to step S5. The threshold here refers to the predetermined threshold described above. Therefore, in step S4, when ultrasonic waves are transmitted and reflected waves are received by the same ultrasonic sensor, and when ultrasonic waves are transmitted and reflected waves are received by different ultrasonic sensors. A threshold value corresponding to each of these is set. In addition, the structure by which a different threshold value is set according to each of ultrasonic sensor US1-US4 may be sufficient. The determination counter indicates the number of times that a series of processing such as transmission of ultrasonic waves, reception of reflected waves, and comparison with a threshold value to be described later is repeated. The determination counter upper limit value is a received pulse signal. This indicates the number of times that the above-described series of processing needs to be repeated in order to determine that the peak is not derived from noise but derived from an obstacle. The determination counter upper limit value is set to 3, for example.

  In step S5, it is determined whether or not the peak height of the received pulse signal is equal to or greater than a threshold value. And when it determines with more than a threshold value (it is Yes at step S5), it moves to step S7. If it is not determined that the threshold value is exceeded (No in step S5), the process proceeds to step S6.

  In step S6, the set determination counter upper limit value is cleared, and empty (no data) is stored in the storage device 5 as distance data to the obstacle (hereinafter referred to as distance data). In addition, when the value of the determination counter is added as will be described later, this value is also cleared. And it returns to step S2 and repeats a flow.

  In step S7, the distance to the obstacle is calculated based on the information on the transmission pulse signal and the information on the reception pulse signal, and the process proceeds to step S8. In step S8, the value of the determination counter is incremented by 1, and the process proceeds to step S9.

  In step S9, it is determined whether or not the value of the determination counter has reached the determination counter upper limit value. If it is determined that the determination counter upper limit value has been reached (Yes in step S9), it is determined that an obstacle has been detected, and the process proceeds to step S11. If it is not determined that the determination counter upper limit value has been reached (No in step S9), the process proceeds to step S10. In step S10, transmission of ultrasonic waves and reception of reflected waves are performed again, and the flow returns to step S5 to repeat the flow.

  In step S11, the obtained distance data to the obstacle is stored in the storage device 5, and the flow returns to step S2 to repeat the flow. The distance data is stored in the storage device 5 for each detection area axis selected as described above. As an example, in the flow when the ultrasonic sensor US1 receives the reflected wave of the ultrasonic wave transmitted from the ultrasonic sensor US1, the distance data is stored in the storage device 5 in association with the information indicating the detection area axis A1. The In the flow when the ultrasonic sensor US2 receives the reflected ultrasonic wave transmitted from the ultrasonic sensor US1, the distance data is stored in the storage device 5 in association with the information indicating the detection area axis CA1. That is, once an obstacle is detected by the ultrasonic sensor, the distance data of the obstacle and the information of the detection area axis are acquired over time and stored in the storage device 5. Therefore, the control device 4 also functions as a detection data acquisition unit.

  For example, the storage of data in the storage device 5 is repeatedly performed for each set described above, and when an obstacle is detected during the one set, the distance data as described above. In addition, information on the detection area axis is stored, and when no obstacle is detected during this one set, for example, information indicating that no data exists is stored.

  In the present embodiment, the configuration in which the threshold value setting in step S4 and the upper limit value of the determination counter are set after the reception of the reflected wave in step S3 is shown, but the present invention is not necessarily limited thereto. For example, a configuration may be used in which a threshold value is set and an upper limit value of a determination counter is set before reception of a reflected wave, and a setting of a threshold value and an upper limit value of a determination counter are simultaneously performed at the time of initialization in step S1. It may be. Further, the threshold value and the determination counter upper limit value may be set according to the noise condition by monitoring the noise.

  Note that this flow ends when the shift position sensor detects that the shift lever has entered a position other than the reverse position (R).

  Returning to FIG. 1, the storage device 5 is a memory in which stored data can be electrically rewritten, and the detection data acquired by the control device 4 over time (that is, the obstacle distance data and the detection area described above). Axis information) is accumulated and stored. Therefore, the storage device 5 functions as a cumulative storage unit in the claims. The storage device 5 stores detection data for the past several times for each detection area axis, and is erased from old detection data each time new detection data is stored.

  Moreover, the control apparatus 4 performs a superimposition display process, when it determines with having detected the obstruction. Here, the flow of the superimposed display process in the control device 4 will be described with reference to FIG. FIG. 4 is a flowchart showing the flow of the superimposed display process in the control device 4. This flow is also started when, for example, the shift position sensor detects that the shift lever has entered the reverse position (R).

  First, in step S21, the current captured image data is acquired from the camera 2, and the process proceeds to step S22. In step S22, information for the past several times is read from the storage device 5, and the process proceeds to step S23. In the present embodiment, information for the past five times is read as an example. Specifically, information for the past five times is read for each of the seven axes of detection area axes A1 to A4 and detection area axes CA1 to CA3.

  In step S23, the relative movement status of the obstacle relative to the host vehicle (hereinafter referred to as the obstacle status) is determined based on the read information for the past five times, and the process proceeds to step S24. Specifically, when the latest information stored in the storage device 5 is the distance data of the obstacle and the information of the detection area axis, the obstacle is detected by the vehicle based on the read old and new distance data. To determine whether it is approaching, moving away, or stopped. Further, when the information indicates that there is no distance data for the past five times stored in the storage device 5, the determination of the obstacle state is not performed.

  In step S24, a display mark determination process is performed, and the process proceeds to step S25. In the display mark determination process, the seven axes of the detection area axes A1 to A4 and the detection area axes CA1 to CA3 are superimposed and displayed in the captured image based on the read information and obstacle status for the past five times. A mark to be used (hereinafter referred to as a display mark) is determined. As the determination of the display mark, the detection area axis for displaying the display mark is selected based on the read detection area axis information. Further, based on the distance data for the past five times, it is determined to display a display mark at a location according to the distance data on the selected detection area axis. Further, based on the obstacle status, for example, when the obstacle is approaching the host vehicle, it is determined that the display mark is displayed in a dangerous color (red, etc.), and the obstacle moves away from the host vehicle. It is determined that the display mark is displayed in a safety color (blue or the like) when the vehicle is moving, and the display mark is displayed in an intermediate color (yellow or the like) when the obstacle is stopped with respect to the host vehicle. In addition, the size of the old and new display marks is determined so that the display marks (that is, older display marks) based on older distance data (that is, older detection data) become smaller, and older distance data. The transmittance of the old and new display marks is determined so that the transmittance of the display mark based on is higher. In addition, when the latest information read for each of the seven axes of the detection area axes A1 to A4 and the detection area axes CA1 to CA3 is information indicating that the detection data does not exist, The display mark is not displayed.

  In step S25, a superimposed image is created by superimposing the display mark determined in step S24 on the captured image, data of this superimposed image is output to the display 3, and the flow returns to step S21 to repeat the flow. In addition, a known method may be used as a method for displaying the mark superimposed on the captured image. The display 3 displays a superimposed image according to the data of the superimposed image.

  Note that this flow ends when the shift position sensor detects that the shift lever has entered a position other than the reverse position (R).

  As described above, the control device 4 superimposes a plurality of new and old display marks corresponding to the new and old detection data on the captured image based on the new and old detection data stored in the storage device 5. In addition, the display marks are changed and displayed so that the display marks can be distinguished from each other. Therefore, the control device 4 also functions as a display control unit in the claims.

  Subsequently, the display of the superimposed image on the display 3 will be described with reference to FIGS. FIG. 5 is a diagram virtually showing the detection area axis in the captured image. FIGS. 6A to 7C are diagrams illustrating an example of a superimposed image in which a display mark is superimposed on a captured image. In the present embodiment, the display mark is an ellipse.

  As shown in FIG. 5, the places where the display marks in the captured image are displayed are places corresponding to the detection area axes A1 to A4 and the detection area axes CA1 to CA3. Further, the seven lines in FIG. 5 corresponding to the detection area axes A1 to A4 and the detection area axes CA1 to CA3 virtually indicate the detection area axes in the captured image, and are displayed in the superimposed image. Shall not be. Note that the detection area axis may also be displayed in the superimposed image.

  For example, when an obstacle is approaching the host vehicle, as shown in FIG. 6A, the display mark on the host vehicle side is displayed larger and the transmittance is displayed lower. Further, when the obstacle is moving in the left-right direction of the host vehicle, as shown in FIG. 6B, the display mark in the direction toward the obstacle is displayed larger and the transmittance is displayed lower. It will be. Note that the superimposed image may be an image with the same viewpoint as the captured image as illustrated in FIG. 6A, or by converting the captured image into an image with a viewpoint looking down on the ground surface from the vertical direction, It is good also as a bird's-eye view image as shown in FIG.6 (b).

  Further, when an obstacle that extends widely in the left-right direction of the host vehicle, such as a wall, is detected, display marks for a plurality of detection area axes are displayed as shown in FIGS. 7 (a) to 7 (c). Will be. FIG. 7A is a display example of a superimposed image when the own vehicle is approaching the wall, and FIG. 7B is a superimposed image when the own vehicle is stopped with respect to the wall. FIG. 7C is a display example of an image, and FIG. 7C is a display example of a superimposed image when the host vehicle is moving away from the wall. In addition, the superimposed images illustrated in FIGS. 7A to 7C are bird's-eye images.

  According to the above configuration, with respect to the obstacle detected by the obstacle sensor 1 (that is, the ultrasonic sensors US1 to US4), a plurality of new and old display marks indicating the obstacle can be individually distinguished from each other. Since the size and transmittance of the camera 2 are changed and superimposed on the captured image of the camera 2 and displayed on the display 3, a display mark whose size and transmittance change as the obstacle moves with time is displayed on the captured image. It is displayed superimposed. Specifically, when an obstacle is approaching the host vehicle, it can be expressed as if the obstacle is approaching by displaying the display mark on the host vehicle side larger. When the obstacle is moving away from the host vehicle, it can be expressed as if the obstacle is moving away by displaying a smaller display mark on the host vehicle side. In addition, when an obstacle is approaching the host vehicle, it is possible to express as if the obstacle is approaching by displaying the transmittance as low as the display mark on the host vehicle side, When the obstacle is moving away from the host vehicle, it is possible to express as if the obstacle is moving away by displaying a higher transmittance as the display mark on the host vehicle side. Therefore, the moving direction of the obstacle can be expressed by the change of these display marks. Therefore, according to the above configuration, when an obstacle existing around the host vehicle is detected and displayed, the user can more easily distinguish the moving direction of the obstacle with respect to the host vehicle.

  Furthermore, according to the above configuration, the detection corresponding to each of the ultrasonic sensors US1 to US4 depending on which ultrasonic wave transmitted from which ultrasonic sensor is received by which ultrasonic sensor. A display mark is displayed at a location according to the distance of the obstacle detected by the ultrasonic sensor on at least one of the seven axes of the area axes A1 to A4 and the detection area axes CA1 to CA3. The detection area axes A1 to A4 and the detection area axes CA1 to CA3 are fixed according to the installation locations and installation directions of the ultrasonic sensors US1 to US4, so that the installation location and imaging direction of the camera 2 are fixed. If it is, the position in the captured image is determined in advance. Therefore, according to the above configuration, the display mark display location in the captured image is limited in advance to a partial area (that is, on the seven axes of detection area axes A1 to A4 and detection area axes CA1 to CA3). However, this display mark can indicate an obstacle in the captured image. In addition, as the in-vehicle display is mainly a small display of about 6.5 to 8 inches, even if the display position of the display mark is limited to the detection area axis, the target is actually the target for the user. The display mark does not appear far away from the obstacle. As a result, the obstacle in the captured image can be calculated by a simpler calculation compared to the process in which the position of the obstacle in the captured image is accurately obtained and the display mark is superimposed on the position corresponding to this position in the captured image. It is possible to display the display mark indicating the position of the object in a superimposed manner.

  In the present embodiment, the size of each display mark is determined so that the older display mark becomes smaller, and the transmittance of each display mark becomes higher so that the older display mark has higher transmittance. However, the present invention is not necessarily limited to this. For example, the moving direction of the obstacle may be expressed only by changing the size of the display mark, or the moving direction of the obstacle may be expressed only by changing the transmittance of the display mark.

  Further, the moving direction of the obstacle may be expressed by changing the color of the display mark (specifically, at least one of hue, saturation, and brightness). As a specific example, the mark on the vehicle side is displayed with a more dangerous hue, a higher saturation, or a higher brightness, so that the obstacle is approaching. When an object is moving away from the host vehicle, the mark on the host vehicle side may be displayed with a safer hue, lower saturation, or lower brightness. Further, the moving direction of the obstacle may be expressed by combining a change in size, a change in transmittance, and a change in color.

  Further, in the present embodiment, the configuration in which the hue of the display mark is selectively used according to the relative movement status of the obstacle with respect to the host vehicle based on the obstacle status determined by the control device 4 is shown. Not exclusively. For example, the control device 4 may not determine the obstacle state, and may not use the hue of the display mark depending on the relative movement state of the obstacle with respect to the host vehicle.

  In the present embodiment, the configuration in which the shape of the display mark is an ellipse is shown, but the present invention is not limited to this. For example, the configuration may be other shapes such as a circle, a rectangle, a star, and a polygon.

  In the present embodiment, the configuration in which four ultrasonic sensors are installed as the obstacle sensor 1 is shown, but the present invention is not necessarily limited thereto. For example, a configuration in which fewer than four ultrasonic sensors are installed may be used, or a configuration in which more than four ultrasonic sensors may be installed. Note that the number of detection area axes also increases or decreases according to the increase or decrease of the number of ultrasonic sensors installed.

  Furthermore, in this embodiment, the obstacle sensor 1 is installed in the rear part of the vehicle to detect obstacles existing around the rear part of the host vehicle, and the camera 2 captures the situation around the rear part of the vehicle and superimposes the rear part of the vehicle. Although the structure which displays is shown, it does not necessarily restrict to this. For example, the obstacle sensor 1 is installed at the front part of the vehicle to detect obstacles existing around the front part of the host vehicle, and the camera 2 captures the situation around the front part of the vehicle and displays a superimposed image of the front part of the vehicle. It may be configured to display, or the obstacle sensor 1 is installed on the side of the vehicle to detect obstacles around the side of the host vehicle, and the camera 2 captures the situation around the side of the vehicle. In this case, a superimposed image of the vehicle side portion may be displayed. Moreover, the structure which combined these may be sufficient. In addition, when displaying the superimposed image of the vehicle front part or the vehicle side part, it may be performed in the same manner as displaying the superimposed image of the vehicle rear part.

  Moreover, in this embodiment, although the ultrasonic sonar is used as the obstacle sensor 1, it is not necessarily limited to this. For example, a configuration in which millimeter wave radar or laser radar is applied as the obstacle sensor 1 may be used, or a device that recognizes an obstacle using an image recognition technology may be applied to the obstacle sensor 1. .

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the technical means disclosed in different embodiments can be appropriately combined. Such embodiments are also included in the technical scope of the present invention.

1 is a block diagram showing a schematic configuration of a vehicle periphery monitoring device 100. FIG. It is a figure which shows an example of the installation of the obstacle sensor 1, and an example of the imaging range of the camera 2. 4 is a flowchart showing a flow of obstacle detection processing in the control device 4. 4 is a flowchart showing a flow of superimposed display processing in the control device 4. It is the figure which showed the detection area axis | shaft in a captured image virtually. (A) And (b) is the figure which showed an example of the superimposed image which superimposed the display mark on the captured image. (A)-(c) is the figure which showed an example of the superimposed image which superimposed the display mark on the captured image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Obstacle sensor (obstacle detection part, distance sensor), 2 camera (imaging part), 3 display (display part), 4 control apparatus (detection data acquisition part, display control part), 5 memory | storage device (accumulation memory | storage part) , 100 Perimeter monitoring device for vehicles

Claims (9)

An imaging unit for imaging the periphery of the vehicle;
An obstacle detection unit for detecting obstacles around the vehicle;
A display unit that displays an image captured by the imaging unit,
When the obstacle detection unit detects an obstacle, the vehicle periphery monitoring device displays a mark indicating the obstacle superimposed on the image,
When an obstacle is detected once by the obstacle detection unit, a detection data acquisition unit that acquires detection data of the obstacle over time;
A cumulative storage unit for accumulating and storing the detection data acquired over time by the detection data acquisition unit;
Based on a plurality of the old and new detection data stored in the cumulative storage unit, the old and new marks corresponding to the new and old detection data are superimposed on the image and displayed. The vehicle periphery monitoring system is provided with a display control unit for changing and displaying the size of each of the marks so that the older one of the marks becomes smaller so that the new and old marks can be distinguished. apparatus. An imaging unit for imaging the periphery of the vehicle;
An obstacle detection unit for detecting obstacles around the vehicle;
A display unit that displays an image captured by the imaging unit,
When the obstacle detection unit detects an obstacle, the vehicle periphery monitoring device displays a mark indicating the obstacle superimposed on the image,
When an obstacle is detected once by the obstacle detection unit, a detection data acquisition unit that acquires detection data of the obstacle over time;
A cumulative storage unit for accumulating and storing the detection data acquired over time by the detection data acquisition unit;
Based on a plurality of the old and new detection data stored in the cumulative storage unit, the old and new marks corresponding to the new and old detection data are superimposed on the image and displayed. A vehicle comprising a display control unit for changing and displaying the transmittance of each of the marks so that the older the mark, the higher the transmittance is , so that the mark can be distinguished from old and new. Perimeter monitoring device. An imaging unit for imaging the periphery of the vehicle;
An obstacle detection unit for detecting obstacles around the vehicle;
A display unit that displays an image captured by the imaging unit,
When the obstacle detection unit detects an obstacle, the vehicle periphery monitoring device displays a mark indicating the obstacle superimposed on the image,
When an obstacle is detected once by the obstacle detection unit, a detection data acquisition unit that acquires detection data of the obstacle over time;
A cumulative storage unit for accumulating and storing the detection data acquired over time by the detection data acquisition unit;
Based on a plurality of the old and new detection data stored in the cumulative storage unit, the old and new marks corresponding to the new and old detection data are superimposed on the image and displayed. For a vehicle, comprising a display control unit that changes and displays at least one of hue, saturation, and brightness of each mark so that the mark can be distinguished from old and new Perimeter monitoring device. An imaging unit for imaging the periphery of the vehicle;
An obstacle detection unit for detecting obstacles around the vehicle;
A display unit that displays an image captured by the imaging unit,
When the obstacle detection unit detects an obstacle, the vehicle periphery monitoring device displays a mark indicating the obstacle superimposed on the image,
When an obstacle is detected once by the obstacle detection unit, a detection data acquisition unit that acquires detection data of the obstacle over time;
A cumulative storage unit for accumulating and storing the detection data acquired over time by the detection data acquisition unit;
Based on a plurality of the old and new detection data stored in the cumulative storage unit, the old and new marks corresponding to the new and old detection data are superimposed on the image and displayed. It is equipped with a display control unit that changes and displays each of the marks so that the new and old marks can be distinguished ,
The obstacle detection unit is a plurality of distance sensors each having a different detection area,
The display control unit
The distance sensor detects the axis of the distance sensor directivity and the axis of the overlap of the detection areas of the distance sensor as the detection area axis that is the area where the mark is displayed. The mark is displayed at a location according to the distance of the obstacle, and
A vehicle periphery monitoring device , wherein a detection area axis for displaying the mark is determined according to which distance sensor of the plurality of distance sensors detects an obstacle . To any one of claims 2-4, characterized in that to enable the old and new distinction of the mark by displaying by changing the size of the individual of the mark to become smaller as the older the mark The vehicle periphery monitoring device described. According to claim 3 or 4, characterized in that to enable the old and new distinction of the mark by displaying by changing the transmittance of each of the marks as the older the mark as the transmittance increases Vehicle periphery monitoring device. 5. The vehicle periphery according to claim 4, wherein the mark can be distinguished from new and old by changing and displaying at least one of hue, saturation, and brightness of each of the marks. Monitoring device. The obstacle detection unit is a plurality of distance sensors each having a different detection area,
The display control unit uses a central axis of directivity of the distance sensor as a detection area axis that is an area for displaying the mark, and follows a distance of an obstacle detected by the distance sensor on the detection area axis. While displaying the mark in the place,
Depending on whether an obstacle is detected in one of the distance sensors of said plurality of distance sensors, any one of claims 1 to 3, characterized by determining the detection area axes for displaying the mark The vehicle periphery monitoring device described in 1. The vehicle periphery monitoring device according to any one of claims 1 to 8, wherein the detection data is information on a distance and a position of an obstacle detected by the obstacle detection unit.

Images