JP7488010B2 - Distance calculation device and distance calculation method - Google Patents

Description

The present invention relates to a distance calculation device and a distance calculation method.

Conventionally, there is technology for measuring distance based on images, etc. Patent Document 1 discloses a distance measuring device equipped with a distance measuring unit that selects and executes a first distance measurement that measures the distance to the target based on an image of the target captured by a stereo camera, and a second distance measurement that measures the distance to the target based on the reflected light after a light emitting unit irradiates the target with a light beam.

JP 2020-112434 A

Here, it is desirable to be able to calculate distance with an inexpensive configuration. For example, if distance could be calculated from an image taken with a monocular camera, costs could be reduced.

The object of the present invention is to provide a distance calculation device and a distance calculation method that can calculate distance with an inexpensive configuration.

The distance calculation device of the present invention includes a video acquisition unit that acquires video data of the area ahead of the vehicle captured by a camera of the vehicle, a speed acquisition unit that acquires speed data of the vehicle, a detection unit that detects a leading vehicle and license plate from an image of the video data and detects the size of a first area, which is an area of pixels in which the license plate is captured, and an estimation unit that estimates a distance along the traveling direction of the vehicle, wherein the estimation unit sets a coefficient based on the size of the first area and the speed data in a plurality of frames captured when the leading vehicle was stopped and the vehicle was traveling, and the estimation unit calculates the distance along the traveling direction based on the coefficient and the size of the first area.

The distance calculation device according to the present invention calculates the distance along the vehicle's traveling direction based on the size of a first region, which is the region of pixels in which the license plate is imaged, and the vehicle's speed data. The distance calculation device according to the present invention has the effect of being able to calculate the distance with an inexpensive configuration.

FIG. 1 is a block diagram of a distance detection device according to an embodiment. FIG. 2 is a diagram showing a state in which the vehicle stops behind the preceding vehicle. FIG. 3 is a diagram showing a frame in which an image of a leading vehicle is captured. FIG. 4 shows a frame in which an image of a leading vehicle is captured. FIG. 5 is a diagram showing the relationship between the distance to the leading vehicle and the imaging range. FIG. 6 is a diagram for explaining detection of a leading vehicle and its license plate. FIG. 7 is a diagram showing the relationship between the distance to the leading vehicle and the width of the first area. FIG. 8 is a diagram for explaining a method for calculating the traveled distance. FIG. 9 is a diagram illustrating the average speed. FIG. 10 is a diagram for explaining a method for calculating the traveled distance. FIG. 11 is a diagram for explaining the criteria for calculating the distance. FIG. 12 is a diagram showing classification of number plates. FIG. 13 is a diagram illustrating the averaging of the calibration curve. FIG. 14 is a flowchart according to the embodiment. FIG. 15 is a flowchart according to the embodiment. FIG. 16 is a flowchart according to a first modified example of the embodiment. FIG. 17 is a block diagram according to a first modified example of the embodiment. FIG. 18 is a flowchart according to a second modified example of the embodiment. FIG. 19 is a block diagram according to a second modification of the embodiment.

The distance calculation device and distance calculation method according to the embodiment of the present invention will be described in detail below with reference to the drawings. Note that the present invention is not limited to these embodiments. Furthermore, the components in the following embodiments include those that can be easily imagined by a person skilled in the art or those that are substantially the same.

[Embodiment]
An embodiment will be described with reference to Fig. 1 to Fig. 15. The embodiment relates to a distance calculation device and a distance calculation method. Fig. 1 is a block diagram of a distance detection device according to the embodiment, Fig. 2 is a diagram showing a state when a vehicle stops behind a preceding vehicle, Fig. 3 is a diagram showing a frame in which the preceding vehicle is imaged, Fig. 4 is a diagram showing a frame in which the preceding vehicle is imaged, Fig. 5 is a diagram showing a relationship between the distance to the preceding vehicle and an image capture range, Fig. 6 is a diagram explaining detection of the preceding vehicle and the license plate, Fig. 7 is a diagram showing a relationship between the distance to the preceding vehicle and the width of the first area, Fig. 8 is a diagram explaining a calculation method of a travel distance, Fig. 9 is a diagram explaining an average speed, and Fig. 10 is a diagram explaining a calculation method of a travel distance.

Figure 11 is a diagram explaining the criteria for calculating distance, Figure 12 is a diagram showing the classification of number markers, Figure 13 is a diagram explaining the averaging of the calibration curve, and Figures 14 and 15 are flowcharts relating to the embodiment.

The distance calculation device 1 of this embodiment shown in FIG. 1 is mounted on a vehicle 100. The vehicle 100 is, for example, an automobile, and may be a hybrid automobile, a plug-in hybrid automobile, or an electric automobile. The vehicle 100 may be a vehicle for transporting cargo or a vehicle for transporting personnel.

The vehicle 100 is equipped with a data measurement unit 110. The data measurement unit 110 includes a camera 111 and a speed sensor 112. The data measurement unit 110 acquires and stores video data generated by the camera 111. The acquired video data is video data captured by the camera 111 of the area ahead of the vehicle 100. The data measurement unit 110 also acquires and stores a speed value detected by the speed sensor 112. The speed sensor 112 is a sensor that detects the traveling speed of the vehicle 100. The speed data includes the detected speed value and the time when the speed value was detected. The video data and the speed data are stored in an in-vehicle device such as a drive recorder or a digital tachograph.

The distance calculation device 1 has a distance estimation unit 10. The distance estimation unit 10 has a video acquisition unit 2, a speed acquisition unit 3, a detection unit 4, an estimation unit 5, and a distance output unit 6. The video acquisition unit 2 communicates with the data measurement unit 110 and acquires video data from the data measurement unit 110. The video acquisition unit 2 may acquire the video data in real time, or may acquire the video data when a trigger condition is met. The video acquisition unit 2 can divide the acquired video data into still images and store them. The speed acquisition unit 3 acquires speed data from the data measurement unit 110. The speed acquisition unit 3 acquires the speed data in real time, for example.

The detection unit 4 detects objects from images of video data. The detection unit 4 acquires the function of detecting objects from images through machine learning such as deep learning. In this embodiment, the detection unit 4 is configured to detect a preceding vehicle and license plate from video data.

As described below, the distance calculation device 1 of this embodiment detects the license plate 210 of the leading vehicle 200 from video data and calculates the distance along the traveling direction based on the size of the license plate 210 in the image. Figure 2 shows a state in which the vehicle 100 stops behind the leading vehicle 200. In the graph of Figure 2, the horizontal axis is time t, and the vertical axis is the speed v of the vehicle 100.

At time _t2 , the preceding vehicle 200 is stopped, and the vehicle 100 is moving while approaching the preceding vehicle 200. The position of the vehicle 100 at time _t2 is defined as _p2 . Note that positions _p0 , _p1 , and _p2 are positions along the traveling direction of the vehicle 100, for example, the position of the tip of the vehicle 100. At time _t2 , frame _F2 of the video data is captured. At the subsequent time _t1 , frame _F1 is captured. The position of the vehicle 100 at time _t1 is defined as _p1 . At time _t1 , the vehicle 100 is still moving. At time _t0 , the vehicle 100 stops at position _p0 . From time _t2 to time _t0 , the preceding vehicle 200 remains stopped.

Fig. 3 shows frame _F2 captured at time _t2 . Fig. 4 shows frame _F1 captured at time _t1 . Frames _F1 and _F2 respectively capture images of a leading vehicle 200 and a license plate 210. In frame _F1 , the license plate 210 is captured over a larger area than in frame _F2 . That is, the size of the license plate 210 in the image changes depending on the relative distance between the vehicle 100 and the leading vehicle 200. The distance calculation device 1 estimates the distance along the traveling direction of the vehicle 100 based on the size of the license plate 210 in the image.

As shown in Fig. 5, the ratio of the length N of the license plate 210 to the imaging range R changes depending on the distance from the vehicle 100 to the preceding vehicle 200. The imaging range R is a range in real space determined according to the angle of view θ of the camera 111. The imaging range R shown in Fig. 5 is a range in the lateral direction of the image, and is a range that can be imaged by the camera 111. For example, for distances La and Lb to the preceding vehicle 200, the imaging ranges _Ra and _Rb are determined according to the angle of view θ. As the distance to the preceding vehicle 200 increases, the ratio of the length N to the imaging range R decreases. In other words, as the distance increases, the apparent size of the license plate 210 in the image decreases.

As shown in FIG. 6, the detection unit 4 detects the preceding vehicle 200 and the license plate 210 from the image. The detection unit 4 detects a first area A1 in which the license plate 210 is imaged from each frame F. The first area A1 is an area of pixels in which the license plate 210 is imaged. The illustrated shape of the first area A1 is rectangular. The detection unit 4 also detects the size of the first area A1. In this embodiment, the detection unit 4 detects the width pW in the horizontal direction X of the image as the size of the first area A1. The width pw is, for example, the number of pixels. The width pW is the apparent width of the license plate 210 when viewed from the vehicle 100.

The detection unit 4 detects the second area A2 in which the preceding vehicle 200 is imaged from each frame F. The second area A2 is an area of pixels in which the preceding vehicle 200 is imaged. The illustrated shape of the second area A2 is rectangular. The detection unit 4 detects the first area A1 and the second area A2 independently. If the entire first area A1 is included in the second area A2, the detection unit 4 determines that the detected license plate 210 is the license plate of the preceding vehicle 200.

7 shows the width pW of the first region A1 according to the distance between the vehicle 100 and the preceding vehicle 200. FIG. 7 shows the width pW ₁ of the first region A1 when the vehicle 100 is at position p ₁ (see FIG. 2 ), and the width pW ₂ of the first region A1 when the vehicle 100 is at position p _2. When the vehicle 100 is close to the preceding vehicle 200, the width pW of the first region A1 is large. As the preceding vehicle 200 becomes farther away from the vehicle 100, the width pW of the first region A1 becomes smaller. Also, as the preceding vehicle 200 becomes farther away from the vehicle 100, the first region A1 converges to the vanishing point VP of the image.

As will be described with reference to Figs. 8 to 10, the estimation unit 5 calculates the travel distance of the vehicle 100 until it stops at position _p0 based on the speed data of the vehicle 100. In Fig. 8 and other figures, the time indicated by a capital T is the time when the speed is detected by the speed sensor 112. The subscript j of the detection time T increases toward the past. The estimation unit 5 calculates the travel distance using the average speed mv. As shown in Fig. 9, the average speed _mvj is the average speed in the period between the detection time _Tj and the detection time Tj ₊₁ . The average speed _mvj is calculated by the following formula (1). Here, _vj is the speed value detected at the detection time _Tj , and vj ₊₁ is the speed value detected at the detection time Tj ₊₁ .
_mvj = ( _vj + vj _{+ 1} ) / 2 (1)

Further, the travel distance _ΔLj during the above period is calculated by the following formula (2).
ΔL _j = mv _j × ( T _j - _{T j + 1} ) (2)

The estimation unit 5 calculates the travel distance _Ln shown in Fig. 10. The travel distance _Ln is the distance traveled by the vehicle 100 from when the preceding vehicle 200 stops until the vehicle 100 stops. The position _Pn , which is the start point of the travel distance _Ln , is the position of the vehicle 100 when the preceding vehicle 200 stops and the first frame _Fn is captured. The end point of the travel distance _Ln is the position _p0 where the vehicle 100 stops. As will be described later, whether the preceding vehicle 200 is stopped is determined based on the video data.

A travel distance _Ln from position _pn to position _p0 is calculated by the following [Equation 1]. Here, the subscript j=s corresponds to the detection time _Ts shown in FIG. 10. The detection time _Ts is, for example, the detection time T of the speed v when it is determined that the vehicle 100 has stopped. The subscript j=e corresponds to the detection time _Te shown in FIG. 10. The detection time _Te is the detection time T when the speed v is first detected after passing position _pn .

As shown in Fig. 11, the reference of the distance L calculated by the distance calculation device 1 is the position _p0 where the vehicle 100 stops. The distance calculation device 1 of this embodiment calculates the traveled distance L _i (i = 1, 2, 3, ...) based on the width of the license plate 210. Here, the actual size of the license plate 210 differs depending on the type of the preceding vehicle 200. Fig. 12 shows the types and sizes of number plates used for automobiles in Japan. As shown in Fig. 12, number plates are divided into three types: large, medium, and small. The size of the number plate is determined depending on the type of number plate.

The distance calculation device 1 normalizes the width pW detected from the image so that the distance L can be calculated using the same coefficients C1 and C2 regardless of the type of the preceding vehicle 200. More specifically, the estimation unit 5 calculates the normalized width W by the following formula (3), where the length N is the horizontal size of the number plate according to the type of vehicle.
W = pW / N (3)

Figure 11 shows the relationship between the normalized width W and the distance L. The normalized width W is the same regardless of the type of preceding vehicle 200. In other words, it is possible to uniquely determine the distance L from the normalized width W.

The estimation unit 5 of the present embodiment estimates the relationship between the normalized width W and the distance L using a linear regression model of a linear function shown in the following formula (4): In formula (4), C1 and C2 are coefficients.
L = C1 + C2 (W / 2) (4)

The estimation unit 5 calculates a normalized width W _n from the width pW _n detected in the frame F _n . The estimation unit 5 also calculates a normalized width W _i and a travel distance L i from the position p ₀ to the position p _i for each frame F _i captured between the position p ₀ and the position p _n . The method of calculating the travel distance _{L i is} the same as the method of calculating the travel distance L _n by the above [Equation 1]. The estimation unit 5 substitutes the combination of the travel distance L _n and the normalized width W _n and the combination of the travel distance L _i and the normalized width W _i into the formula (4) to determine the coefficients C1 and C2.

Here, it is considered that the distance between the preceding vehicle 200 and the position _p0 varies every time the vehicle stops. Therefore, the estimation unit 5 of this embodiment performs active calibration to determine the coefficients C1 and C2 multiple times, and averages the coefficients C1 and C2. As shown on the left side of FIG. 13, the estimation unit 5 calculates multiple calibration curves. The estimation unit 5 averages the coefficients C1 and C2 calculated in each calibration curve to determine the coefficients D1 and D2. Note that, when the values of the coefficients C1 and C2 in one calibration curve are significantly different from those in the other calibration curves, the coefficients C1 and C2 of that one calibration curve are omitted. The estimation unit 5 performs statistical calculations, and does not use the coefficients C1 and C2 that make the variance larger than a specified value in the calculation of the coefficients D1 and D2.

The estimation unit 5 uses the averaged coefficients D1 and D2 to calculate the distance L according to the following formula (5). In detail, the detection unit 4 detects the size pW of the first area A1 and the type of the leading vehicle 200 from the video image. The estimation unit 5 calculates the normalized width W based on the detection result of the detection unit 4, and calculates the distance L by substituting the width W into formula (5).
L = D1 + D2 (W / 2) (5)

The distance calculation device 1 according to this embodiment can roughly calculate the distance from the vehicle 100 to the preceding vehicle 200 based on image data and speed data. In other words, the distance L can be calculated without adding an expensive distance sensor to the vehicle 100.

The method of calculating distance by the distance calculation device 1 will be described with reference to FIG. 14. In step S10, the estimation unit 5 starts checking whether the vehicle 100 has stopped. In step S10, the estimation unit 5 checks, for example, whether the current speed of the vehicle 100 is 0. After step S10 is executed, the process proceeds to step S20.

In step S20, the estimation unit 5 determines whether the stop of the vehicle 100 has been detected. For example, the estimation unit 5 determines that the stop of the vehicle 100 has been detected when the detected speed of the vehicle 100 is 0 for a predetermined time. If a positive determination is made in step S20 that the vehicle 100 has stopped, the process proceeds to step S30, and if a negative determination is made, the process returns to step S10.

In step S30, the estimation unit 5 rewinds the video data. For example, the estimation unit 5 acquires an image of the last frame before the time when it is determined that the vehicle 100 is stopped. After step S30 is executed, the process proceeds to step S40.

In step S40, the estimation unit 5 instructs the detection unit 4 to detect the preceding vehicle 200. The detection unit 4 executes a detection process for the preceding vehicle 200 from the image acquired in step S30. When the detection unit 4 detects the preceding vehicle 200, it attaches a label indicating the type of vehicle to the preceding vehicle 200. The label is, for example, one of a "large" label indicating a vehicle with a large number plate, a "medium" label indicating a vehicle with a medium number plate, and a "small" label indicating a vehicle with a small number plate. Note that the detection unit 4 may attach an "unknown" label to the detected preceding vehicle 200 if it cannot determine the type of vehicle. After step S40 is executed, the process proceeds to step S50.

In step S50, the estimation unit 5 determines whether the preceding vehicle 200 detected in step S40 is a target vehicle. For example, the estimation unit 5 determines that the preceding vehicle 200 is a target vehicle if the label attached to the preceding vehicle 200 is any of "large", "medium", and "small". If a positive determination is made in step S50 that the preceding vehicle is a target vehicle, the process proceeds to step S60, and if a negative determination is made, the flowchart ends.

In step S60, the estimation unit 5 rewinds the video of the video data. For example, the estimation unit 5 obtains an image of the frame captured one frame before the current target frame. After step S60 is executed, the process proceeds to step S70.

In step S70, the estimation unit 5 instructs the detection unit 4 to check whether the preceding vehicle 200 is stopped. The detection unit 4 determines whether the preceding vehicle 200 is stopped, for example, based on the image of the frame acquired in step S60 and the image of the frame captured one frame later. The detection unit 4 detects, for example, whether the positional relationship between the preceding vehicle 200 and surrounding objects has changed. After step S70 is executed, the process proceeds to step S80.

In step S80, the estimation unit 5 determines whether or not the preceding vehicle 200 has been detected as stopped. The estimation unit 5 performs the determination in step S80 based on the detection result of the detection unit 4. If the result of the determination in step S80 is a positive determination that the preceding vehicle 200 has stopped, the process proceeds to step S90, and if a negative determination is made, the process returns to step S60.

In step S90, the estimation unit 5 rewinds the video of the video data. For example, the estimation unit 5 obtains an image of a frame captured one frame before the current target frame. After step S90 is executed, the process proceeds to step S100.

In step S100, the estimation unit 5 commands the detection unit 4 to detect the preceding vehicle 200 and the license plate 210. In response to the command, the detection unit 4 executes a detection process for the preceding vehicle 200 and the license plate 210 from the image acquired in step S90. If the detection unit 4 detects the license plate 210, it records information about the position and size of the first area A1. If the detection unit 4 detects the preceding vehicle 200, it records information about the position and size of the second area A2, and a label related to the type of the preceding vehicle 200. Once step S100 has been executed, the process proceeds to step S110.

In step S110, the estimation unit 5 determines whether or not both the preceding vehicle 200 and the license plate 210 have been detected from one image. The estimation unit 5 also determines whether the detected license plate 210 belongs to the preceding vehicle 200. If the result of the determination in step S110 is a positive determination that the preceding vehicle 200 and the license plate 210 have been detected, the process proceeds to step S120, and if a negative determination is made, the process proceeds to step S130.

In step S120, the estimation unit 5 stores the size and speed value of the detected license plate 210. The estimation unit 5 calculates the width pW of the first area A1 in the horizontal direction X of the image as the size of the license plate 210. Furthermore, the estimation unit 5 calculates a normalized width W based on the type label of the preceding vehicle 200. The estimation unit 5 also acquires a speed value required to calculate the travel distance from the time the frame F to be detected is captured to the time the vehicle 100 stops. After step S120 is executed, the process proceeds to step S130.

In step S130, the estimation unit 5 rewinds the video of the video data. For example, the estimation unit 5 obtains an image of a frame captured one frame before the target frame. After step S130 is executed, the process proceeds to step S140.

In step S140, the estimation unit 5 instructs the detection unit 4 to check whether the preceding vehicle 200 is traveling. The detection unit 4 detects whether the preceding vehicle 200 is traveling, for example, based on the image of the frame acquired in step S130 and the image of the frame captured one frame later. After step S140 is executed, the process proceeds to step S150.

In step S150, the estimation unit 5 determines whether or not the preceding vehicle 200 is detected. If a positive determination is made in step S150 that the preceding vehicle 200 is moving, the process proceeds to step S160, and if a negative determination is made, the process returns to step S100 and detects the preceding vehicle 200 and the license plate 210.

In step S160, the estimation unit 5 creates a calibration curve. The creation of the calibration curve in step S160 is described with reference to FIG. 15.

In step S210, the estimation unit 5 creates a new calibration curve based on the width W and speed value stored in step S120. After step S210 is executed, the process proceeds to step S220.

In step S220, the estimation unit 5 determines whether or not the calibration curve has been created. If the number of data items read for the width W and speed value is insufficient, the coefficients C1 and C2 cannot be calculated, and the calibration curve may not be created. If it is determined that the calibration curve has been created, the process proceeds to step S230; if it is determined that the calibration curve has been created, the process returns to step S210. If it is determined that the calibration curve has been created, data for the speed value and width W are added, for example, and the calibration curve is created again.

In step S230, the estimation unit 5 determines whether a calibration curve has been created in the past. If it is determined that a calibration curve has been created in the past, the process proceeds to step S240, and if it is determined that a calibration curve has been created in the past, the process proceeds to step S250.

In step S240, the estimation unit 5 creates an average calibration curve. The estimation unit 5 averages multiple calibration curves to determine the coefficients D1 and D2. After step S240 is executed, the process proceeds to step S250.

In step S250, the estimation unit 5 outputs the calibration curve. When step S250 is executed, the flowchart ends. When the creation of the calibration curve in step S160 is completed, the flowchart in FIG. 14 ends.

As described above, the distance calculation device 1 according to this embodiment has a video acquisition unit 2, a speed acquisition unit 3, a detection unit 4, and an estimation unit 5. The video acquisition unit 2 acquires video data captured by the camera 111 of the vehicle 100 of the vehicle 100. The speed acquisition unit 3 acquires speed data of the vehicle 100. The detection unit 4 detects the preceding vehicle 200 and the license plate 210 from the image of the video data. The detection unit 4 further detects the size of a first area A1, which is an area of pixels in which the license plate 210 is captured. The estimation unit 5 estimates the distance L along the traveling direction of the vehicle 100.

The estimation unit 5 sets the coefficients C1 and C2 based on the size of the first area A1 in multiple frames F captured when the leading vehicle 200 was stopped and the vehicle 100 was traveling, and on the speed data. The estimation unit 5 calculates the distance along the traveling direction based on the coefficients C1 and C2 and the size of the first area A1. According to the distance calculation device 1 of this embodiment, the distance can be calculated with an inexpensive configuration.

The estimation unit 5 of this embodiment calculates the travel distance _Ln of the vehicle 100 from when the preceding vehicle 200 stops to when the vehicle 100 stops behind the preceding vehicle 200 based on the speed data, and sets the coefficients C1 and C2 based on the travel distance _Ln . Thus, the estimation unit 5 can accurately estimate the relationship between the change in the inter-vehicle distance between the preceding vehicle 200 and the vehicle 100 and the change in the size of the first area A1.

The detection unit 4 of this embodiment detects the type of the preceding vehicle 200. The estimation unit 5 normalizes the size of the first area A1 according to the size N of the license plate 210 that is determined according to the type of the preceding vehicle 200. The estimation unit 5 sets the coefficients C1 and C2 based on the normalized size W of the first area A1. Therefore, the distance calculation device 1 of this embodiment can estimate the distance L using the same coefficients C1 and C2 regardless of the type of the preceding vehicle 200.

[First Modification of the Embodiment]
A first modified example of the embodiment will be described with reference to Fig. 16 and Fig. 17. Fig. 16 is a flowchart according to the first modified example of the embodiment, and Fig. 17 is a block diagram according to the first modified example of the embodiment. A driving support device 20 including a distance calculation device 1 is mounted on a vehicle 100 according to the first modified example of the embodiment. The driving support device 20 outputs an alarm based on the distance L calculated by the distance calculation device 1 and the result of behavior detection.

As shown in FIG. 17, the driving assistance device 20 has a distance estimation unit 10, a behavior detection unit 7, an abnormality detection unit 8, and an alarm output unit 9. The distance estimation unit 10 is configured similarly to the distance estimation unit 10 in the above embodiment. The behavior detection unit 7 detects the behavior of an object based on video data. The object to be detected is, for example, a moving object present in the vicinity of the vehicle 100. The moving object includes, for example, a leading vehicle 200 and other vehicles such as a bicycle. The moving object may also include a pedestrian or an animal.

The behavior detection unit 7 is configured to detect a moving object from images of the video data, and to detect the behavior of the moving object. The behavior detection unit 7 detects the behavior of the moving object based on multiple frames in which the same moving object is imaged. For example, the behavior detection unit 7 detects changes over time in the moving object, such as changes in the state or position of the moving object. The behavior detection unit 7 records the position of the detected moving object, the type of moving object, and the behavior of the moving object. For example, the behavior detection unit 7 detects that the moving object is moving from right to left. For example, the behavior detection unit 7 detects that a bicycle in front of the vehicle 100 is swaying unsteadily.

The abnormality detection unit 8 detects abnormalities based on the detection result of the behavior detection unit 7 and the calculation result of the distance estimation unit 10. The alarm output unit 9 outputs an alarm based on the detection result of the abnormality detection unit 8.

The operation of the driving assistance device 20 according to the first modified example of the embodiment will be described with reference to FIG. 16. In step S310, the speed acquisition unit 3 acquires speed data from the data measurement unit 110. After step S310 is executed, the process proceeds to step S330.

In step S320, the video acquisition unit 2 acquires video data from the data measurement unit 110. After step S320 is executed, the process proceeds to steps S330 and S350.

In step S330, the distance estimation unit 10 creates a calibration curve. The estimation unit 5 creates the calibration curve using the procedure described in the above embodiment. After step S330 is executed, the process proceeds to step S340.

In step S340, the distance estimation unit 10 estimates the distance between the vehicle 100 and the preceding vehicle 200. The distance estimation unit 10 estimates the distance L based on the calibration curve created in step S330. The driving assistance device 20 uses the estimated distance L as the distance between the vehicle 100 and the preceding vehicle 200. After step S340 is executed, the process proceeds to step S370.

In step S350, the behavior detection unit 7 executes behavior detection. The behavior detection unit 7 detects the behavior of the moving object based on the video data acquired in step S320. After step S350 is executed, the process proceeds to step S360.

In step S360, the abnormality detection unit 8 determines whether the behavior detected in step S350 is a dangerous behavior. If the behavior is determined to be a dangerous behavior in a positive manner, the process proceeds to step S370, and if the behavior is determined to be a dangerous behavior in a negative manner, the process proceeds to step S400.

In step S370, the abnormality detection unit 8 determines whether the distance L estimated in step S340 is equal to or less than a specified value. The specified value is, for example, a value for determining whether or not the dangerous behavior of the moving object affects the safe driving of the vehicle 100. If a positive determination is made that the distance L is equal to or less than the specified value, the process proceeds to step S380, and if a negative determination is made, the process proceeds to step S400.

In step S380, the abnormality detection unit 8 determines whether or not an abnormality has been detected. For example, the abnormality detection unit 8 determines that an abnormality has been detected when the behavior of the detected moving object is dangerous and the distance L is equal to or less than a specified value. If a positive determination is made that an abnormality has been detected, the process proceeds to step S390, and if a negative determination is made, the process proceeds to step S400.

In step S390, the warning output unit 9 issues a warning. The warning output unit 9 issues a warning to the driver of the vehicle 100 by, for example, light or sound. When step S390 is executed, the flowchart ends.

In step S400, the abnormality detection unit 8 determines that it is safe. When step S400 is executed, the flowchart ends.

[Second Modification of the Embodiment]
A second modified example of the embodiment will be described with reference to Fig. 18 and Fig. 19. Fig. 18 is a flowchart according to the second modified example of the embodiment, and Fig. 19 is a block diagram according to the second modified example of the embodiment. The second modified example of the embodiment differs from the first modified example in that, for example, the second modified example has a related behavior detection unit 11 instead of the behavior detection unit 7.

As shown in FIG. 19, the data measurement unit 110 according to the second modified example has a function of acquiring and storing acceleration data generated by an acceleration sensor 113. The driving assistance device 20 has a distance estimation unit 10, a related behavior detection unit 11, an abnormality detection unit 8, and an alarm output unit 9.

The related behavior detection unit 11 detects the relationship between the behavior of the vehicle 100 and the behavior of the detected moving object. The related behavior detection unit 11 is, for example, artificial intelligence configured to detect related behavior by machine learning based on teacher data. The driving assistance device 20 according to the second modified example detects an abnormality based on the relationship between the behavior of the moving object and the behavior of the vehicle 100.

The operation of the driving assistance device 20 according to the second modified example of the embodiment will be described with reference to FIG. 18. In step S410, the data measurement unit 110 acquires acceleration data from the acceleration sensor 113. After step S410 is executed, the process proceeds to step S460.

In step S420, the speed acquisition unit 3 acquires speed data from the data measurement unit 110. After step S420 is executed, the process proceeds to steps S440 and S460.

In step S430, the video acquisition unit 2 acquires video data from the data measurement unit 110. After step S430 is executed, the process proceeds to step S440.

In step S440, the distance estimation unit 10 creates a calibration curve. The estimation unit 5 creates the calibration curve using the procedure described in the above embodiment. After step S440 is executed, the process proceeds to step S450.

In step S450, the distance estimation unit 10 estimates the distance between the vehicle 100 and the preceding vehicle 200. The distance estimation unit 10 estimates the distance L based on the calibration curve created in step S440. The driving assistance device 20 uses the estimated distance L as the distance between the vehicle 100 and the preceding vehicle 200. After step S450 is executed, the process proceeds to step S460.

In step S460, the related behavior detection unit 11 detects related behavior. The related behavior detection unit 11 detects the relationship between the behavior of the moving object and the behavior of the vehicle 100 based on the acceleration data acquired in step S410, the speed data acquired in step S420, the video data acquired in step S430, and the distance L estimated in step S450. After step S460 is executed, the process proceeds to step S470.

In step S470, the anomaly detection unit 8 determines whether or not an anomaly has been detected. The anomaly detection unit 8 makes the determination in step S470 based on the detection result by the related behavior detection unit 11 in step S460. If a positive determination is made that an anomaly has been detected, the process proceeds to step S480, and if a negative determination is made, the process proceeds to step S490.

In step S480, the alarm output unit 9 issues an alarm. When step S480 is executed, the flowchart ends.

In step S490, the abnormality detection unit 8 determines that it is safe. When step S490 is executed, the flowchart ends.

The distance calculation method according to the above embodiment and each modified example includes steps S320 and S430 of acquiring video data, steps S310 and S420 of acquiring speed data, a detection step S100, and an estimation step.

In steps S320 and S430 for acquiring video data, video data is acquired that is captured by the camera 111 of the vehicle 100, ahead of the vehicle 100. In steps S310 and S420 for acquiring speed data, the speed data of the vehicle 100 is acquired. In step S100 for detection, the preceding vehicle 200 and the license plate 210 are detected from the image of the video data. Furthermore, in step S100 for detection, the size of the first area A1 is detected.

The estimation steps include step S160 for setting coefficients C1 and C2, and steps S340 and S450 for calculating distance L. In step S160 for setting coefficients C1 and C2, coefficients C1 and C2 are set based on the size pW of first area A1 in multiple frames captured when leading vehicle 200 was stopped and vehicle 100 was traveling, and on speed data. In steps S340 and S450 for calculating distance L, distance L along the traveling direction of vehicle 100 is calculated based on coefficients C1 and C2 and size pW of first area A1.

[Third Modification of the Embodiment]
A third modified example of the embodiment will be described. The distance L calculated by the estimation unit 5 may be used as the remaining distance to the average stop position. A scene in which the vehicle 100 approaches the stopped preceding vehicle 200 from behind and stops will be described. In this case, the position where the distance L is 0 is a stop position where a safe inter-vehicle distance can be secured for the driver of the vehicle 100. The driving support device 20 may issue an alarm when it is predicted that the vehicle 100 cannot stop at the position where the distance L = 0. The travel distance until the vehicle 100 stops can be calculated, for example, based on the speed, acceleration, and distance L of the vehicle 100 calculated during deceleration.

The contents disclosed in the above embodiments and variations can be implemented in any suitable combination.

REFERENCE SIGNS LIST 1 Distance calculation device 2 Video acquisition unit 3 Speed acquisition unit 4 Detection unit 5 Estimation unit 6 Distance output unit 7 Behavior detection unit 8 Abnormality detection unit 9 Warning output unit 10 Distance estimation unit 11 Related behavior detection unit 20 Driving support device 100 Vehicle 110: Data measurement unit 111: Camera 112: Speed sensor 113: Acceleration sensor 200: Leading vehicle 210: License plate A1: First area A2: Second area F, _F1 , _F2 Frame L Distance N Length of license plate _p0 , _p1 , _p2 , _pn Vehicle position R, R _a , R _b Imaging range pW: Width of first area W: Normalized width T _j Detection time

Claims (4)

a video acquisition unit that acquires video data captured by a camera of the vehicle;
A speed acquisition unit that acquires speed data of the vehicle;
a detection unit that detects a leading vehicle and a license plate from an image of the video data and detects a size of a first region that is a pixel region in which the license plate is captured;
an estimation unit that estimates a distance along a traveling direction of the vehicle;
Equipped with
the distance along the traveling direction is a distance from the vehicle to the preceding vehicle,
the estimation unit sets a coefficient based on a size of the first area in a plurality of frames captured when the preceding vehicle is stopped and the preceding vehicle is traveling and based on the speed data;
the coefficient indicates a relationship between a size of the first region and a distance along the traveling direction,
The distance calculation device, wherein the estimation unit calculates the distance along the traveling direction based on the coefficient and a size of the first region. 2. The distance calculation device according to claim 1, wherein the estimation unit calculates a travel distance of the vehicle from when the preceding vehicle stops to when the vehicle stops behind the preceding vehicle based on the speed data, and sets the coefficient based on the travel distance. The detection unit detects a type of the preceding vehicle,
The estimation unit normalizes a size of the first region according to a size of a license plate determined according to a type of the preceding vehicle,
The distance calculation device according to claim 1 , wherein the estimation unit sets the coefficient based on a normalized size of the first region. acquiring video data of an area ahead of the vehicle captured by a camera of the vehicle;
acquiring speed data of the vehicle;
detecting a leading vehicle and a license plate from an image of the video data, and detecting a size of a first region which is a pixel region in which the license plate is captured;
estimating a distance along a direction of travel of the vehicle;
Including,
The estimating step includes:
setting a coefficient based on a size of the first area in a plurality of frames captured when the preceding vehicle was stopped and the preceding vehicle was traveling, and on the speed data;
calculating a distance along the travel direction based on the coefficient and a size of the first region;
Including,
the distance along the traveling direction is a distance from the vehicle to the preceding vehicle,
The coefficient indicates a relationship between the size of the first region and a distance along the traveling direction.
Distance calculation method.

Images