JP7720800B2 - Deviation amount detection method, deviation amount detection device, and driver monitoring system - Google Patents

Description

This disclosure relates to a deviation detection method, a deviation detection device, and a driver monitoring system.

Vehicles equipped with a driver monitoring system are fitted with cameras that capture images of occupants. Due to errors made by the camera installer, the camera's installation position may shift from its correct position. The correct installation position for a camera is, for example, the installation position indicated by the design value. If the camera's installation position shifts from its correct installation position, for example, the image of the occupant captured by the camera may become unclear, which may result in a deterioration in the monitoring accuracy of the driver monitoring system.

There is a deviation detection device that detects the deviation between the installation position of an on-board camera attached to a vehicle and the correct installation position of the on-board camera (see Patent Document 1). In this deviation detection device, an electronic control unit extracts feature points of objects captured in images taken by the on-board camera and calculates the deviation from the feature points. Examples of objects include a window frame, a vehicle seat, or a headrest.

JP 2010-181209 A

Changes in the state of ambient light, which is light entering the vehicle from outside, can cause changes in the color, brightness, or shadow of an object captured in an image captured by a camera. Changes in the color, etc. of an object can cause changes in the appearance of the object's shape.
In the displacement amount detection device disclosed in Patent Document 1, the appearance of the shape of an object captured in an image changes as the ambient light changes, which can degrade the accuracy with which the electronic control unit extracts feature points of the object.This degradation in the accuracy with which the electronic control unit calculates the displacement amount can also degrade.

The present disclosure has been made to solve the above-mentioned problems, and aims to provide a deviation amount detection method and device that can suppress deterioration in deviation amount calculation accuracy due to changes in ambient light conditions.

The deviation detection method disclosed herein includes a step in which a lighting device illuminates a structure on the vehicle with light to reduce the effects of ambient light, which is light entering the vehicle from outside the vehicle; a step in which an imaging device captures an image of the structure illuminated by the lighting device; and a step in which a deviation detection device calculates the deviation between the installation position of the imaging device and the correct installation position of the imaging device based on an image of the structure captured by the imaging device.

This disclosure makes it possible to reduce the deterioration in accuracy of calculating the amount of deviation due to changes in the ambient light conditions.

1 is a configuration diagram showing a deviation amount detection device 3 and a driver monitoring system 4 according to a first embodiment. FIG. 2 is a hardware configuration diagram showing hardware of a deviation amount detection device 3 according to the first embodiment. FIG. 2 is a hardware configuration diagram showing hardware of the driver monitoring system 4 according to the first embodiment. FIG. 10 is a hardware configuration diagram of a computer in the case where components of the deviation amount detection device 3 other than the illumination device 10 are realized by software, firmware, or the like. Figure 5A is an explanatory diagram showing the captured image when the imaging device 1 is installed in the correct installation position, and Figure 5B is an explanatory diagram showing the captured image when the imaging device 1 is installed with an angle in the vehicle height direction. Figure 6A is an explanatory diagram showing the captured image when the imaging device 1 is installed in the correct installation position, and Figure 6B is an explanatory diagram showing the captured image when the imaging device 1 is installed at an angle in the vehicle width direction. Figure 7A is an explanatory diagram showing an image captured when the imaging device 1 is installed in the correct installation position, and Figure 7B is an explanatory diagram showing an image captured when the imaging device 1 is installed in a rotated state within the X-Y plane. 4 is a flowchart showing a method for detecting a deviation amount according to the first embodiment. FIG. 2 is an explanatory diagram showing an example of a captured image GP. Figure 10A is an explanatory diagram showing a light source 10a whose light-emitting surface is installed at a position where it comes into contact with a window frame a, which is a structure 2, and Figure 10B is an explanatory diagram showing a light source 10a whose light-emitting surface is installed at a position away from the window frame a, which is a structure 2. Figure 11A is an explanatory diagram showing a state in which the shadow of an object caused by ambient light is cast inside the window frame a, and Figure 11B is an explanatory diagram showing a captured image GP in which the shadow of an object caused by ambient light is reflected. Figure 12A is an explanatory diagram showing a state in which the lighting device 10 prevents shadows of objects caused by ambient light from entering the inside of the window frame a, and Figure 12B is an explanatory diagram showing a captured image GP in which shadows of objects caused by ambient light are not reflected. FIG. 10 is an explanatory diagram showing an example of light that creates a pattern on an irradiated surface. FIG. 10 is an explanatory diagram showing an example in which light from a light source 10a is flashing. FIG. 10 is a configuration diagram showing a deviation amount detection device 3 and a driver monitoring system 4 according to a second embodiment. Figure 16A is an explanatory diagram showing a reflector 10c whose reflective surface is installed at a position where it comes into contact with a window frame a, which is a structure 2, and Figure 16B is an explanatory diagram showing a reflector 10c whose reflective surface is installed at a position away from the window frame a, which is a structure 2. FIG. 10 is a configuration diagram showing a deviation amount detection device 3 and a driver monitoring system 4 according to a third embodiment. FIG. 10 is a hardware configuration diagram showing hardware of a deviation amount detection device 3 according to a third embodiment. 10 is an explanatory diagram showing an example in which light-emitting points 60a, 60b, and 60c of a lighting device 60 are attached to the inside of a window frame a. 10 is a flowchart showing a method for detecting a deviation amount according to a third embodiment.

To explain this disclosure in more detail, the following describes an embodiment of the disclosure with reference to the accompanying drawings.

Embodiment 1.
FIG. 1 is a configuration diagram showing a deviation amount detection device 3 and a driver monitoring system 4 according to the first embodiment.
FIG. 2 is a hardware configuration diagram showing the hardware of the deviation amount detection device 3 according to the first embodiment.
FIG. 3 is a hardware configuration diagram showing the hardware of the driver monitoring system 4 according to the first embodiment.

In FIG. 1, the imaging device 1 is installed, for example, on the center console, steering column, A-pillar, rearview mirror, dashboard, instrument panel, or ceiling of a vehicle.
The imaging device 1 is a camera that captures an image of a structure 2 inside a vehicle, a driver of the vehicle, etc. The camera may be, for example, a visible light camera or an infrared camera.
The imaging device 1 outputs a captured image of a structure 2 to a displacement amount detection device 3, and outputs a captured image of a vehicle driver or the like to a driver monitoring system 4.
The captured image output from the imaging device 1 to the deviation amount detection device 3 is an image used by the deviation amount detection device 3 when detecting the amount of deviation between the installation position of the imaging device 1 and the correct installation position. Therefore, the captured image is, for example, an image captured by the imaging device 1 when the imaging device 1 is attached to a vehicle.
The captured image output from the imaging device 1 to the driver monitoring system 4 is an image that the driver monitoring system 4 uses to detect the driver's inattentive driving, etc. Therefore, the captured image is, for example, an image captured by the imaging device 1 while the driver is driving the vehicle.

The structure 2 is, for example, a window frame, an assist grip, a headrest, a B-pillar, a door frame, a shoulder anchor, or a sun visor of a vehicle.
In the first embodiment, the captured image of the imaging device 1 is used in the driver monitoring system 4, and therefore the imaging device 1 needs to be installed so that, for example, the face of the driver of the vehicle is captured when the driver is seated in the driver's seat, and the structure 2 is captured when the driver is not seated in the driver's seat. If the captured image of the imaging device 1 is not used in the driver monitoring system 4, the imaging device 1 does not need to be installed so that the face of the driver is captured.

The deviation amount detection device 3 detects the amount of deviation between the installation position of the imaging device 1 and the correct installation position of the imaging device 1. The correct installation position of the imaging device 1 is, for example, the installation position indicated by the design value.
The deviation amount detection device 3 outputs the deviation amount to the driver monitoring system 4 .

The driver monitoring system 4 acquires the captured image from the imaging device 1 and acquires the amount of deviation from the deviation amount detection device 3 .
The driver monitoring system 4 monitors the state of the vehicle occupant based on the amount of deviation and the captured image. The state of the occupant may be, for example, the driver's inattentive driving or the driver's drowsy driving.
For example, if the driver monitoring system 4 detects that the driver is looking away from the road or is drowsy at the wheel, it outputs warning information to the vehicle's electronic control unit or the like to warn the driver.

In embodiment 1, an example is described in which the driver monitoring system 4 detects whether the driver is looking away from the road or whether the driver is drowsy at the wheel. However, the driver monitoring system 4 is not limited to detecting whether the driver is looking away from the road or whether the driver is drowsy at the wheel, and may, for example, detect the behavior of vehicle occupants other than the driver. In this case, the vehicle occupants are photographed by the imaging device 1.

The deviation amount detection device 3 includes an illumination device 10 , a position detection unit 11 , and a deviation amount calculation unit 12 .
The lighting device 10 is installed outside the vehicle.
The lighting device 10 includes a light source 10a that irradiates the structure 2 with light to suppress the influence of ambient light, which is light that enters the vehicle from outside.
The light source 10a is a surface light source, such as a display that emits white light or a display that emits infrared light.
1, the brightness of the light-emitting surface of the light source 10a is assumed to be uniform. Uniform brightness means, for example, that the difference between the maximum and minimum luminance measurement values on the light-emitting surface of the surface light source is within ±10%.
1, the light source 10a is a surface light source. However, this is merely an example, and the light source 10a may be, for example, a point light source.
1 includes an illumination device 10. However, this is merely an example, and the illumination device 10 may be provided outside the misalignment amount detection device 3.

The position detection unit 11 is realized by, for example, a position detection circuit 21 shown in FIG.
The position detection unit 11 acquires an image of the structure 2 captured by the imaging device 1 .
The position detection unit 11 detects the structure image position, which is the position of the structure 2 on the image, and outputs structure image position information indicating the structure image position to the displacement amount calculation unit 12 .

The deviation amount calculation unit 12 is realized by, for example, a deviation amount calculation circuit 22 shown in FIG.
The deviation amount calculation unit 12 acquires structure image position information from the position detection unit 11. The deviation amount calculation unit 12 also acquires, from an external source, reference image position information indicating a reference image position, which is the correct position on the image of the structure when the installation position of the imaging device 1 is correct.
The deviation amount calculation unit 12 calculates the amount of deviation between the installation position of the imaging device 1 and the correct installation position based on the reference image position indicated by the reference image position information and the structure image position indicated by the structure image position information.
The deviation amount calculation unit 12 outputs the calculated deviation amount to the driver monitoring system 4 .

The driver monitoring system 4 includes a sensing unit 31 and a warning output unit 32 .
The sensing unit 31 is realized by, for example, a sensing circuit 41 shown in FIG.
The sensing unit 31 acquires the captured image from the imaging device 1 and acquires the amount of deviation from the deviation amount calculation unit 12 of the deviation amount detection device 3 .
The sensing unit 31 monitors the state of the vehicle occupant based on the amount of displacement and the captured image.
The sensing unit 31 monitors the state of the occupants, and detects, for example, whether the driver is distracted while driving or whether the driver is drowsy while driving.

The warning output unit 32 is realized by, for example, a warning output circuit 42 shown in FIG.
If the sensing unit 31 detects, for example, that the driver is looking away from the road or is drowsy at the wheel, the warning output unit 32 outputs warning information indicating a warning to the driver to the vehicle's electronic control unit or the like.

1, it is assumed that the position detection unit 11 and the misalignment calculation unit 12, which are components of the misalignment detection device 3 other than the illumination device 10, are each realized by dedicated hardware as shown in Fig. 2. In other words, it is assumed that a part of the misalignment detection device 3 is realized by a position detection circuit 21 and a misalignment calculation circuit 22.
Each of the position detection circuit 21 and the deviation calculation circuit 22 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

The components of the deviation amount detection device 3 other than the lighting device 10 are not limited to those realized by dedicated hardware, but may be realized by software, firmware, or a combination of software and firmware.
The software or firmware is stored as a program in the memory of a computer. A computer refers to hardware that executes the program, such as a CPU (Central Processing Unit), central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor).

FIG. 4 is a hardware configuration diagram of a computer in the case where the components of the deviation amount detection device 3 other than the illumination device 10 are realized by software, firmware, or the like.
When the components of the deviation amount detection device 3 other than the illumination device 10 are realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures of the position detection unit 11 and the deviation amount calculation unit 12 is stored in the memory 51. Then, a processor 52 of the computer executes the program stored in the memory 51.

Furthermore, Figure 2 shows an example in which all of the components of the misalignment detection device 3 other than the lighting device 10 are realized by dedicated hardware, while Figure 4 shows an example in which all of the components of the misalignment detection device 3 other than the lighting device 10 are realized by software, firmware, etc. However, this is merely an example, and some of the components of the misalignment detection device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

Figure 5A is an explanatory diagram showing the captured image when the imaging device 1 is installed in the correct installation position, and Figure 5B is an explanatory diagram showing the captured image when the imaging device 1 is installed with an angle in the vehicle height direction.
5A and 5B, the X axis is parallel to the vehicle width direction, the Y axis is parallel to the vehicle height direction, and the Z axis is parallel to the vehicle front-rear direction.
The captured images in Figures 5A and 5B show the driver's face. If the installation position of the imaging device 1 is correct, the driver's face will be captured in the correct position in the captured image. In the example of Figure 5A, the correct position is approximately the center of the captured image, and the driver's face is captured in approximately the center of the captured image.
If the imaging device 1 is installed with an angle in the vehicle height direction, the driver's face will appear in the captured image at a position shifted in the vehicle height direction from the correct position, as shown in FIG. 5B.
If the angle in the vehicle height direction is large, the driver's face will appear to be facing upward or downward, which may make it difficult for the driver monitoring system 4 to detect the driver's distracted driving, etc.

Figure 6A is an explanatory diagram showing the captured image when the imaging device 1 is installed in the correct installation position, and Figure 6B is an explanatory diagram showing the captured image when the imaging device 1 is installed at an angle in the vehicle width direction.
6A and 6B, the X-axis, Y-axis, and Z-axis are the same as those in FIGS. 5A and 5B, respectively.
The captured images in Figures 6A and 6B show the driver's face. If the installation position of the imaging device 1 is correct, the driver's face will be captured in the correct position in the captured image. In the example of Figure 6A, the correct position is approximately the center of the captured image, and the driver's face is captured in approximately the center of the captured image.
If the imaging device 1 is installed at an angle in the vehicle width direction, the driver's face will appear in the captured image at a position shifted in the vehicle width direction from the correct position, as shown in FIG. 6B.
If the angle in the vehicle width direction is large, the driver's face will appear to be facing left or right, which may make it difficult for the driver monitoring system 4 to detect the driver's distracted driving, etc.

Figure 7A is an explanatory diagram showing an image captured when the imaging device 1 is installed in the correct installation position, and Figure 7B is an explanatory diagram showing an image captured when the imaging device 1 is installed in a rotated state within the X-Y plane.
7A and 7B, the X-axis, Y-axis, and Z-axis are the same as those in FIGS. 5A and 5B, respectively.
The captured images in Figures 7A and 7B show the driver's face. If the installation position of the imaging device 1 is correct, the driver's face will be captured in the correct position in the captured image without rotation. In the example of Figure 7A, the correct position is approximately the center of the captured image, and the driver's face is captured in the correct position without rotation. The state without rotation means that the driver's face is captured so that the line connecting the driver's left eye and right eye is approximately parallel to the X-axis.
If the imaging device 1 is installed in a state where it is rotated within the XY plane, the face of the driver will appear rotated in the captured image as shown in FIG. 7B.
If the angle of rotation is large, the driver's face may appear to be facing upward, downward, left, right, or a combination of these directions, which may make it difficult for the driver monitoring system 4 to detect the driver's distracted driving, etc.

FIG. 8 is a flowchart showing a method for detecting the amount of deviation according to the first embodiment.
For example, an installer of the imaging device 1 installs the imaging device 1 in a vehicle (step ST1 in FIG. 8).
Here, an installation worker installs the imaging device 1 inside the vehicle. However, in a situation where the imaging device 1 is already installed inside the vehicle, the installation worker does not need to install the imaging device 1 inside the vehicle.
Next, the installer installs the lighting device 10 outside the vehicle (step ST2 in FIG. 8).
Here, the installer installs the lighting device 10 on the exterior of the vehicle. However, in a situation where the lighting device 10 has already been installed on the exterior of the vehicle, the installer does not need to install the lighting device 10 on the exterior of the vehicle.

Next, the installer operates the lighting device 10 .
When the installer operates the lighting device 10, the lighting device 10 irradiates the structure 2 with light to suppress the effects of ambient light, which is light entering the vehicle from outside the vehicle (step ST3 in Figure 8).
The light for suppressing the influence of ambient light means light having a brightness that is capable of suppressing the influence of ambient light.
Next, the installer operates the imaging device 1 .
The installer operates the imaging device 1, which then captures an image of the structure 2 illuminated by the lighting device 10 (step ST4 in FIG. 8).
The deviation amount detection device 3 calculates the amount of deviation between the installation position of the imaging device 1 and the correct installation position of the imaging device 1 based on the image of the structure 2 captured by the imaging device 1 .

Next, the operation of the deviation amount detecting device 3 and the driver monitoring system 4 shown in FIG. 1 will be described.
As described above, the imaging device 1 photographs the structure 2 inside the vehicle when light is irradiated onto the structure 2 by the lighting device 10, and outputs the captured image GP, which is an image of the structure 2, to the deviation amount detection device 3.
9 is an explanatory diagram showing an example of a captured image GP. The captured image GP shown in Fig. 9 is an image captured by an imaging device 1 installed on a dashboard, for example, and shows structures 2 such as a window frame a, an assist grip b, a headrest c, a sun visor d, a B-pillar e, a door frame f, and a shoulder anchor g of the vehicle.
The captured image GP output from the imaging device 1 to the deviation amount detection device 3 is basically an image captured when the driver is not seated in the driver's seat. However, even if the driver is seated in the driver's seat, the captured image GP may be an image captured when the driver is seated in the driver's seat, as long as the driver is seated in a manner that does not block the structure 2.

The position detection unit 11 acquires the captured image GP from the imaging device 1 (step ST5 in FIG. 8).
The position detection unit 11 detects the structure image position, which is the position of the structure 2 on the image, from the captured image GP (step ST6 in FIG. 8).
The position detection unit 11 outputs structure image position information indicating the structure image position to the displacement amount calculation unit 12 .

The process of detecting the structure image position by the position detection unit 11 will be described in detail below.
The position detection unit 11 externally acquires reference image position information indicating a reference image position, which is the correct position on the image of the structure 2 when the installation position of the imaging device 1 is correct, and a reference image G _REF . The reference image G _REF is an image when the installation position of the imaging device 1 is correct.
In the first embodiment, the reference image position information and the reference image G _REF are each provided from outside the deviation amount detection device 3. However, this is merely an example, and the reference image position information and the reference image G _REF may each be stored in the internal memory of the position detection unit 11.
Here, an example will be described in which the position detection unit 11 detects the position of a structure image by performing template matching.
In template matching, for example, SAD (Sum of Absolute Difference) shown in the following equation (1) is used.

In equation (1), I(x, y) is a pixel value of the captured image GP, T(x, y) is a pixel value of the reference image _GREF , and (x, y) is the reference image position indicated by the reference image position information.
w is the width of the reference image G _REF , and h is the height of the reference image G _REF . Each of the width w and height h is a given value.
_dx is the scanning position of the x coordinate in the captured image GP, and _dy is the scanning position of the y coordinate in the captured image GP.

First, the position detection unit 11 calculates SAD( _dx , _dy ) shown in equation (1) assuming that the scanning position of the captured image GP is ( _dx , _dy ).
The position detection unit 11 repeatedly calculates SAD ( _dx , _dy ) by rotating the reference image _GREF or changing the scanning position ( _dx , _dy ).
Next, the position detection unit 11 identifies the smallest SAD ( _dx , _dy ) from among the multiple SADs ( _dx , _dy ) that have been repeatedly calculated.
The position detection unit 11 outputs structure image position information indicating the structure image position to the deviation amount calculation unit 12, where the scanning position ( _dx , _dy ) at the minimum SAD ( _dx , _dy ) is the structure image position, which is the position of the captured image GP.
Furthermore, the position detection unit 11 outputs the amount of rotation of the reference image G _REF when the minimum SAD(d _x , d _y ) is calculated to the deviation amount calculation unit 12 as the deviation amount Δφ _roll in the rotation direction of the imaging device 1 .

The deviation amount calculation unit 12 acquires the structure image position information and the deviation amount Δφ _roll in the rotation direction from the position detection unit 11 .
Moreover, the deviation amount calculation unit 12 acquires the reference image position information and the reference image G _REF from the outside.
In the first embodiment, the reference image position information and the reference image G _REF are each provided from outside the deviation amount detection device 3. However, this is merely an example, and the reference image position information and the reference image G _REF may each be stored in an internal memory of the deviation amount calculation unit 12.

The deviation amount calculation unit 12 moves the reference image G _REF horizontally, vertically, or rotates it so that the reference image position indicated by the reference image position information matches the structure image position indicated by the structure image position information.
That is, the deviation amount calculation unit 12 rotates the reference image G _REF by the deviation amount Δφ _roll in the rotation direction.
Furthermore, the deviation amount calculation unit 12 moves the reference image G _REF horizontally or vertically so that the reference image position (x, y) coincides with the scanning position (d _x , d _y ) at the minimum SAD (d _x , d _y ).
The deviation amount calculation unit 12 calculates the horizontal movement amount Δd _x (=x−d _x ) and the vertical movement amount Δd _y (=y−d _y ) of the reference _{image G REF when} the reference image position and the structure image position coincide with each other.

The deviation calculation unit 12 calculates a deviation Δφ _x between the installation position of the imaging device 1 and the correct installation position from the horizontal movement amount Δd _x of the reference image G _REF , as shown in the following equation (2) (step ST7 in FIG. 8 ). The deviation Δφ _x is the amount of optical axis deviation between the optical axis ΦX _REF in the X-axis direction of the imaging device 1 when the imaging device 1 is installed in the correct position and the optical axis in the X-axis direction of the imaging device 1.
The deviation calculation unit 12 also calculates a deviation _Δφy between the installation position of the imaging device 1 and the correct installation position from the vertical movement Δd _y of the reference image G _REF , as shown in the following equation (3) (step ST7 in FIG. 8 ). The deviation _Δφy is the amount of optical axis deviation between the optical axis ΦY _REF in the Y-axis direction of the imaging device 1 when the imaging device 1 is installed in the correct position and the optical axis of the imaging device 1 in the Y-axis direction.

In equations (2) and (3), FL is a known focal length of the imaging device 1. The focal length is, for example, the distance between an image sensor (not shown) included in the imaging device 1 and a lens (not shown) included in the imaging device 1.

The deviation amount calculation unit 12 outputs the deviation amount Δφ _x in the X-axis direction, the deviation amount Δφ _y in the Y-axis direction, and the deviation amount Δφ _roll in the rotational direction to the driver monitoring system 4 .

The sensing unit 31 of the driver monitoring system 4 acquires a captured image GP' from the imaging device 1. The captured image GP' is an image captured while the driver is driving the vehicle.
Furthermore, the sensing unit 31 acquires the amount of deviation Δφ _x in the X-axis direction, the amount of deviation Δφ _y in the Y-axis direction, and the amount of deviation Δφ _roll in the rotation direction from the deviation amount calculation unit 12 .
The sensing unit 31 performs passenger sensing to identify the position of the driver's head shown in the captured image GP', for example.
When performing occupant sensing, the sensing unit 31 takes into consideration the possibility that the installation position of the imaging device 1 may be shifted from the correct installation position. For this reason, when sensing the position of the driver's head, for example, the sensing unit 31 senses a position that is shifted by the amounts of shift Δφ _x , Δφ _y , Δφ _roll from the position of the driver's head in the captured image GP′.
Specifically, if the coordinates of the driver's head position in the captured image GP' are (x, y, z), the coordinates (x', y', z') of the position sensed by the sensing unit 31 are expressed as follows:

The sensing unit 31 monitors the state of the vehicle occupant based on the pixel value of the coordinates (x', y', z') in the captured image GP'.
The sensing unit 31 monitors the state of the occupants, and detects, for example, whether the driver is distracted while driving or whether the driver is drowsy while driving.
The process of detecting whether the driver is looking away from the road or whether the driver is drowsy at the wheel is a known technique, and therefore a detailed description thereof will be omitted.
If the sensing unit 31 detects, for example, that the driver is looking away from the road or is drowsy at the wheel, the warning output unit 32 outputs warning information indicating a warning to the driver to the vehicle's electronic control unit or the like.

In the displacement amount detection device 3 shown in FIG. 1, a light source 10 a of an illumination device 10 irradiates light onto a structure 2 .
Illuminating the structure 2 with light highlights the boundary between the structure 2 and other structures, thereby suppressing the influence of ambient light. Suppressing the influence of ambient light makes it easier to detect the position of the structure image from the captured image GP.

When the structure 2 inside the vehicle that is photographed by the imaging device 1 is, for example, a window frame a, the light source 10a of the lighting device 10 is installed on one of both sides of the window frame a, opposite the side facing the imaging device 1, as shown in Fig. 10. In the example of Fig. 10, the light source 10a is installed on the outside of the vehicle.
In particular, in the example of FIG. 10A, the light source 10a is installed at a position where the light emitting surface thereof comes into contact with the window frame a, which is the structure 2.
10B , the light emitting surface of light source 10a is installed at a position away from window frame a, which is structure 2. When the light emitting surface of light source 10a is installed at a position away from window frame a, lighting device 10 may be supported by, for example, a tripod or may be suspended from the ceiling of a parking lot where the vehicle is parked. When the light emitting surface of light source 10a is installed at a position away from structure 2, it is desirable that the distance between the light emitting surface and structure 2 be as short as possible so that workers and the like do not get between the light emitting surface of light source 10a and structure 2.
FIG. 10A is an explanatory diagram showing a light source 10a having a light-emitting surface disposed at a position where it comes into contact with a window frame a, which is a structure 2. FIG.
FIG. 10B is an explanatory diagram showing a light source 10a whose light-emitting surface is installed at a position away from a window frame a, which is the structure 2.

11A , if the lighting device 10 is not installed, a shadow of, for example, the scenery outside the vehicle, a worker, or an object caused by ambient light may appear inside the window frame a, which is the structure 2. In FIG. 11A , the area surrounded by the dashed line is the imaging area of the imaging device 1.
In this case, the image GP captured by the imaging device 1 may include the scenery outside the vehicle, a worker, or the shadow of an object.
In a captured image GP in which shadows of objects due to ambient light are reflected, it may be difficult to detect the position of the structure 2 because the boundary between the structure 2 and the shadows of the objects is unclear.
FIG. 11A is an explanatory diagram showing a state in which a shadow of an object caused by ambient light is cast inside a window frame a.
FIG. 11B is an explanatory diagram showing a captured image GP in which a shadow of an object caused by ambient light is reflected.
11B, "0" indicates the brightness value of the pixels constituting the window frame a, the pixels constituting the scenery outside the vehicle, the pixels constituting the worker, and the pixels constituting the shadow of the object, while "1" indicates the brightness value of the pixels other than these pixels.
11A and 11B, the shapes of the shadows of the objects and the like are deformed, so the shapes of the shadows of the objects and the like shown in Fig. 11A do not completely match the shapes of the shadows of the objects and the like shown in Fig. 11B.

On the other hand, when the lighting device 10 is installed, the scenery outside the vehicle, the shadow of a worker, or an object caused by ambient light is prevented from entering the inside of the window frame a, as shown in FIG. 12A . In FIG. 12A , the area surrounded by a dashed line is the capture area of the imaging device 1. Therefore, as shown in FIG. 12B , the scenery outside the vehicle, the shadow of a worker, or an object caused by ambient light is prevented from appearing in the captured image GP. Furthermore, the light source 10a of the lighting device 10 irradiates the structure 2 with light, thereby highlighting the boundary between the structure 2 and other structures. Therefore, the position of the structure 2 captured in the captured image GP can be easily detected.
Therefore, by providing the deviation amount detection device 3 with the illumination device 10, it is possible to suppress deterioration in the calculation accuracy of the deviation amounts Δφ _x , Δφ _y , and Δφ _roll due to changes in the state of the ambient light.
FIG. 12A is an explanatory diagram showing a state in which the lighting device 10 prevents shadows of objects caused by ambient light from entering inside the window frame a.
FIG. 12B is an explanatory diagram showing a captured image GP in which shadows of objects caused by ambient light and the like are not reflected.
12B, "0" is the brightness value of the pixels that make up window frame a, and "1" is the brightness value of the pixels other than the pixels that make up window frame a.
12A and 12B, the shape of the window frame a is deformed, so the shape of the window frame a shown in Fig. 12A does not completely match the shape of the window frame a shown in Fig. 12B.

In the above-described first embodiment, the deviation amount detection method is configured to include a process in which the lighting device 10 illuminates the vehicle structure 2 with light to suppress the effects of ambient light, which is light entering the vehicle from outside the vehicle; a process in which the imaging device 1 captures an image of the structure 2 illuminated by the lighting device 10; and a process in which the deviation amount detection device 3 calculates the amount of deviation between the installation position of the imaging device 1 and the correct installation position of the imaging device 1 based on the image of the structure 2 captured by the imaging device 1. Therefore, the deviation amount detection method can suppress deterioration in the accuracy of calculating the deviation amount due to changes in the ambient light state.

In the deviation amount detection device 3 shown in Fig. 1, the brightness of the light emitting surface of the light source 10a is uniform. However, this is merely an example, and the light irradiated from the light source 10a may be light that generates a pattern on the irradiated surface of the structure 2, as shown in Fig. 13.
13 is an explanatory diagram showing an example of light that produces a pattern on an irradiated surface. In the example of Fig. 13, a plurality of patterns each having a diamond shape are arranged vertically and horizontally.
By drawing a pattern on the light-emitting surface of light source 10a, a pattern appears on the surface irradiated to structure 2. Furthermore, by engraving a pattern on the light-emitting surface of light source 10a, a pattern appears on the surface irradiated to structure 2. Furthermore, by disposing a sheet between the light-emitting surface of light source 10a and the structure 2, in which the transmittance of the patterned portion differs from the transmittance of the non-patterned portion, a pattern appears on the surface irradiated to structure 2.
By making the light emitted from the light source 10a to be light that generates a pattern on the irradiated surface of the structure 2, it becomes easier to distinguish between the light emitted from the light source 10a and ambient light, which is light other than the light emitted from the light source 10a, thereby improving the accuracy with which the position detection unit 11 detects the position of the structure image.

In the displacement amount detection device 3 shown in Fig. 1, the light source 10a continuously emits light. However, this is merely an example, and the light source 10a may also emit light that flashes on and off onto the structure 2, as shown in Fig. 14.
FIG. 14 is an explanatory diagram showing an example in which the light from the light source 10a is blinking.
14 , among the imaging frames of the imaging device 1, in the first and third frames, the light source 10a irradiates the structure 2 with light. Among the imaging frames of the imaging device 1, in the second frame, the light source 10a does not irradiate the structure 2 with light.
The position detection unit 11 can easily detect the structure 2 shown in the captured image GP by, for example, detecting the difference between the captured image GP of the first frame and the captured image GP of the second frame.

In the deviation amount detection device 3 shown in FIG. 1, the position detection unit 11 detects the structure image position, which is the position of the structure 2 on the image. However, this is merely one example, and instead of detecting the position of the structure 2 on the image, the position detection unit 11 may extract feature points of the structure 2 shown in the captured image GP using a method similar to that of the deviation amount detection device described in Patent Document 1. In this case, the deviation amount calculation unit 12 must calculate the deviation amount from the feature points of the structure 2 using a method similar to that of the deviation amount detection device described in Patent Document 1. Note that in this case as well, the captured image GP is an image of the structure 2 captured by the imaging device 1 when light is irradiated onto the structure 2 by the lighting device 10.

In the displacement detection device 3 shown in FIG. 1, the structure 2 is assumed to be a window frame a, and the lighting device 10 irradiates light onto the window frame a. However, the structure 2 is not limited to the window frame a, and the structure 2 may be, for example, an assist grip b, a headrest c, a sun visor d, a B-pillar e, a door frame f, or a shoulder anchor g. Therefore, the lighting device 10 may be configured to irradiate light onto the assist grip b, etc.

Embodiment 2.
In the second embodiment, a description will be given of a displacement amount detecting device 3 in which an illumination device 10 includes a light source 10b and a reflector 10c.

Fig. 15 is a configuration diagram showing a deviation amount detection device 3 and a driver monitoring system 4 according to embodiment 2. In Fig. 15, the same reference numerals as in Fig. 1 indicate the same or corresponding parts, and therefore description thereof will be omitted.
In the displacement amount detecting device 3 shown in FIG. 15, the lighting device 10 includes a light source 10b and a reflector 10c.
The light source 10b irradiates the window frame a, which is the structure 2, with light for suppressing the influence of ambient light.
The light source 10b is a surface light source, and the brightness of the light-emitting surface of the surface light source is uniform.
15, the light source 10b is a surface light source. However, this is merely an example, and the light source 10b may be, for example, a point light source, or may be, for example, an incandescent lamp or an LED (Light Emitting Diode).

The reflector 10c is, for example, a screen, paper, cloth, a wooden board, a resin board, or a metal board, but this is merely an example, and the reflector 10c may be a combination of the screen, paper, cloth, wooden board, resin board, and metal board.
The reflector 10c has a flat light reflecting surface.
The reflecting plate 10 c is installed on one of both sides of the structure 2 , opposite to the side facing the imaging device 1 .
The reflector 10c reflects the light that has been emitted from the light source 10b and passed through the inside of the window frame a toward the window frame a. The light reflected by the reflector 10c passes through the inside of the window frame a and reaches the imaging device 1.
15, the reflecting surface of the reflector 10c is flat. However, this is merely an example, and the reflecting surface of the reflector 10c may be, for example, a spherical surface or a surface having projections and recesses.
In the displacement amount detection device 3 shown in Fig. 15, the light source 10b is installed inside the vehicle. However, this is merely an example, and the light source 10b may be installed outside the vehicle. In this case, the reflector 10c reflects the light from the light source 10b toward the window frame a. The light reflected by the reflector 10c passes through the window frame a and reaches the imaging device 1.

Next, the operation of the deviation amount detecting device 3 shown in FIG. 15 will be described.
However, apart from the illumination device 10, the deviation amount detection device 3 is the same as that shown in Fig. 1. Therefore, only the operation of the illumination device 10 will be described here.
When the structure 2 inside the vehicle that is photographed by the imaging device 1 is, for example, a window frame a, the reflector 10c of the lighting device 10 is installed on one of both sides of the window frame a opposite to the side facing the imaging device 1, as shown in Fig. 16. In the example of Fig. 16, the reflector 10c is installed on the outside of the vehicle.

16A, a light source 10b, which is a part of the lighting device 10, is installed inside the vehicle, and a reflector 10c, which is also a part of the lighting device 10, is installed outside the vehicle. The reflective surface of the reflector 10c is installed in a position where it comes into contact with a window frame a, which is the structure 2.
16B , a light source 10b that is a part of the lighting device 10 is installed outside the vehicle, and a reflector 10c that is also a part of the lighting device 10 is installed outside the vehicle. The reflective surface of the reflector 10c is installed at a position away from a window frame a that is a structure 2. When the reflective surface of the reflector 10c is installed at a position away from the window frame a, the reflector 10c may be supported by, for example, a tripod or may be suspended from the ceiling of a parking lot where the vehicle is parked. When the reflective surface of the reflector 10c is installed at a position away from the structure 2, it is desirable that the distance between the reflective surface and the structure 2 be as short as possible so that workers or the like do not get between the reflective surface of the reflector 10c and the structure 2.
FIG. 16A is an explanatory diagram showing a reflector 10c having a reflecting surface disposed at a position where it comes into contact with a window frame a, which is a structure 2. FIG.
FIG. 16B is an explanatory diagram showing a reflector 10c whose reflecting surface is located at a position away from a window frame a, which is the structure 2.

When the lighting device 10 including the light source 10b and the reflector 10c is installed, the scenery outside the vehicle, the shadow of a worker, or an object caused by ambient light is prevented from entering the inside of the window frame a, as shown in FIG. 12A . Therefore, as shown in FIG. 12B , the scenery outside the vehicle, the shadow of a worker, or an object caused by ambient light is prevented from being reflected in the captured image GP. Furthermore, the reflector 10c reflects light toward the structure 2, thereby highlighting the boundary between the structure 2 and other structures, thereby suppressing the influence of ambient light. Suppressing the influence of ambient light improves the accuracy with which the position detection unit 11 detects the position of the structure 2.
Therefore, by providing the deviation amount detection device 3 with the illumination device 10, it is possible to suppress deterioration in the calculation accuracy of the deviation amounts Δφ _x , Δφ _y , and Δφ _roll due to changes in the state of the ambient light.

As described above, the misalignment detection device 3 shown in FIG. 15, like the misalignment detection device 3 shown in FIG. 1, can suppress deterioration in the accuracy of misalignment calculation due to changes in the ambient light conditions.

In the deviation amount detection device 3 shown in Fig. 15, the brightness of the light emitting surface of the light source 10b is uniform. However, this is only one example, and the light irradiated from the light source 10b may be light that generates a pattern on the irradiated surface of the structure 2, as shown in Fig. 13.
By drawing a pattern on the light-emitting surface of light source 10b, a pattern appears on the surface irradiated to structure 2. Furthermore, by engraving a pattern on the light-emitting surface of light source 10b, a pattern appears on the surface irradiated to structure 2. Furthermore, by disposing a sheet between the light-emitting surface of light source 10b and the structure 2, in which the transmittance of the patterned portion differs from the transmittance of the non-patterned portion, a pattern appears on the surface irradiated to structure 2.
By making the light emitted from light source 10b to be light that creates a pattern on the irradiated surface of structure 2, it becomes easier to distinguish between the light emitted from light source 10b and ambient light, which is light other than the light emitted from light source 10b, thereby improving the accuracy with which position detection unit 11 detects the position of structure 2.
The light emitted from the light source 10b may be light that produces a pattern on the irradiated surface of the structure 2, but the reflector 10c may have a pattern formed thereon. Even if the reflector 10c has a pattern formed thereon, the pattern will still be produced on the irradiated surface of the structure 2.
The reflecting surface of the reflector 10c is made of a plurality of materials with different reflectivities, so that a pattern is applied to the reflector 10c. Also, the reflecting surface of the reflector 10c is made spherical or uneven, so that a pattern is applied to the reflector 10c.

In the deviation amount detection device 3 shown in Fig. 15, the light source 10b continuously emits light. However, this is merely an example, and the light source 10b may be configured to flash the light that it emits to the structure 2, as shown in Fig. 14.
The position detection unit 11 can easily detect the structure 2 shown in the captured image GP by, for example, detecting the difference between the captured image GP of the first frame and the captured image GP of the second frame.

In the displacement detection device 3 shown in Figure 15, the structure 2 is assumed to be a window frame a, and the reflector 10c reflects light from the light source 10b toward the window frame a. However, the structure 2 is not limited to the window frame a, and the structure 2 may be, for example, an assist grip b, a headrest c, a sun visor d, a B-pillar e, a door frame f, or a shoulder anchor g. Therefore, the reflector 10c may reflect light from the light source 10b toward the assist grip b, etc.

Embodiment 3.
In the third embodiment, a displacement amount detecting device 3 including an illumination device 60 having a plurality of light emitting points will be described.

Fig. 17 is a configuration diagram showing a deviation amount detection device 3 and a driver monitoring system 4 according to embodiment 3. In Fig. 17, the same reference numerals as in Figs. 1 and 15 indicate the same or corresponding parts, and therefore description thereof will be omitted.
Fig. 18 is a hardware configuration diagram showing the hardware of the deviation amount detection device 3 according to embodiment 3. In Fig. 18, the same reference numerals as in Fig. 2 indicate the same or corresponding parts, and therefore the description thereof will be omitted.

The lighting device 60 is installed, for example, inside a vehicle.
The illumination device 60 has N light-emitting points, where N is an integer equal to or greater than 3. For convenience of explanation, the deviation amount detection device 3 shown in FIG. 17 will be described assuming that N=3.
Each of the light-emitting points 60a, 60b, and 60c is a light-emitting portion for outputting light emitted from the light source of the lighting device 60 to the outside.
In the deviation amount detection device 3 shown in FIG. 17, the three light emitting points 60a, 60b, and 60c are not aligned in a straight line, and therefore when the three light emitting points 60a, 60b, and 60c are connected, a triangle is formed.

The position detection unit 61 is realized by, for example, a position detection circuit 71 shown in FIG.
The position detection unit 61 acquires images of the three light emitting points 60 a , 60 b , and 60 c captured by the imaging device 1 .
The position detection unit 61 detects the light emitting point image positions, which are the positions of the three light emitting points 60a, 60b, and 60c on the image.
The position detection unit 61 outputs light emitting point image position information indicating the light emitting point image position to the deviation amount calculation unit 62 .

The deviation amount calculation section 62 is realized by, for example, a deviation amount calculation circuit 72 shown in FIG.
The deviation amount calculation unit 62 acquires light-emitting point image position information from the position detection unit 61. The deviation amount calculation unit 62 also acquires, from the outside, reference image position information indicating reference image positions, which are correct positions on the image of the light-emitting points 60 a, 60 b, and 60 c when the installation position of the imaging device 1 is correct.
The deviation amount calculation unit 62 calculates the amount of deviation between the installation position of the imaging device 1 and the correct installation position based on the reference image position indicated by the reference image position information and the light-emitting point image position indicated by the light-emitting point image position information.
The deviation amount calculation unit 62 outputs the calculated deviation amount to the driver monitoring system 4 .

17, it is assumed that the position detection unit 61 and the deviation amount calculation unit 62, which are components of the deviation amount detection device 3 other than the illumination device 60, are each realized by dedicated hardware as shown in Fig. 18. In other words, it is assumed that a part of the deviation amount detection device 3 is realized by a position detection circuit 71 and a deviation amount calculation circuit 72.
Each of the position detection circuit 71 and the deviation amount calculation circuit 72 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

The components of the deviation amount detection device 3 other than the lighting device 60 are not limited to those realized by dedicated hardware, but may also be realized by software, firmware, or a combination of software and firmware.
When the components of the deviation amount detection device 3 other than the illumination device 60 are realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures of the position detection unit 61 and the deviation amount calculation unit 62 is stored in a memory 51 shown in Fig. 4. Then, a processor 52 shown in Fig. 4 executes the program stored in the memory 51.

Furthermore, Figure 18 shows an example in which all of the components of the misalignment detection device 3 other than the lighting device 60 are realized by dedicated hardware, while Figure 4 shows an example in which all of the components of the misalignment detection device 3 other than the lighting device 60 are realized by software, firmware, etc. However, this is merely an example, and some of the components of the misalignment detection device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

FIG. 20 is a flowchart showing a method for detecting the amount of deviation according to the third embodiment.
For example, an installer of the imaging device 1 installs the imaging device 1 in a vehicle (step ST11 in FIG. 20).
Here, an installation worker installs the imaging device 1 inside the vehicle. However, in a situation where the imaging device 1 is already installed inside the vehicle, the installation worker does not need to install the imaging device 1 inside the vehicle.
Next, the installer installs the lighting device 60 having three or more light-emitting points in the vehicle (step ST12 in FIG. 20).
Here, the installer installs the lighting device 60 inside the vehicle. However, in a situation where the lighting device 60 has already been installed inside the vehicle, the installer does not need to install the lighting device 60 inside the vehicle.
In the deviation amount detection device 3 shown in Fig. 17, an illumination device 60 having three light-emitting points 60a, 60b, and 60c is installed inside a window frame a, as shown in Fig. 19. However, this is merely an example, and an illumination device 60 having three light-emitting points 60a, 60b, and 60c may also be installed outside the vehicle.
FIG. 19 is an explanatory diagram showing an example in which light-emitting points 60a, 60b, and 60c of a lighting device 60 are attached to the inside of a window frame a.

Next, the installer operates the lighting device 60 .
The installer operates the lighting device 60, causing the lighting device 60 to emit light from the light-emitting points 60a, 60b, and 60c (step ST13 in FIG. 20). The light-emitting points 60a, 60b, and 60c emit light toward the imaging device 1.
Next, the installer operates the imaging device 1 .
The installer operates the imaging device 1, which then captures images of the light-emitting points 60a, 60b, and 60c that are emitting light (step ST14 in FIG. 20).
The deviation amount detection device 3 calculates the amount of deviation between the installation position of the imaging device 1 and the correct installation position of the imaging device 1 based on the images of the light emitting points 60 a , 60 b , and 60 c captured by the imaging device 1 .

Next, the operation of the deviation amount detecting device 3 and the driver monitoring system 4 shown in FIG. 17 will be described.
As described above, when light is irradiated from the three light-emitting points 60a, 60b, and 60c, the imaging device 1 captures the three light-emitting points 60a, 60b, and 60c, and outputs the captured image GP, which is an image of the light-emitting points 60a, 60b, and 60c, to the deviation amount detection device 3.
Regardless of whether or not ambient light is incident on the vehicle, the brightness of the light from the light-emitting points 60 a, 60 b, and 60 c is higher than the brightness of the areas around the light-emitting points 60 a, 60 b, and 60 c. Therefore, the shapes of the light-emitting points 60 a, 60 b, and 60 c captured in the captured image GP are clear.
The captured image GP output from the imaging device 1 to the deviation amount detection device 3 is basically an image captured when the driver is not seated in the driver's seat. However, even if the driver is seated in the driver's seat, the captured image GP may be an image captured when the driver is seated in the driver's seat, as long as the driver is seated in a manner that does not block the structure 2.

The position detection unit 61 acquires the captured image GP from the imaging device 1 (step ST15 in FIG. 20).
The position detection unit 61 detects the light emitting point image positions, which are the positions of the light emitting points 60a, 60b, and 60c on the image, from the captured image GP (step ST16 in FIG. 20).
The position detection unit 61 outputs light emitting point image position information indicating the light emitting point image position to the deviation amount calculation unit 62 .
The process of detecting the light emitting point image position by the position detector 61 is similar to the process of detecting the structure image position by the position detector 11 shown in FIG. 1, and therefore a detailed description thereof will be omitted.

The deviation amount calculation unit 62 acquires the light emitting point image position information and the deviation amount Δφ _roll in the rotation direction from the position detection unit 61 .
In addition, the deviation amount calculation unit 62 acquires, from the outside, reference image position information indicating the reference image positions, which are the correct positions on the image of the light-emitting points 60a, 60b, and 60c when the installation position of the imaging device 1 is correct, and the reference image G _REF .
In the third embodiment, the light-emitting point image position information and the reference image G _REF are each provided from outside the deviation amount detection device 3. However, this is merely an example, and the reference image position information and the reference image G _REF may each be stored in an internal memory of the deviation amount calculation unit 62.

The deviation amount calculation unit 62 moves the reference image G _REF horizontally, vertically, or rotates it so that the reference image position indicated by the reference image position information matches the light-emitting point image position indicated by the light-emitting point image position information.
That is, the deviation amount calculation unit 62 rotates the reference image G _REF by the deviation amount Δφ _roll in the rotation direction.
Furthermore, the deviation amount calculation unit 62 moves the reference image G _REF horizontally or vertically so that the reference image position (x, y) coincides with the scanning position (d _x , d _y ) at the minimum SAD (d _x , d _y ).
The deviation amount calculation unit 62 calculates the horizontal movement amount Δd _x (=x−d _x ) and the vertical movement amount Δd _y (=y−d _y ) of the reference _{image G REF when} the reference image position and the light emitting point image position coincide with each other.

The deviation amount calculation unit 62 calculates the deviation amount _Δφx between the installation position of the imaging device 1 and the correct installation position from the horizontal movement amount _Δdx of the reference image _GREF , as shown in the above formula (2) (step ST17 in FIG. 20). The deviation amount _Δφx is the amount of optical axis deviation between the optical axis _ΦXREF in the X-axis direction of the imaging device 1 when the imaging device 1 is installed in the correct position and the optical axis in the X-axis direction of the imaging device 1.
The deviation calculation unit 62 also calculates the deviation _Δφy between the installation position of the imaging device 1 and the correct installation position from the vertical movement Δd _y of the reference image G _REF , as shown in the above equation (3) (step ST17 in FIG. 20 ). The deviation _Δφy is the amount of optical axis deviation between the optical axis ΦY _REF in the Y-axis direction of the imaging device 1 when the imaging device 1 is installed in the correct position and the optical axis of the imaging device 1 in the Y-axis direction.

The deviation amount calculation unit 62 outputs the deviation amount Δφ _x in the X-axis direction, the deviation amount Δφ _y in the Y-axis direction, and the deviation amount Δφ _roll in the rotational direction to the driver monitoring system 4 .

In the above-described third embodiment, the deviation amount detection method is configured to include the steps of: an illumination device 60 having three or more light-emitting points emitting light from the light-emitting points; an imaging device 1 photographing the light-emitting points emitting light; and a deviation amount detection device 3 calculating the deviation amount between the installation position of the imaging device 1 and the correct installation position of the imaging device 1 based on the image of the light-emitting points captured by the imaging device 1. Therefore, the deviation amount detection method can suppress deterioration in the accuracy of calculating the deviation amount due to changes in the ambient light conditions.

In the misalignment detection device 3 shown in Figure 17, the lighting device 60 has three light-emitting points 60a, 60b, and 60c. However, this is merely an example, and the misalignment detection device 3 may be equipped with three or more lighting devices 60 each with a single light-emitting point.

Note that this disclosure allows for free combinations of the embodiments, modifications to any of the components of the embodiments, or the omission of any of the components in the embodiments.

1 Imaging device, 2 Structure, 3 Deviation amount detection device, 4 Driver monitoring system, 10 Lighting device, 10a Light source, 10b Light source, 10c Reflector, 11 Position detection unit, 12 Deviation amount calculation unit, 21 Position detection circuit, 22 Deviation amount calculation circuit, 31 Sensing unit, 32 Warning output unit, 41 Sensing circuit, 42 Warning output circuit, 51 Memory, 52 Processor, 60 Lighting device, 60a, 60b, 60c Light-emitting point, 61 Position detection unit, 62 Deviation amount calculation unit, 71 Position detection circuit, 72 Deviation amount calculation circuit.

Claims (13)

In order to suppress the influence of ambient light, which is light entering the vehicle from outside the vehicle,
a step in which a lighting device irradiates the vehicle structure with light emitted from a surface light source installed on one of both sides of the vehicle structure opposite to the side facing the imaging device ;
a step of capturing an image of a structure of the vehicle illuminated with the illumination light by the lighting device using the imaging device ;
and a step in which a deviation amount detection device calculates a deviation amount between an installation position of the imaging device and a correct installation position of the imaging device based on an image of a structure of the vehicle captured by the imaging device. In order to suppress the influence of ambient light, which is light entering the vehicle from outside the vehicle,
a step in which the lighting device reflects light emitted from the light source toward the vehicle structure using a reflector installed on one of both sides of the vehicle structure opposite to the side facing the imaging device, thereby irradiating the vehicle structure with the light;
a step of capturing an image of a structure of the vehicle illuminated with the illumination light by the lighting device using the imaging device;
and a step in which a deviation amount detection device calculates a deviation amount between an installation position of the imaging device and a correct installation position of the imaging device based on an image of a structure of the vehicle captured by the imaging device. an imaging device that captures an image of a vehicle structure;
A lighting device that irradiates light onto a structure of the vehicle,
a surface light source that irradiates the illumination light onto a structure of the vehicle;
the surface light source is installed on one of both sides of a structure of the vehicle opposite to a side facing the imaging device; and
In order to suppress the influence of ambient light, which is light entering the vehicle from outside the vehicle,

a position detection unit that acquires an image of the vehicle structure captured by the imaging device when the illumination light from the surface light source of the lighting device is irradiated onto the vehicle structure, and detects a structure image position that is a position on the image of the vehicle structure;
and a deviation amount calculation unit that calculates a deviation amount between the installation position of the imaging device and the correct installation position based on a reference image position that is a correct position on an image of a structure of the vehicle when the installation position of the imaging device is the correct installation position and the structure image position detected by the position detection unit. an imaging device that captures an image of a vehicle structure;
A lighting device that irradiates light onto a structure of the vehicle,
A light source and
a reflector that reflects the light emitted from the light source toward a structure of the vehicle,
the lighting device, wherein the reflector is installed on one of both sides of a structure of the vehicle opposite to a side facing the imaging device;
In order to suppress the influence of ambient light, which is light entering the vehicle from outside the vehicle,
a position detection unit that acquires an image of the vehicle structure captured by the imaging device when the illumination light reflected by the reflector of the illumination device is irradiated onto the vehicle structure, and detects a structure image position, which is the position of the vehicle structure on the image;
and a deviation amount calculation unit that calculates a deviation amount between the installation position of the imaging device and the correct installation position based on a reference image position that is a correct position on an image of a structure of the vehicle when the installation position of the imaging device is the correct installation position and the structure image position detected by the position detection unit. 4. The displacement amount detecting device according to claim 3 , wherein the brightness of the light emitting surface of said surface light source is uniform. 4. The deviation amount detecting device according to claim 3 , wherein the light emitted from the surface light source is light that produces a pattern on the illuminated surface of the vehicle structure. 4. The deviation amount detecting device according to claim 3 , wherein the surface light source blinks the light that is irradiated onto the vehicle structure. 5. The deviation amount detecting device according to claim 4 , wherein the reflector is installed at a position where it comes into contact with a structure of the vehicle or at a position away from the structure of the vehicle . the light source is a surface light source,
5. The displacement amount detecting device according to claim 4 , wherein the brightness of the light emitting surface of said surface light source is uniform. 5. The deviation amount detection device according to claim 4 , wherein the light emitted from the light source is light that produces a pattern on the irradiated surface of the vehicle structure, or the reflector has a pattern. 5. The deviation amount detecting device according to claim 4 , wherein the light source blinks the irradiated light that is irradiated onto the vehicle structure. 5. The displacement detection device according to claim 4 , wherein the reflector is a screen, paper, cloth, a wooden board, a resin board, or a metal board. 5. The deviation amount detecting device according to claim 3 , wherein the vehicle structure is a window frame attached to the vehicle.