JP7711615B2 - Vehicle imaging system and vehicle back door control system - Google Patents

Description

The present invention relates to a vehicle imaging system that captures images to check the status of a user behind a back door installed at the rear of a vehicle, and a vehicle back door control system that uses the vehicle imaging system to control the opening and closing of the back door.

For example, Patent Document 1 discloses a method for controlling an operated object that allows a user to open and close the left and right doors of a vehicle even when carrying luggage in both hands. In this method, an image of the surroundings of the left and right doors is captured by a camera (image capture device) attached to the vehicle, and the door is automatically opened or closed when a predetermined line of sight movement of the user is detected from the captured image.

For example, Patent Document 2 discloses an installation structure for an imaging device for capturing images of the rear of a vehicle. In the installation structure for an imaging device in Patent Document 2, one imaging device is attached to the ceiling inside the vehicle cabin so that the area behind the back door is the imaging range. When the back door is fully open, it is possible to directly capture an image of the area behind the back door, and when the back door is fully closed, it is possible to capture an image of the area behind the back door through the glass of the back door.

JP 2019-121183 A JP 2014-151778 A

The controlled object control method described in Patent Document 1 is intended for the left and right doors of a vehicle to assist users getting in and out of the vehicle, but does not consider the back door of a vehicle where users do not get in and out of the vehicle. Therefore, consider the case where the controlled object control method described in Patent Document 1 is applied to the automatic opening and closing of a flip-up back door installed at the rear of a vehicle to assist with loading and unloading luggage, etc. In this case, a camera needs to be installed so that the line of sight of a user within a specified distance behind the back door can be detected whether the back door is fully closed or fully open.

For example, consider the case where the mounting structure of Patent Document 2 is applied and one camera (imaging device) is mounted on the ceiling of the vehicle interior (luggage compartment) so that the rear of the back door is the imaging range. In this case, whether the back door is fully closed or fully open, the position and attitude of the camera do not change, and an image with the rear of the back door as the imaging range can be obtained. When the back door is fully open, it is possible to directly image the user behind the back door, and when the back door is fully closed, it is possible to image the user behind the back door through the glass of the back door. However, since the camera is installed on the ceiling of the vehicle interior (luggage compartment), the camera is installed much further forward than the back door. Therefore, in order to make the imaging range through the glass when the back door is fully closed such that the user's line of sight can be confirmed, the camera needs to be placed at a position much lower than the top of the glass (because the user's height is generally higher than the height of the vehicle). In this case, when the driver of the vehicle checks the rear through the rearview mirror inside the vehicle, the camera protruding downward from the ceiling of the vehicle interior gets in the way, which is undesirable because it reduces the rear visibility.

If the camera is installed on the back door instead of on the ceiling of the vehicle interior (luggage compartment) in order to prevent the camera (imaging device) from protruding downward from the ceiling of the vehicle interior, the position and orientation of the camera will change depending on whether the back door is fully closed or fully open, so two cameras would be necessary. For example, the first camera needs to be installed so that the rear of the back door is its imaging range when the back door is fully closed, and the second camera needs to be installed so that the rear of the back door is its imaging range when the back door is fully open. In this case, the first camera can be positioned very close to the glass of the back door, so it is possible to place this first camera close to the top edge of the glass of the back door, which prevents the camera from protruding downward into the vehicle interior. However, a second camera is also needed when the back door is fully open, which is not preferable as it requires two cameras.

The present invention was devised in light of these points, and aims to provide an imaging system for a vehicle and a back door control system for a vehicle that suppresses deterioration of rear visibility and can properly capture images of a user who is within a specified distance behind the back door with a single imaging device, regardless of whether the back door is fully closed or fully open.

To solve the above problems, the first aspect of the present invention is an imaging system for a vehicle that acquires an image for recognizing the state of a user behind a flip-up back door provided at the rear of the vehicle, the imaging system having an imaging device provided on the back door, on the inside of the vehicle and near the upper end of the back door glass, and that moves in response to the opening and closing of the back door, and an imaging range adjustment mechanism that brings a user who is within a predetermined distance behind the back door into the imaging range of the imaging device, regardless of whether the back door is fully closed or fully open.

The second aspect of the present invention is a vehicle imaging system according to the first aspect of the present invention, in which the imaging range adjustment mechanism is a half mirror with a predetermined reflectance and transmittance that transmits light from a user within the predetermined distance behind the back door to the imaging device when the back door is fully closed, and reflects and guides light from a user within the predetermined distance behind the back door to the imaging device when the back door is fully open.

The third aspect of the present invention is the vehicle imaging system according to the second aspect of the present invention, in which the half mirror is attached to a portion of the back door glass that faces the imaging device, or the portion of the back door glass that faces the imaging device is processed to be the half mirror.

The fourth aspect of the present invention is the vehicle imaging system according to the first aspect, in which the imaging range adjustment mechanism is a wide-angle lens that captures within the imaging range a user who is within the specified distance behind the back door, regardless of whether the back door is in the fully closed state or the fully open state.

The fifth aspect of the present invention is a vehicle imaging system according to the first aspect of the present invention, in which the imaging range adjustment mechanism is an attitude control device that can change the attitude of the imaging device provided on the back door to a substantially horizontal state so that a user within the specified distance behind the back door is within the imaging range, regardless of whether the back door is in the fully closed state or the fully open state.

The sixth aspect of the present invention is a vehicle imaging system according to any one of the first to fifth aspects, further comprising a lighting device capable of illuminating a user within the specified distance behind the back door, whether the back door is in the fully closed state or the fully open state.

Next, the seventh invention of the present invention is a vehicle back door control system that includes any one of the vehicle imaging systems of the first to fifth inventions, a back door drive device capable of operating the back door from the fully closed state to the fully open state and from the fully open state to the fully closed state, a back door opening/closing state detection device capable of detecting whether the back door is in the fully closed state or the fully open state, and a control device that detects whether the back door is in the fully closed state or the fully open state using the back door opening/closing state detection device, obtains an image of a user who is within the specified distance behind the back door using the vehicle imaging system, and controls the back door drive device to change from the fully closed state to the fully open state when the back door is in the fully closed state based on the recognition result of the state of the user captured in the image, and controls the back door drive device to change from the fully closed state to the fully open state when the back door is in the fully open state.

The eighth aspect of the present invention is a vehicle back door control system according to the seventh aspect of the present invention, further comprising a lighting device capable of illuminating a user within the specified distance behind the back door, whether the back door is in the fully closed state or the fully open state.

Next, the ninth invention of the present invention is a vehicle back door control system comprising: the vehicle imaging system of the second or third invention; a back door drive device capable of operating the back door from the fully closed state to the fully open state and from the fully open state to the fully closed state; a back door opening/closing state detection device capable of detecting whether the back door is in the fully closed state or the fully open state; and a control device that detects whether the back door is in the fully closed state or the fully open state using the back door opening/closing state detection device, obtains an image of a user who is within the specified distance behind the back door using the vehicle imaging system, and controls the back door drive device to change from the fully closed state to the fully open state when the back door is in the fully closed state based on the recognition result of the state of the user captured in the image, and controls the back door drive device to change from the fully closed state to the fully open state when the back door is in the fully open state. The control device is a vehicle back door control system that, when the back door is fully open, creates a normal image by inverting the acquired image vertically and horizontally, and controls the back door drive device based on the recognition result of the user's state captured in the normal image.

The tenth aspect of the present invention is the vehicle back door control system according to the ninth aspect, further comprising a lighting device capable of illuminating a user within the specified distance behind the back door, whether the back door is in the fully closed state or the fully open state.

According to the first invention, by having an imaging range adjustment mechanism, simply by attaching one imaging device to the back door, users who are within a predetermined distance behind the back door can be captured within the imaging range of the imaging device, regardless of whether the back door is fully closed or fully open. In addition, since the imaging device is provided on the inside of the vehicle and near the upper end of the back door glass, protrusion of the imaging device downward inside the vehicle cabin is suppressed, and deterioration of rear visibility is suppressed. Therefore, deterioration of rear visibility is suppressed, and a single imaging device can appropriately capture images of users who are within a predetermined distance behind the back door, regardless of whether the back door is fully closed or fully open.

According to the second invention, the imaging range adjustment mechanism can be appropriately realized by using a half mirror with a predetermined reflectance and transmittance.

According to the third invention, the imaging range adjustment mechanism can be appropriately realized by attaching a half mirror to the back door glass or by forming a half mirror by applying half mirror processing to a predetermined location on the back door glass.

According to the fourth invention, the imaging range adjustment mechanism can be appropriately realized by using a wide-angle lens.

According to the fifth aspect of the invention, the imaging range adjustment mechanism can be appropriately realized by using an attitude control device that changes the attitude of the imaging device to a substantially horizontal state regardless of whether the back door is fully closed or fully open.

According to the sixth, eighth and tenth inventions, when a user within a specified distance behind a back door and the surroundings are dark and an image with appropriate brightness cannot be obtained, an image with appropriate brightness can be obtained by using a lighting device.

According to the seventh invention, by applying any one of the vehicle imaging systems according to the first to fifth inventions to a vehicle back door control system, it is possible to appropriately realize a vehicle back door control system that automatically opens and closes the back door based on the state of a user behind the back door.

According to the ninth invention, when an image reflected by a half mirror is acquired while the back door is fully open, the image is inverted vertically and horizontally. Therefore, a normal image is created by inverting the acquired image vertically and horizontally, and the normal image is used to recognize the user's state, thereby enabling recognition to be completed in a shorter time.

1 is a diagram illustrating an example of the configuration and arrangement of a vehicle imaging system and a vehicle back door control system having the vehicle imaging system according to a first embodiment, as viewed from the left side of the vehicle. FIG. 1 is a diagram illustrating an example of the arrangement of a vehicle imaging system according to a first embodiment, as viewed from the rear of the vehicle. FIG. 2 is an enlarged view of the periphery of the vehicle imaging system and the vehicle back door control system in FIG. 1 . 10A and 10B are diagrams illustrating an example of an imaging range of the imaging device and an imaging range captured by a half mirror when the back door is in a fully closed state in the first embodiment. 10A and 10B are diagrams illustrating an example of an imaging range by the imaging device and reflection by a half mirror when the back door is fully open in the first embodiment. FIG. 5 is an enlarged view of the periphery of the imaging device in FIG. 4 . FIG. 6 is an enlarged view of the periphery of the imaging device in FIG. 5 . 10A and 10B are diagrams illustrating an example of an imaging range of the imaging device and an imaging range captured by a half mirror when the back door is in a fully closed state in the first embodiment. 10A and 10B are diagrams illustrating an example of an imaging range by the imaging device and reflection by a half mirror when the back door is fully open in the first embodiment. 5 is a flowchart illustrating an example of a processing procedure of a control device that controls opening and closing of a back door in the first embodiment. 13A and 13B are diagrams illustrating an example of the configuration and arrangement of a vehicle imaging system and a vehicle back door control system according to a second embodiment, and an example of the imaging range of the imaging device when the back door is in a fully closed state. 13A and 13B are diagrams illustrating an example of the configuration and arrangement of a vehicle imaging system and a vehicle back door control system according to a second embodiment, and an example of the imaging range of the imaging device when the back door is in a fully open state. 13A and 13B are diagrams illustrating an example of the configuration and arrangement of a vehicle imaging system and a vehicle back door control system according to a third embodiment, and an example of the imaging range of the imaging device when the back door is in a fully closed state. 13A and 13B are diagrams illustrating an example of the configuration and arrangement of a vehicle imaging system and a vehicle back door control system according to a third embodiment, and an example of the imaging range of the imaging device when the back door is in a fully open state.

The following describes embodiments of the present invention with reference to the drawings. In the drawings, when "front", "rear", "left", "right", "up" and "down" are used, "front" and "rear" refer to the front-rear direction of the vehicle, "left" and "right" refer to the left-right direction of the vehicle, and "up" and "down" refer to the top-bottom direction of the vehicle. Hereinafter, first to third embodiments of a vehicle imaging system 2 (2B, 2C) that acquires images for recognizing the state of a user U behind a flip-up back door 81 provided at the rear of a vehicle 80, and a vehicle back door control system 1 (1B, 1C) that recognizes a user U using the vehicle imaging system 2 (2B, 2C) and automatically opens and closes the back door 81 will be described.

[First embodiment (FIGS. 1 to 10)]
[Configuration and layout of the vehicle imaging system 2 (FIGS. 1 to 3)]
First, a vehicle imaging system 2 and a vehicle back door control system 1 in the first embodiment will be described with reference to Figures 1 to 10. As shown in Figures 1 and 3, the vehicle imaging system 2 in the first embodiment includes an imaging device 11, a half mirror 12, an illumination device 13, and the like.

The imaging device 11 is, for example, a CCD camera, and based on imaging instructions from the control device 50, captures an image of a preset imaging range to create a color or monochrome image, and outputs the created image to the control device 50. As shown in FIG. 3, the imaging device 11 is mounted on the back door 81 by a connecting arm 14, and only one imaging device 11 is provided on the inside of the vehicle and near the upper end of the back door glass 82. As shown in the examples when the back door 81 is fully closed (see FIG. 4) and when the back door 81 is fully open (see FIG. 5), the imaging device 11 moves in response to the opening and closing of the back door 81.

The half mirror 12 (corresponding to the imaging range adjustment mechanism) is set to a predetermined reflectance and transmittance (e.g., reflectance A%, transmittance B%), and reflects A% of incident light and transmits B%. The half mirror 12 is, for example, rectangular and has a predetermined thickness, and is attached (attached) to the rear door glass 82 at approximately the center in the left-right direction, the upper end, and the inside of the vehicle, as shown in Figures 2 and 3. Note that the half mirror 12 shown in this embodiment is separate from the rear door glass 82 and has a predetermined thickness, but it may be a sticker-like half mirror with almost no thickness, or a part of the rear door glass 82 may be made into a half mirror by applying half mirror processing (coating, etc.) to the part of the rear door glass 82 facing the imaging device 11.

The lighting device 13 is provided adjacent to the imaging device 11 (attached to the imaging device 11). The optical axis of the lighting device 13 is set to be approximately the same as the optical axis of the imaging device 11, and the illumination range of the lighting device 13 is set to be approximately the same as the imaging range of the imaging device 11. The output (illumination light) of the lighting device 13 is turned on and off based on a lighting control signal from the control device 50.

The connecting arm 14 holds the imaging device 11 on the inside of the back door 81, near the upper end of the back door glass 82.

[Imaging range of the vehicle imaging system 2 (FIGS. 4 to 9)]
The position and attitude (angle) of the imaging device 11 are adjusted so that, when the back door 81 is in a fully closed state, light from a user U within a predetermined distance L1 behind the back door 81 is transmitted through the half mirror 12 and guided to the imaging device 11, as shown in Fig. 4 and Fig. 6. In other words, the optical axis of the imaging device 11 transmitted through the half mirror 12 is adjusted to be the optical axis J1 as shown in Fig. 4 and Fig. 6. As a result, when the back door 81 is in a fully closed state, the imaging range of the imaging device 11 becomes the imaging range S1 shown in Fig. 4, Fig. 6, and Fig. 8. Note that the range within the imaging range S1 that is not blocked by the edge of the back door glass 82 or the like is the effective imaging range S1A, and in the example of Fig. 4, the imaging range S1 and the effective imaging range S1A are substantially the same. 4, when the back door 81 is in a fully closed state, the position and attitude (angle) of the imaging device 11 are adjusted so that the face of the user U, who is within a predetermined distance L1 behind the back door 81, falls within the effective imaging range S1A of the imaging device 11. Although not shown, light reflected by the half mirror 12 is also guided to the imaging device 11. Therefore, in the image acquired by the imaging device 11, an image contained in the transmitted light (in this case, an image including the user U) and an image contained in the reflected light (in this case, an image of approximately the interior of the luggage compartment of the vehicle) overlap.

The position and attitude (angle) of the imaging device 11 are further adjusted so that, when the back door 81 is fully open, the light from the user U who is within a predetermined distance L1 behind the back door 81 is reflected by the half mirror 12 and guided to the imaging device 11, as shown in FIG. 5 and FIG. 7. In other words, the optical axis of the imaging device 11 reflected by the half mirror 12 is adjusted to be the optical axis J2 as shown in FIG. 5 and FIG. 7. As a result, when the back door 81 is fully open, the imaging range of the imaging device 11 becomes the imaging range S2 shown in FIG. 5, FIG. 7, and FIG. 9. Note that the range within the imaging range S2 that is not blocked by the edge of the back door 81 is the effective imaging range S2A. As shown in FIG. 5, when the back door 81 is fully open, the position and attitude (angle) of the imaging device 11 are adjusted so that the face of the user U who is within a predetermined distance L1 behind the back door 81 is included in the effective imaging range S2A of the imaging device 11. Although not shown in the figure, light transmitted through the half mirror 12 is also guided to the imaging device 11. Therefore, in the image captured by the imaging device 11, an image contained in the reflected light (in this case, an image containing the user U) and an image contained in the transmitted light (in this case, an image of substantially the sky) overlap.

The lighting device 13 is positioned adjacent to the imaging device 11 (almost in contact with the imaging device 11), and the attitude (angle) of the lighting device 13 is set to be almost the same as that of the imaging device 11. When the back door 81 is fully closed, the optical axis of the lighting device 13 is set to be almost the same as the optical axis J1 of the imaging device 11 shown in Figures 4 and 6. When the back door 81 is fully closed, the illumination range of the lighting device 13 is set to be almost the same as the effective imaging range S1A of the imaging device 11 shown in Figure 4. As a result, when the back door 81 is fully closed, the illumination light emitted from the lighting device 13 illuminates and brightens the imaging subject within the effective imaging range S1A of the imaging device 11.

The optical axis of the lighting device 13 is further configured to be approximately the same as the optical axis J2 of the imaging device 11 shown in Figures 5 and 7 when the back door 81 is fully open. Furthermore, the illumination range of the lighting device 13 is configured to be approximately the same as the effective imaging range S2A of the imaging device 11 shown in Figure 5 when the back door 81 is fully open. As a result, when the back door 81 is fully open, the illumination light emitted from the lighting device 13 illuminates and brightens the imaging subject within the effective imaging range S2A of the imaging device 11.

When an imaging device is mounted on the ceiling 83 of the vehicle 80, rather than near the upper end of the back door glass 82 of the back door 81, as in the case of imaging device 211 shown by the dotted line in FIG. 3, in order to obtain an imaging range S1 equivalent to that of imaging device 11, it is necessary to mount the imaging device at a fairly low position. When an imaging device is mounted on the ceiling 83 of the vehicle 80, as in imaging device 211, the imaging device 211 protruding downward from the ceiling 83 gets in the way when the driver checks the rear through the rearview mirror inside the vehicle, deteriorating the rear visibility, which is not preferable.

[Configuration and layout of vehicle back door control system 1 (Figs. 1 to 3)]
Next, a description will be given of the vehicle back door control system 1. As shown in Fig. 1 and Fig. 3, the vehicle back door control system 1 includes a vehicle imaging system 2, a back door drive device 61, a back door open/close state detection device 62, a control device 50, and the like.

The tailgate drive device 61 is, for example, an electric motor, and is capable of operating the tailgate 81 from a fully closed state to a fully open state, or from a fully open state to a fully closed state, based on a drive signal from the control device 50.

The back door open/closed state detection device 62 is, for example, a limit switch, which outputs a fully closed signal to the control device 50 when the back door 81 is fully closed, and outputs a fully open signal to the control device 50 when the back door 81 is fully open. The back door open/closed state detection device 62 is provided on the back door 81 or in the vicinity of the back door 81.

The control device 50 includes a CPU, RAM, ROM (storage device), timer, etc., which are not shown, and controls the back door drive device 61 to open and close the back door 81 according to the processing procedure of the flowchart described later. The control device 50 receives an image from the imaging device 11 and a detection signal from the back door open/close state detection device 62. The control device 50 also outputs an imaging instruction to the imaging device 11 and outputs a lighting control signal to the lighting device 13. The control device 50 also has an antenna 51 for communicating with the electronic key 63 held by the user U. As will be described in the flowchart described later, the control device 50 communicates with the electronic key 63 via the antenna 51 and can determine whether the user U holding the electronic key 63 is within a predetermined distance L1 behind the back door 81.

[Processing procedure for controlling the vehicle back door by the control device 50 (FIG. 10)]
Next, an example of a processing procedure for [vehicle back door control] (automatic opening/closing control of the back door) by the control device 50 will be described with reference to Fig. 10. The control device 50 starts the processing shown in Fig. 10 at a predetermined time interval (an interval of several tens of ms to several hundreds of ms) and proceeds to step S010.

In step S010, the control device 50 determines whether the back door drive device is in operation to move from a fully open state to a fully closed state (whether the drive signal output was started in step S090B at the time of the previous or previous startup). If the back door drive device is in operation to move from a fully open state to a fully closed state (Yes), the control device 50 proceeds to step S015, and if not (No), the control device 50 proceeds to step S020.

When the process proceeds to step S015, the control device 50 determines whether the back door is fully closed or not based on the detection signal from the back door open/close state detection device. If the back door is fully closed (Yes), the control device 50 proceeds to step S030, and if the back door is not fully closed (No), the control device 50 ends the process shown in FIG. 10.

If the process proceeds to step S020, the control device 50 determines whether the back door drive device is in operation to move from a fully closed state to a fully open state (whether the drive signal output was started in step S090A at the time of the previous or previous startup). If the back door drive device is in operation to move from a fully closed state to a fully open state (Yes), the control device 50 proceeds to step S025, and if not (No), the control device 50 proceeds to step S035.

When the process proceeds to step S025, the control device 50 determines whether the back door is fully open or not based on the detection signal from the back door open/closed state detection device. If the back door is fully open (Yes), the control device 50 proceeds to step S030, and if the back door is not fully open (No), the control device 50 ends the process shown in FIG. 10.

If processing proceeds to step S030, the control device 50 stops the operation of the back door drive device and proceeds to step S035.

When step S090A in FIG. 10 is executed, the back door drive device starts to drive from the fully closed state to the fully open state, and when step S090B is executed, the drive device starts to drive from the fully open state to the fully closed state. However, the control device 50 outputs a drive signal to the back door drive device in step S090A or step S090B and ends the processing of FIG. 10, and even after the processing ends, the drive signal is still input to the back door drive device. Then, when a predetermined time has elapsed and the next processing of FIG. 10 is started, the control device 50 stops the back door drive device in step S030 if the back door drive device is operating in steps S010 to S030, or if the back door has moved to a position where the back door drive device should be stopped.

If the process proceeds to step S035, the control device 50 communicates with the electronic key 63 via the antenna 51 as shown in FIG. 1, and determines whether a user U carrying the electronic key 63 is present within a predetermined distance L1 behind the back door. If the control device 50 determines that a user carrying the electronic key is present within a predetermined distance behind the back door (Yes), the process proceeds to step S040, otherwise (No), the process ends as shown in FIG. 10. Note that the process from step S035 onwards is only executed when the back door is in a fully closed or fully open state, and is not executed when the back door drive device is operating in a fully open state -> fully closed state or in a fully closed state -> fully open state (when the back door is operating halfway between a fully closed state and a fully open state).

If the process proceeds to step S040, the control device 50 outputs an image capture instruction to the imaging device 11 to capture an image of the imaging range with the imaging device 11, acquires the image from the imaging device 11, and proceeds to step S045. Note that the image captured in the acquired image is a superposition of an image contained in the light that has passed through the half mirror 12 (transmitted image) and an image contained in the light that has been reflected by the half mirror 12 (reflected image). The control device 50 extracts an image of the user U and performs face authentication using face authentication, which will be described later.

In step S045, the control device 50 determines whether the brightness of the user U captured in the acquired image is sufficient to authenticate the face of the user pre-stored in the storage device. If the control device 50 determines that the image is dark (does not have sufficient brightness) (Yes), the process proceeds to step S050, and if not (No), the process proceeds to step S060.

If the process proceeds to step S050, the control device 50 outputs an illumination control signal to the illumination device to set the illumination device ON and irradiate illumination light, then outputs an image capture instruction to recapture the image, sets the illumination device OFF, and proceeds to step S060.

If the process proceeds to step S060, the control device 50 determines whether the back door is fully open. If the back door is fully open (Yes), the control device 50 proceeds to step S065, and if the back door is not fully open (No), the control device 50 proceeds to face authentication in step S070.

If the process proceeds to step S065, the control device 50 creates a normal image by inverting the acquired image vertically and horizontally, and proceeds to face authentication in step S070.

User U registers his/her face in advance in the storage device of the control device 50. When registering, user U stands within a predetermined distance L1 behind the back door 81 with the back door 81 fully closed as shown in FIG. 4, and registers an image of himself/herself captured using the imaging device 11. In this case, since the back door 81 is fully closed, a normal image transmitted through the half mirror 12 is registered. Furthermore, if the lens of the imaging device 11 is a wide-angle lens such as a fisheye lens, the image is distorted in a curved shape depending on the position in the image, so the control device 50 corrects the distortion depending on the position in the image and registers it in the storage device.

In contrast, the image acquired when the back door is fully open is an image reflected by the half mirror 12, as shown in Figs. 5 and 7, and is therefore inverted vertically and horizontally. If this inverted image is compared with a registered normal image for face authentication, the authentication time may become longer. Therefore, when an inverted image is acquired, in step S065, a normal image is created by inverting the inverted image vertically and horizontally, and the authentication time is shortened by comparing it with the registered normal image.

If the process proceeds to step S070, the control device 50 corrects the distortion according to the position within the image, compares the image after the distortion correction with an image of the user U registered in the storage device, and performs face authentication (recognition of the user's state), and proceeds to step S075. Note that the face authentication process is an existing process, and detailed explanation will be omitted.

In step S075, the control device 50 determines whether or not the user U in the captured image matches the user U registered in the storage device as a result of the facial authentication. If there is a match (Yes), the control device 50 proceeds to step S080, and if there is no match (No), the control device 50 ends the process shown in FIG. 10.

If the process proceeds to step S080, the control device 50 determines whether the back door is fully closed. If the back door is fully closed (Yes), the control device 50 proceeds to step S090A, and if the back door is not fully closed (No), the control device 50 proceeds to step S090B. If the process has proceeded to the determination of step S080, the back door is either fully closed or fully open in steps S010 to S030, so step S090B is executed if the back door is fully open.

If the process proceeds to step S090A, the control device 50 outputs a drive signal to the back door drive device so that the back door moves from a fully closed state to a fully open state, and ends the process shown in FIG. 10.

If the process proceeds to step S090B, the control device 50 outputs a drive signal to the back door drive device so that the back door moves from a fully open state to a fully closed state, and ends the process shown in FIG. 10.

As described above, in the first embodiment, the vehicle imaging system 2 having one imaging device 11 provided near the upper end of the back door glass 82 of the back door 81, the half mirror 12, and the lighting device 13 can appropriately capture the user U who is within a predetermined distance L1 behind the back door 81 within the imaging ranges S1 and S2 of the imaging device 11, regardless of whether the back door 81 is in a fully open or fully closed state. The vehicle back door control system 1 having the vehicle imaging system 2, the back door drive device 61, the back door open/closed state detection device 62, and the control device 50 can recognize the user U who is within a predetermined distance L1 behind the back door 81 and appropriately open/close the back door 81 based on the recognition result. In other words, deterioration of rear visibility is suppressed, and the user who is within a predetermined distance behind the back door can be appropriately imaged with one imaging device, regardless of whether the back door is in a fully closed or fully open state.

[Second embodiment]
[Configuration and arrangement of the vehicle imaging system 2B and the vehicle back door control system 1B, and the imaging range of the vehicle imaging system 2B (FIGS. 11 and 12)]
Next, a vehicle imaging system 2B and a vehicle back door control system 1B in the second embodiment will be described with reference to Fig. 11 and Fig. 12. In the vehicle imaging system 2B in the second embodiment, the half mirror 12 is omitted from the vehicle imaging system 2 in the first embodiment, and instead, the angle of the imaging range SB of the imaging device 11B is made very large (ultra) wide angle. The following mainly describes the differences from the first embodiment.

As in the first embodiment, the imaging device 11B is, for example, a CCD camera, and based on an imaging command from the control device 50, captures an image of a preset imaging range to create a color or monochrome image, and outputs the created image to the control device 50. Also, as in the first embodiment, the imaging device 11B is provided on the back door 81 by a connecting arm 14, and only one imaging device 11B is provided on the inside of the vehicle and near the upper end of the back door glass 82. As shown in the examples when the back door 81 is fully closed (see FIG. 11) and when the back door 81 is fully open (see FIG. 12), the imaging device 11B moves in response to the opening and closing of the back door 81.

However, the imaging device 11B is provided with a wide-angle lens 11Z (corresponding to an imaging range adjustment mechanism, e.g., a fisheye lens) so as to obtain the imaging range SB shown in Figs. 11 and 12. The optical axis JB of the imaging device 11B is adjusted so as to face downward when the back door 81 is fully closed as shown in Fig. 12, and is adjusted so as to pass, for example, near the lower end of the back door glass 82. As a result, when the back door 81 is fully closed, it is possible to obtain an effective imaging range SBA (an imaging range that is not blocked by the back door, etc.) within the imaging range SB as shown in Fig. 11. Also, when the back door 81 is fully open, it is possible to obtain an effective imaging range SBB (an imaging range that is not blocked by the back door, etc.) within the imaging range SB as shown in Fig. 12.

As in the first embodiment, the lighting device 13B is provided adjacent to the imaging device 11B, and the optical axis of the lighting device 13B is set to be approximately the same as the optical axis of the imaging device 11B. However, the illumination range of the lighting device 13B in the second embodiment is set to a larger angle than the illumination range of the lighting device 13 in the first embodiment, and is set to an illumination range equivalent to the imaging range SB of the imaging device 11B.

The processing procedure of the control device 50 is the same as that of the first embodiment shown in the flowchart of FIG. 10 except that the up/down/left/right image inversion in step S065 is omitted, but the rest of the processing procedure is the same, so a description of the processing procedure is omitted.

In the second embodiment, the imaging range SB is wider than in the first embodiment, resulting in an image with a larger curvature and a larger amount of image data. This increases the correction range and amount of correction in the image correction process before performing face authentication of the user, but the half mirror 12 can be omitted. As in the first embodiment, deterioration of rear visibility is suppressed, and a single imaging device can appropriately capture an image of a user who is within a specified distance behind the back door, regardless of whether the back door is fully closed or fully open.

[Third embodiment]
[Configuration and arrangement of the vehicle imaging system 2C and the vehicle back door control system 1C, and the imaging range of the vehicle imaging system 2C (FIGS. 13 to 14)]
Next, a vehicle imaging system 2C and a vehicle back door control system 1C in the third embodiment will be described with reference to Fig. 13 and Fig. 14. In the vehicle imaging system 2C in the third embodiment, the half mirror 12 is omitted from the vehicle imaging system 2 in the first embodiment, and instead, the imaging device 11C is made movable. The imaging device 11C is provided on the back door 81 via a posture control device 15 that can change the posture of the imaging device 11C to a substantially horizontal state regardless of whether the back door 81 is in a fully closed state or a fully open state. Below, differences from the first embodiment will be mainly described.

As in the first embodiment, the imaging device 11C is, for example, a CCD camera, and based on an imaging command from the control device 50, captures an image of a preset imaging range to create a color or monochrome image, and outputs the created image to the control device 50. Also, as in the first embodiment, the imaging device 11C is provided on the back door 81 via the attitude control device 15 (corresponding to the imaging range adjustment mechanism), and only one imaging device 11C is provided on the inside of the vehicle and near the upper end of the back door glass 82. As shown in the examples when the back door 81 is fully closed (see FIG. 11) and when the back door 81 is fully open (see FIG. 12), the imaging device 11B moves in response to the opening and closing of the back door 81.

The attitude control device 15 is, for example, a simple arm, and holds the imaging device 11C so that the imaging device 11C can rotate freely around a rotation axis extending in the left-right direction of the vehicle. A weight is attached to the lower part of the imaging device 11C. As a result, whether the back door 81 is fully closed or fully open, the attitude of the imaging device 11C is maintained in a substantially horizontal state, and the optical axis JC of the imaging device 11C is also maintained in a substantially horizontal direction. As shown in FIG. 13, when the back door 81 is fully closed, it is possible to obtain an effective imaging range SCA (an imaging range that is not blocked by the back door 81, etc.) within the imaging range SC. Also, as shown in FIG. 14, when the back door 81 is fully open, it is possible to obtain an effective imaging range SCB (an imaging range that is not blocked by the back door 81, etc.) within the imaging range SC.

The attitude control device 15 may have an electric motor controlled by the control device 50. The imaging device 11C may be configured to be rotatable around a rotation axis extending in the left-right direction of the vehicle, and the rotation angle may be changed by the electric motor.

As in the first embodiment, the lighting device 13C is provided adjacent to the imaging device 11C, and the optical axis of the lighting device 13C is set to be approximately the same as the optical axis of the imaging device 11C. The illumination range of the lighting device 13C is set to be the same as the imaging range SC of the imaging device 11C.

The processing procedure of the control device 50 is the same as that of the first embodiment shown in the flowchart of FIG. 10 except that the up/down/left/right image inversion in step S065 is omitted, and a process is added to control the attitude of the imaging device 11C using an electric motor depending on whether the back door is fully closed or fully open before acquiring the image in step S040 (if the attitude control device 15 has an electric motor). Note that the rest of the processing procedure is the same, so a description of the processing procedure is omitted.

In the third embodiment, compared to the first embodiment, a posture control device 15 is required to set the posture of the imaging device 11C to a substantially horizontal state, but the half mirror 12 can be omitted. In addition, if the posture control device 15 is configured with an arm and a weight, the posture control device 15 can be realized with a simpler configuration. And, as in the first embodiment, deterioration of rear visibility is suppressed, and a single imaging device can appropriately capture an image of a user who is within a specified distance behind the back door, regardless of whether the back door is fully closed or fully open.

● [Other]
The vehicle imaging systems 2, 2B, and 2C and the vehicle back door control systems 1, 1B, and 1C of the present invention are not limited to the appearance, configuration, structure, processing procedure, etc. described in the present embodiment, and various changes, additions, and deletions are possible within the scope of the present invention. For example, the processing procedure of the control device 50 is not limited to the processing of the flowchart shown in FIG.

In the description of this embodiment, an example was described in which the vehicle imaging systems 2, 2B, and 2C have lighting devices 13, 13B, and 13C, but the lighting devices 13, 13B, and 13C may be omitted. Also, in the description of this embodiment, an example was described in which the lighting devices are positioned adjacent to the imaging devices, but the lighting devices may be positioned in any position as long as they can brightly illuminate the imaging range (effective imaging range) of the imaging devices.

In the description of this embodiment, when the half mirror 12 is used, an example in which the half mirror 12 is provided on the inside of the vehicle has been described, but the half mirror may be provided on the outside of the vehicle. When the half mirror 12 is used, the shape, thickness, reflectance, transmittance, etc. of the half mirror are not particularly limited. Also, in the processing procedure of the first embodiment (FIG. 10), when the back door is fully open, a normal image is created by flipping the image up and down and left and right in step S065, but step S065 may be omitted and the up and down and left and right inverted image may be used as the image for face recognition.

In the description of this embodiment, the flowchart in Figure 10 has been used to explain an example of recognizing the state of user U by recognizing the face of user U (face recognition in which the face is compared with the face of a registered user), but it is not limited to face recognition as long as the state of user U can be recognized, such as the direction of movement of the face or the direction of gaze movement of user U.

REFERENCE SIGNS LIST 1, 1B, 1C Vehicle back door control system 2, 2B, 2C Vehicle imaging system 11, 11B, 11C Imaging device 11Z Wide-angle lens 12 Half mirror 13 Illumination device 14 Connection arm 15 Attitude control device 50 Control device 51 Antenna 61 Back door drive device 62 Back door open/closed state detection device 63 Electronic key 80 Vehicle 81 Back door 82 Back door glass 83 Ceiling J1, J2 Optical axis JB, JC Optical axis L1 Predetermined distance S1, S2 Imaging range S1A, S2A Effective imaging range SB, SC Imaging range SBA, SBB Effective imaging range SCA, SCB Effective imaging range U User

Claims (7)

An image capturing system for a vehicle that captures an image for recognizing a state of a user behind a flip-up back door provided at a rear part of the vehicle,
an imaging device that is provided on the back door, attached to the inside of the vehicle and forward of an upper end of the back door glass in the vehicle front-rear direction , and moves in response to opening and closing of the back door;
an imaging range adjustment mechanism that includes a user who is within a predetermined distance behind the back door within an imaging range of the imaging device, regardless of whether the back door is in a fully closed state or a fully open state;
It has
The imaging range adjustment mechanism includes:
a half mirror having a predetermined reflectance and transmittance, which transmits light from a user within the predetermined distance behind the back door when the back door is in the fully closed state and guides it to the imaging device, and which reflects light from a user within the predetermined distance behind the back door and guides it to the imaging device when the back door is in the fully open state;
Vehicle imaging system. 2. The vehicle imaging system according to claim 1 ,
The half mirror is attached to a portion of the back door glass provided in the back door facing the imaging device.
Alternatively, a portion of the back door glass provided in the back door facing the imaging device is processed to be the half mirror.
Vehicle imaging system. An image capturing system for a vehicle that captures an image for recognizing a state of a user behind a flip-up back door provided at a rear part of the vehicle,
an imaging device that is provided on the back door, attached to the inside of the vehicle and forward of an upper end of the back door glass in the vehicle front-rear direction, and moves in response to opening and closing of the back door;
an imaging range adjustment mechanism that includes a user who is within a predetermined distance behind the back door within an imaging range of the imaging device, regardless of whether the back door is in a fully closed state or a fully open state;
It has
The imaging range adjustment mechanism includes:
an attitude control device capable of changing the attitude of the imaging device provided on the back door to a substantially horizontal state so that a user within the predetermined distance behind the back door is within the imaging range regardless of whether the back door is in the fully closed state or the fully open state;
Vehicle imaging system. The vehicle imaging system according to any one of claims 1 to 3 ,
Further, the vehicle has a lighting device capable of illuminating a user who is within the predetermined distance behind the back door, regardless of whether the back door is in the fully closed state or the fully open state.
Vehicle imaging system. An imaging system for a vehicle according to any one of claims 1 to 4 ,
A back door drive device capable of operating the back door from the fully closed state to the fully open state and from the fully open state to the fully closed state;
a back door open/close state detection device capable of detecting whether the back door is in the fully closed state or the fully open state;
a control device that detects whether the back door is in the fully closed state or the fully open state using the back door opening/closing state detection device, acquires an image of a user within the predetermined distance behind the back door using the vehicle imaging system, and controls the back door drive device to change from the fully closed state to the fully open state based on a recognition result of the state of the user captured in the image when the back door is in the fully closed state, and controls the back door drive device to change from the fully open state to the fully closed state when the back door is in the fully open state;
It has
A vehicle back door control system. The vehicle imaging system according to claim 1 or 2 ,
A back door drive device capable of operating the back door from the fully closed state to the fully open state and from the fully open state to the fully closed state;
a back door open/close state detection device capable of detecting whether the back door is in the fully closed state or the fully open state;
a control device that detects whether the back door is in the fully closed state or the fully open state using the back door opening/closing state detection device, acquires an image of a user within the predetermined distance behind the back door using the vehicle imaging system, and controls the back door drive device to change from the fully closed state to the fully open state based on a recognition result of the state of the user captured in the image when the back door is in the fully closed state, and controls the back door drive device to change from the fully open state to the fully closed state when the back door is in the fully open state;
having
The control device includes:
When the back door is in the fully open state, a normal image is created by inverting the acquired image vertically and horizontally, and the back door driving device is controlled based on the recognition result of the state of the user captured in the normal image.
A vehicle back door control system. 7. The vehicle back door control system according to claim 6 ,
Further, the vehicle has a lighting device capable of illuminating a user who is within the predetermined distance behind the back door, regardless of whether the back door is in the fully closed state or the fully open state.
A vehicle back door control system.