JP7715060B2 - Apparatus, method and program for detecting getting off of vehicle - Google Patents

Description

The present invention relates to technology for detecting the exit behavior of a vehicle occupant.

Monitoring devices that allow drivers to check what is happening inside the vehicle (particularly in the rear seats) have been known for some time. Patent Document 1 describes an invention for a monitoring device that displays an image of the rear seats. In this invention, the image display begins only if there is someone in the rear seat. Furthermore, Patent Document 2 describes an invention for a device that detects the seating position of a passenger. In this invention, it detects whether the passenger is wearing their seat belt in the correct position.

JP 2008-221989 A Japanese Patent Application Laid-Open No. 2008-199515

As mentioned above, systems have been proposed for monitoring passengers and ensuring their safety, but no technology has been known to prevent passengers in the front or rear seats (especially children) from getting out of the vehicle without thoroughly checking their surroundings when the vehicle comes to a stop.

In light of the above background, the present invention aims to provide technology that prevents passengers in the front passenger seat or rear seats from inadvertently exiting a vehicle.

The vehicle exit behavior detection device of the present invention comprises an image acquisition unit that acquires an image including an occupant captured by a camera; a base position determination unit that determines the base position of the occupant's hands based on the occupant's seated posture while the vehicle is in motion; an area determination unit that determines an area for detecting exit behavior based on the door handle position and the base position; an exit behavior detection unit that detects exit behavior when the occupant's hands enter the area or are estimated to enter the area based on the image; and an output unit that outputs information indicating the occupant's exit behavior.

When an occupant exits a vehicle, they need to operate the door handle, so it is possible to detect the occupant's hands from images captured by a camera and detect the exit behavior based on the position of the occupant's hands. However, if the exit behavior is detected when the occupant touches the door handle, it will delay any actions such as issuing a warning based on the exit behavior detection, so it is preferable to detect the exit behavior when the occupant is about to put their hand on the door handle.

In order to detect exit behavior based on the position of a passenger's hand before it reaches the door handle, it is necessary to know how far the passenger's hand moves from a normal posture toward the door handle. However, determining the position of a passenger's hand when in a normal posture is difficult. This is because passengers in the front passenger seat or rear seat do not have the driving task of operating the steering wheel, and therefore have a high degree of freedom in where they place their hands. In addition, the position of the seats differs depending on the vehicle model, and the way passengers sit varies depending on the seat position. This makes it difficult to determine whether a passenger has moved their hand toward the door handle.

In this invention, the base position of the occupant's hands is determined based on the occupant's seated posture while the vehicle is moving. The exit behavior detection device then determines an area for detecting exit behavior based on the base position and the position of the door handle. This makes it possible to appropriately detect whether the occupant is exiting the vehicle based on whether the occupant's hands have entered, or are estimated to enter, that area. The base position is the estimated position where the occupant's hands would normally be placed. The seating posture is a concept that includes not only the posture when seated, but also the seated position.

In the vehicle exit behavior detection device of the present invention, the base position determination unit may determine the base position based on the position of the occupant's waist and the position of the shoulder on the door side. With this configuration, the position when the occupant's hands are placed on their thighs can be determined as the base position.

In the vehicle exit behavior detection device of the present invention, the base position determination unit may determine the base position based on an image captured when the vehicle is traveling at a speed equal to or greater than a predetermined threshold. Here, the threshold may be, for example, the legal speed limit, or may be determined based on the legal speed limit. When the vehicle is traveling smoothly at a constant speed, the seated posture of the occupants is considered to be stable.

The alighting behavior detection device of the present invention includes a stop detection unit that detects when a vehicle is stopped, and the alighting behavior detection unit may detect alighting behavior while the stop detection unit is detecting that the vehicle is stopped. Alighting behavior is typically performed when a vehicle is stopped, so it is desirable to detect alighting behavior while the vehicle is detected to be stopped. By not detecting alighting behavior when the vehicle is not stopped, it is possible to reduce false detections of alighting behavior.

The vehicle alighting behavior detection method of the present invention is a method for detecting an occupant's alighting behavior using an alighting behavior detection device, and includes the steps of: the alighting behavior detection device acquiring an image including the occupant captured by a camera; the alighting behavior detection device determining the base position of the occupant's hands based on the occupant's seated posture while the vehicle is moving; the alighting behavior detection device determining an area for detecting alighting behavior based on the door handle position and the base position; the alighting behavior detection device detecting alighting behavior when the occupant's hands enter the area or are estimated to enter the area based on the image; and the alighting behavior detection device outputting information indicating the occupant's alighting behavior.

The program of the present invention is a program for detecting an occupant's exit behavior, and causes a computer to execute the following steps: acquiring an image including the occupant captured by a camera; determining the base position of the occupant's hands based on the occupant's seated posture while the vehicle is moving; determining an area for detecting exit behavior based on the door handle position and the base position; detecting the occupant's exit behavior when the occupant's hands enter the area based on the image, or when it is estimated that the occupant's hands will enter the area; and outputting information indicating the occupant's exit behavior.

This invention makes it possible to accurately determine whether an occupant is exiting the vehicle.

1 is a diagram illustrating a configuration of an alighting behavior detection device according to an embodiment. FIG. 2 is a diagram showing a detection region R. 10A and 10B are diagrams illustrating an operation of the getting-off behavior detection device according to the embodiment to determine a detection area. 10A and 10B are diagrams illustrating an operation of the alighting behavior detection device according to the embodiment for detecting alighting behavior.

The following describes the exit behavior detection device of this embodiment with reference to the drawings. Note that the following description merely shows one preferred embodiment and is not intended to limit the invention described in the claims.

Figure 1 shows the configuration of an alighting behavior detection device 1 according to an embodiment. The alighting behavior detection device 1 is connected to a camera 20 that captures images of the interior of the vehicle, and detects the alighting behavior of rear seat passengers based on images captured by the camera 20. The camera 20 is installed near the rear monitor and captures images of the rear seats. Note that while this embodiment uses rear seat passengers as an example, the alighting behavior detection device of the present invention can of course also be applied to detecting the alighting behavior of a passenger in the front passenger seat.

The alighting behavior detection device 1 has an image acquisition unit 10, a calculation unit 11, a memory unit 16, and an output unit 17. The image acquisition unit 10 has a function of acquiring image data from a camera 20. The calculation unit 11 has a function of detecting alighting behavior based on camera images. The calculation unit 11 has a base position determination unit 12, an area determination unit 13, alighting behavior detection unit 14, and a stop detection unit 15.

The base position determination unit 12 determines the base position of the hands of a passenger sitting in the rear seat. As described above, the hand position of a passenger who does not have a driving task has a high degree of freedom, so the base position determination unit 12 determines the base position of the hands based on the passenger's sitting posture and sitting position. In this embodiment, the base position is determined based on the position of the passenger's shoulder on the door handle side and their waist. The position of the shoulder on the door handle side is the horizontal coordinate of the base position, and the position of the waist is the vertical coordinate of the base position.

The area determination unit 13 has the function of determining an area (hereinafter referred to as the "detection area") for detecting exit behavior based on the base position and the door handle position. Specifically, the detection area is determined to be an area centered on the door handle and with a radius that is X% of the distance between the door handle position and the base position.

Figure 2 is a diagram showing the detection area R. With the door handle H at its center and a circle C whose radius is the distance L between the position of the door handle H and the base position B, the detection area R is an area with a radius of L x X (%). The position of the door handle H can be initially set based on the shooting direction when the camera 20 is installed. Note that, although the detection area R here is a circular area centered on the door handle, the detection area R does not have to be circular. For example, it could be an elliptical area that includes the straight line connecting the door handle H and the base position B.

The coefficient X, which determines the radius of the detection area R, is less likely to miss disembarking behavior if it is set to a large value, but false detections will also increase. The coefficient X needs to be set appropriately, and a value between 60% and 70%, for example, is desirable. The coefficient X may also be changed depending on the situation when the vehicle is stopped. For example, if the vehicle is stopped on the road, a larger value may be set than if the vehicle is parked in a parking lot at home. This is because there are other vehicles traveling on the road, so greater caution is required. The area determination unit 13 stores data on the determined area in the memory unit 16.

The alighting behavior detection unit 14 has the function of detecting whether an occupant is alighting based on images acquired by the image acquisition unit 10. The alighting behavior detection unit 14 detects an occupant alighting when the occupant's hand, captured in images successively acquired by the camera 20, enters the detection area. The alighting behavior detection unit 14 performs alighting behavior detection process when the vehicle stop detection unit 15 detects that the vehicle has stopped.

The vehicle stop detection unit 15 receives data on the vehicle's traveling speed from the vehicle speed sensor 21 and determines whether the vehicle is stopped. The vehicle stop detection unit 15 detects that the vehicle is stopped when the vehicle speed is 0. In this specification, stopping is a concept that includes parking.

The output unit 17 has the function of outputting information indicating that an exit action has occurred to in-vehicle equipment. In this embodiment, the output unit 17 outputs data indicating that an exit action has occurred to the speaker 22 or the locking device 23.

A variety of applications are possible when an exit behavior is detected. For example, the driver can be notified via speaker 22 that a rear seat passenger is about to exit the vehicle. The rear seat passenger can also be warned to carefully check the outside of the vehicle. As another example, the rear seat door can be locked by locking device 23 to prevent it from opening. The processing performed in response to the detection of an exit behavior can be switched depending on the age of the rear seat passenger. If the passenger is young, the driver can be notified or the door can be locked, and if the passenger is older, a warning can be given to the passenger who is about to exit the vehicle.

Figure 3 is a diagram showing the operation of the alighting behavior detection device 1 of the embodiment to determine the detection area. The alighting behavior detection device 1 acquires an image from the camera 20 (S10). The alighting behavior detection device 1 acquires vehicle speed data from the vehicle speed sensor 21 and determines whether the vehicle speed is equal to or greater than a predetermined threshold (S11). If the vehicle speed is less than the predetermined threshold (NO in S11), the process returns to acquiring an image from the camera 20 (S10).

If the vehicle speed is equal to or greater than a predetermined threshold (YES in S11), the alighting behavior detection device 1 records the images at that time (S12). The alighting behavior detection device 1 determines whether a predetermined amount of images has been accumulated (S13). If the predetermined amount of images (e.g., 5 minutes) has not been accumulated (NO in S13), the above-described processing (S10-S12) is repeated. If the predetermined amount of images has been accumulated (YES in S13), the alighting behavior detection device 1 determines the base position of the occupant's hands based on the accumulated images (S14), and determines a detection area for determining alighting behavior based on the base position and the position of the door handle, and stores this in the memory unit 16 (S15). Note that the detection area can be stored in the memory unit 16 in association with the occupant; it is not necessary to perform the operation shown in FIG. 3 every time.

Figure 4 is a diagram showing the operation of the alighting behavior detection device 1 of the embodiment to detect alighting behavior. The alighting behavior detection device 1 acquires an image from the camera 20 (S20). The alighting behavior detection device 1 acquires vehicle speed data from the vehicle speed sensor 21 and determines whether the vehicle is stopped (S21). If the vehicle is stopped (YES in S21), it determines whether the occupant's hands shown in the image from the camera 20 are within the detection area (S22). If the occupant's hands are within the detection area (YES in S22), the alighting behavior detection device 1 detects that the occupant is performing alighting behavior and outputs data indicating that alighting behavior has occurred (S23). If the occupant's hands are not within the detection area (NO in S22), the process returns to the process of acquiring an image from the camera 20 (S20).

The configuration of the alighting behavior detection device 1 of this embodiment has been described above. An example of the hardware for the alighting behavior detection device 1 is a computer equipped with an ECU, RAM, ROM, a hard disk, a display, a communication interface, etc. The alighting behavior detection device 1 is realized by storing a program having modules that realize each of the above-mentioned functions in RAM or ROM and executing the program with the ECU. Such programs are also within the scope of the present invention.

The exit behavior detection device 1 of this embodiment determines the base position of the hands of rear seat passengers while the vehicle is moving, and determines a detection area for detecting exit behavior based on the base position and the position of the door handle. This makes it possible to appropriately detect exit behavior of rear seat passengers at an early stage. Furthermore, according to this embodiment, exit behavior can be detected with little learning data, even when considering machine learning, regardless of the vehicle model or seat position.

In the above embodiment, an example of the configuration of the alighting behavior detection device was described, but the alighting behavior detection device of the present invention can be modified in various ways. Modifications are described below.

(Timing of detecting disembarking behavior)
In the above-described embodiment, detection of disembarking behavior is performed when the vehicle is stopped, but detection of disembarking behavior may be performed not only when the vehicle is completely stopped but also when the vehicle speed reaches a predetermined threshold (for example, 4 km/h or less). When traveling at a low speed, it is possible to disembark if one wishes to do so, but it is desirable to prevent such unreasonable disembarking.

In addition, if the rear seat door handle is locked by the child lock to prevent it from being opened, exiting the vehicle may not be detected.

(Target of getting off behavior detection)
It is also possible to configure the system so that if the passenger in the back seat is an adult, the system does not detect the passenger getting out of the vehicle. The determination of whether the passenger is an adult can be made by machine learning or by determining the height of the passenger's head.

(Detection of getting off the vehicle)
In the above embodiment, an example has been described in which an exit behavior is detected when the occupant's hand enters the detection area, but an exit behavior may be detected only when the occupant's hand remains in the detection area for a predetermined number of frames (e.g., three frames), thereby reducing false detection of an exit behavior.

In addition, even if the occupant's hands are not within the detection area, the exit behavior detection device may detect an exit behavior when it is estimated from the time-series images from the camera 20 that the occupant is moving toward the detection area and entering the detection area. Machine learning such as a Kalman filter or dynamic Bayesian may also be used to analyze the time-series images.

(Determining the door handle position)
In the above embodiment, the door handle position is initially set, but the door handle position may be determined based on a camera image. For example, the position of the occupant's hand when the door is opened may be set as the door handle position.

(Determining the base position)
In the above embodiment, an example was given in which the base position of the occupant's hands was determined based on the positions of the occupant's shoulders and waist, but the base position may be determined by other methods. For example, the position of the occupant's eyes or ears on the door handle side may be used instead of the shoulder position. Also, the position of the seat may be used instead of the waist position.

1 Exit behavior detection device 10 Image acquisition unit 11 Calculation unit 12 Base position determination unit 13 Area determination unit 14 Exit behavior detection unit 15 Stop detection unit 16 Storage unit 17 Output unit 20 Camera 21 Vehicle speed sensor 22 Speaker 23 Locking device B Base position H Handle R Detection area

Claims (6)

an image acquisition unit that acquires an image including an occupant captured by a camera;
a base position determination unit that determines a base position of the occupant's hands based on the seating posture of the occupant while the vehicle is running;
an area determination unit that determines an area for detecting an exit behavior based on a door handle position and a base position;
an alighting behavior detection unit that detects an alighting behavior when a hand of an occupant enters the area or when it is estimated that the hand of an occupant will enter the area based on the image;
an output unit that outputs information indicating the disembarking behavior of the occupant;
A disembarking behavior detection device comprising: The exit behavior detection device according to claim 1, wherein the base position determination unit determines the base position based on the position of the occupant's waist and the position of the shoulder on the door side. The vehicle getting-off behavior detection device according to claim 1 or 2, wherein the base position determination unit determines the base position based on an image captured while the vehicle is traveling at a speed equal to or greater than a predetermined threshold. A vehicle stop detection unit is provided to detect when the vehicle is stopped,
The vehicle getting-off behavior detection device according to claim 1 , wherein the vehicle getting-off behavior detection unit detects a vehicle getting-off behavior while the vehicle stop detection unit detects a vehicle stop. A method for detecting an occupant's getting-off behavior by an getting-off behavior detection device, comprising:
a step in which the disembarking behavior detection device acquires an image including an occupant taken by a camera;
a step in which the vehicle exit behavior detection device determines a base position of the occupant's hands based on the seating posture of the occupant while the vehicle is moving;
a step in which the vehicle getting-out behavior detection device determines an area for detecting a vehicle getting-out behavior based on a door handle position and a base position;
a step in which the vehicle exit behavior detection device detects a vehicle exit behavior when a hand of an occupant enters the area or when it is estimated that the hand of an occupant will enter the area based on the image;
a step in which the alighting behavior detection device outputs information indicating the alighting behavior of an occupant;
The method for detecting alighting behavior comprises: A program for detecting an occupant's exit behavior, the program comprising:
acquiring an image including an occupant captured by a camera;
determining a base position of the occupant's hands based on the seating posture of the occupant while the vehicle is moving;
determining a region for detecting an exit behavior based on a door handle position and a base position;
detecting an exit behavior when a hand of an occupant enters the area or when it is estimated that the hand of an occupant will enter the area based on the image;
outputting information indicating the disembarking behavior of the occupant;
A program that executes the following.