JP7184533B2 - Notification device - Google Patents

Description

The present disclosure relates to a vehicle notification device .

2. Description of the Related Art A technology is known in which a vehicle is equipped with a camera, and a warning is issued to a driver when a moving object such as a pedestrian is detected from an image captured by the camera.

Patent Literature 1 discloses the following. That is, pedestrians existing on the sidewalk in front of the vehicle are detected based on the image input from the camera, and danger zones are set for each degree of danger based on the pedestrian status of the pedestrian and the running status of the vehicle. Then, depending on which of the danger zones the pedestrian is in, the degree of danger is determined and an alarm is issued.

Patent Document 2 discloses the following. That is, the approaching object detection start condition is that the vehicle is running or stopped so that it can be started. Then, when the approaching object detection start condition is satisfied, detection of an approaching object approaching the vehicle is started, and when the approaching object detection start condition is not satisfied, detection of the approaching object is not started.

JP 2009-098956 A JP-A-2005-316746

However, in a vehicle such as a bus where passengers are present, consideration has not been sufficiently given to the notification issued to the driver in order to enhance the safety of the passengers.

An object of the present disclosure is to provide a notification device that further enhances the safety of passengers getting on and off a vehicle.

A notification device according to one aspect includes a video reception unit that receives input of a video in which an exterior of a vehicle having a door is captured, and a first detection in the vicinity of the door in the video according to the state of the door. A detection area setting unit for setting an area, a moving object detection unit for detecting a moving object in the first detection area, and a notification unit for notifying when the moving object is detected in the first detection area. and wherein the detection area setting unit sets the first detection area during a period from when the operation of closing the door is started until the door is closed.

In addition, these general or specific aspects may be realized by devices, methods, integrated circuits, computer programs, or recording media, and any of the systems, devices, methods, integrated circuits, computer programs, and recording media may be implemented. may be implemented in any combination.

According to the present disclosure, it is possible to further enhance the safety of passengers getting on and off the vehicle.

It is a figure which shows the example of the articulated bus concerning one embodiment. It is a figure which shows the example of the installation position of the camera in the articulated bus|bath which concerns on one embodiment. It is a block diagram which shows the structural example of the alerting|reporting apparatus which concerns on one Embodiment. FIG. 10 is a diagram showing an example of a scene determination table according to one embodiment; FIG. FIG. 5 is a diagram showing an example of detection areas in a scene of a stop at a bus stop according to one embodiment; FIG. 7 is a diagram showing an example of a detection area in a scene during boarding and alighting according to one embodiment; FIG. 7 is a diagram showing an example of a detection area in a scene in which the front door is in the middle of being closed according to one embodiment; FIG. 7 is a diagram showing an example of a detection area in a scene in which the middle door is in the middle of being closed according to one embodiment; FIG. 7 is a diagram showing an example of a detection area in a scene in which the rear door is in the middle of being closed according to one embodiment; FIG. 4 is a diagram showing an example of detection areas in a scene of starting from a bus stop according to one embodiment; FIG. 5 is a diagram showing an example of a notification image on a monitor according to one embodiment; FIG. 10 is a diagram showing a modification of the notification image on the monitor according to one embodiment;

Embodiments of the present disclosure will be described below with reference to the drawings.

Note that when similar elements are described separately, reference numerals such as "camera 10A" and "camera 10B" are used, and when similar elements are described without distinction, "camera 10" is used. In some cases only common numbers of reference signs are used.

In addition, in the following embodiments, the constituent elements (including element steps and the like) are not necessarily essential unless otherwise specified or clearly considered essential in principle.

(one embodiment)
<Overview>
FIG. 1 is a diagram showing an example of an articulated bus. FIG. 2 is a diagram showing an example of installation positions of cameras on an articulated bus. An articulated bus will be described below as an example, but the present embodiment can also be applied to a single bus.

Since the articulated bus 1 has many blind spots from the driver, it is equipped with a plurality of cameras 10 for photographing the surroundings S of the articulated bus 1. - 特許庁For example, as shown in FIG. 2, the articulated bus 1 has a front camera 10A at the front of the leading vehicle 2 that captures the front of the leading vehicle 2, and at the rear of the rear vehicle 3 that captures the rear of the rear vehicle 3. A rear camera 10B is provided. The articulated bus 1 also has a left front side camera 10C on the left side of the leading vehicle 2 that captures the left side of the leading vehicle 2, and a left rear side camera 10C that captures the left side of the rear vehicle 3 on the left side of the rear vehicle 3. A camera 10D is provided. The articulated bus 1 also has a right front side camera 10E on the right side of the leading vehicle 2 that captures the right side of the leading vehicle 2, and a right rear side camera 10E that captures the right side of the rear vehicle 3 on the right side of the rear vehicle 3. A camera 10F is provided.

Images captured by each camera 10 (hereinafter referred to as “camera images 120”) are displayed on a monitor 11 (see FIG. 3) installed near the driver. The driver grasps the situation of the surroundings S of the articulated bus 1 by viewing the camera image 120 displayed on the monitor 11 .

By the way, as shown in FIG. 1, there may be passengers (hereinafter referred to as "last-minute passengers") rushing to board the bus that is stopped at the bus stop just before the door 4 for getting on and off is closed. . The driver confirms whether or not there is a last-minute passenger from the camera image 120, but in order to further improve safety, it is necessary for the driver to be able to more reliably recognize the last-minute passenger. Therefore, in the present embodiment, a notification device and notification method for notifying the driver when a rushing passenger is detected will be described.

<Configuration of notification device>
Next, the configuration of the notification device 20 will be described with reference to FIG.

Notification device 20 includes video reception unit 101 , scene determination unit 102 , detection area setting unit 103 , moving object detection unit 104 , and notification unit 105 . The scene determination unit 102, the detection area setting unit 103, the moving object detection unit 104, and the notification unit 105 are realized by the processor 21, which is an integrated circuit, LSI (Large Scale Integration), FPGA (Field Programmable Gate Array), or the like. or be implemented by a computer program running on an integrated circuit. Note that the image reception unit 101 may be implemented by a communication I/F (Interface) 22 that connects the camera 10 and the notification device 20 .

The image reception unit 101 receives a camera image 120 (for example, a moving image frame image) transmitted in real time from the camera 10 and temporarily stores it in the memory 23 . Note that the memory 23 may be a volatile semiconductor memory or a non-volatile semiconductor memory.

The scene determination unit 102 determines the scene of the articulated bus 1 from the information 30 regarding the state of the articulated bus 1 (hereinafter referred to as “bus state information”), and generates scene information including the determination result. Scene determination section 102 outputs the generated scene information to detection area setting section 103 . Details of the bus state information 30 and scene determination will be described later.

The detection area setting unit 103 sets the detection area of the moving object in the camera image 120 based on the scene information input from the scene determination unit 102, and combines the scene information and the area information indicating the set detection area. Generate detection configuration information that contains: Detection area setting section 103 outputs the generated detection setting information to moving body detection section 104 . The position, size and shape of the detection area, and the camera image 120 for which the detection area is set may differ according to the scene information. Details of the detection area will be described later.

The moving object detection unit 104 sets a detection area in the camera image 120 based on the detection setting information input from the detection area setting unit 103, and detects a moving object in the set detection area. The moving body detection unit 104 may detect a moving body using an existing technology such as an optical flow method. When a moving object is detected in the detection area, the moving object detection unit 104 generates detection result information including scene information and area information indicating the detection area where the moving object is detected, and outputs the detection result information to the notification unit 105 .

Further, the moving object detection unit 104 may change the detection accuracy of moving objects according to the scene information and area information in the detection setting information, or may detect only moving objects moving in a certain direction. For example, when the detection setting information includes predetermined scene information and area information, the moving body detection unit 104 that employs the optical flow method can detect only moving bodies having vector components in a predetermined direction. good. Details of the moving object detection unit 104 will be described later.

The notification unit 105 notifies the driver when the detection result information is input from the moving body detection unit 104 . For example, the notification unit 105 outputs a signal of an image indicating notification (hereinafter referred to as "notification screen") to the monitor 11, or outputs a signal of sound indicating notification (hereinafter referred to as "notification sound") to the speaker 12. do. Details of the notification unit 105 will be described later.

<Scene judgment>
Next, details of the bus state information and scene determination will be described with reference to the scene determination table 130 shown in FIG.

The bus status information 30 includes vehicle speed information, brake information, left blinker information, right blinker information, door switch information, door status information, and kneeling information.

The vehicle speed information indicates the running speed of the bus. The brake information indicates whether or not the connection bus 1 is being braked (ON/OFF). The left winker information indicates whether or not the left winker operation of the connection bus 1 is being performed (ON/OFF). The right winker information indicates whether or not the right winker operation of the connection bus 1 is being performed (ON/OFF).

The door switch information indicates whether the opening/closing switch of the door 4 operated by the driver is "open" or "closed". In addition, when a plurality of doors 4A, 4B, and 4C exist in the articulated bus 1, door switch information corresponding to each of the doors 4A, 4B, and 4C exists.

The door state information indicates whether the door 4 is actually "open" or "closed". The door state is detected, for example, by a sensor that turns ON when the door 4 is closed and turns OFF when the door 4 is open. In addition, when there are a plurality of doors 4 on the articulated bus 1, door status information corresponding to each door 4 exists.

The kneeling information indicates whether or not the articulated bus 1 is in kneeling operation (that is, the vehicle height of the articulated bus 1 is being raised or lowered).

As shown in the first column of FIG. 4, the scene determination unit 102 determines that the vehicle speed information is "0 km/h", the brake information is "ON", the left blinker information is "ON", the right blinker information is "OFF", and the door When the switch information is "closed" for all the doors 4, the door state information is "closed" for all the doors 4, and the kneeling information is "not operating" (that is, the articulated bus 1 is not rising or lowering). If there is, it is determined to be a scene from when the vehicle stops at the bus stop until the door 4 opens (hereinafter referred to as "the scene where the vehicle stops at the bus stop"). In this case, the scene determination unit 102 generates scene information indicating a scene in which the vehicle stops at the bus stop, and outputs the scene information to the detection area setting unit 103 .

As shown in the second column of FIG. 4, the scene determination unit 102 determines that the vehicle speed information is "0 km/h", the door switch information is "open" for any door 4, and the door state information is "open" for any door 4. When the "open state" and the kneeling information are "not operated", it is determined that the scene is a scene in which the door 4 is opened at the bus stop to allow passengers to get on and off (hereinafter referred to as "the scene during boarding and alighting"). In this case, the scene determination unit 102 generates scene information indicating the scene during boarding and alighting, and outputs the scene information to the detection area setting unit 103 .

As shown in the third column of FIG. 4, the scene determination unit 102 determines that the vehicle speed information is "0 km/h", the door switch information is "closed" for all doors 4, and the door state information is "open" for any door 4. state" and the kneeling information is "unoperated", it is determined to be a scene from the driver's operation to close the door 4 to the actual closing of the door 4 (hereinafter referred to as "scene during door closing"). do. In this case, scene determination section 102 generates scene information indicating a scene in which the door is being closed, and outputs the scene information to detection area setting section 103 . The scene determination unit 102 includes door information indicating which door 4 is being closed in the scene information indicating the scene in which the door is being closed.

As shown in the fourth column of the table in FIG. 4, the scene determination unit 102 determines that the vehicle speed information is "0 km/h", the left blinker information is "OFF", the right blinker information is "ON", and the door switch information is all doors. 4 is "closed", the door state information is "closed" for all doors 4, and the kneeling information is "unoperated", the scene from the closing of the door 4 to the departure of the articulated bus 1 from the bus stop ( hereinafter referred to as "the scene of starting from the bus stop"). In this case, the scene determination unit 102 generates scene information indicating the scene of starting from the bus stop, and outputs the scene information to the detection area setting unit 103 .

In the above description, the reason why the kneeling information is set to "non-operating" is that during the kneeling operation, the camera 10 also moves, which complicates the detection of a moving object by, for example, the optical flow method. is.

<Detection area and notification>
Next, referring to FIGS. 5, 6, 7A, 7B, 7C, and 8, how the detection area is set, the moving object is detected, and the notification is performed in each scene shown in FIG. explain what is done.

<<Scene where the bus stops at the bus stop>>
The detection area setting unit 103 sets the detection area as shown in FIG. 5 when the scene information indicates "a scene of a vehicle stopping at a bus stop". That is, the detection area setting unit 103 sets the left rear detection area 201 covering the left rear of the articulated bus 1 with respect to the camera image 120 of the left front side camera 10C.

When the moving object detection unit 104 detects a moving object in the left rear detection area 201 , the moving object detection unit 104 outputs detection result information to the notification unit 105 . In the left rear detection area 201, the moving body detection unit 104 may detect only moving bodies having vector components in the same direction as the traveling direction of the articulated bus 1. FIG. As a result, the moving object detection unit 104 can detect only moving objects approaching the articulated bus 1 .

In the detection result information input from the moving body detection unit 104, the notification unit 105 notifies the speaker 12 of to output a notification sound signal.

As a result, the notification device 20 can notify the driver that a mobile object (for example, a bicycle) is approaching from the left rear of the articulated bus 1 by notification. When the driver receives the notification, he/she can prevent contact between the passenger and the moving object by suspending the operation of opening the door 4, for example.

<<Scene while getting on and off>>
The detection area setting unit 103 sets the detection area as shown in FIG. 6 when the scene information indicates the "scene of boarding/alighting". That is, the detection area setting unit 103 sets the front detection area 202 covering the front of the leading vehicle 2 with respect to the camera image 120 of the front camera 10A. Further, the detection area setting unit 103 sets a rear detection area 203 covering the rear of the rear vehicle 3 with respect to the camera image 120 of the rear camera 10B. Further, the detection area setting unit 103 sets a leading left detection area 204 covering the left side of the leading vehicle 2 with respect to the camera image 120 of the left front side camera 10C. Further, the detection area setting unit 103 sets a rear left detection area 205 covering the left side of the rear vehicle 3 with respect to the camera image 120 of the left rear side camera 10</b>D of the rear vehicle 3 .

When the moving object detection unit 104 detects a moving object in at least one of the front detection area 202, the rear detection area 203, the front left detection area 204, and the rear left detection area 205, the detection unit 105 notifies the detection unit 105 of the detection. Output result information.

In the detection result information input from the moving body detection unit 104, the notification unit 105, when the scene information indicates the "scene of getting on and off" and the detection area indicates, for example, the "head left detection area 204", As shown, a signal (for example, a signal for changing color) for displaying the notification image 304 corresponding to the leading left detection area 204 is output to the monitor 11 . In FIG. 9 , the notification image 302 corresponds to the front detection area 202 , the notification image 303 to the rear detection area 203 , and the notification image 305 to the rear left detection area 205 .

Thereby, the notification device 20 can notify the driver of the detection area where passengers are present by the notification image. Note that the notification unit 105 does not have to output the notification sound signal to the speaker 12 in the scene of boarding and alighting.

<<Scene in the middle of door closing>>
When the scene information indicates "a scene in which the door is being closed", the detection area setting unit 103 sets the detection areas 202, 203, 204, and 205 described with reference to FIG. A door detection area 220 covering the vicinity of the door 4 in the process of closing is set for the camera image 120 of . Specifically, when the front door 4A is in the process of being closed, the detection area setting unit 103 sets a front door detection area 220A covering the vicinity of the front door 4A, as shown in FIG. 7A.

Similarly, when the middle door 4B is in the middle of closing, the detection area setting unit 103 sets a middle door detection area 220B covering the vicinity of the middle door 4B as shown in FIG. 7B. In this case, as shown in FIG. 7C, a rear door detection area 220C covering the vicinity of the rear door 4C is set. The door detection area 220 is set for all the doors 4 that are in the process of closing. For example, when the front door 4A, middle door 4B, and rear door 4C are all in the process of being closed, front door detection area 220A, middle door detection area 220B, and rear door detection area 220C are set.

As shown in FIGS. 7A, 7B, and 7C, the door detection area 220 is set in a range extending in a direction away from the door 4 with the door 4 at the center. Further, the boundary L1 of the door detection area 220 farthest from the door side of the bus 1 is farthest from the boundary L2 of the front left detection area 204 (or the rear left detection area 205) from the door side of the bus 1. is also set far away. As a result, the moving object detection unit 104 can detect a customer running into the door 4 from a distance.

As shown in FIG. 7A, the moving object detection unit 104 detects a moving object in at least one of a front detection area 202, a rear detection area 203, a front left detection area 204, a rear left detection area 205, and a door detection area 220. is detected, detection result information is output to the notification unit 105 . As shown in FIGS. 7A, 7B, and 7C, when the front door 4A, middle door 4B, and/or rear door 4C is in the middle of closing, the moving object detection unit 104 detects the area near the door 4 that is in the middle of closing. In a predetermined detection area, only a moving object approaching the direction of the closing door may be detected. For example, in the detection areas 221A, 221B, and 221C in the vicinity of the closing door 4 including at least a part of the front left detection area 204 and the rear left detection area 205, the moving object detection unit 104 Only moving objects having a vector component in a direction approaching the center line C of the door 4 may be detected.

In the detection result information input from the moving object detection unit 104, the notification unit 105 notifies the speaker 12 of Outputs a notification sound signal.

As a result, the notification device 20 can notify the driver that there is a passenger running into the door 4 by notification. When the driver hears the notification sound, the driver, for example, opens the door 4 to allow the rushing passenger to board the articulated bus 1, or temporarily suspends the start of the articulated bus 1, thereby avoiding the rushing passenger. Contact with the articulated bus 1 can be prevented.

On the other hand, in the detection result information input from the moving body detection unit 104, the notification unit 105, when the scene information indicates the "scene in the process of closing the door" and the area information indicates, for example, the "head left detection area 204", As shown in FIG. 9, a signal (for example, a signal for changing color) for displaying the notification image 304 corresponding to the leading left detection area 204 is output to the monitor 11 . Similarly, the notification unit 105 outputs a signal to display the notification image 302 for the front detection area 202, the rear notification image 303 for the rear detection area 203, and the notification image 305 for the rear left detection area 205. Output.

If the detection result information indicates that the front door 4A and/or the middle door 4B is in the process of being closed and the area information indicates the "front left detection area 204", the notification unit 105 outputs a notification sound to the speaker 12. signal may be output. Alternatively, if the detection result information indicates that the rear door 4C is in the process of being closed and the area information indicates the "rear left detection area", the notification unit 105 may output a notification sound signal to the speaker 12. .

As a result, the notification device 20 can notify the driver of the presence or absence of passengers by the notification sound and the detection area where the passengers are present by the notification screen.

When the detection area is the "door detection area 220", the notification unit 105 outputs a notification sound signal to the speaker 12, and also outputs a notification sound signal to the monitor 11 as shown in the example of FIG. A signal (for example, a signal that changes color) for displaying door notification images 320A, 320B, and 320C corresponding to detection areas 220A, 220B, and 220C, respectively, may be output. As a result, the notification device 20 can notify the driver of which door 4 there is a last-minute passenger using the door notification images 320A, 320B, and 320C.

<<Scene of departure from the bus stop>>
The detection area setting unit 103 sets the detection area as shown in FIG. 8 when the scene information is "the scene of starting from the bus stop". That is, the detection area setting unit 103 defines a front detection area 202 for the camera image 120 of the front camera 10A, and a right rear detection area 206 covering the right rear of the rear vehicle 3 for the camera image 120 of the rear camera 10B. set.

When the moving object detection unit 104 detects a moving object in at least one of the forward detection area 202 and the right rear detection area 206 , the moving object detection unit 104 outputs detection result information to the notification unit 105 . In the right rear detection area 206, the moving body detection unit 104 may detect only moving bodies having vector components in the same direction as the traveling direction of the articulated bus 1. FIG. Thereby, the moving object detection unit 104 can detect a moving object approaching the articulated bus 1 from the rear right.

In the detection result information input from the moving object detection unit 104, the notification unit 105 detects that the scene information indicates the “departure scene from the bus stop” and the area information indicates the “front detection area 202” or the “right rear detection area 206”. If the brake information is "OFF", a notification signal is output to the speaker 12.

As a result, the notification device 20 indicates that a mobile object (for example, an automobile) is approaching from the right rear of the articulated bus 1 and/or that a mobile object (for example, a passenger getting off) exists in front of the articulated bus 1. This can be notified to the driver by notification. Upon receiving the notification, the driver can temporarily suspend the start of the articulated bus 1 to prevent contact with the moving object.

<Summary of this embodiment>
In the present embodiment, the notification device 20 detects the camera image 120 in which the outside of the articulated bus 1 is captured from the start of the operation to close the door 4 until the door 4 is closed. A door detection area 220 is set in the vicinity of the door 4, and when a moving object is detected in the door detection area 220, a predetermined notification (output of notification sound, for example) is performed. As a result, the driver can easily know the passengers rushing in just before the door 4 of the articulated bus 1 closes.

The embodiment of the present disclosure has been described above.

Note that the relationship between the camera 10, the camera image 120, and the detection area in the present disclosure is merely an example. For example, the detection area setting unit 103 may set a detection area for one camera image (panorama image) generated by combining a plurality of camera images 120 .

It should be noted that the present disclosure is an example of a case where the vehicle drives on the left side. can also be applied in the case of right-hand traffic. In addition, the present disclosure is an example in which the vehicle is a bus, and the embodiments of the present disclosure can also be applied to other vehicles (for example, taxis, courtesy cars, etc.) provided with doors for passengers to get on and off.

The notification device of the present disclosure can be used in vehicles.

1 articulated bus 2 leading vehicle 3 rear vehicle 4, 4A, 4B, 4C door 10, 10A, 10B, 10C, 10D, 10E, 10F camera 11 monitor 12 speaker 20 notification device 101 image reception unit 102 scene determination unit 103 detection area Setting unit 104 Moving object detection unit 105 Notification unit 130 Scene determination table 201 Left rear detection area 202 Front detection area 203 Rear detection area 204 Head left detection area 205 Rear left detection area 206 Right rear detection area 220, 220A, 220B, 220C Door detection area

Claims (5)

a video reception unit that receives an input of a video in which the outside of a vehicle having a door is captured;
a detection area setting unit that sets a first detection area in the vicinity of the door in the image according to the state of the door;
a moving body detection unit that detects a moving body in the first detection area;
a notification unit that performs notification when the moving body is detected in the first detection area;
with
The detection area setting unit sets the first detection area during a period from when the operation of closing the door is started until the door is closed.
Notification device. The detection area setting unit further sets a second detection area, and sets a farthest boundary of the first detection area from the door-side vehicle side surface of the second detection area from the door-side vehicle side surface. Set farther than the farthest boundary,
The moving body detection unit detects a moving body in the second detection area,
The notification unit notifies when the moving body is detected in the second detection area,
The notification device according to claim 1. Notification performed when the moving object is detected in the first detection area includes notification by sound,
Notification when the moving object is detected in the second detection area includes notification by an image,
The notification device according to claim 2. The moving object detection unit detects only a moving object moving in a predetermined direction in the second detection area,
The notification device according to claim 2 or 3. The notification includes notification by sound,
The notification device according to claim 1.

Images