JP7579428B2 - Monitoring device, saddle-type vehicle, information terminal, monitoring method, and program - Google Patents

Description

The present invention relates to monitoring technology for monitoring abnormalities in a driver and/or a saddle-type vehicle.

Patent Document 1 proposes a technology that analyzes the circumstances of an accident based on video and audio before and after the accident transmitted from a drive recorder installed in the vehicle, and transmits the analysis results to an emergency rescue center.

JP 2011-123814 A

In the technology described in Patent Document 1, to analyze the accident situation, advanced (complex) processing is required for the video and audio captured by the drive recorder. However, for example, in a saddle-type vehicle, from the viewpoint of suppressing increases in vehicle costs, it is desirable to accurately detect abnormalities of the driver and/or vehicle (such as a fall) using a simple configuration and processing.

Therefore, the present invention aims to provide an advantageous technology for accurately detecting abnormalities in a driver and/or vehicle with a simple configuration and processing.

In order to achieve the above object, a monitoring device according to one aspect of the present invention is mounted on a saddle-riding vehicle and monitors the driver of the saddle-riding vehicle and/or abnormalities in the saddle-riding vehicle, and includes an acquisition means for acquiring situation information indicating the status of the driver and the saddle-riding vehicle, and a determination means for determining that the abnormality has occurred when a predetermined situation is detected based on the situation information acquired by the acquisition means, and the situation information includes a first image acquired by a first image capturing unit capable of capturing an image of the driver seated in the seat of the saddle-riding vehicle and provided in the saddle-riding vehicle, and an image of the surroundings of the saddle-riding vehicle. and a second image obtained by a second photographing unit provided in the saddle-riding vehicle and capable of photographing the surrounding area, the predetermined situation includes a first situation in which the driver is not seated in the seat and a second situation in which the driver is lying on the road around the saddle-riding vehicle, and when the determination means detects the first situation based on the first image obtained by the acquisition means and detects the second situation based on the second image obtained by the acquisition means, it determines that the abnormality is that the saddle-riding vehicle has fallen over while the driver is driving the saddle-riding vehicle.

The present invention provides, for example, an advantageous technology for accurately detecting abnormalities in a driver and/or vehicle with a simple configuration and processing.

Other features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings, in which the same or similar components are designated by the same reference numerals.

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
A diagram showing an example of an information terminal mounted on a saddle-type vehicle. FIG. 1 is a block diagram showing an example of the configuration of a system including a monitoring device. FIG. 1 shows an example of the arrangement of a photographing unit (camera) in a vehicle. A flowchart showing a monitoring process for monitoring an abnormality while a vehicle is being driven. FIG. 1 is a diagram showing an example of an image obtained by an imaging unit; FIG. 1 is a diagram showing an example of an image obtained by an imaging unit;

Below, an embodiment of the present invention will be described with reference to the drawings. The present invention is not limited to the following embodiment, and includes changes and modifications of the configuration within the scope of the spirit of the present invention. Furthermore, not all of the combinations of features described in this embodiment are necessarily essential to the present invention. Note that identical components are given the same reference numbers and their description will be omitted.

First Embodiment
An example of application of the monitoring device according to the present invention to a saddle-type vehicle will be described below. The monitoring device according to the present invention is a device for monitoring an abnormality (e.g., a fall) of a driver and/or a vehicle and detecting the abnormality. In the following embodiment, a motorcycle will be described as an example of a saddle-type vehicle, but the monitoring device according to the present invention can also be applied to other types of saddle-type vehicles such as three-wheeled vehicles and vehicles other than saddle-type vehicles such as four-wheeled vehicles. The monitoring device may be configured as an in-vehicle device provided in a vehicle, but in this embodiment, it can be configured as an information terminal (mobile terminal). An information terminal is, for example, a device having various functions such as a communication function and a calling function, such as a smartphone or tablet terminal that has become widely used in recent years, and can be configured to be detachable from a saddle-type vehicle V by a dedicated holding mechanism HM, as shown in FIG. 1. FIG. 1 shows an example in which an information terminal 20 as a mobile terminal is mounted on a saddle-type vehicle V using a dedicated holding mechanism HM.

Figure 2 is a block diagram showing an example configuration of a system 100 including a monitoring device of this embodiment. The system 100 of this embodiment may include an in-vehicle device 10 mounted on a vehicle V, and an information terminal 20 owned by a driver (user) of the vehicle V. In this embodiment, the information terminal 20 may be understood as a monitoring device that monitors abnormalities in the driver and/or the vehicle V. The in-vehicle device 10 and the information terminal 20 may be connected to each other so that they can communicate with each other using a communication method such as Bluetooth (registered trademark) or Wi-Fi (registered trademark), but may also be connected to each other so that they can communicate with each other via the Internet.

First, a configuration example of the in-vehicle device 10 mounted on the vehicle V will be described. The in-vehicle device 10 may include, for example, a processing unit 11, a memory unit 12, an image capture unit 13, a status detection sensor 14, and a communication unit 15, which are communicatively connected to each other via a system bus 16. The processing unit 11 is, for example, an ECU (Electronic Control Unit), and may include a processor represented by a CPU, a storage device such as a semiconductor memory, an interface with an external device, etc. The storage unit 12 stores programs executed by the processor and data used by the processor for processing, etc., and the processing unit 11 can read out and execute the programs stored in the storage unit 12 into a storage device such as a memory.

The photographing unit 13 includes, for example, a camera, and is provided on the vehicle V to photograph a part of the vehicle V (the driver) and/or the surroundings of the vehicle V to obtain an image (video). For example, the photographing unit 13 may photograph to repeatedly (for example, periodically) obtain still images, or may photograph to obtain a video. The photographing unit 13 of this embodiment may include multiple cameras 13a to 13f arranged on the vehicle V as shown in FIG. 3. FIG. 3 is a view of the vehicle V as a saddle-type vehicle from above, and shows an example of the arrangement of the photographing unit 13 (cameras 13a to 13f) on the vehicle V. In the figure, the fan-shaped dashed lines indicate the photographing direction (photographing range) of each camera, with "FR" indicating the front of the vehicle V and "RR" indicating the rear of the vehicle.

In the example shown in FIG. 3, cameras 13a to 13b are cameras (first imaging units) arranged on vehicle V so as to be able to capture an image of the driver driving vehicle V. Camera 13a is arranged on vehicle V so as to capture an image of the rear of vehicle V, and captures the front of the driver seated on seat 32. Camera 13b is arranged on vehicle V so as to capture an image of the front of vehicle V, and captures the back of the driver seated on seat 32. In the case of this embodiment, cameras 13a to 13b can be arranged so as to be able to capture the handlebars 31 (left and right grips), seat 32 (driver's seat), and ignition key 33. Cameras 13c to 13f are cameras (second imaging units) arranged on vehicle V so as to be able to capture the surroundings of vehicle V. Camera 13c can capture the front of vehicle V, camera 13d can capture the rear of vehicle V, camera 13e can capture the left side of vehicle V, and camera 13f can capture the right side of vehicle V.

The status detection sensor 14 is a sensor that detects various states of the vehicle V. For example, the status detection sensor 14 may include a position sensor (GPS sensor) that detects the current position of the vehicle V, a speed sensor that detects the speed of the vehicle V, an acceleration sensor that detects the acceleration of the vehicle V, and an attitude sensor (tilt sensor) that detects the attitude (tilt) of the vehicle V. The status detection sensor 14 may also include a sensor that detects the ignition state (on or off) of the vehicle V, a sensor that detects the position of a side stand to allow the vehicle V to stand on its own when stopped, and the like.

The communication unit 15 communicates with the information terminal 20 using a communication method such as Bluetooth (registered trademark) or Wi-Fi (registered trademark). Specifically, the communication unit 15 can have a function as a transmitter that transmits information to the information terminal 20 and a function as a receiver that receives information from the information terminal 20.

Next, the configuration of the information terminal 20 configured as a monitoring device will be described. As described above, the information terminal 20 is a device (apparatus) that can transmit and receive information to and from the in-vehicle device 10 of the vehicle V, such as a smartphone or tablet terminal, and may include a processing unit 21, a memory unit 22, an alarm unit 23, an input/output unit 24, and a communication unit 25 that are communicatively connected to each other via a system bus 26.

The processing unit 21 includes a processor such as a CPU, a storage device such as a semiconductor memory, an interface with an external device, etc. The storage unit 22 stores programs executed by the processor and data used by the processor for processing, etc., and the processing unit 21 can read the programs stored in the storage unit 22 into a storage device such as a memory and execute them. In the case of this embodiment, the storage unit 22 stores an application program (hereinafter, sometimes referred to as a monitoring program) for monitoring an abnormality (e.g., a rollover) of the driver and/or the vehicle V, and the processing unit 21 can read the monitoring program stored in the storage unit 22 into a storage device such as a memory and execute it. In other words, the processing unit 21 can be a computer that executes the monitoring program.

The processing unit 21 of this embodiment may be provided with an acquisition unit 21a and a judgment unit 21b. The acquisition unit 21a acquires situation information indicating the situation of the driver and the situation of the vehicle V from the photographing unit 13 and/or the state detection sensor 14. The situation information is information indicating the current situation of the driver and the vehicle V, and may include an image (first image) obtained by photographing the driver while driving with the cameras 13a to 13b (first photographing unit), and/or an image (second image) obtained by photographing the surroundings of the vehicle V with the cameras 13c to 13f (second photographing unit). The situation information may also include at least one of the position, speed, acceleration, and attitude of the vehicle V detected by the various sensors of the state detection sensor 14. The situation information may also include information regarding the ignition state of the vehicle V and information regarding the position of the side stand. The judgment unit 21b judges whether or not an abnormality (for example, an abnormality during driving) has occurred in the driver and/or the vehicle V based on the situation information acquired by the acquisition unit 21a. For example, when a predetermined situation is detected based on the situation information acquired by the acquisition unit 21a, the determination unit 21b can determine that the abnormality has occurred. The predetermined situation may be understood as a determination item used to determine (estimate) the occurrence of an abnormality (falling), and the details thereof will be described later.

The notification unit 23 notifies the occurrence of an abnormality. For example, if the information terminal 20 has a speaker, the notification unit 23 can notify the occurrence of an abnormality by outputting sound from the speaker. The notification unit 23 may also communicate with the in-vehicle device 10 of the vehicle V via the communication unit 25 and notify the occurrence of an abnormality by outputting sound from a speaker mounted on the vehicle V or by turning on or blinking the vehicle V's lighting devices (headlights, taillights) or hazard lights. The notification unit 23 may notify the occurrence of an abnormality by contacting an external organization (e.g., a fire department, a police station, an insurance company, etc.) about the occurrence of the abnormality using a telephone function or email function mounted on the information terminal 20.

The input/output unit 24 outputs information to the driver (user) and receives information input from the driver. For example, the input/output unit 24 may be a unit including a display unit (display) and an input unit (buttons), or may be a touch panel display. The communication unit 25 communicates with the in-vehicle device 10 of the vehicle V using a communication method such as Bluetooth (registered trademark) or Wi-Fi (registered trademark). Specifically, the communication unit 25 may have a function as a transmitter that transmits information to the in-vehicle device 10 and a function as a receiver that receives information from the in-vehicle device 10.

[Monitoring process]
Next, a flow of processing (monitoring processing) performed by the processing unit 21 of the information terminal 20 when the monitoring program is executed will be described. FIG. 4 is a flowchart showing the monitoring processing performed by the processing unit 21. As described above, the processing unit 21 of this embodiment acquires situation information obtained by the photographing unit 13 and/or the state detection sensor 14, and judges that an abnormality has occurred when a predetermined situation is detected based on the situation information. The predetermined situation may be configured by a predetermined combination of multiple types of situations detected from the situation information. That is, the processing unit 21 can judge that an abnormality has occurred when a predetermined combination of multiple types of situations is detected from the situation information. Each of the multiple situations may be understood as a judgment item used to judge (estimate) the occurrence of an abnormality (fall). Note that this flowchart starts when a user (e.g., a driver) starts (launches) the monitoring program, and can be repeatedly executed unless the user stops the monitoring program. That is, this flowchart can be newly started from step S11 even after it has ended. During execution of this flowchart, in the in-vehicle device 10 of the vehicle V, the photographing unit 13 photographs the vehicle and the status detection sensor 14 detect the vehicle.

In step S11, the processing unit 21 determines whether the driver is driving the vehicle V. Specifically, the processing unit 21 (acquisition unit 21a) acquires an image (first image) from the cameras 13a to 13b of the photographing unit 13. The processing unit 21 (determination unit 21b) then performs a known image processing technique on the image (first image) acquired from the cameras 13a to 13b, and determines, based on the image, whether the driver is gripping the steering wheel 31 (at least one of the left and right grips) and/or whether the driver is seated in the seat 32 (driver's seat) of the vehicle V. If it is detected that the driver is gripping the steering wheel 31 and/or that the driver is seated in the seat 32 of the vehicle V, it is determined that the driver is driving the vehicle V and step S11 is repeated. On the other hand, if the images from the cameras 13a to 13b detect a situation in which the driver is not gripping the steering wheel 31 and/or the driver is not seated in the seat 32 of the vehicle V, it is determined that the driver is not driving the vehicle V (first situation) and the process proceeds to step S12.

In step S12, the processing unit 21 judges whether or not a driver has been detected around the vehicle. Specifically, the processing unit 21 (acquisition unit 21a) acquires an image (second image) from the cameras 13c to 13f of the photographing unit 13 by the acquisition unit 21a. Then, the processing unit 21 (determination unit 21b) performs a known image processing technique on the image (second image) acquired from the cameras 13c to 13f, and judges whether or not a driver has been detected around the vehicle based on the image. As an example, the processing unit 21 can determine whether or not the driver is captured in the image acquired from the cameras 13c to 13f in this step S12 by extracting and storing characteristic points of the driver, such as the color and pattern of the helmet and clothes, based on the images acquired from the cameras 13a to 13b while driving the vehicle V. If the driver is captured in the image acquired from the cameras 13c to 13f, it is judged that the driver has been detected around the vehicle V and proceeds to step S13. On the other hand, if the driver is not captured in the images acquired from the cameras 13c to 13f, it is determined that the driver has not been detected around the vehicle V, and the process proceeds to step S17.

In step S13, the processing unit 21 judges whether the driver is lying on the road in the image (second image) acquired in step S12. Specifically, the processing unit 21 (judgment unit 21b) judges whether the situation in which the driver is lying on the road (second situation) has been detected based on the image acquired in step S12 by performing a known image processing technique on the image. If the situation in which the driver is lying on the road (second situation) is detected, the process proceeds to step S14, and the processing unit 21 judges that an abnormality has occurred while the driver is driving the vehicle V and the driver has been thrown from the vehicle V. Then, the process proceeds to step S15, and the processing unit 21 notifies the occurrence of the abnormality by the notification unit 23. As described above, the notification mode by the notification unit 23 includes output of sound from the information terminal 20 or the speaker of the vehicle V, lighting or blinking of the lighting device or hazard lamp of the vehicle V, and contacting an external agency of the occurrence of the abnormality by using a telephone function or an email function.

On the other hand, if step S13 detects that the driver is standing or walking on the road, but does not detect that the driver is lying on the road (second situation), the process proceeds to step S16, where the processing unit 21 (judgment unit 21b) judges that the driver is safe and ends the process. In this case, since there is a possibility that the driver himself may report the occurrence of an abnormality, or that no abnormality has occurred, in this embodiment, the process ends without reporting the occurrence of an abnormality.

In step S17, which is reached when it is determined in step S12 that the driver has not been detected around the vehicle V, the processing unit 21 determines whether the ignition of the vehicle V is on (third situation). Specifically, the processing unit 21 (acquisition unit 21a) acquires an image (first image) from the cameras 13a to 13b of the photographing unit 13. As an example, FIG. 5A shows an image 41 acquired by the camera 13b of the photographing unit 13. The image 41 shown in FIG. 5A includes images of the steering wheel 31, the seat 32, and the ignition key 33. The processing unit 21 (determination unit 21b) then performs a known image processing technique on the image acquired from the cameras 13a to 13b (image 41 in FIG. 5A) to detect from the image whether the position (rotation position) of the ignition key 33 is in the on position or the off position, and determines whether the ignition of the vehicle V is on. If it is determined that the ignition of the vehicle V is off, the process proceeds to step S18, where the processing unit 21 (determination unit 21b) determines that the driver has normally dismounted from the vehicle V (i.e., no abnormality has occurred) and ends the process. On the other hand, if it is determined that the ignition of the vehicle V is on (third situation), the process proceeds to step S19.

Here, in step S17 of this embodiment, the processing unit 21 determines whether or not the ignition of the vehicle V is on based on the images acquired from the cameras 13a to 13b, but this is not limited to the above. If an ignition detection sensor that detects the ignition state of the vehicle V is provided in the vehicle V as one of the state detection sensors 14, the processing unit 21 may determine whether or not the ignition of the vehicle V is on based on ignition state information acquired from the ignition detection sensor. For example, if a sensor that detects the position (rotation position) of the ignition key 33 is used as the ignition detection sensor, the processing unit 21 may determine whether or not the ignition of the vehicle V is on based on state information indicating the position of the ignition key 33 acquired from the sensor. In addition, if the vehicle V is an electric vehicle (EV), the processing unit 21 may use a sensor that detects the state (remaining charge or output amount) of the battery installed in the vehicle V as an ignition detection sensor, and determine whether or not the ignition of the vehicle V is on based on status information indicating the state of the battery obtained from the sensor.

In step S19, the processing unit 21 judges whether or not the vehicle V is in a state of overturning (fourth situation). Specifically, the processing unit 21 (acquisition unit 21a) acquires an image (second image) acquired from the cameras 13c to 13f of the photographing unit 13. The processing unit 21 (determination unit 21b) then performs a known image processing technique on the image (second image) acquired from the cameras 13c to 13f, thereby judging whether or not the vehicle V is in a state of overturning based on the image. As an example, Figure 5B shows an image 42 acquired by the camera 13c of the photographing unit 13 when the vehicle V has overturned. The image 42 shown in Figure 5B includes an image of another vehicle OV. The processing unit 21 extracts the feature points of the object (object) in the image 42 as a known image processing technique to identify what structure the object is (another vehicle OV in FIG. 5B), and calculates the amount of inclination (inclination amount) of the object (other vehicle OV) in the image 42 relative to the inclination of the object (other vehicle OV) that can be obtained by the camera 13c under normal circumstances. If the calculated inclination amount is equal to or greater than a threshold, it can be determined that the vehicle V is in a rollover state. If it is determined that the vehicle V is not in a rollover state, the process proceeds to step S20, and the processing unit 21 (determination unit 21b) determines that the driver forgot to turn off the ignition when leaving the vehicle V (i.e., no abnormality has occurred), and ends the process. For example, if the ignition is on for a predetermined time after it is determined that the vehicle V is not in a rollover state, the processing unit 21 can determine that the driver forgot to turn off the ignition when leaving the vehicle V. On the other hand, if it is determined that the vehicle V is in an overturned state (situation 4), the processing proceeds to step S14, and after the processing unit 21 (judgment unit 21b) determines that an abnormality has occurred, the processing unit 21 notifies the occurrence of the abnormality via the notification unit 23 in step S15.

Here, in step S19 of this embodiment, the processing unit 21 determines whether the vehicle V is in a tipped-over state based on the images acquired from the cameras 13c to 13f, but this is not limited to the above. For example, if a stand detection sensor that detects the state (position) of the side stand of the vehicle V is provided on the vehicle V as one of the state detection sensors 14, the processing unit 21 may determine whether the vehicle V is in a tipped-over state based on the state information of the side stand acquired from the stand detection sensor. Also, if a posture sensor (tilt sensor) that detects the posture (tilt) of the vehicle V is provided on the vehicle V as one of the state detection sensors 14, the processing unit 21 may determine whether the vehicle V is in a tipped-over state based on the state information (posture information of the vehicle V) acquired from the posture sensor.

As described above, the monitoring device (information terminal 20) of this embodiment uses images (first images) obtained by cameras 13a-13b (first imaging unit) of the imaging unit 13 capable of imaging the driver driving the vehicle V to monitor (detect, determine) abnormalities in the driver and/or vehicle V. This makes it possible to accurately detect the abnormalities with a simple configuration and processing.

Second Embodiment
In the above embodiment, the processing unit 21 (the determination unit 21b) may determine the type of abnormality based on the situation information acquired by the acquisition unit 21a from the photographing unit 13 and/or the state detection sensor 14, and may change the notification mode by the notification unit 23 according to the type of abnormality. For example, in the flowchart of FIG. 4, the type (mode) of the abnormality may be different between a case where the occurrence of an abnormality is determined by detecting a situation in which the driver is lying on the road in step S13 (hereinafter referred to as the first case) and a case where the occurrence of an abnormality is determined by detecting a situation in which the vehicle V is in a rollover state in step S19 (hereinafter referred to as the second case). Specifically, the abnormality in the first case is that the driver is lying on the road. Therefore, as an example, a notification mode may be taken in which a voice to request an ambulance is output to the surroundings of the vehicle V, or a call is made to the fire department to request an ambulance. On the other hand, the abnormality in the second case is that fuel may be leaking from the vehicle V. Therefore, a notification mode may be adopted in which a voice message is output to those around the vehicle V requesting that the vehicle V leave the vehicle V, or a call is made to the fire department to request a fire engine. In the second case, a voice message may be output to those around the vehicle V requesting that the ignition of the vehicle V be turned off.

Third Embodiment
In the above embodiment, an example has been described in which the information terminal 20 functions as a monitoring device that monitors abnormalities in the driver and/or the vehicle V, but the in-vehicle device 10 may function as a monitoring device. In this case, the functions of the acquisition unit 21a and the determination unit 21b included in the processing unit 21 of the information terminal 20 may be provided in the processing unit 11 of the in-vehicle device 10. In addition, the function of the notification unit 23 of the information terminal 20 may also be provided in the in-vehicle device 10.

<Other embodiments>
Furthermore, a program for realizing one or more functions described in the above embodiment can be supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device can read and execute the program. The present invention can also be realized in such an embodiment.

Summary of the embodiment
1. The monitoring device of the above embodiment is
A monitoring device (e.g., 20) for monitoring an abnormality of a driver and/or a vehicle (e.g., V),
An acquisition means (e.g., 21a) for acquiring situation information indicating the situation of the driver and the vehicle;
a determination means (e.g., 21b) for determining that the abnormality has occurred when a predetermined situation is detected based on the situation information acquired by the acquisition means;
Equipped with
The situation information includes a first image obtained by a first image capturing unit (eg, 13a to 13b) capable of capturing an image of the driver who is driving the vehicle.
According to this embodiment, the occurrence of an abnormality in the driver and/or vehicle can be accurately detected (determined) with a simple configuration and processing.

2. In the above embodiment,
the predetermined situation includes a first situation in which the driver is not driving the vehicle;
The determination means is capable of detecting the first situation based on the first image.
According to this embodiment, by simply using the first image obtained by the first imaging unit capable of imaging the driver, the first situation in which the driver is not driving the vehicle can be accurately detected as a judgment item for determining whether an abnormality has occurred.

3. In the above embodiment,
The first situation includes a situation in which the driver is not gripping a steering wheel (e.g., 31) and/or a situation in which the driver is not seated in a driver's seat (e.g., 32) of the vehicle.
According to this embodiment, by simply detecting the driver's seating status and/or the driver's grip on the steering wheel using the first image, the first situation in which the driver is not driving the vehicle can be accurately detected as a judgment item for determining whether an abnormality has occurred.

4. In the above embodiment,
The situation information includes a second image obtained by a second photographing unit (e.g., 13c to 13f) capable of photographing the surroundings of the vehicle,
The predetermined situation further includes a second situation in which the driver is lying down;
The determination means is capable of detecting the second situation based on the second image, and determines that the abnormality has occurred when the second situation is detected in addition to the first situation.
According to this embodiment, by detecting a combination of multiple types of situations, abnormalities in the driver and/or vehicle can be accurately detected, and the driver's situation can also be detected based on the second image, making it possible to take appropriate measures (processing, notification) according to the driver's situation.

5. In the above embodiment,
The determining means determines that the driver is safe when the driver is not lying down in the second image.
According to this embodiment, it is possible to appropriately take measures (processing, notification) according to the driver's condition detected based on the second image.

6. In the above embodiment,
The predetermined situation further includes a third situation in which an ignition of the vehicle is on;
The determination means is capable of detecting the third situation based on the situation information.
According to this embodiment, the vehicle situation can be detected with high accuracy, and appropriate measures (processing, notification) can be taken according to the situation.

7. In the above embodiment,
The judgment means is capable of detecting the third situation based on information indicating at least one of an image obtained by a camera (e.g., 13a-13b) capable of photographing an ignition key (e.g., 33), the position of the ignition key (e.g., 33), and the state of a battery installed in the vehicle.
According to this embodiment, the third situation in which the ignition of the vehicle is on can be detected with high accuracy.

8. In the above embodiment,
The determining means determines that the driver has exited the vehicle when the ignition is off.
According to this embodiment, it is possible to accurately determine whether or not the driver has appropriately exited the vehicle, based on a combination of a plurality of types of detected situations.

9. In the above embodiment,
The predetermined situation further includes a fourth situation in which the vehicle is in a rollover state,
The determining means determines that the abnormality has occurred when detecting the third situation and the fourth situation in addition to the first situation.
According to this embodiment, the occurrence of an abnormality in the driver and/or the vehicle can be detected with high accuracy based on a combination of multiple types of detected situations.

10. In the above embodiment,
The determining means determines that the driver has forgotten to turn off the ignition if the vehicle is not in an overturned state.
According to this embodiment, it is possible to recognize that the driver has forgotten to turn off the vehicle ignition, based on a combination of a plurality of types of detected situations.

11. In the above embodiment,
a notification means (e.g., 23) for notifying the user of the abnormality when the determination means determines that the abnormality has occurred;
The determining means determines the type of the abnormality based on the situation information acquired by the acquiring means, and changes the manner of notification by the notifying means depending on the type of the abnormality.
According to this embodiment, the occurrence of an abnormality can be appropriately notified depending on the type of abnormality that has occurred in the driver and/or vehicle, making it possible to appropriately deal with (process) the abnormality.

12. In the above embodiment,
The vehicle is a saddle-type vehicle.
According to this embodiment, it is possible to accurately detect the occurrence of an abnormality in a saddle type vehicle.

The invention is not limited to the above-described embodiments, and various modifications and variations are possible within the scope of the invention.

This application claims priority based on Japanese patent application No. 2021-049129, filed on March 23, 2021, the entire contents of which are incorporated herein by reference.

V: vehicle, 10: in-vehicle device, 13: image capture unit, 14: status detection sensor, 20: information terminal (monitoring device), 21: processing unit, 21a: acquisition unit, 21b: judgment unit, 23: notification unit

Claims (13)

A monitoring device that is mounted on a saddle-riding type vehicle and monitors an abnormality of a driver of the saddle-riding type vehicle and/or an abnormality of the saddle-riding type vehicle,
an acquisition unit for acquiring situation information indicating a situation of the driver and the saddle riding type vehicle;
a determination means for determining that the abnormality has occurred when a predetermined situation is detected based on the situation information acquired by the acquisition means;
Equipped with
the situation information includes a first image that is obtained by a first image capturing unit that is capable of capturing an image of the driver seated on a seat of the saddle riding type vehicle and that is provided in the saddle riding type vehicle, and a second image that is obtained by a second image capturing unit that is capable of capturing an image of the surroundings of the saddle riding type vehicle and that is provided in the saddle riding type vehicle,
the predetermined situation includes a first situation in which the driver is not seated in the seat and a second situation in which the driver is lying on a road around the saddle riding type vehicle,
The monitoring device is characterized in that, when the judgment means detects the first situation based on the first image acquired by the acquisition means and detects the second situation based on the second image acquired by the acquisition means, the judgment means judges that the abnormality is that the saddle-type vehicle has rolled over while the driver is driving the saddle-type vehicle. The monitoring device according to claim 1, characterized in that the first situation also includes a situation in which the driver is not gripping the steering wheel. The monitoring device according to claim 1 or 2, characterized in that the judgment means judges that the driver is safe when the first situation is detected based on the first image and the second situation is not detected based on the second image. the predetermined situation further includes a third situation in which an ignition of the saddle riding type vehicle is on, and a fourth situation in which the saddle riding type vehicle is in a toppled state,
The monitoring device according to any one of claims 1 to 3, characterized in that when the judgment means detects the third situation and the fourth situation based on the situation information acquired by the acquisition means instead of the second situation, it judges that a fall of the saddle-type vehicle has occurred while the driver is driving the saddle-type vehicle. The monitoring device according to claim 4, characterized in that the situation information includes information indicating at least one of an image obtained by a camera capable of photographing an ignition key, the position of the ignition key, and the state of a battery installed in the saddle-type vehicle. The monitoring device according to claim 4 or 5, characterized in that, when the ignition is in the off position, the determination means determines that the saddle-type vehicle has not fallen over while the driver is driving the saddle-type vehicle and that the driver has dismounted from the saddle-type vehicle. The monitoring device according to any one of claims 4 to 6, characterized in that the determination means determines that the driver has forgotten to turn off the ignition when it detects the first situation and the third situation but does not detect the fourth situation. a notification means for notifying an abnormality when the determination means determines that the abnormality has occurred,
The monitoring device according to any one of claims 1 to 7, characterized in that the judgment means determines the type of the abnormality based on the situation information acquired by the acquisition means, and changes the notification mode by the notification means depending on the type of the abnormality. The monitoring device according to any one of claims 1 to 8, characterized in that the first and second imaging units are provided on the saddle-type vehicle. A saddle-type vehicle equipped with a monitoring device according to any one of claims 1 to 9. An information terminal communicably connected to a saddle-type vehicle,
10. An information terminal comprising the monitoring device according to claim 1, and configured to acquire the situation information from the saddle-type vehicle. 1. A monitoring method for monitoring an abnormality in a driver of a saddle-type vehicle and/or in the saddle-type vehicle, comprising:
an acquisition step of acquiring situation information indicating a situation of the driver and the saddle riding type vehicle;
a determination step of determining that the abnormality has occurred when a predetermined situation is detected based on the situation information acquired in the acquisition step;
Including,
the situation information includes a first image that is obtained by a first image capturing unit that is capable of capturing an image of the driver seated on a seat of the saddle riding type vehicle and that is provided in the saddle riding type vehicle, and a second image that is obtained by a second image capturing unit that is capable of capturing an image of the surroundings of the saddle riding type vehicle and that is provided in the saddle riding type vehicle,
the predetermined situation includes a first situation in which the driver is not seated in the seat and a second situation in which the driver is lying on a road around the saddle riding type vehicle,
The monitoring method is characterized in that, in the judgment process, the first situation is detected based on the first image acquired in the acquisition process, and if the second situation is detected based on the second image acquired in the acquisition process, it is determined that the abnormality is that the saddle-type vehicle has rolled over while the driver is driving the saddle-type vehicle. A program for causing a computer to execute each step of the monitoring method described in claim 12.

Images