JP7742975B2 - Smoking determination device and smoking determination method - Google Patents

Description

This disclosure relates to a smoking determination device and a smoking determination method.

Technology has been known in the past to determine whether a person is smoking based on an image of their face (for example, Patent Document 1).

International Publication No. 2013/157466

When determining whether a person is smoking from a captured image, for example, there is a possibility that a behavior similar to a smoking behavior may be mistakenly detected as a smoking behavior, or that an object resembling a cigarette may be mistakenly detected as a cigarette.
Therefore, the conventional technology has a problem in that it may determine that a non-smoker who clearly does not smoke is a smoker.

This disclosure has been made to solve the above-mentioned problems and aims to provide a smoking detection device that prevents non-smokers from being mistakenly determined to be smoking.

The smoking determination device of the present disclosure comprises a smoker determination unit that determines whether a subject, whose smoking status is to be determined, is a smoker based on smoker determination information for determining whether the subject is a smoker; and a smoking determination unit that, if the smoker determination unit determines that the subject is a smoker, makes a smoking determination as to whether the subject is a smoker based on an image in which at least the face of the subject is captured, and does not make a smoking determination if the smoker determination unit determines that the subject is a non-smoker.

This disclosure makes it possible to prevent non-smokers from being mistakenly identified as smoking.

FIG. 1 is a diagram showing an example of the configuration of a smoking determination system including a smoking determination device according to a first embodiment; 1 is a diagram showing an example of the configuration of a smoking determination system in a case where the information acquisition device for smoker determination is a camera in embodiment 1. FIG. 4 is a flowchart for explaining the operation of the smoking determination device according to the first embodiment. 10 is a flowchart for explaining the operation of a smoker determination unit in the smoking determination device in the first embodiment when the smoker determination information is a captured image. 1 is a diagram showing an example of the configuration of a smoking determination system in a case where a smoker determination information acquisition device is a smoker setting device in embodiment 1. FIG. 10 is a flowchart for explaining the operation of a smoker determination section in the smoking determination device in the case where the smoker determination information is smoker setting information in the first embodiment. 1 is a diagram showing an example of the configuration of a smoking determination system in a case where a smoker determination information acquisition device is a personal authentication information acquisition device in embodiment 1. FIG. 10 is a flowchart for explaining the operation of a smoker determination unit in the smoking determination device in the case where the smoker determination information is personal authentication information in the first embodiment. 1 is a diagram showing an example of the configuration of a smoking determination system in a case where a smoker determination information acquisition device is a nicotine dependence determination device in embodiment 1. FIG. 10 is a flowchart for explaining the operation of a smoker determination section in the smoking determination device in the case where the smoker determination information is nicotine dependence information in the first embodiment. 1 is a diagram showing an example of the configuration of a smoking determination system in a case where the information acquisition device for smoker determination is a temperature measurement device in embodiment 1. FIG. 10 is a flowchart for explaining the operation of a smoker determination unit in the smoking determination device in the first embodiment when the smoker determination information is heat source information. 1 is a diagram showing an example of the configuration of a smoking determination system in a case where the information acquisition device for smoker determination is a breathing detection device in embodiment 1. FIG. 10 is a flowchart for explaining the operation of a smoker determination unit in the smoking determination device in the first embodiment when the smoker determination information is breathing information. FIG. 10 is a diagram showing an example of the configuration of a smoking determination system in embodiment 1 that has a smoking determination device that has the function of determining whether or not a driver is wearing a mask and is configured not to perform a smoking determination if it is determined that the driver is wearing a mask. 1 is a flowchart for explaining the operation of a smoking determination device in embodiment 1 that has a function of determining whether a driver is wearing a mask, and is configured not to make a smoking determination if it is determined that the driver is wearing a mask. 17A and 17B are diagrams illustrating an example of a hardware configuration of the smoking determination device according to embodiment 1. FIG.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.
Embodiment 1.
FIG. 1 is a diagram showing an example of the configuration of a smoking determination system 100 including a smoking determination device 1 according to the first embodiment.
The smoking determination device 1 according to the first embodiment determines whether a person (hereinafter referred to as the "subject") to be determined as to whether or not he or she is currently smoking is currently smoking.
In the following first embodiment, as an example, the smoking determination device 1 is mounted on a vehicle (not shown) and determines whether the driver of the vehicle is smoking. That is, in the following first embodiment, the subject is the driver of the vehicle.
In the first embodiment, the "smoking state" refers to a state in which a cigarette is held in the mouth.

The smoking determination device 1 is connected to a camera 2 , a smoker determination information acquisition device 3 , and an alarm device 4 .
The smoking determination device 1, the camera 2, the smoker determination information acquisition device 3, and the alarm device 4 constitute a smoking determination system 100.

The camera 2 is mounted on a vehicle. For example, the smoking determination device 1 and the camera 2 may constitute a so-called DMS (Driver Monitoring System).
The camera 2 is installed, for example, in the center of the vehicle's instrument panel, a meter panel, a steering column, an A-pillar, or near the center console, and is installed so as to be able to capture an image of at least the face of the occupant.
The camera 2 is a visible light camera or an infrared camera.
1, only one camera 2 is installed in the vehicle, but this is merely an example. A plurality of cameras 2 may be installed in the vehicle.
The camera 2 outputs the captured image to the smoking determination device 1 .

The information acquisition device 3 for determining whether a driver is a smoker is provided in the vehicle or outside the vehicle, and acquires information (hereinafter referred to as "information for determining whether a driver is a smoker") that the smoking determination device 1 uses to determine whether the driver is a smoker.
The details of the smoker determination information acquisition device 3 will be described later with specific examples.

The warning device 4 is, for example, an audio output device installed in a vehicle, and outputs, for example, an alarm sound or a warning voice.
The warning device 4 is not limited to an audio output device, but may be, for example, a display device that displays a warning message or a warning icon, or a vibration generating device that generates vibrations.
The alarm device 4 is provided in the vehicle or outside the vehicle. For example, the alarm device 4 is installed in the center of the vehicle's instrument panel, meter panel, steering column, A-pillar, or around the center console. The alarm device 4 may be mounted on a navigation device (not shown), for example. Furthermore, the alarm device 4 may be provided on a PC (personal computer, not shown) provided in a management center (not shown), or on a mobile terminal (not shown) carried by a monitor.
1, only one alarm device 4 is connected to the smoking determination device 1, but this is merely an example. Multiple alarm devices 4 may be connected to the smoking determination device 1.

The smoking determination device 1 includes a smoker determination unit 11 , a smoking determination unit 12 , and an alarm control unit 13 .
The smoker determination unit 11 acquires smoker determination information from the smoker determination information acquisition device 3, and determines whether the driver is a smoker based on the acquired smoker determination information.
When the smoker determination unit 11 determines that the driver is a smoker, the smoking determination unit 12 determines whether or not the driver is smoking (hereinafter referred to as "smoking determination") based on the captured image output from the camera 2. When the smoker determination unit 11 determines that the driver is not a smoker, that is, the driver is a non-smoker, the smoking determination unit 12 does not perform the smoking determination.
When the smoking determination unit 12 determines that the driver is smoking as a result of making the smoking determination, the warning control unit 13 outputs information for outputting a warning (hereinafter referred to as "warning output control information") to the warning device 4. In accordance with the warning output control information, the warning device 4 outputs, for example, a warning sound or warning voice, displays a warning message or a warning icon, or generates a vibration.

<Information for determining whether a smoker is a captured image>
The smoker determination information acquisition device 3 may be any device that can acquire smoker determination information.
For example, the smoker determination information acquisition device 3 is a camera. In this case, the smoker determination information is an image captured by the camera.
Hereinafter, a detailed description will be given of an example of the configuration of the smoking determination device 1 according to the first embodiment, assuming that the smoker determination information acquisition device 3 is a camera and the smoker determination information is a captured image.
FIG. 2 is a diagram showing an example of the configuration of the smoking determination system 100 in the first embodiment when the smoker determination information acquisition device 3 is a camera 3a.
The camera 3a is installed so as to be able to capture at least the face of the occupant and captures an image of at least the face of the occupant. The camera 3a is a visible light camera or an infrared camera. Note that, for convenience, the camera 2 and the camera 3a are illustrated separately in FIG. 2, but the camera 2 and the camera 3a may be the same camera.

In this case, in the smoking determination device 1, the smoker determination unit 11 acquires a captured image from the camera 3a as smoker determination information.
The smoker determination unit 11 then determines whether or not the driver is a smoker based on the captured image acquired from the camera 3a.
In detail, the smoker determination unit 11 acquires time-series captured images output from the camera 3a during a preset period (hereinafter referred to as the "smoking determination period"). For example, the smoker determination unit 11 stores the captured images output from the camera 3a in time-series in a buffer of the smoker determination unit 11 or in a storage unit (not shown). The smoker determination unit 11 may acquire captured images for the smoking determination period from the stored time-series captured images. The captured images stored by the smoker determination unit 11 are deleted at a preset timing. For example, the smoker determination unit 11 deletes captured images stored prior to the smoking determination period, going back from the present.

When the smoker determination unit 11 acquires the time-series captured images output from the camera 3a during the smoking determination period, it determines whether the driver's mouth is no longer detected at a preset period (hereinafter referred to as the "mouth detection period") based on the acquired time-series captured images. The smoker determination unit 11 may use a known image recognition technique to determine whether the driver's mouth is detectable.
When the smoker determination unit 11 determines that the driver's mouth is no longer detected during the mouth detection period, it calculates the time during which the driver's mouth is no longer detected (hereinafter referred to as "mouth non-detection time").
The smoker determination unit 11 then compares the calculated mouth non-detection time with a preset threshold value (hereinafter referred to as the "mouth non-detection time determination threshold value").
The smoker determination unit 11 determines that the driver is a smoker when the calculated mouth non-detection time is equal to or greater than the mouth non-detection time determination threshold value.
If the calculated mouth non-detection time is less than the mouth non-detection time determination threshold, the smoker determination unit 11 determines that the driver is a non-smoker.

To give a specific example, suppose the smoking determination period is 10 hours, the mouth detection cycle is 15 seconds, the calculated mouth non-detection time, i.e., the total time during which a mouth is not detected at 15-second intervals, is 3 hours, and the mouth non-detection time determination threshold is 2 hours.
In this case, the mouth non-detection time of 3 hours exceeds the mouth non-detection time determination threshold of 2 hours.
Therefore, the smoker determination unit 11 determines that the driver is a smoker.

If the mouth is not detected during the mouth detection period and the total time during which the mouth is not detected exceeds the threshold for determining mouth non-detection time, it is estimated that the driver has placed a cigarette in their mouth, i.e., smoked a cigarette.
For example, when the driver's mouth is not detected in the captured image, in other words, when the driver's mouth is hidden, the driver may be eating something in addition to holding a cigarette in his/her mouth. However, the action of bringing food to the mouth when eating is not expected to occur as periodically and for as long as the action of holding a cigarette in the mouth when smoking.
The smoker determination unit 11 periodically determines whether or not the mouth has not been detected on the captured image for a certain period of time, thereby excluding the possibility that the driver is taking the action of eating something and determining whether or not the driver is taking the action of putting a cigarette in his/her mouth.
The mouth detection period is set to, for example, a period that is generally assumed as the period during which a person puts a cigarette in his mouth to smoke one cigarette.
The threshold value for determining the mouth non-detection time is set appropriately by an administrator or the like.

The smoker determination unit 11 outputs the determination result (hereinafter referred to as the "smoker determination result") of whether the driver is determined to be a smoker or a non-smoker to the smoking determination unit 12.

As described above, if the smoker determination unit 11 determines that the driver is a smoker, the smoking determination unit 12 makes a smoking determination based on the captured image output from the camera 2. If the smoker determination unit 11 determines that the driver is not a smoker, i.e., that the driver is a non-smoker, the smoking determination unit 12 does not make the above smoking determination. Note that the smoking determination unit 12 can determine from the smoker determination result whether the smoker determination unit 11 has determined that the driver is a smoker or a non-smoker.

In detail, when the smoker determination unit 11 determines that the driver is a smoker, the smoking determination unit 12 determines whether the driver is smoking based on the captured image and a trained model (hereinafter referred to as the "machine learning model"). In the first embodiment, the machine learning model used by the smoking determination unit 12 in determining whether the driver is a smoker is also referred to as the "first machine learning model."
The first machine learning model is a model that has been trained to receive an image of at least a person's face as input and output information indicating whether the person is smoking. The first machine learning model is generated in advance and stored in a location that can be referenced by the smoking determination unit 12.
The smoking determination unit 12 inputs the captured image output from the camera 2 into the first machine learning model to obtain information indicating whether or not the driver is smoking. If the smoking determination unit 12 obtains information indicating that the driver is smoking, it determines that the driver is smoking, and if the smoking determination unit 12 obtains information indicating that the driver is not smoking, it determines that the driver is not smoking.

If the smoking determination unit 12 determines that the driver is smoking, it outputs information that the driver is smoking (hereinafter referred to as "smoking information") to the alarm control unit 13.

As described above, if the smoking determination unit 12 determines that the driver is smoking as a result of making the smoking determination, in other words, if smoking information is output from the smoking determination unit 12, the alarm control unit 13 outputs alarm output control information to the alarm device 4.

For example, if the alarm device 4 is an audio output device, the alarm output control information is information that causes the audio output device to output an alarm sound, a warning message warning against smoking, or a warning message urging caution when driving. For example, if the alarm device 4 is a display device, the alarm output control information is information that causes the display device to display a warning message warning against smoking, or a warning message urging caution when driving. For example, the alarm output control information may be information that causes the display device to display an icon, such as a cigarette icon. For example, if the alarm device 4 is a vibration generating device, the alarm output control information is information that causes the vibration generating device to generate vibrations that warn against smoking.

The operation of the smoking determination device 1 according to the first embodiment will be described.
Here, the operation of the smoking determination apparatus 1 will be described assuming that the smoker determination information acquisition device 3 is a camera 3a as shown in FIG.

FIG. 3 is a flowchart for explaining the operation of the smoking determination device 1 according to the first embodiment.
3 each time an image is output from camera 2 after the vehicle is powered on. More specifically, for example, a control unit (not shown) of smoking determination device 1 detects that an image has been output from camera 2, and outputs operation instructions to smoker determination unit 11, smoking determination unit 12, and alarm control unit 13 of smoking determination device 1.
For example, when the power supply of the vehicle is turned off, the smoking determination device 1 ends the operation shown in the flowchart of FIG.

The smoker determination unit 11 acquires smoker determination information from the smoker determination information acquisition device 3, and determines whether the driver is a smoker based on the acquired smoker determination information (step ST10).
The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12 .

The smoking determination unit 12 determines whether or not the smoker determination unit 11 has determined in step ST10 that the driver is a smoker (step ST20).
If the smoker determination unit 11 determines in step ST20 that the driver is a smoker (if "YES" in step ST20), the smoking determination unit 12 makes a smoking determination based on the captured image output from the camera 2 (step ST30).
If the smoking determination unit 12 determines, as a result of the smoking determination, that the driver is smoking, it outputs smoking occurrence information to the alarm control unit 13 .

If the smoking determination unit 12 determines in step ST30 that the driver is smoking, in other words, if smoking information is output from the smoking determination unit 12, the alarm control unit 13 outputs alarm output control information to the alarm device 4 (step ST40).
It should be noted that the alarm control unit 13 does not output the alarm output control information unless the smoking occurrence information is output from the smoking determination unit 12 .

If, in step ST20, the smoker determination unit 11 determines that the driver is a non-smoker (if "NO" in step ST20), the operation of the smoking determination device 1 skips the processing of steps ST30 and ST40 and terminates the processing shown in the flowchart of Figure 3.

Fig. 4 is a flowchart for explaining an example of the details of the process of step ST10 in Fig. 3. That is, Fig. 4 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 in the first embodiment when the smoker determination information is a captured image.
The smoker determining unit 11 acquires captured images from the camera 3a as information for smoker determination. More specifically, the smoker determining unit 11 acquires time-series captured images output from the camera 3a during a smoking determination period (step ST101a).

When the smoker determination unit 11 acquires the time-series captured images output from the camera 3a during the smoking determination period, it determines whether the driver's mouth is no longer detected in the mouth detection period based on the acquired time-series captured images, and if it determines that the driver's mouth is no longer detected in the mouth detection period, it calculates the mouth non-detection time (step ST102a).

The smoker determination unit 11 compares the calculated mouth non-detection time with a mouth non-detection time determination threshold, and determines whether the mouth non-detection time is equal to or greater than the mouth non-detection time determination threshold (step ST103a).
If the mouth non-detection time is equal to or greater than the mouth non-detection time determination threshold (YES in step ST103a), the smoker determination unit 11 determines that the driver is a smoker (step ST104a), and outputs the smoker determination result to the smoking determination unit 12.
On the other hand, if the mouth non-detection time is less than the mouth non-detection time determination threshold ("NO" in step ST103a), the smoker determination unit 11 determines that the driver is a non-smoker (step ST105a). Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

In this way, the smoking determination device 1 determines whether the driver is a smoker based on the smoker determination information, and if it determines that the driver is a smoker, it makes a smoking determination based on the captured image to determine whether the driver is smoking, and if it determines that the driver is a non-smoker, it does not make a smoking determination.

In more detail, for example, if the smoker determination information acquisition device 3 is a camera 3a and the smoker determination information is a captured image, the smoking determination device 1 calculates, based on the captured image, the mouth non-detection time during which the driver's mouth was not detected during the smoking determination period if the driver's mouth was not detected at the mouth detection cycle during the smoking determination period, and determines that the driver is a smoker if the mouth non-detection time is equal to or greater than a mouth non-detection time determination threshold.If the mouth non-detection time is not equal to or greater than the mouth non-detection time determination threshold, the smoking determination device 1 determines that the driver is a non-smoker.
If the smoking determination device 1 determines that the driver is a smoker, it performs a smoking determination based on the captured image to determine whether the driver is smoking. However, if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

This allows the smoking determination device 1 to prevent mistakenly determining that a non-smoker is smoking.

If the smoking determination device 1 does not determine whether a driver is a smoker or a non-smoker, and instead performs a smoking determination on non-smokers as well, there is a possibility that it may mistakenly determine that a non-smoker who clearly does not smoke is smoking.
This is because, when determining whether a person is smoking from a captured image, for example, a behavior similar to a smoking behavior may be mistakenly detected as a smoking behavior, or an object similar to a cigarette may be mistakenly detected as a cigarette. For example, cigarettes and stick-shaped candy have similar shapes. Therefore, the behavior of holding a stick-shaped candy in one's mouth may be mistakenly detected as the behavior of holding a cigarette in one's mouth.
As a result, a false alarm warning about smoking may be issued based on a false determination that the person is smoking. Such false alarms are particularly annoying for non-smokers who clearly do not smoke.
In contrast, as described above, the smoking determination device 1 determines whether the driver is a smoker or a non-smoker, and performs a smoking determination only when it determines that the driver is a smoker.
This allows the smoking determination device 1 to prevent erroneous determination that a non-smoker is smoking.
For non-smokers who never smoke, the function of determining whether or not a person is smoking is itself an unnecessary function. The smoking determination device 1 disables the function of determining whether or not a person is smoking, which is an unnecessary function for non-smokers, in other words, does not determine whether or not a person is smoking, thereby reducing the inconvenience that may be caused to non-smokers as described above.

In the above description, the smoker determination information acquisition device 3 is a camera 3a, for example.
However, the information acquisition device 3 for determining whether a driver is a smoker may be any device capable of acquiring information for determining whether a driver is a smoker, and the smoking determination device 1 can determine whether a driver is a smoker or not based on the information for determining whether a driver is a smoker obtained from various information acquisition devices 3 for determining whether a driver is a smoker.

<Information for determining whether a smoker is smoker setting information>
For example, the smoker determination information acquisition device 3 may be a smoker setting device that allows the driver to set that he or she is a smoker. In this case, the smoker determination information is information set in the smoker setting device that indicates whether or not the driver of the vehicle is a smoker (hereinafter referred to as "smoker setting information").
The following describes in detail the smoking determination device 1 when the smoker determination information acquisition device 3 is a smoker setting device and the smoker determination information is smoker setting information.
FIG. 5 is a diagram showing an example of the configuration of the smoking determination system 100 in the first embodiment when the smoker determination information acquisition device 3 is the smoker setting device 3b.
The configuration example of the smoking determination device 1 is similar to the configuration example described with reference to FIGS. 1 and 2, and therefore the same reference numerals are used.

The smoker setting device 3b is mounted on, for example, a vehicle and has a setting button. The smoker setting device 3b may be provided in a navigation device provided in the center of an instrument panel or a meter panel.
For example, when the driver gets into the vehicle, he or she operates a setting button to input setting information for setting whether the driver is a smoker or a non-smoker. The smoker setting device 3b accepts the setting information input by the driver and generates smoker setting information.
After generating the smoker setting information, the smoker setting device 3 b outputs the generated smoker setting information to the smoking determination device 1 .

In the smoking determination device 1, the smoker determination unit 11 acquires smoker setting information as smoker determination information from the smoker setting device 3b.
Then, the smoker determination unit 11 determines whether or not the driver is a smoker based on the smoker setting information acquired from the smoker setting device 3b.
Specifically, when the driver is set as a smoker in the smoker setting information, the smoker determination unit 11 determines that the driver is a smoker.
If the driver is set as a non-smoker in the smoker setting information, the smoker determination unit 11 determines that the driver is a non-smoker.
The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12 .

Details of the smoking determination unit 12 and the alarm control unit 13 are the same as those already explained, so duplicate explanations will be omitted.

Even when the smoker determination information is smoker setting information, the basic operation of the smoking determination device 1 is the same as when the smoker determination information is a captured image.
Even when the smoker determination information is smoker setting information, the basic operation of the smoking determination device 1 is the same as the operation of the smoking determination device 1 described using the flowchart of FIG.
However, the details of the processing of step ST10 in FIG. 3 by the smoker determination unit 11 are different from the details of the processing by the smoker determination unit 11 when the smoker determination information is smoker setting information, as explained above.
FIG. 6 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 in the first embodiment when the smoker determination information is smoker setting information.
If the smoker determination information is smoker setting information, the smoking determination device 1 performs the process in step ST10 of FIG. 3, specifically the process described in the flowchart of FIG.

The smoker determination unit 11 acquires smoker setting information as information for smoker determination from the smoker setting device 3b (step ST101b).

The smoker determination unit 11 determines whether the driver is set as a smoker in the smoker setting information (step ST102b).

If it is determined that the driver is set as a smoker (YES in step ST102b), the smoker determination unit 11 determines that the driver is a smoker (step ST103b), and outputs the smoker determination result to the smoking determination unit 12.
On the other hand, if it is determined that the driver is set as a non-smoker ("NO" in step ST102b), the smoker determination unit 11 determines that the driver is a non-smoker (step ST104b), and outputs the smoker determination result to the smoking determination unit 12.

Thus, for example, if the information acquisition device 3 for smoker determination is a smoker setting device 3b and the information for smoker determination is smoker setting information, the smoking determination device 1 determines that the driver is a smoker if the driver is set as a smoker in the smoker setting information.
If the smoking determination device 1 determines that the driver is a smoker, it performs a smoking determination based on the captured image to determine whether the driver is smoking. However, if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

This allows the smoking determination device 1 to prevent mistakenly determining that a non-smoker is smoking.

<Information used to determine whether a smoker is a personal authentication information>
Furthermore, for example, the smoker determination information acquisition device 3 may be a personal authentication information acquisition device that acquires personal authentication information for personal authentication of the driver. In this case, the smoker determination information is personal authentication information acquired by the personal authentication information acquisition device.
The following describes in detail the smoking determination device 1 when the smoker determination information acquisition device 3 is a personal authentication information acquisition device and the smoker determination information is personal authentication information.
FIG. 7 is a diagram showing an example of the configuration of the smoking determination system 100 in the first embodiment when the smoker determination information acquisition device 3 is the personal authentication information acquisition device 3c.
The configuration example of the smoking determination device 1 is similar to the configuration examples described in FIGS. 1, 2, and 5, and therefore the same reference numerals are used.

The personal authentication information acquisition device 3c may be, for example, the camera 2, a reader that reads a card key such as a contactless IC card, or a biometric information acquisition device that reads biometric information. The personal authentication information acquisition device 3c may be any device that can acquire personal authentication information for performing any known personal authentication.
For example, when the driver gets into the vehicle, the driver causes the personal authentication information acquisition device 3c to read the personal authentication information. Upon acquiring the personal authentication information, the personal authentication information acquisition device 3c outputs the acquired personal authentication information to the smoking determination device 1.

In the smoking determination device 1, the smoker determination unit 11 acquires personal authentication information as smoker determination information from the personal authentication information acquisition device 3c.
The smoker determination unit 11 then performs personal authentication of the driver using a known personal authentication method based on the personal authentication information acquired from the personal authentication information acquisition device 3c.
When the smoker determination unit 11 performs personal authentication, it compares information that can identify the authenticated driver (e.g., ID or facial image, etc.) with information that can identify a person (e.g., ID or facial image, etc.) registered in the smoker registration list, and determines whether the driver is a smoker or a non-smoker.

The smoker registration list is a list of registered smokers, and lists information that can identify smokers. The smoker registration list is generated in advance by an administrator or the like, and is held by the smoker determination unit 11 or stored in a location that can be referenced by the smoker determination unit 11.

The smoker determination unit 11 determines that a driver who has been personally authenticated is a smoker when the driver is registered in the smoker registration list, more specifically, when information that can identify the personally authenticated driver is registered in the smoker registration list.
The smoker determination unit 11 determines that a driver who has been personally authenticated is a non-smoker if the driver is not registered in the smoker registration list, more specifically, if information that can identify the personally authenticated driver is not registered in the smoker registration list.
The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12 .

Details of the smoking determination unit 12 and the alarm control unit 13 are the same as those already explained, so duplicate explanations will be omitted.

Even if the smoker determination information is personal authentication information, the basic operation of the smoking determination device 1 is the same as when the smoker determination information is a captured image or smoker setting information.
Even when the smoker determination information is personal authentication information, the basic operation of the smoking determination device 1 is the same as the operation of the smoking determination device 1 described using the flowchart of FIG.
However, the details of the processing of step ST10 in Figure 3 by the smoker determination unit 11 differ from the details of the processing by the smoker determination unit 11 when the smoker determination information is a captured image or smoker setting information, as described above.
FIG. 8 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 in the first embodiment when the smoker determination information is personal authentication information.
If the smoker determination information is personal authentication information, the smoking determination device 1 performs the process in step ST10 of FIG. 3, specifically the process described in the flowchart of FIG.

The smoker determining unit 11 acquires personal authentication information as smoker determining information from the personal authentication information acquisition device 3c (step ST101c).
The smoker determination unit 11 performs personal authentication of the driver using a known personal authentication method based on the personal authentication information acquired from the personal authentication information acquisition device 3c (step ST102c).

After performing personal authentication, the smoker determination unit 11 compares information that can identify the authenticated driver with information that can identify people registered in the smoker registration list. Specifically, the smoker determination unit 11 determines whether the driver who has been authenticated in step ST102c is registered in the smoker registration list (step ST103c).
If it is determined in step ST103c that the personally authenticated driver is registered in the smoker registration list (YES in step ST103c), the smoker determination unit 11 determines that the driver is a smoker (step ST104c).Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.
If it is determined in step ST103c that the authenticated driver is not registered in the smoker registration list ("NO" in step ST103c), the smoker determination unit 11 determines that the driver is a non-smoker (step ST105c). Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

In this way, for example, if the information acquisition device 3 for smoker determination is a personal authentication information acquisition device 3c and the information for smoker determination is personal authentication information, the smoking determination device 1 determines that the driver is a smoker based on the personal authentication information if the driver is registered in the smoker registration list.
If the smoking determination device 1 determines that the driver is a smoker, it performs a smoking determination based on the captured image to determine whether the driver is smoking. However, if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

This allows the smoking determination device 1 to prevent mistakenly determining that a non-smoker is smoking.

<Information used to determine whether a smoker is nicotine dependence information>
Furthermore, for example, the information acquisition device 3 for determining whether a smoker is present may be a nicotine dependence determination device for determining whether a driver is presently nicotine dependent. In this case, the information for determining whether a smoker is present is nicotine dependence information indicating the degree of nicotine dependence of the driver.
Hereinafter, the smoking determination device 1 will be described in detail in the case where the smoker determination information acquisition device 3 is a nicotine dependence determination device and the smoker determination information is nicotine dependence information.
FIG. 9 is a diagram showing an example of the configuration of the smoking determination system 100 in the first embodiment when the smoker determination information acquisition device 3 is the nicotine dependence determination device 3d.
The configuration example of the smoking determination device 1 is similar to the configuration examples described in FIGS. 1, 2, 5, and 7, and therefore the same reference numerals are used.

The nicotine dependence assessment device 3d acquires the results of a nicotine dependence test, such as the Fagerstrom Tobacco Dependence Test, administered to subjects including drivers, and assesses the subject's nicotine dependence. The nicotine dependence test is, for example, a test in which subjects answer questions and assess the subject's nicotine dependence based on the total score. The nicotine dependence assessment device 3d assesses the subject's nicotine dependence so that the higher the score on the nicotine dependence test, the higher the subject's nicotine dependence.
Here, the nicotine dependence determination device 3d is assumed to be mounted on a vehicle. For example, when a driver gets into the vehicle and turns on the vehicle, the nicotine dependence determination device 3d displays a nicotine dependence test on a display device (not shown). The display device is, for example, a touch panel display provided in a navigation device.
The driver inputs answers to questions in the nicotine dependence test by touching the display device. The nicotine dependence determination device 3d acquires the results of the nicotine dependence test performed by the driver and determines the driver's nicotine dependence.
The nicotine dependence determination device 3 d then outputs nicotine dependence information indicating the determined nicotine dependence to the smoking determination device 1 .

For example, the nicotine dependence determination device 3d may cause the driver to take a nicotine dependence test only if the driver has not taken a nicotine dependence test in the past.
In this case, for example, the nicotine dependence determination device 3d has a personal authentication function, and stores information that can identify the driver who underwent the nicotine dependence test (the driver's personal authentication result) and information that associates the determined nicotine dependence of the driver (hereinafter referred to as "test result information") in a storage unit (not shown). The storage unit may be provided in the nicotine dependence determination device 3d, or may be provided in a location outside the nicotine dependence determination device 3d that can be referenced by the nicotine dependence determination device 3d.
When the nicotine dependence determination device 3d performs personal authentication of the driver when the driver gets into the vehicle, it refers to the storage unit and determines whether test result information including the personal authentication result is stored. If the test result information is stored, it is determined that the driver has previously taken a nicotine dependence test, and the nicotine dependence level set in the test result information is set as the driver's nicotine dependence. In this case, the nicotine dependence determination device 3d does not display the nicotine dependence test on the display device.
If the test result information is not stored, the nicotine dependence determination device 3d determines that the driver has never taken a nicotine dependence test, displays the nicotine dependence test on the display device, and makes the driver take the nicotine dependence test.
In this way, if the driver has taken a nicotine dependence test in the past, the nicotine dependence determination device 3d uses the results to determine the driver's nicotine dependence and does not require the driver to take the nicotine dependence test, thereby preventing the driver from having to go through the hassle of taking the nicotine dependence test multiple times.

In the smoking determination device 1, the smoker determination unit 11 acquires nicotine dependence information as smoker determination information from the nicotine dependence determination device 3d.
Then, the smoker determination unit 11 determines whether the driver is a smoker or a non-smoker based on the nicotine dependence information acquired from the nicotine dependence determination device 3d.
The smoker determination unit 11 determines that the driver is a smoker when it is estimated that the driver has a high level of nicotine dependence based on the nicotine dependence information.
On the other hand, when it is estimated based on the nicotine dependence information that the driver has a low level of nicotine dependence, the smoker determination unit 11 determines that the driver is a smoker.
For example, the smoker determination unit 11 estimates that the driver's nicotine dependence is high if the driver's nicotine dependence indicated in the nicotine dependence information is equal to or greater than a predetermined threshold (hereinafter referred to as the "threshold for determining dependence"), and estimates that the driver's nicotine dependence is low if the driver's nicotine dependence is less than the threshold for determining dependence.
The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12 .

Details of the smoking determination unit 12 and the alarm control unit 13 are the same as those already explained, so duplicate explanations will be omitted.

Even when the smoker determination information is nicotine dependence information, the basic operation of the smoking determination device 1 is the same as when the smoker determination information is a captured image, smoker setting information, or personal authentication information.
Even when the smoker determination information is nicotine dependence information, the basic operation of the smoking determination device 1 is the same as the operation of the smoking determination device 1 described using the flowchart of FIG.
However, the details of the processing of step ST10 in Figure 3 by the smoker determination unit 11 differ from the details of the processing by the smoker determination unit 11 when the smoker determination information is an image capture, smoker setting information, or personal authentication information, as described above.
FIG. 10 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 in the first embodiment when the smoker determination information is nicotine dependence information.
If the smoker determination information is nicotine dependence information, the smoking determination device 1 performs the process described in detail in the flowchart of FIG. 10 in step ST10 of FIG.

The smoker determination unit 11 acquires nicotine dependence information as smoker determination information from the nicotine dependence determination device 3d (step ST101d).
The smoker determination unit 11 estimates whether the driver's level of nicotine dependence is high or low based on the nicotine dependence information (step ST102d).
If it is estimated in step ST102d that the driver has a high level of nicotine dependence (YES in step ST102d), the smoker determination unit 11 determines that the driver is a smoker (step ST103d).Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.
If it is estimated in step ST102d that the driver's nicotine dependence is low ("NO" in step ST102d), the smoker determination unit 11 determines that the driver is a non-smoker (step ST104d). Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

In this way, for example, if the information acquisition device 3 for determining whether a driver is a smoker is a nicotine dependence determination device 3d and the information for determining whether a driver is a smoker is nicotine dependence information, the smoking determination device 1 determines that the driver is a smoker if it estimates that the driver has a high level of nicotine dependence based on the nicotine dependence information.
If the smoking determination device 1 determines that the driver is a smoker, it performs a smoking determination based on the captured image to determine whether the driver is smoking. However, if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

This allows the smoking determination device 1 to prevent mistakenly determining that a non-smoker is smoking.

<Information used to determine whether a smoker is a heat source>
Furthermore, for example, the smoker determination information acquisition device 3 may be a temperature measurement device that measures the temperature around the face of the driver. In this case, the smoker determination information is information about the temperature around the face of the vehicle driver measured by the temperature measurement device (hereinafter referred to as "heat source information").
The following describes in detail the smoking determination device 1 when the smoker determination information acquisition device 3 is a temperature measurement device and the smoker determination information is heat source information.
FIG. 11 is a diagram showing an example of the configuration of the smoking determination system 100 in the first embodiment when the smoker determination information acquisition device 3 is a temperature measurement device 3e.
The configuration example of the smoking determination device 1 is similar to the configuration examples described in FIGS. 1, 2, 5, 7, and 9, and therefore the same reference numerals are used.

The temperature measuring device 3e may be, for example, a thermal source camera or a temperature sensor. The temperature measuring device 3e may be any device capable of acquiring heat source information. Here, as an example, the temperature measuring device 3e is assumed to be a thermal source camera. A thermal source camera is a so-called thermograph that generates a heat source image according to the amount of infrared rays emitted from an object.
The temperature measuring device 3 e outputs heat source information, more specifically, a heat source image here, to the smoking determination device 1 .

In the smoking determination device 1, the smoker determination unit 11 acquires heat source information from the temperature measurement device 3e as smoker determination information.
Then, the smoker determination unit 11 determines whether or not the driver is a smoker based on the heat source information acquired from the temperature measuring device 3e.
In detail, the smoker determination unit 11 acquires the time-series heat source information output from the temperature measuring device 3e during the smoking determination period. For example, the smoker determination unit 11 stores the heat source information output from the temperature measuring device 3e in a buffer of the smoker determination unit 11 or in a storage unit (not shown) in time series. The smoker determination unit 11 may acquire the heat source information for the smoking determination period from the stored time-series heat source information. The heat source information stored by the smoker determination unit 11 is deleted at a preset timing. For example, the smoker determination unit 11 deletes the heat source information stored before the smoking determination period, going back from the present.

When the smoker determination unit 11 acquires the time-series heat source information output from the temperature measuring device 3e during the smoking determination period, it determines, based on the acquired time-series heat source information, whether a high-temperature area above a predetermined threshold (hereinafter referred to as the ``temperature determination threshold'') has been detected around the driver's face for a predetermined time (hereinafter referred to as the ``high-temperature determination time'').
The smoker determination unit 11 may, for example, acquire a captured image from the camera 2, perform known image recognition processing on the acquired captured image and the heat source image, and identify which area on the heat source image corresponds to the area around the driver's mouth. Since the installation position and angle of view of the camera 2 and the installation position and angle of view of the heat source camera are known in advance, the smoker determination unit 11 can associate the area on the captured image with the area on the heat source image. Note that the arrow from the camera 2 to the smoker determination unit 11 is omitted in Figure 11.

When the smoker determination unit 11 determines that a high temperature area equal to or higher than the temperature determination threshold has been detected around the face of the driver for a period equal to or longer than the high temperature determination time, the smoker determination unit 11 determines that the driver is a smoker.
On the other hand, if there is no high temperature area around the driver's face that is above the temperature judgment threshold, or if there is a high temperature area around the driver's face that is above the temperature judgment threshold but the high temperature area is not detected for more than the high temperature judgment time, the smoker determination unit 11 determines that the driver is a non-smoker.

The method by which the smoker determination unit 11 determines whether a driver is a smoker or a non-smoker based on the time-series heat source information is not limited to the above-described method. The smoker determination unit 11 may determine whether a driver is a smoker or a non-smoker based on the time-series heat source information and in accordance with other conditions.
For example, the smoker determination unit 11 may determine whether the frequency of occurrence of high temperature areas around the driver's face is equal to or greater than a predetermined threshold (hereinafter referred to as the "high temperature occurrence frequency determination threshold") based on the acquired time-series heat source information.
In this case, the smoker determination unit 11 determines that the driver is a smoker if the frequency of appearance of high temperature areas around the face of the driver is equal to or greater than the high temperature appearance frequency determination threshold value.
If the frequency of appearance of high temperature areas around the face of the driver is less than the high temperature appearance frequency determination threshold, the smoker determination unit 11 determines that the driver is a non-smoker.
The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12 .

Details of the smoking determination unit 12 and the alarm control unit 13 are the same as those already explained, so duplicate explanations will be omitted.

Even if the information for determining a smoker is heat source information, the basic operation of the smoking determination device 1 is the same as when the information for determining a smoker is a captured image, smoker setting information, personal authentication information, or nicotine dependence information.
Even when the smoker determination information is heat source information, the basic operation of the smoking determination device 1 is the same as the operation of the smoking determination device 1 described using the flowchart of FIG.
However, the details of the processing of step ST10 in Figure 3 by the smoker determination unit 11 differ from the details of the processing by the smoker determination unit 11 when the smoker determination information is an image capture, smoker setting information, personal authentication information, or nicotine dependence information, as described above.
FIG. 12 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 in the first embodiment when the smoker determination information is heat source information.
When the smoker determination information acquisition device 3 is the temperature measurement device 3e, the smoking determination device 1 performs the process described in detail in the flowchart of FIG. 12 in step ST10 of FIG.

The smoker determination unit 11 acquires heat source information from the temperature measurement device 3e as information for smoker determination. More specifically, the smoker determination unit 11 acquires the time-series heat source information output from the temperature measurement device 3e during the smoking determination period (step ST101e).
Based on the time-series heat source information, the smoker determination unit 11 determines whether a high-temperature area above the temperature determination threshold has been detected around the driver's face for a period of time longer than the high-temperature determination time, or whether the frequency of occurrence of high-temperature areas around the driver's face is above the high-temperature occurrence frequency determination threshold (step ST102e).

If it is determined in step ST102e that a high-temperature area equal to or greater than the temperature determination threshold has been detected around the driver's face for the high-temperature determination time or longer, or if it is determined that the frequency of occurrence of high-temperature areas around the driver's face is equal to or greater than the high-temperature occurrence frequency determination threshold ("YES" in step ST102e), the smoker determination unit 11 determines that the driver is a smoker (step ST103e).The smoker determination unit 11 then outputs the smoker determination result to the smoking determination unit 12.

If it is determined in step ST102e that there is no high-temperature area around the driver's face that is equal to or higher than the temperature determination threshold, if it is determined that even if there is a high-temperature area around the driver's face that is equal to or higher than the temperature determination threshold, the high-temperature area has not been detected for the high-temperature determination time or longer, or if it is determined that the frequency of high-temperature areas around the driver's face is less than the high-temperature appearance frequency determination threshold ("NO" in step ST102e), the smoker determination unit 11 determines that the driver is a non-smoker (step ST104e).The smoker determination unit 11 then outputs the smoker determination result to the smoking determination unit 12.

Thus, for example, if the information acquisition device 3 for determining whether a smoker is a temperature measuring device 3e and the information for determining whether a smoker is heat source information, the smoking determination device 1 determines that the driver is a smoker based on the time-series heat source information if it detects a high-temperature area around the driver's face that is above the temperature determination threshold for a period of time or longer for high-temperature determination, or if the frequency of occurrence of high-temperature areas around the driver's face is above the high-temperature occurrence frequency determination threshold.
If the smoking determination device 1 determines that the driver is a smoker, it performs a smoking determination based on the captured image to determine whether the driver is smoking. However, if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

This allows the smoking determination device 1 to prevent mistakenly determining that a non-smoker is smoking.

<Respiration information is used to determine whether a smoker is a smoker>
Furthermore, for example, the smoker determination information acquisition device 3 may be a breathing detection device that detects the breathing of the driver. In this case, the smoker determination information is information related to the breathing of the vehicle driver (hereinafter referred to as "breathing information") set by the breathing detection device.
The following describes in detail the smoking determination device 1 when the smoker determination information acquisition device 3 is a breathing detection device and the smoker determination information is breathing information.
FIG. 13 is a diagram showing an example of the configuration of the smoking determination system 100 in the first embodiment when the smoker determination information acquisition device 3 is a breathing detection device 3f.
The configuration example of the smoking determination device 1 is similar to the configuration examples described in Figures 1, 2, 5, 7, 9, and 11, and therefore the same reference numerals are used.

The breathing detection device 3f is, for example, a millimeter wave sensor. The millimeter wave sensor transmits radio waves, such as millimeter waves, into the vehicle interior and receives the reflected waves from an object. The millimeter wave sensor can detect the presence, position, speed, etc. of an object associated with the reflected waves based on the time from when the radio waves are transmitted to when the reflected waves are received, thereby detecting movement that can be estimated as a person's breathing. Note that this is merely an example, and the breathing detection device 3f may be, for example, a camera 2 or a breathing sensor attached to clothing. The breathing detection device 3f may be any device that can detect the driver's breathing.
The breathing detection device 3f outputs the detected breathing information of the driver to the smoking determination device 1. The breathing information includes, for example, the waveform of the driver's breathing.

In the smoking determination device 1, the smoker determination unit 11 acquires breathing information from the breathing detection device 3f as information for smoker determination.
Then, the smoker determination unit 11 determines whether or not the driver is a smoker based on the breathing information acquired from the breathing detection device 3f.
In detail, the smoker determination unit 11 acquires time-series respiratory information output from the breathing detection device 3f during the smoking determination period. For example, the smoker determination unit 11 stores the respiratory information output from the breathing detection device 3f in time series in a buffer of the smoker determination unit 11 or in a storage unit (not shown). The smoker determination unit 11 may acquire respiratory information for the smoking determination period from the stored time-series respiratory information. The respiratory information stored by the smoker determination unit 11 is deleted at a preset timing. For example, the smoker determination unit 11 deletes respiratory information stored before the smoking determination period, going back from the present.

When the smoker determination unit 11 acquires the time-series breathing information output from the breathing detection device 3f during the smoking determination period, it detects breathing that is estimated to be breathing associated with smoking behavior (hereinafter referred to as ``smoking breathing'') based on the acquired time-series breathing information.
Generally, a person's breathing during smoking, in other words, the act of smoking a cigarette, is composed of a single inhalation phase and a single exhalation phase. It is estimated that the breathing during smoking is a breathing with a duration and amplitude that is a predetermined times longer or larger than that of normal breathing. The smoker determination unit 11 can detect the driver's smoking breath by determining the timing of the driver's inhalation and exhalation from the breathing information.

The smoker determination unit 11 determines that the driver is a smoker if the number of smoking breaths during the smoking determination period is equal to or greater than a preset number (hereinafter referred to as the "smoking breath determination threshold").
On the other hand, if the number of smoking breaths during the smoking determination period is less than the smoking breath determination threshold, the smoker determination unit 11 determines that the driver is a non-smoker.
The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12 .

Details of the smoking determination unit 12 and the alarm control unit 13 are the same as those already explained, so duplicate explanations will be omitted.

Even if the information for determining whether a smoker is breathing information, the basic operation of the smoking determination device 1 is the same as when the information for determining whether a smoker is an image, smoker setting information, personal authentication information, nicotine dependence information, or heat source information.
Even when the information for determining whether a smoker is present as breathing information, the basic operation of the smoking determination apparatus 1 is the same as that of the smoking determination apparatus 1 described using the flowchart of FIG.
However, the details of the processing of step ST10 in Figure 3 by the smoker determination unit 11 differ from the details of the processing by the smoker determination unit 11 when the smoker determination information is an image capture, smoker setting information, personal authentication information, nicotine dependence information, or heat source information, as described above.
FIG. 14 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 in the first embodiment when the smoker determination information is breathing information.
If the smoker determination information is breathing information, the smoking determination device 1 performs the process in step ST10 of FIG. 3, specifically the process described in the flowchart of FIG.

The smoker determination unit 11 acquires breathing information as information for determining whether a smoker is present from the breathing detection device 3 f. More specifically, the smoker determination unit 11 acquires time-series breathing information output from the breathing detection device 3 f during the smoking determination period (step ST101 f).
The smoker determining unit 11 determines whether the number of smoking breaths during the smoking determination period is equal to or greater than the smoking breath determination threshold value, based on the time-series breathing information (step ST102f).

If it is determined in step ST102f that the number of smoking breaths during the smoking determination period is equal to or greater than the smoking breath determination threshold ("YES" in step ST102f), the smoker determination unit 11 determines that the driver is a smoker (step ST103f).The smoker determination unit 11 then outputs the smoker determination result to the smoking determination unit 12.

If it is determined in step ST102f that the number of smoking breaths during the smoking determination period is less than the smoking breath determination threshold ("NO" in step ST102f), the smoker determination unit 11 determines that the driver is a non-smoker (step ST104e).The smoker determination unit 11 then outputs the smoker determination result to the smoking determination unit 12.

Thus, for example, if the information acquisition device 3 for determining whether a driver is a smoker is a breathing detection device 3f and the information for determining whether a driver is a smoker is breathing information, the smoking determination device 1 determines that the driver is a smoker based on the time-series breathing information if the number of smoking breaths during the smoking determination period is equal to or greater than the smoking breath determination threshold value.
If the smoking determination device 1 determines that the driver is a smoker, it performs a smoking determination based on the captured image to determine whether the driver is smoking. However, if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

This allows the smoking determination device 1 to prevent mistakenly determining that a non-smoker is smoking.

Furthermore, in the above-described first embodiment, the possibility that the driver is wearing a mask has not been taken into consideration. However, it cannot be said with certainty that the driver is not wearing a mask. In the first embodiment, the term "mask" refers to a nonwoven fabric, gauze, or other protective device that covers the nose or mouth.
If a driver is wearing a mask, it is not assumed that the driver is smoking. In other words, if a driver is wearing a mask, just like if the driver is a non-smoker, the function to determine whether or not the driver is smoking is itself an unnecessary function.
Therefore, in the above-described embodiment 1, the smoking determination device 1 may have a function of determining whether or not the driver is wearing a mask, and may be configured to not make a smoking determination if it determines that the driver is wearing a mask.

Figure 15 is a diagram showing an example configuration of a smoking determination system 100a in embodiment 1, which is equipped with a smoking determination device 1a that has the function of determining whether the driver is wearing a mask and is configured not to make a smoking determination if it is determined that the driver is wearing a mask.
In FIG. 15, the same components as those in the smoking determination system 100 described with reference to FIG. 1 are denoted by the same reference numerals, and redundant description will be omitted.
The smoking determination system 100a shown in FIG. 15 differs from the smoking determination system 100 described with reference to FIG. 1 in that the smoking determination device 1a includes a mask wearing determination unit 14.
1. Furthermore, the specific operation of the smoking determination unit 12a in the smoking determination device 1a is different from the specific operation of the smoking determination unit 12 in the smoking determination device 1 shown in FIG.
15, the information acquisition device 3 for smoker determination may be a camera 3a, a smoker setting device 3b, a personal authentication information acquisition device 3c, a nicotine dependence determination device 3d, a temperature measurement device 3e, or a breathing detection device 3f. In other words, the information for smoker determination acquired by the smoking determination device 1a shown in FIG. 15 may be a captured image, smoker setting information, personal authentication information, nicotine dependence information, heat source information, or breathing information.

The mask wearing determination unit 14 determines whether the driver is wearing a mask based on the captured image output from the camera 2.
The mask wearing determiner 14 determines whether or not the driver is wearing a mask based on, for example, the captured image acquired from the camera 2 and a machine learning model. In the following description, the machine learning model used by the mask wearing determiner 14 to determine whether or not the driver is wearing a mask is also referred to as a "second machine learning model."
The second machine learning model has been trained to receive an image of a person's face as input and output information indicating whether the person is wearing a mask (hereinafter referred to as "mask wearing information"). The mask wearing information includes, for example, information indicating whether the person is wearing a mask. Note that this is merely an example, and the mask wearing information may include, for example, information indicating the color, pattern, or type of mask (nonwoven fabric, gauze, etc.) in addition to the information indicating whether the person is wearing a mask. The mask wearing information needs to include at least information indicating whether the person is wearing a mask. In other words, the second machine learning model needs to be a model that has been trained to receive, at least, an image of a person's face as input and output mask wearing information including information indicating whether the person is wearing a mask.

When the captured image is input into the second machine learning model and mask wearing information indicating that a mask is being worn is obtained, the mask wearing determination unit 14 determines that the driver is wearing a mask. On the other hand, when the captured image is input into the second machine learning model and mask wearing information indicating that a mask is not being worn is obtained, the mask wearing determination unit 14 determines that the driver is not wearing a mask.

The above-described method is merely an example, and the mask wearing determiner 14 may use other methods to determine whether the driver is wearing a mask based on the captured image.
For example, the mask wearing determiner 14 may determine whether the driver is wearing a mask based on whether or not a nose feature point can be extracted from the captured image. If the mask wearing determiner 14 cannot extract the nose feature point, it determines that the driver is wearing a mask.
The mask wearing determination unit 14 may extract feature points indicating parts of the driver's face (specifically, the nose) in the captured image using a known technique such as a method using Haar-Like features or pattern matching.

When the mask wearing determination unit 14 determines whether the driver is wearing a mask or not, it outputs the result (hereinafter referred to as "mask determination result") to the smoking determination unit 12a.
The mask determination result includes information indicating whether the driver is wearing a mask.

When the mask wearing determination unit 14 determines that the driver is wearing a mask, the smoking determination unit 12a does not make a smoking determination regardless of whether the smoker determination unit 11 determines that the driver is a smoker.
The smoking determination unit 12a performs a smoking determination only when the mask wearing determination unit 14 determines that the driver is not wearing a mask.
In addition, the smoking determination unit 12a can determine from the mask determination result output from the mask wearing determination unit 14 whether the mask wearing determination unit 14 has determined that the driver is wearing a mask or not wearing a mask.

Figure 16 is a flowchart for explaining the operation of the smoking determination device 1a in embodiment 1, which has the function of determining whether the driver is wearing a mask or not, and is configured not to make a smoking determination if it is determined that the driver is wearing a mask.
The specific contents of the processes of steps ST10, ST20, ST30, and ST40 in Fig. 16 are the same as the specific contents of the processes of steps ST10, ST20, ST30, and ST40 in Fig. 1, respectively, and therefore will not be described again. Note that the details of the process of step ST10 in Fig. 16 are the contents already shown and explained in the flowcharts of Fig. 4, 6, 8, 10, 12, or 14, for example.

The mask wearing determination unit 14 determines whether the driver is wearing a mask based on the captured image output from the camera 2 (step ST25).
The mask wearing determination unit 14 outputs the mask determination result to the smoking determination unit 12a.

The smoking determining unit 12a determines whether or not the mask wearing determining unit 14 has determined in step ST25 that the driver is wearing a mask (step ST27).
If the smoking determination unit 12a determines that the mask wearing determination unit 14 has determined that the driver is wearing a mask ("YES" in step ST27), the smoking determination unit 12a proceeds to the processing of step ST30 and makes a smoking determination.
If the smoking determination unit 12a determines that the mask wearing determination unit 14 has determined that the driver is not wearing a mask ("NO" in step ST27), the operation of the smoking determination device 1a skips the processing of steps ST30 and ST40 and terminates the processing shown in the flowchart of Figure 16.

In the flowchart of FIG. 16, the smoking determination device 1a performs step ST25 after processing step ST10, but this is merely an example. For example, the smoking determination device 1a may perform step ST10 and step ST25 in parallel. Alternatively, the smoking determination device 1a may perform steps ST25, ST27, ST10, ST20, ST30, and ST40 in this order. That is, if the smoking determination device 1a determines, as a result of processing step ST25, that the driver is not wearing a mask ("NO" in step ST27), it may perform step ST10, and if it determines that the driver is a smoker ("YES" in step ST20), it may perform step ST30.

In this way, for example, the smoking determination device 1a may be configured to determine whether or not the driver is wearing a mask based on the captured image, and if it determines that the driver is wearing a mask, not to perform a smoking determination regardless of whether or not it determines that the driver is a smoker.

This allows the smoking determination device 1 to prevent unnecessary smoking determinations from being made when the driver is wearing a mask. As a result, the smoking determination device 1 can prevent erroneous warnings about smoking from being issued to drivers who are wearing masks and who are not expected to be smoking, thereby reducing the annoyance caused to drivers by false warnings.

17A and 17B are diagrams illustrating an example of the hardware configuration of the smoking determination device 1 according to embodiment 1. FIG.
In the first embodiment, the functions of the smoker determination unit 11, the smoking determination units 12 and 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit (not shown) are realized by the processing circuit 1001. That is, the smoking determination device 1 is equipped with the processing circuit 1001 for determining whether a subject is a smoker or a non-smoker based on the smoker determination information, and for controlling the device to make a smoking determination based on the captured image acquired from the camera 2 only if the subject is a smoker or only if the subject is not wearing a mask and is a smoker.
The processing circuit 1001 may be dedicated hardware as shown in FIG. 17A, or may be a processor 1004 that executes a program stored in memory as shown in FIG. 17B.

When the processing circuit 1001 is dedicated hardware, the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

When the processing circuit is a processor 1004, the functions of the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit (not shown) are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in memory 1005. The processor 1004 reads and executes the program stored in memory 1005 to perform the functions of the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit (not shown). In other words, the smoking determination device 1 includes memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST10 to ST40 of FIG. 3 or steps ST10 to ST40 of FIG. 16 described above. In addition, the program stored in memory 1005 can also be said to cause the computer to execute the processing procedures or methods of the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit not shown. Here, the memory 1005 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read-Only Memory), or a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc).

The functions of the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and the control unit (not shown) may be partially realized by dedicated hardware and partially realized by software or firmware. For example, the function of the alarm control unit 13 may be realized by a processing circuit 1001 as dedicated hardware, and the functions of the smoker determination unit 11, the smoking determination unit 12, 12a, the mask wearing determination unit 14, and the control unit (not shown) may be realized by the processor 1004 reading and executing programs stored in the memory 1005.
The smoking determination device 1 also includes an input interface device 1002 and an output interface device 1003 that perform wired or wireless communication with devices such as a camera 2, a smoker determination information acquisition device 3, or an alarm device 4.

In the above-described first embodiment, the "state of smoking" was assumed to be a state in which a cigarette is held in the mouth. In other words, a subject is considered to be smoking if the subject actually has a cigarette in their mouth. However, this is merely an example, and the "state of smoking" may also include, for example, a state in which the subject does not have a cigarette in their mouth but has a cigarette in their hand.

In addition, in the first embodiment described above, the smoking determination device 1, 1a is equipped with the alarm control unit 13, but this is merely an example, and the smoking determination device 1, 1a is not required to have the alarm control unit 13. The function of the alarm control unit 13 may be provided by another device outside the smoking determination device 1, 1a and connected to the smoking determination device 1, 1a.
If the smoking determination device 1 is configured not to have an alarm control unit 13, the processing of step ST40 is omitted in the operation of the smoking determination device 1 described using the flowchart of Figure 3 and the operation of the smoking determination device 1a described using the flowchart of Figure 16.

In addition, in the above-described first embodiment, the smoking determination device 1, 1a is an on-board device mounted on a vehicle, and the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit not shown are provided in the on-board device.
Without being limited to this, for example, the system may be configured with an on-board device and a server, with some of the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit not shown being mounted on the on-board device of the vehicle, and the others being provided on a server connected to the on-board device via a network.
Furthermore, for example, the smoker determination unit 11, the smoking determination units 12 and 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit (not shown) may all be provided in the server.

In the first embodiment, the subject is the driver of the vehicle, but this is merely an example. The subject may be a passenger other than the driver of the vehicle.
Furthermore, the subject is not limited to a vehicle occupant, but may be, for example, a occupant of a moving body other than a vehicle, or a person present in a room or the like.

Furthermore, when the faces of multiple people are captured in the image captured by the camera 2, the smoking determination device 1 can determine whether each of the multiple people is a smoker or not, and can perform smoking determination only on those people who are determined to be smokers.
When the faces of multiple people are captured in the image captured by the camera 2, the smoking determination device 1a determines whether each of the multiple people is a smoker and whether they are wearing a mask, and can perform smoking determination only on those people who are not wearing a mask and are determined to be smokers.

As described above, according to embodiment 1, the smoking determination device 1, 1a is configured to include a smoker determination unit 11 that determines whether or not a subject, whose smoking status is to be determined, is a smoker based on smoker determination information for determining whether or not the subject is a smoker, and a smoking determination unit 12 that, if the smoker determination unit 11 determines that the subject is a smoker, makes a smoking determination as to whether or not the subject is a smoker based on an image in which at least the face of the subject is captured, and does not make a smoking determination if the smoker determination unit 11 determines that the subject is a non-smoker.
Therefore, the smoking determination device 1, 1a can prevent erroneous determination that a non-smoker is smoking.

In detail, for example, the information for determining whether a person is a smoker is an image captured, and in the smoking determination device 1, 1a, the smoker determination unit 11 calculates, based on the captured image, if the subject's mouth is no longer detected at the mouth detection cycle during the smoking determination period, the mouth non-detection time during which the subject's mouth was no longer detected during the smoking determination period, and if the mouth non-detection time is equal to or greater than the mouth non-detection time determination threshold, determines that the subject is a smoker.
When the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 performs a smoking determination as to whether the subject is smoking or not based on a captured image in which at least the face of the subject is captured, and when the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination.
Therefore, the smoking determination device 1, 1a can prevent erroneous determination that a non-smoker is smoking.

Also, for example, the information for determining whether a subject is a smoker is smoker setting information that indicates whether the subject is a smoker or not, and is generated based on the subject's input of instruction information that causes the subject to register whether the subject is a smoker or a non-smoker.In the smoking determination device 1, 1a, the smoker determination unit 11 determines that the subject is a smoker if the subject is set as a smoker in the smoker setting information.
When the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 performs a smoking determination as to whether the subject is smoking or not based on a captured image in which at least the face of the subject is captured, and when the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination.
Therefore, the smoking determination device 1, 1a can prevent erroneous determination that a non-smoker is smoking.

Also, for example, the information for determining a smoker is information for personal authentication, and in the smoking determination device 1, 1a, the smoker determination unit 11 performs personal authentication of the subject based on the information for personal authentication, and if the subject who has been personally authenticated is registered in a smoker registration list in which smokers are registered, it determines that the subject is a smoker.
When the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 performs a smoking determination as to whether the subject is smoking or not based on a captured image in which at least the face of the subject is captured, and when the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination.
Therefore, the smoking determination device 1, 1a can prevent erroneous determination that a non-smoker is smoking.

Also, for example, the information for determining whether a person is a smoker is nicotine dependence information that indicates the subject's level of nicotine dependence, and in the smoking determination device 1, 1a, the smoker determination unit 11 determines that the subject is a smoker if it is estimated that the subject has a high level of nicotine dependence based on the nicotine dependence information.
When the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 performs a smoking determination as to whether the subject is smoking or not based on a captured image in which at least the face of the subject is captured, and when the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination.
Therefore, the smoking determination device 1, 1a can prevent erroneous determination that a non-smoker is smoking.

Furthermore, for example, the information for determining whether a smoker is a heat source is heat source information regarding the temperature around the subject's face, and in the smoking determination device 1, 1a, the smoker determination unit 11 determines that the subject is a smoker based on the time-series heat source information if it detects a high-temperature area above the temperature determination threshold around the subject's face for a period of time or longer for high-temperature determination, or if the frequency of occurrence of high-temperature areas around the subject's face is above the high-temperature occurrence frequency determination threshold.
When the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 performs a smoking determination as to whether the subject is smoking or not based on a captured image in which at least the face of the subject is captured, and when the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination.
Therefore, the smoking determination device 1, 1a can prevent erroneous determination that a non-smoker is smoking.

Also, for example, the information for determining whether a person is a smoker is breathing information of the subject, and in the smoking determination device 1, 1a, the smoker determination unit 11 detects smoking breaths that are estimated to be breathing associated with smoking behavior based on time-series breathing information, and if the number of smoking breaths during the smoking determination period is equal to or greater than the smoking breath determination threshold, determines that the subject is a smoker.
When the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 performs a smoking determination as to whether the subject is smoking or not based on a captured image in which at least the face of the subject is captured, and when the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination.
Therefore, the smoking determination device 1, 1a can prevent erroneous determination that a non-smoker is smoking.

Furthermore, according to embodiment 1, the smoking determination device 1a includes, in addition to the smoker determination unit 11 and the smoking determination unit 12, a mask wearing determination unit 14 that determines whether or not the subject is wearing a mask based on the captured image, and if the mask wearing determination unit 14 determines that the subject is wearing a mask, the smoking determination unit 12a does not make a smoking determination regardless of whether or not the smoker determination unit 11 has determined that the subject is a smoker.
This allows the smoking determination device 1a to prevent erroneous determination that a non-smoker is smoking, and also to prevent unnecessary smoking determination from being made when the subject is wearing a mask.

Furthermore, according to embodiment 1, the smoking determination device 1, 1a can be configured to include an alarm control unit 13 that outputs alarm output control information for outputting an alarm when the smoking determination unit 12 determines that the subject is smoking.
As a result, the smoking determination device 1, 1a can notify the subject or a person other than the subject that the subject is smoking.

Note that this disclosure allows for modifications to any of the components of the embodiments or the omission of any of the components of the embodiments.

The smoking determination device disclosed herein can prevent non-smokers from being mistakenly determined to be smoking.

1, 1a Smoking determination device, 11 Smoker determination unit, 12, 12a Smoking determination unit, 13 Alarm control unit, 14 Mask wearing determination unit, 2, 3a Camera, 3 Information acquisition device for smoker determination, 3b Smoker setting device, 3c Information acquisition device for personal authentication, 3d Nicotine dependence determination device, 3e Temperature measurement device, 3f Breath detection device, 1001 Processing circuit, 1002 Input interface device, 1003 Output interface device, 1004 Processor, 1005 Memory.

Claims (12)

a smoker determination unit that determines whether a subject, whose smoking status is to be determined, is a smoker based on smoker determination information for determining whether the subject is a smoker;
and a smoking determination unit that, when the smoker determination unit determines that the subject is a smoker, makes a smoking determination as to whether the subject is smoking based on an image in which at least the face of the subject is captured, and does not make the smoking determination when the smoker determination unit determines that the subject is a non-smoker. the smoker determination information is the captured image,
The smoking determination device of claim 1, characterized in that, if the subject's mouth is not detected at a mouth detection cycle during a smoking determination period based on the captured image, the smoker determination unit calculates a mouth non-detection time during which the subject's mouth was not detected during the smoking determination period, and if the mouth non-detection time is equal to or greater than a mouth non-detection time determination threshold, determines that the subject is a smoker. The information for determining whether the subject is a smoker is smoker setting information that indicates whether the subject is a smoker or a non-smoker, and is generated based on input of setting information by the subject that causes the subject to set whether the subject is a smoker or a non-smoker,
The smoking determination device according to claim 1 , wherein the smoker determination unit determines that the subject is a smoker if the subject is set as the smoker in the smoker setting information. the smoker determination information is personal authentication information,
The smoking determination device of claim 1, characterized in that the smoker determination unit performs personal authentication of the subject based on the personal authentication information, and determines that the subject is a smoker if the subject who has been personally authenticated is registered in a smoker registration list in which the smoker is registered. the information for determining whether the subject is a smoker is nicotine dependence information indicating a degree of nicotine dependence of the subject,
The smoking determination device according to claim 1 , wherein the smoker determination unit determines that the subject is a smoker when the subject is estimated to have a high level of nicotine dependence based on the nicotine dependence information. the smoker determination information is heat source information relating to the temperature around the face of the subject,
The smoking determination device of claim 1, characterized in that the smoker determination unit determines that the subject is a smoker if, based on the time-series heat source information, a high-temperature area above a temperature determination threshold is detected around the face of the subject for a period of time or longer for high-temperature determination, or if the frequency of appearance of the high-temperature area around the face of the subject is above a high-temperature appearance frequency threshold. the information for determining whether the smoker is a person is respiratory information of the person;
The smoking determination device according to claim 1, characterized in that the smoker determination unit detects smoking breaths that are estimated to be breaths associated with smoking behavior based on the time-series breathing information, and determines that the subject is a smoker if the number of smoking breaths during a smoking determination period is equal to or greater than a smoking breath determination threshold. a mask wearing determination unit that determines whether the subject is wearing a mask based on the captured image;
8. The smoking determination device according to claim 1, wherein the smoking determination unit does not perform the smoking determination if the mask wearing determination unit determines that the subject is wearing the mask, regardless of whether the smoker determination unit determines that the subject is a smoker. 8. The smoking determination device according to claim 1 , further comprising: an alarm control unit that outputs alarm output control information for outputting an alarm when the smoking determination unit determines that the subject is smoking. The smoking determination device according to any one of claims 1 to 7 , wherein the subject is a driver of a vehicle. The smoking determination unit makes the smoking determination based on a machine learning model that receives the captured image in which at least the face of a person is captured as an input and outputs information indicating whether the person is smoking, and the captured image in which at least the face of the subject is captured. a step in which the smoker determination unit determines whether or not a subject is a smoker based on smoker determination information for determining whether or not the subject is a smoker;
a step of, when the smoker determination unit determines that the subject is a smoker, making a smoking determination as to whether the subject is smoking based on an image in which at least the face of the subject is captured, and not making the smoking determination when the smoker determination unit determines that the subject is a non-smoker.