JP7554021B2 - In-cabin monitoring system and vehicle interior detection method - Google Patents

Description

This disclosure relates to an in-cabin monitoring system that monitors the interior of a vehicle, and an in-cabin detection method.

Conventionally, an in-cabin monitoring system that monitors the interior of a vehicle is known. As an example of this type of in-cabin monitoring system, Patent Document 1 discloses a technology for detecting an object based on the change in shading of an image when a predetermined amount of light is irradiated onto the object and an image is captured, and the change in shading of an image when an amount of light smaller than the predetermined amount of light is irradiated onto the object and an image is captured. This technology makes it possible to detect an object even when it is irradiated with disturbance light.

JP 2006-155483 A

However, the disturbance light irradiated into the vehicle interior varies depending on the weather, season, time of day, vehicle orientation, and vehicle interior. Therefore, the technology described in Patent Document 1 requires analysis of changes in shading corresponding to various disturbance lights, which makes it difficult to detect the interior of the vehicle.

The present disclosure aims to provide an in-cabin monitoring system that can easily detect the inside of a vehicle.

In order to achieve the above-mentioned object, an in-cabin monitoring system according to one embodiment of the present disclosure includes a camera that captures an image of the interior of a vehicle cabin, a light source that irradiates light into the interior of the vehicle cabin, and a control unit that controls the camera and the light source, wherein the light source is capable of emitting a first light that is emitted when the camera is used to check for the presence or absence of a person, and a second light that is stronger than the first light, and the control unit captures an image of the interior of the vehicle cabin with the camera in a state in which the light source is emitting the second light before a person gets into the vehicle and after a person gets out of the vehicle, and further, when the control unit receives a door unlock signal for the vehicle or a door lock signal for the vehicle, the control unit captures an image of the interior of the vehicle cabin with the camera in a state in which the light source is emitting the first light before emitting the second light, and determines the presence or absence of a person in the vehicle cabin.
In order to achieve the above-mentioned object, an in-cabin monitoring system according to one embodiment of the present disclosure includes a camera that captures images of the interior of a vehicle's cabin, a light source that irradiates light into the interior of the vehicle cabin, a control unit that controls the camera and the light source, and a memory unit that stores multiple images captured in the past, wherein the light source is capable of emitting a first light that is emitted when the camera is used to check for the presence or absence of a person, and a second light that is stronger than the first light, and the control unit captures images of the interior of the vehicle cabin with the camera while the light source is emitting the second light before a person gets into the vehicle and after a person gets out of the vehicle, and further, the control unit selects an image from the multiple images stored in the memory unit that has a layout that is closest to the layout of the interior of the vehicle cabin in the currently captured image, and determines the difference in the state of objects placed in the vehicle cabin by comparing the selected image with the currently captured image.

In order to achieve the above-mentioned object, an interior detection method according to one aspect of the present disclosure is an interior detection method for detecting the interior of a vehicle, the method including the steps of: capturing an image of the interior of the vehicle cabin by irradiating a second light stronger than a first light irradiated when checking for the presence or absence of a person with a camera before a person gets into the vehicle; capturing an image of the interior of the vehicle cabin by irradiating the second light stronger than the first light irradiated when checking for the presence or absence of a person with the camera after a person has disembarked from the vehicle; comparing an image of the interior of the vehicle cabin captured before the person gets into the vehicle with an image of the interior of the vehicle cabin captured after the person has disembarked from the vehicle; and, when a door unlock signal or a door lock signal for the vehicle is received, capturing an image of the interior of the vehicle cabin with the camera in a state in which a light source that irradiates light into the vehicle cabin is caused to emit the first light before emitting the second light, and determining the presence or absence of a person in the vehicle cabin .

These comprehensive or specific aspects may be realized as a system, method, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, method, integrated circuit, computer program, and recording medium.

The in-cabin monitoring system disclosed herein allows for easy detection of the interior of the vehicle.

1 is a schematic diagram of an in-cabin monitoring system according to an embodiment, viewed from the side inside a vehicle cabin. FIG. 2 is a diagram showing a camera and a light source provided in the in-cabin monitoring system according to the embodiment. 1 is a block diagram showing an in-cabin monitoring system according to an embodiment. FIG. 3A and 3B are diagrams illustrating light emitted by a light source of the in-cabin monitoring system according to the embodiment. 4 is a flowchart showing a vehicle interior detection method according to an embodiment.

In today's world where rental car businesses and car sharing are becoming more common, there is a demand for technology that can detect items left behind in the vehicle, such as mobile phones and wallets, as well as dirt on the seats. In order to detect items left behind or dirt in the vehicle, it is conceivable to use an in-cabin monitoring system, for example, to detect items inside the vehicle.

One example of a method for detecting an object is a conventional technology that detects an object based on the change in shading of an image when a predetermined amount of light is irradiated onto the object and an image is captured, and the change in shading of an image when an amount of light smaller than the predetermined amount of light is irradiated onto the object and an image is captured. With this technology, it is possible to detect an object even when disturbance light is irradiated onto the object.

However, the disturbance light shining into the vehicle interior varies depending on the weather, season, time of day, vehicle orientation, and vehicle interior. Therefore, conventional technology requires analysis of changes in light and shade corresponding to various disturbance lights, which complicates detection of the interior of the vehicle.

The in-cabin monitoring system disclosed herein has the following configuration to reduce the effects of external light and enable easy detection of the interior of the vehicle.

An in-cabin monitoring system according to one aspect of the present disclosure includes a camera that captures an image of the interior of a vehicle cabin, a light source that irradiates light into the interior of the vehicle cabin, and a control unit that controls the camera and the light source. The light source is capable of emitting a first light that is emitted when the camera checks for the presence or absence of a person, and a second light that is stronger than the first light. The control unit causes the camera to capture an image of the interior of the vehicle cabin before a person gets in the vehicle and after a person gets out of the vehicle, with the light source emitting the second light.

In this way, by capturing an image of the interior of the vehicle while emitting the second light that is stronger than the first light, it is possible to suppress the effects of external light and detect the interior of the vehicle. In addition, by capturing an image of the interior of the vehicle before a person gets in and after a person gets out, it is possible to easily detect the difference between the interior of the vehicle before and after a person gets out.

The control unit may also capture an image of the interior of the vehicle cabin with the camera when a door unlock signal for the vehicle is received before a person gets into the vehicle, and capture an image of the interior of the vehicle cabin with the camera when a door lock signal for the vehicle is received after a person gets out of the vehicle.

This allows images of the interior of the vehicle to be captured when the door is unlocked before getting in and when the door is locked after getting out. This makes it easy to detect the difference between the interior of the vehicle before getting in and after getting out.

The control unit may also determine the difference in the state of objects placed within the vehicle interior by comparing an image captured within the vehicle interior upon receiving the door unlock signal with an image captured within the vehicle interior upon receiving the door lock signal.

This makes it easy to detect differences in the interior of the vehicle before and after boarding.

The difference in the state of the object placed in the vehicle interior may be at least one of the presence or absence of an object placed in the vehicle interior and the presence or absence of damage or dirt on the object placed in the vehicle interior.

This makes it possible to detect whether or not there are objects placed inside the vehicle cabin, and whether or not the objects placed inside the vehicle cabin are damaged or dirty.

The control unit may also output an abnormality signal if it determines that the condition of an object placed in the vehicle cabin is different.

This allows the vehicle user to be notified that there is something wrong with the condition of an item placed inside the vehicle cabin.

In addition, when the control unit receives the door unlock signal or the door lock signal, the control unit may image the interior of the vehicle cabin with the camera while the light source is emitting the first light before emitting the second light, and determine whether or not a person is present in the vehicle cabin.

In this way, by determining whether or not a person is present before emitting the second light, it is possible to avoid irradiating people with the strong second light.

The control unit may also not cause the light source to emit the second light when it determines that there is a person in the vehicle cabin, and may cause the light source to emit the second light and capture an image of the interior of the vehicle cabin with the camera when it determines that there is no person in the vehicle cabin.

This makes it possible to prevent the second light from being irradiated onto people. Also, by emitting the second light, which is a strong light, and capturing an image, it is possible to capture an image of the interior of the vehicle while suppressing the effects of external light.

The in-cabin monitoring system may further include a memory unit that stores a plurality of images captured in the past, and the control unit may select an image from the plurality of images stored in the memory unit that has a layout closest to the layout of the interior of the vehicle cabin of the currently captured image, and may determine the difference in the state of objects placed within the vehicle cabin by comparing the selected image with the currently captured image.

As a result, even if the layout of the vehicle interior has been changed by the user, differences in the state of objects placed within the vehicle interior can be easily detected using an image with the interior layout that is closest to the captured image.

The control unit may also determine whether or not maintenance is required within the vehicle interior by comparing images captured in the past with images currently captured using the camera.

In this way, by comparing images captured in the past with images captured currently, it is possible to appropriately determine whether maintenance is required inside the vehicle.

The camera and the light source may be provided on the ceiling, overhead console, rearview mirror, dashboard, steering column, rearview mirror, or A-pillar, B-pillar, C-pillar, or D-pillar inside the vehicle.

This allows the camera and light source to be used to properly capture images of the state of objects placed inside the vehicle cabin. This allows the system to properly detect differences in the state of objects placed inside the vehicle cabin.

A vehicle interior detection method according to one aspect of the present disclosure is a vehicle interior detection method for detecting the interior of a vehicle, and includes the steps of capturing an image of the interior of the vehicle interior by irradiating a light stronger than the light irradiated when checking for the presence or absence of a person with a camera before a person gets into the vehicle, capturing an image of the interior of the vehicle interior by irradiating a light stronger than the light irradiated when checking for the presence or absence of a person with the camera after a person has left the vehicle, and comparing an image of the interior of the vehicle interior captured before the person gets into the vehicle with an image of the interior of the vehicle interior captured after the person has left the vehicle.

In this way, by capturing an image of the interior of the vehicle using light that is stronger than the light used to check for the presence of occupants, the effects of external light can be suppressed and the interior of the vehicle can be detected. In addition, by comparing the images of the interior of the vehicle before a person gets in and after a person gets out, it is possible to easily detect the differences between the interior of the vehicle before and after a person gets out.

The following describes in detail the embodiments of the present disclosure with reference to the drawings. Note that the embodiments described below are examples, and the present disclosure is not limited to these embodiments. In other words, the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, and order of steps shown in the following embodiments are examples, and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in an independent claim that indicates a superordinate concept are described as optional components.

The figures are schematic diagrams and are not necessarily precise illustrations. In each figure, the same components are given the same reference numerals.

(Embodiment)
[In-cabin monitoring system configuration]
The configuration of an in-cabin monitoring system according to an embodiment will be described with reference to FIGS. 1 to 3. FIG.

Figure 1 is a schematic diagram of an in-cabin monitoring system 1 according to an embodiment, viewed from the side inside a vehicle compartment 2. Figure 2 is a diagram showing a camera 20 and a light source 30 provided in the in-cabin monitoring system 1. Figure 3 is a block diagram showing the configuration of the in-cabin monitoring system 1.

As shown in Figures 1 to 3, the in-cabin monitoring system 1 includes a camera 20, a light source 30, and a control unit 50 that controls the camera 20 and the light source 30. The in-cabin monitoring system 1 also includes a housing 10.

The housing 10 is provided in the center of the ceiling 7 inside the vehicle interior 2. The housing 10 is a resin case and has an opening 11. The camera 20 and the light source 30 are disposed in the opening 11 of the housing 10, and the control unit 50 is disposed inside the housing 10. The housing 10 may be provided in an overhead console located above the windshield 4. The housing 10 may also be provided in a room mirror, a dashboard, a steering column, a rearview mirror, or an A-pillar, B-pillar, C-pillar, or D-pillar. In other words, the camera 20 and the light source 30 may be provided in a position where they can capture an image of the occupant 3 or the seat.

The light source 30 is a device that irradiates light into the vehicle interior 2. The light source 30 has a plurality of light-emitting elements that emit visible light or near-infrared light. The light source 30 is provided in the opening 11 of the housing 10, and is arranged in a direction that allows the light source 30 to irradiate the seat, the foot floor, and the occupant 3 with visible light or near-infrared light. The light source 30 is capable of emitting a first light L1 and a second light L2.

Figure 4 is a diagram showing the light emitted by the light source 30 of the in-cabin monitoring system 1. Figure 4 (a) shows the light source 30 emitting the first light L1, and Figure 4 (b) shows the light source 30 emitting the second light L2. Figures 4 (a) and (b) show a state in which there is nothing left behind in the vehicle compartment 2 and no occupant 3 is present.

The light source 30 emits the first light L1 when the camera 20 checks for the presence or absence of a person in the vehicle interior 2. The intensity of the first light L1 is usually the intensity emitted when capturing an image of a person. The illuminance of the light irradiated by the first light L1 is, for example, 100 lux or more and less than 2000 lux for visible light, and 0.1 W/m ² or more and less than 1 W/m ² for near-infrared light with a wavelength of 940 nm. In addition, the light source 30 emits the second light L2 to suppress the influence of disturbance light when capturing an image of the vehicle interior 2 with the camera 20. The second light L2 is light that is stronger (light with a stronger emission intensity) than the first light L1. The illuminance of the light irradiated by the second light L2 is, for example, 2000 lux or more and 100,000 lux or less for visible light, and 1 W/m ² or more and 10 W/m ² or less for near-infrared light with a wavelength of 940 nm. The light source 30 emits the first light L1 or the second light L2 based on a light emission command from the control unit 50, which will be described later.

The camera 20 is a device that captures images of the interior of the vehicle cabin 2 or the occupant 3. The camera 20 is provided in the opening 11 of the housing 10. The camera 20 is positioned so that the seat, the floor under the feet, and the occupant 3 are located within the lens angle of view α of the camera 20, that is, in an orientation that allows the seat, the floor under the feet, and the occupant 3 to be captured.

The camera 20 is, for example, a visible light camera that detects visible light, a near-infrared light camera that detects near-infrared light, or a camera that can detect both. A GS (global shutter) type CMOS image sensor or CCD image sensor can be used as the camera 20. The camera 20 captures an image of the interior of the vehicle compartment 2 or the occupant 3 based on an image capture command from the control unit 50. The control unit 50 can capture an image of the interior of the vehicle compartment 2 while suppressing the effects of ambient light, for example, by shortening the emission time of the second light L2, which is a strong light, and the exposure time of the camera 20.

The control unit 50 is composed of a processor such as a CPU (Central Processing Unit), a storage unit 53 having a volatile memory and a non-volatile memory, and a program stored in the storage unit 53. The functional configuration of the control unit 50 is realized by executing the above-mentioned program. The control unit 50 is provided inside the housing 10, but is not limited to this and may be provided outside the housing 10. The control unit 50 controls the operation of the camera 20, the light source 30, etc. while communicating with an ECU (Electronic Control Unit) 90 of the vehicle 9.

The control unit 50 acquires a door unlock signal s1 and a door lock signal s2, which are signals for opening and closing the lock of the doors of the vehicle 9. The door unlock signal s1 is input to the control unit 50 before a person gets into the vehicle 9. The door lock signal s2 is input to the control unit 50 after a person gets out of the vehicle 9. For example, the door unlock signal s1 and the door lock signal s2 are input to the ECU 90 by an operation input by the user of the vehicle 9, and are input to the control unit 50 via the ECU 90. The door unlock signal s1 and the door lock signal s2 acquired by the control unit 50 may be the signal itself received by the ECU 90, or may be a signal output from the ECU 90 in conjunction with the signal received by the ECU 90.

In this embodiment, the second light L2 is used to capture an image of the interior of the vehicle compartment 2 before a person gets into the vehicle 9 and after the person gets out of the vehicle 9. However, the control unit 50 checks whether there is a person in the vehicle compartment 2 before emitting the second light L2 to avoid exposing people to too much strong light. Specifically, the control unit 50 captures an image of the interior of the vehicle compartment 2 with the camera 20 while emitting the first light L1, which is weaker than the second light L2. The control unit 50 determines whether there is a person in the vehicle compartment 2 based on the image captured with the first light L1. Determining the presence or absence of a person based on an image can be achieved using well-known technology.

When the control unit 50 determines that there is a person in the vehicle compartment 2, it does not cause the light source 30 to emit the second light L2, and does not capture an image of the interior of the vehicle compartment 2. When the control unit 50 determines that there is no person in the vehicle compartment 2 based on the image captured with the first light L1, it causes the light source 30 to emit the second light L2, and captures an image of the interior of the vehicle compartment 2 with the camera 20.

For example, before a person gets into the vehicle 9 and when the control unit 50 receives a door lock release signal s1, the control unit 50 causes the light source 30 to emit the second light L2 and captures an image of the interior of the vehicle compartment 2 with the camera 20. Also, after a person gets out of the vehicle 9 and when the control unit 50 receives a door lock signal s2, the control unit 50 causes the light source 30 to emit the second light L2 and captures an image of the interior of the vehicle compartment 2 with the camera 20.

The control unit 50 determines the difference in the state of the objects placed in the vehicle interior 2 by comparing a first image im1 taken when the vehicle interior 2 is imaged upon receipt of the door unlock signal s1 with a second image im2 taken when the vehicle interior 2 is imaged upon receipt of the door lock signal s2. Specifically, the control unit 50 determines the difference in the state of the objects placed in the vehicle interior 2 by obtaining the difference between the first image im1 and the second image im2 (for example, the difference in color, brightness, etc. of multiple pixels constituting the image).

The difference in the state of the object is at least one of the presence or absence of an object placed in the vehicle interior 2, and the presence or absence of damage or dirt on the object placed in the vehicle interior 2. The object placed in the vehicle interior 2 may be an object that has been brought into the vehicle interior 2 from outside, such as a mobile phone, wallet, or trash, or it may be an object that has been placed in advance in the vehicle interior 2, such as a seat or the floor under the feet. Note that in this embodiment, it is not necessary to specify the name of the object, and it is sufficient to determine the difference in the state of the object based on at least whether there is a difference between images im1 and im2.

If the control unit 50 determines that the condition of an object placed in the vehicle compartment 2 is different, it outputs an abnormality signal indicating that something is wrong and notifies the user of the vehicle 9.

In the in-cabin monitoring system 1 of this embodiment, the light source 30 can emit a first light L1 that is emitted when the camera 20 checks for the presence or absence of a person, and a second light L2 that is stronger than the first light L1. Before a person gets on the vehicle 9 and after a person gets off the vehicle 9, the light source 30 is made to emit the second light L2, and the camera 20 captures an image of the interior of the vehicle compartment 2.

In this way, by capturing an image of the interior of the vehicle compartment 2 while emitting the second light L2 that is stronger than the first light L1, it is possible to suppress the effects of external light and detect the interior of the vehicle compartment 2. In addition, by capturing an image of the interior of the vehicle compartment 2 before a person gets in and after a person gets out, it is possible to easily detect the difference in the interior of the vehicle compartment 2 before and after a person gets out.

[In-vehicle detection method]
The vehicle interior detection method according to the embodiment will be described with reference to FIG.

Figure 5 is a flowchart showing an interior vehicle detection method according to an embodiment.

Here, an example of detecting the state of objects in the vehicle compartment 2 using the in-cabin monitoring system 1 will be described. Detection of the state of objects in the vehicle compartment 2 is performed when the vehicle 9 is parked.

The in-cabin monitoring system 1 receives a door unlock signal s1 before a person gets into the vehicle 9 (step S11). For example, the control unit 50 receives the door unlock signal s1 via the ECU 90 in response to a key operation by the user.

When the control unit 50 receives the door lock release signal s1, it checks whether or not there is a person in the vehicle compartment 2. Specifically, the control unit 50 causes the light source 30 to emit the first light L1 to irradiate the interior of the vehicle compartment 2, and captures an image of the interior of the vehicle compartment 2 with the camera 20 (step S12). The emission time of the first light L1 is, for example, 0.1 ms or more and 10 ms or less.

The control unit 50 uses the image acquired when the first light L1 is emitted to determine whether or not there is a person in the vehicle compartment 2 (step S13).

When the control unit 50 determines that there is a person in the vehicle compartment 2 (Yes in S13), it does not cause the light source 30 to emit the second light L2, does not capture an image of the inside of the vehicle compartment 2, and returns to step S11. At this time, the control unit 50 outputs a signal to the ECU 90 urging the person in the vehicle compartment 2 to leave the vehicle compartment 2, and the ECU 90 may notify the person in the vehicle compartment 2 to leave the vehicle compartment 2 using an audio device or the like in the vehicle compartment 2.

When the control unit 50 determines that no one is present in the vehicle compartment 2 (No in S13), it causes the light source 30 to emit the second light L2 to irradiate the interior of the vehicle compartment 2, and captures an image of the interior of the vehicle compartment 2 with the camera 20 (step S14). The emission time of the second light L2 is, for example, 0.1 ms or more and 10 ms or less.

The control unit 50 stores the image acquired when the second light L2 is emitted as the first image im1 in the storage unit 53 (step S15). The first image im1 is the first still image captured during the period of use of the vehicle 9, and the capture date and time are also stored in the storage unit 53.

Next, we will explain what happens after the user has finished driving vehicle 9.

The in-cabin monitoring system 1 receives a door lock signal s2 after the person exits the vehicle 9 (step S16). For example, the control unit 50 receives the door lock signal s2 via the ECU 90 in response to a key operation by the user.

When the control unit 50 receives the door lock signal s2, it checks whether or not there is a person in the vehicle compartment 2. Specifically, the control unit 50 causes the light source 30 to emit the first light L1 to irradiate the interior of the vehicle compartment 2, and captures an image of the interior of the vehicle compartment 2 with the camera 20 (step S17). The emission time of the first light L1 is, for example, 0.1 ms or more and 10 ms or less.

The control unit 50 uses the image acquired when the first light L1 is emitted to determine whether or not there is a person in the vehicle compartment 2 (step S18).

If the control unit 50 determines that there is a person in the vehicle compartment 2 (Yes in S18), it does not cause the light source 30 to emit the second light L2, does not capture an image of the inside of the vehicle compartment 2, and returns to step S16. At this time, the control unit 50 outputs a signal to the ECU 90 urging the person in the vehicle compartment 2 to leave the vehicle compartment 2, and the ECU 90 may notify the person in the vehicle compartment 2 to leave the vehicle compartment 2 using audio equipment or the like in the vehicle compartment 2.

When the control unit 50 determines that no one is present in the vehicle compartment 2 (No in S18), it causes the light source 30 to emit the second light L2 to irradiate the interior of the vehicle compartment 2, and captures an image of the interior of the vehicle compartment 2 with the camera 20 (step S19). The emission time of the second light L2 is, for example, 0.1 ms or more and 10 ms or less.

The control unit 50 stores the image acquired when the second light L2 is emitted as the second image im2 in the memory unit 53 (step S20). The second image im2 is the last still image captured during the period of use of the vehicle 9, and the capture date and time are also stored in the memory unit 53.

Next, the control unit 50 compares the first image im1 with the second image im2 and determines whether there is a difference in the state of the objects inside the vehicle compartment 2 (step S21).

If the control unit 50 determines that there is a difference in the state of the items in the vehicle interior 2 (Yes in S21), it outputs an abnormality signal indicating that there is an abnormality to the ECU 90 (step S22). If the control unit 50 determines that there is no difference in the state of the items in the vehicle interior 2 (No in S21), it outputs a normal signal indicating that there is no abnormality to the ECU 90 (step S23). Information based on the abnormality signal and the normal signal is presented to the user's communication terminal, for example, by Internet communication or by communication using Bluetooth (registered trademark).

By executing these steps S11 to S23, the vehicle interior detection method in the in-cabin monitoring system 1 is realized.

The vehicle interior detection method of this embodiment includes the steps of capturing an image of the interior of the vehicle interior 2 by irradiating a light stronger than the light irradiated by the camera 20 when checking for the presence or absence of a person before a person gets into the vehicle 9, capturing an image of the interior of the vehicle interior 2 by irradiating a light stronger than the light irradiated by the camera 20 when checking for the presence or absence of a person after a person gets out of the vehicle 9, and comparing an image im1 captured when the interior of the vehicle interior 2 is captured before the person gets into the vehicle 9 with an image im2 captured when the interior of the vehicle interior 2 is captured after the person gets out of the vehicle 9.

In this way, by capturing an image of the interior of the vehicle compartment 2 using light that is stronger than the light used to check for the presence of a person, it is possible to suppress the effects of external light and perform detection inside the vehicle compartment 2. In addition, by comparing the images of the interior of the vehicle compartment 2 before a person gets in and after a person gets out, it is possible to easily detect the differences inside the vehicle compartment 2 before and after a person gets out.

Next, we will explain a variation of the embodiment.

Although the first image im1 and the second image im2 are stored in the memory unit 53, the images stored in the memory unit 53 are not limited to these. For example, the memory unit 53 may accumulate and store a plurality of images captured in the past, such as images captured when the vehicle was new. The plurality of images may include images of the interior of the vehicle compartment 2 in a state where the layout has been changed, such as the position of the seats or the angle of the backrest. The control unit 50 may then select, from the plurality of images stored in the memory unit 53, an image having a layout that is closest to the layout of the interior of the vehicle compartment 2 in the currently captured image, and determine the difference in the state of the objects placed in the vehicle compartment 2 by comparing the selected image with the currently captured image.

The control unit 50 determines the difference in the state of the objects inside the passenger compartment 2 based on the images captured at the beginning and end of the usage period of the vehicle 9, but is not limited to this. For example, the control unit 50 may determine whether or not maintenance is required inside the passenger compartment 2 by comparing images captured in the past using the camera 20 with images captured currently. Images captured in the past are, for example, images captured when the vehicle was new. Maintenance inside the passenger compartment 2 is, for example, repair or replacement of seat covers, floor mats, etc. The control unit 50 may notify the user of the result of the determination of whether or not maintenance is required.

(Other embodiments)
As described above, the embodiments and the modified examples of the embodiments have been described as examples of the technology disclosed in this application. However, the technology in the embodiments and the modified examples of the embodiments is not limited to this, and can be applied to embodiments in which modifications, replacements, additions, omissions, etc. are appropriately made. In addition, it is also possible to combine the components described in each of the above embodiments to create a new embodiment.

In the above embodiment, an example has been described in which the control unit 50 captures an image of the interior of the vehicle compartment 2 with the camera 20 when it receives the door unlock signal s1 and the door lock signal s2, respectively, but this is not limited to the example. For example, the control unit 50 may receive a pre-imaging instruction signal instead of the door unlock signal s1 and a post-imaging instruction signal instead of the door lock signal s2, and capture an image of the interior of the vehicle compartment 2 with the camera 20 when it receives each of these signals.

In the above embodiment, an example has been shown in which the camera 20 and the light source 30 are provided on the ceiling 7, but this is not limiting. The camera 20 and the light source 30 may be provided on at least one of the ceiling 7, the overhead console, the room mirror, the dashboard, the steering column, the rearview mirror, and the A-pillar, B-pillar, C-pillar, and D-pillar in the vehicle interior 2. A plurality of light sources 30 may be provided in the vehicle interior 2. A plurality of cameras 20 may be provided in the vehicle interior 2.

Although the embodiments of the present disclosure have been described in detail with reference to the drawings, the functions of the above-mentioned devices and each processing unit can be realized by a computer program.

A computer that realizes the above-mentioned functions through a program includes an input device such as a touchpad, an output device such as a display or speaker, a processor or CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a storage device such as a hard disk drive or SSD (Solid State Drive), a reading device that reads information from recording media such as a DVD-ROM (Digital Versatile Disk Read Only Memory) or USB (Universal Serial Bus) memory, and a network card that communicates via a network, and each part is connected by a bus.

The reading device then reads the program from the recording medium on which it is recorded and stores it in the storage device. Alternatively, the network card communicates with a server device connected to the network, and stores in the storage device the program for implementing the functions of each of the above devices that has been downloaded from the server device.

The processor or CPU then copies the program stored in the storage device to the RAM, and sequentially reads and executes the instructions contained in the program from the RAM, thereby realizing the functions of each of the above devices.

The in-cabin monitoring system disclosed herein can be used to monitor the interior of a vehicle.

REFERENCE SIGNS LIST 1 In-cabin monitoring system 2 Vehicle compartment 3 Occupant 4 Windshield 7 Ceiling 9 Vehicle 10 Housing 11 Opening 20 Camera 30 Light source 50 Control unit 53 Memory unit 90 ECU
im1, im2: image L1: first light L2: second light s1: door lock release signal s2: door lock signal α: lens angle of view

Claims (10)

A camera for capturing an image of the interior of a vehicle;
A light source that irradiates light into the vehicle interior;
A control unit that controls the camera and the light source;
Equipped with
The light source is capable of emitting a first light that is emitted when the camera is used to check for the presence or absence of a person, and a second light that is stronger than the first light,
the control unit captures an image of the interior of the vehicle cabin with the camera in a state in which the light source emits the second light before a person gets in the vehicle and after a person gets out of the vehicle ,
Furthermore, when the control unit receives a door lock release signal or a door lock signal of the vehicle, the control unit captures an image of the interior of the vehicle compartment with the camera in a state in which the light source emits the first light before emitting the second light, and determines whether or not a person is present in the vehicle compartment.
In-cabin monitoring system. 2. The in-cabin monitoring system according to claim 1, wherein the control unit captures an image of the interior of the vehicle cabin with the camera when the door lock release signal is received before a person gets into the vehicle, and captures an image of the interior of the vehicle cabin with the camera when the door lock signal is received after a person gets out of the vehicle. 3. The in-cabin monitoring system according to claim 2, wherein the control unit determines a difference in a state of an object placed in the vehicle interior by comparing an image captured in response to the door unlock signal with an image captured in response to the door lock signal. The in-cabin monitoring system according to claim 3 , wherein the difference in the state of the object placed in the vehicle interior is at least one of the presence or absence of an object placed in the vehicle interior and the presence or absence of damage or dirt on the object placed in the vehicle interior. The in-cabin monitoring system according to claim 3 or 4, wherein the control unit outputs an abnormality signal when it determines that a state of an object placed in the vehicle compartment is different. 2. The in-cabin monitoring system according to claim 1, wherein the control unit, when determining that there is a person in the vehicle cabin, does not cause the light source to emit the second light, and, when determining that there is no person in the vehicle cabin, causes the light source to emit the second light and captures an image of the interior of the vehicle cabin with the camera . A camera for capturing an image of the interior of a vehicle;
A light source that irradiates light into the vehicle interior;
A control unit that controls the camera and the light source;
A storage unit for storing a plurality of images captured in the past;
Equipped with
The light source is capable of emitting a first light that is emitted when the camera is used to check for the presence or absence of a person, and a second light that is stronger than the first light,
the control unit captures an image of the interior of the vehicle cabin with the camera in a state in which the light source emits the second light before a person gets in the vehicle and after a person gets out of the vehicle ,
Furthermore, the control unit selects an image having a layout closest to the layout of the interior of the vehicle interior of the currently captured image from the plurality of images stored in the storage unit, and compares the selected image with the currently captured image to determine a difference in the state of the objects placed in the vehicle interior.
In-cabin monitoring system. The in-cabin monitoring system according to any one of claims 1 to 7, wherein the control unit determines whether or not maintenance is required within the vehicle cabin by comparing an image captured in the past using the camera with an image currently captured. The in-cabin monitoring system according to any one of claims 1 to 8 , wherein the camera and the light source are provided on a ceiling, an overhead console, a room mirror, a dashboard, a steering column, a rearview mirror, or an A-pillar, a B-pillar, a C-pillar, or a D-pillar inside the vehicle. A vehicle interior detection method for detecting an interior of a vehicle, comprising:
a step of capturing an image of the interior of the vehicle by irradiating a second light, the second light being stronger than a first light irradiated when checking for the presence or absence of a person with a camera, before the person gets into the vehicle;
After a person gets off the vehicle, the camera irradiates the vehicle with the second light, which is stronger than the first light, to check for the presence or absence of a person, and captures an image of the interior of the vehicle.
comparing an image captured inside the vehicle cabin before a person gets into the vehicle with an image captured inside the vehicle cabin after the person gets out of the vehicle;
a step of capturing an image of the interior of the vehicle cabin with the camera in a state in which a light source that irradiates light into the vehicle cabin is caused to emit the first light before emitting the second light when a door unlock signal or a door lock signal of the vehicle is received, and determining whether or not a person is present in the vehicle cabin;
A vehicle interior detection method comprising:

Images