JP7683145B2 - Display control device and display control method - Google Patents

Description

The present invention relates to a display control device and a display control method.

A display control technology is known that captures an image of the rear of a vehicle using a camera mounted on the vehicle and displays the captured image on a display device or the like, causing the display device or the like to function as an electronic mirror.

There is also known technology that uses a camera mounted on a vehicle to capture images of passengers in the back seats, and displays the captured images of the passengers on a display device or the like along with an image of the rear of the vehicle (see, for example, Patent Documents 1 and 2).

JP 2019-199153 A JP 2020-095406 A

In the technology disclosed in Patent Document 1, an image of the rear seats is displayed within an image of the rear of the vehicle displayed on a display device or the like. In this case, since the image of the rear seats also includes an image of the rear of the vehicle, it creates a situation where there is a landscape within a landscape, which creates a problem in that it is difficult for users such as the driver to distinguish between the image of the rear of the vehicle and the image of the rear seats.

In the technology disclosed in Patent Document 2, a composite image is displayed on a display device, etc., by combining an image of the rear of the vehicle with an image of the subject, which is an image of the backseat, excluding the background image. In this case, the scenery behind the vehicle can be removed from the image of the backseat, but there is a problem in that it is difficult to distinguish whether a person in the composite image is inside or outside the vehicle.

As such, in conventional image display devices that display images of the outside and inside of a vehicle on a display device, it was difficult for users, such as drivers, to distinguish between images of the outside and inside of the vehicle.

One embodiment of the present invention has been made in consideration of the above problems, and in a display control device that displays images of the outside and inside of the vehicle on a display device, the device allows a user, such as a driver, to easily distinguish between the images of the outside and the images of the inside of the vehicle.

In order to solve the above problems, a display control device according to one embodiment of the present invention has a first imaging unit that generates a first captured image of the rear of a vehicle , a second imaging unit that generates a second captured image of an occupant in a rear seat , and a display control unit that displays the first captured image and the second captured image on a display unit, and when simultaneously displaying the first captured image and the second captured image, the display control unit displays the second captured image within the range of the first captured image displayed on the display unit, changes the color scheme of the background of the occupant in the second captured image, and makes the background of the occupant whose color scheme has been changed transparent so that the first captured image can be viewed.

According to one embodiment of the present invention, in a display control device that displays images of the outside and the inside of a vehicle on a display device, a user such as a driver can easily distinguish between images of the outside and the inside of the vehicle.

FIG. 1 is a diagram illustrating an example of the configuration of a display control system according to an embodiment. FIG. 1 is a diagram showing an image of a conventional rearview mirror. FIG. 13 is a diagram showing an example of a display image that displays a rear image and an image of an occupant. FIG. 2 is a diagram showing an example of a display image displayed by a display device according to an embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a display control device according to an embodiment. 4 is a flowchart illustrating an example of processing of the display control device according to the first embodiment. FIG. 2 is a diagram showing an example of a display image according to the first embodiment; 10 is a flowchart illustrating an example of processing of a display control device according to a second embodiment. 13 is a flowchart showing another example of the process of the display control device according to the second embodiment. FIG. 11 is a diagram showing an example of a display area of a second captured image according to the second embodiment. FIG. 13 is a diagram illustrating an example of the configuration of a display control system according to a third embodiment. 13 is a flowchart illustrating an example of processing of a display control device according to a third embodiment. FIG. 13 is a diagram for explaining an example of a display image according to a third embodiment. FIG. 13 is a diagram illustrating an example of the configuration of a display control system according to a fourth embodiment. 13 is a flowchart illustrating an example of processing of a display control device according to a fourth embodiment. FIG. 13 is a diagram for explaining a second captured image according to the fourth embodiment.

The following describes an embodiment of the present invention with reference to the attached drawings.

<Example of display control system configuration>
Fig. 1 is a diagram showing an example of the configuration of a display control system according to an embodiment. The display control system 1 is a system that, for example, in a vehicle 10 such as an automobile, causes a display device 13 to function as an electronic mirror by displaying a first captured image of the rear of the vehicle 10 and a second captured image of an occupant inside the vehicle 10 on the display device 13. In the example of Fig. 1, the display control system 1 includes a display control device 100, an exterior camera 11, an interior camera 12, and a display device 13, which are mounted on the vehicle 10.

The display control device 100 is, for example, an electronic control device having a computer configuration such as an ECU (Electric Control Unit) mounted on the vehicle 10. The display control device 100 realizes, for example, functional configurations such as a first imaging unit 120, a second imaging unit 130, and a display control unit 110 by executing a predetermined program on the computer provided in the display control device 100. Note that at least a part of the above functional configurations may be realized by hardware.

The first imaging unit 120, for example, controls the exterior camera 11 that captures images of the outside of the vehicle 10 to execute a first imaging process that generates a first captured image of the outside of the vehicle. Preferably, the first imaging unit 120 captures an image of the rear of the vehicle 10 using the exterior camera 11, and generates a first captured image by flipping the captured image horizontally. Note that the process of flipping the captured image horizontally may be executed by the display control unit 110.

The second imaging unit 130 executes a second imaging process, for example, by controlling the in-vehicle camera 12 that captures images of the interior of the vehicle 10 to generate a second captured image of the interior of the vehicle. Preferably, the second imaging unit 130 uses the in-vehicle camera 12 to capture images of passengers in the rear seats of the vehicle 10, and generates a second captured image by flipping the captured image horizontally. Note that the process of flipping the captured image horizontally may be executed by the display control unit 110.

The display control unit 110 generates a display image that simultaneously displays the first captured image and the second captured image, and executes a display control process that causes the display device 13 to function as an electronic mirror (rear-view mirror) by displaying the generated display image on the display device 13.

The display device (display unit) 13 is provided, for example, in a position similar to that of a conventional rearview mirror (indoor rearview mirror), and displays a display image according to control from the display control unit 110. The display device 13 is equipped with a display device such as, for example, a liquid crystal display or an organic EL (Electro Luminescence) display.

Figure 2 is a diagram showing an image of a conventional rearview mirror. A conventional rearview mirror (indoor rearview mirror) 201 is installed, for example, at the upper center inside the front windshield 202 of the vehicle 10, and is used by the driver of the vehicle to check the condition of the rear of the vehicle 203 or passengers in the rear seats of the vehicle 204, etc.

Figure 3 shows an example of a display image that displays an image of the rear and an image of an occupant. The display device 13 is provided, for example, at the top center of the inside of the front windshield 202 of the vehicle 10, and displays a first captured image of the rear of the vehicle 10 and a second captured image of an occupant in the rear seat of the vehicle 10.

In the example of FIG. 3, similar to the technology disclosed in Patent Document 2, a display image 303 is displayed on the display device 13, which is a composite of a first captured image 301 of the rear of the vehicle 10 and an image 302 of an occupant in the back seat, obtained by removing the background image from a second captured image of the occupant. In this case, there is a problem that it is difficult for the driver or the like who drives the vehicle 10 to distinguish whether a person shown in the display image 303 is a person inside the vehicle 10 or a person outside the vehicle 10.

As with the technology disclosed in Patent Document 1, it is also possible to display a second captured image of the passengers in the rear seats of the vehicle 10 in a PinP (Picture in Picture) format within a first captured image 301 of the rear of the vehicle 10. However, in this case, since the second captured image of the passengers in the rear seats also includes an image of the rear of the vehicle 10, it creates a situation where there is a landscape within a landscape, making it difficult for the driver to distinguish between the image of the rear of the vehicle 10 and the image of the rear seats.

Therefore, the display control unit 110 according to this embodiment includes a display mode correction unit 111 and a display image generation unit 112, as shown in FIG. 1.

When the display control unit 110 simultaneously displays the first captured image and the second captured image on the display device 13, the display mode correction unit 111 executes a display mode correction process to correct a part or all of the display mode of at least one of the first captured image and the second captured image. For example, when the display control unit 110 simultaneously displays the first captured image and the second captured image, the display mode correction unit 111 executes a color scheme change process to change the color scheme of the background of the occupant captured in the second captured image. In addition, the display mode correction unit 111 executes a transparency process to make the area where the color scheme has been changed transparent. For example, when the display mode correction unit 111 changes the color scheme of the background of the occupant captured in the second captured image, the display mode correction unit 111 makes the background of the occupant whose color scheme has been changed transparent so that the first captured image can be viewed. Note that the display mode correction process may include various processes such as a blurring process and a mosaic process in addition to the color scheme change process and the transparency process.

The display image generating unit 112 executes a display image generating process to generate a display image by, for example, synthesizing the first captured image with the second captured image on which the display mode correcting unit 111 has executed color scheme change processing, transparency processing, and the like. The display image generating unit 112 also outputs the generated display image to the display device 13, thereby displaying the display image on the display device 13.

Figure 4 is a diagram showing an example of a display image displayed by a display device according to one embodiment. This figure shows an example of a display image 401 when the display control device 100 according to this embodiment simultaneously displays a first captured image 402 and a second captured image 403 on the display device 13. In the example of Figure 4, the display control unit 110 displays a display image 401 on the display device 13 in which a second captured image 403 captured of the rear seat of the vehicle 10 is displayed in PinP format within a first captured image 402 captured of the rear of the vehicle 10.

In the example of FIG. 4, the display control unit 110 uses the display mode correction unit 111 to change the color scheme of the background 405 of the occupant 404 in the second captured image 403. For example, the display control unit 110 may use the display mode correction unit 111 to fill in the background 405 of the occupant 404 in the second captured image 403 with a predetermined color. This allows the driver of the vehicle 10 to easily distinguish between the first captured image 402 captured behind the vehicle 10 and the second captured image 403 captured in the rear seat of the vehicle 10.

Preferably, the display control unit 110 uses the display mode correction unit 111 to perform transparency processing on the background 405 of the occupant 404 in the second captured image 403. For example, the display control unit 110 uses a known image processing technique to make the background 405, which has been filled in with a predetermined color using the display mode correction unit 111, transparent so that the first captured image 402 can be viewed. This allows the display control unit 110 to reduce the area of the first captured image 402 that is hidden by the second captured image 403 in the display image 401 as shown in FIG. 4.

As described above, according to this embodiment, the display control device 100, which displays images outside the vehicle and images inside the vehicle on the display device 13, allows a user such as a driver to easily distinguish between images outside the vehicle and images inside the vehicle.

<Hardware Configuration>
5 is a diagram showing an example of a hardware configuration of a display control device according to an embodiment. The display control device 100 has a computer configuration, and includes, for example, a CPU (Central Processing Unit) 501, a memory 502, a storage device 503, a GPU (Graphics Processing Unit) 504, a communication device 505, an I/F (Interface) 506, and a bus 507.

The CPU 501 is a processor that realizes each functional configuration of the display control device 100 as shown in FIG. 1 by executing a predetermined program stored in a storage medium such as the storage device 503. The memory 502 includes, for example, a RAM (Random Access Memory), which is a volatile memory used as a work area for the CPU 501, and a ROM (Read Only Memory), which is a non-volatile memory that stores programs for starting the CPU 501. The storage device 503 is, for example, a non-volatile, large-capacity storage device such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive).

The GPU 504 is an image processing processor that executes various processes mainly related to image processing at higher speeds than the CPU 501. The communication device 505 includes one or more communication interfaces for connecting the display control device 100 to a communication network. For example, the communication device 505 includes a communication interface for connecting the display control device 100 to an in-vehicle network of the vehicle 10. The communication device 505 may also include a wireless communication device for connecting the display control device 100 to a WAN (Wide Area Network) such as the Internet.

The I/F 506 is an interface for connecting external devices such as the exterior camera 11, the interior camera 12, and the line of sight detection device 1101 described below to the display control device 100. The bus 507 is commonly connected to each of the above components and transmits, for example, address signals, data signals, and various control signals.

<Processing flow>
Next, the process flow of the display control method according to the present embodiment will be described.

[First embodiment]
6 is a flowchart showing an example of processing of the display control device according to the first embodiment. The display control device 100 repeatedly executes the processing shown in FIG.

In step S601, the first imaging unit 120 controls the exterior camera 11 to generate a first captured image, for example, of the rear of the vehicle 10. Preferably, the first imaging unit 120 horizontally inverts the image captured by the exterior camera 11 to generate the first captured image to be displayed on the display device 13.

In step S602, in parallel with the processing of step S601, the second imaging unit 130 controls the in-vehicle camera 12 to generate a second captured image, for example, of an occupant in the rear seat of the vehicle 10. Preferably, the second imaging unit 130 horizontally inverts the image captured by the in-vehicle camera 12 to generate the second captured image to be displayed on the display device 13.

In step S603, the display control unit 110 determines whether or not an occupant is captured in the second captured image generated by the second imaging unit 130. If an occupant is captured in the second captured image, the display control unit 110 transitions the process to step S604. On the other hand, if an occupant is not captured in the second captured image, the display control unit 110 transitions the process to step S606.

When the process proceeds to step S604, the display control unit 110 uses the display mode correction unit 111 to change the color scheme of the background of the occupant in the second captured image. As an example, the display control unit 110 uses the display mode correction unit 111 to fill in the background of the occupant in the second captured image with a predetermined color (e.g., a blue background, etc.).

In step S605, the display control unit 110 uses the display image generation unit 112 to generate a display image that simultaneously displays the first captured image and the second captured image in which the color scheme of the occupant has been changed.

On the other hand, when the process proceeds from step S603 to step S606, the display control unit 110 uses the display image generation unit 112 to generate a display image that displays the first captured image. As a result, the display control unit 110 stops displaying the second captured image on the display device 13.

In step S607, the display control unit 110 outputs the display image generated by the display image generation unit 112 to the display device 13. As a result, the display device 13 displays, for example, a display image 701 as shown in FIG. 7.

Figure 7 is a diagram showing an example of a display image according to the first embodiment. This figure shows an example of a display image 701 displayed by the display device 13 when there is an occupant in the rear seat of the vehicle 10.

In the example of FIG. 7, the display control device 100 displays on the display device 13 a display image 701 in which a second captured image 703 of an occupant in the rear seat of the vehicle 10 is displayed in a pin-on-pin-display manner within a first captured image 702 of the rear of the vehicle 10. In addition, in the second captured image 703, the background 705 of the occupant 704 is filled in with a predetermined color. Therefore, the driver of the vehicle 10 can easily distinguish between the occupant 704 sitting in the rear seat of the vehicle 10 and a person 706 outside the vehicle who is behind the vehicle 10.

The display image 701 shown in FIG. 7 is an example. For example, the display control device 100 may display on the display device 13 a display image in which a first captured image 702 capturing an image of the rear of the vehicle 10 and a second captured image 703 capturing an image of an occupant in the rear seat of the vehicle 10 are displayed side by side.

As described above, according to the first embodiment, the display control device 100 displays an image outside the vehicle and an image inside the vehicle on the display device 13, allowing a user such as a driver to easily distinguish between the image outside the vehicle and the image inside the vehicle.

In step S604 in FIG. 6, the display control unit 110 may use the display mode correction unit 111 to execute a process of blurring the background instead of changing the color scheme of the background of the occupant in the second captured image. Even in this case, the driver of the vehicle 10 can easily distinguish between the occupant 704 sitting in the rear seat of the vehicle 10 and the person 706 outside the vehicle who is behind the vehicle 10.

Second Embodiment
In the second embodiment, an example will be described in which the display control device 100 performs transparency processing on a background 705 of an occupant 704 in a second captured image 703 in a display image 701 as shown in FIG.

Figure 8 is a flowchart showing an example of the processing of the display control device according to the second embodiment. Note that, among the processing shown in Figure 8, the processing of steps S601 to S604, S606, and S607 is similar to the processing of steps S601 to S604, S606, and S607 in the first embodiment, and therefore a description thereof will be omitted here.

In step S801, the display control unit 110 uses the display mode correction unit 111 to perform a transparency process that makes the background of the occupant in the second captured image transparent so that the first captured image can be captured.

In step S802, the display control unit 110 uses the display image generation unit 112 to generate a display image that displays the second captured image within the first captured image.

Through the above processing, the display control device 100 can display, for example, a display image 401 as shown in FIG. 4 on the display device 13.

In the example of FIG. 4, the display control device 100 displays on the display device 13 a display image 401 in which a second captured image 403 of an occupant in the rear seat of the vehicle 10 is displayed in PinP format within the range of a first captured image 402 of the rear of the vehicle 10. In addition, in the second captured image 703, the background 705 of the occupant 704 is filled in with a predetermined color and is made transparent so that the first captured image 402 can be seen.

In the first embodiment, the driver of the vehicle 10 cannot see the area of the first captured image 702 in which the second captured image 703 is displayed, as shown in FIG. 7. On the other hand, in the second embodiment, the driver of the vehicle 10 can see the first captured image 402 in the background 405 of the occupant 404 in the area in which the second captured image 403 is displayed, as shown in FIG. 4.

Note that the process shown in FIG. 8 is just an example, and the display control device 100 may display the display image 401 shown in FIG. 4 on the display device 13 using, for example, a different processing procedure as shown in FIG. 9.

Figure 9 is a flowchart showing another example of the processing of the display control device according to the second embodiment. Note that, among the processing shown in Figure 9, the processing of steps S601 to S603, S606, and S607 is similar to the processing of steps S601 to S603, S606, and S607 in the first embodiment, and therefore a description thereof will be omitted here.

In step S901, the display control unit 110 uses the display mode correction unit 111 to change the color scheme (hue) of the display area 1002 that displays the second captured image of the first captured image 1001, for example, as shown in FIG. 10.

In step S902, the display control unit 110, for example, uses the display image generation unit 112 to extract an image of the occupant from the second captured image, excluding the background image.

In step S903, the display control unit 110 uses the display image generation unit 112 to generate a display image that displays the image of the occupant extracted in step S902 in the display area 1002 of the second captured image within the first captured image 1001 as shown in FIG. 10.

By the processing of FIG. 9, the display control unit 110 can display a display image similar to the display image 401 shown in FIG. 4 on the display device 13.

[Third embodiment]
In the third embodiment, an example of processing will be described in which the display control device 100 detects the line of sight of a driver who drives a vehicle 10 and changes the display method of the second captured image in accordance with the driver's line of sight.

Fig. 11 is a diagram showing an example of the configuration of a display control system according to the third embodiment. As shown in Fig. 11, the display control system 1 according to the third embodiment has a gaze detection device 1101 and a gaze detection unit 1102 in addition to the configuration of the display control system 1 according to the embodiment described in Fig. 1.

The gaze detection device 1101 is a device that detects the gaze of a specific user. For example, the gaze detection device 1101 extracts the pupils from a facial image of the user using a known image analysis technique, and detects the user's gaze or viewpoint from the pupils' movement, direction, etc.

The gaze detection unit 1102 is realized, for example, by a program executed by the CPU 501, and uses the gaze detection device 1101 to detect that the driver of the vehicle 10 is gazing at the first captured image displayed on the display device 13. For example, when the driver's gaze remains on the first captured image for more than a predetermined time, the gaze detection unit 1102 determines that the driver is gazing at the first captured image.

The display control unit 110 according to the third embodiment reduces or makes transparent the display area of the second captured image when the driver of the vehicle 10 is gazing at the first captured image displayed on the display device 13. Note that the functional configuration of the display control system 1 other than the above may be the same as the functional configuration of the display control system 1 according to the embodiment described in FIG. 1.

<Processing flow>
Fig. 12 is a flowchart showing an example of processing of a display control device according to the third embodiment. Among the processing shown in Fig. 12, the processing of steps S601 to S603, S606, and S607 is the same as the processing of the first embodiment described in Fig. 6, so the description will be omitted here. Also, detailed description of the processing contents similar to those of the first and second embodiments will be omitted here.

In step S1201, the display control unit 110 uses the display mode correction unit 111 to change the color scheme of the background of the occupant in the second captured image, and performs a transparent process on the background of the occupant whose color scheme has been changed. Note that this process corresponds to the processes of steps S604 and S801 in FIG. 8.

In steps S1202 and S1203, the gaze detection unit 1102 detects the gaze of the driver of the vehicle 10 using the gaze detection device 1101, and determines whether the driver is gazing at the first captured image displayed on the display device 13. For example, as shown in FIG. 13(A), when the driver's gaze (point of view) 1302 remains on the first captured image 1301 for more than a predetermined time, the gaze detection unit 1102 determines that the driver is gazing at the first captured image.

As a specific example, the gaze detection unit 1102 stores the detection result of the driver's gaze 1302 by the gaze detection device 1101 in chronological order in the memory 502 or the like. In addition, the gaze detection unit 1102 detects that the driver is gazing at the first captured image from the driver's gaze 1302 detected in step S1202 and the driver's gaze 1302 stored in chronological order in the memory 502 or the like.

If the driver is gazing at the first captured image, the display control unit 110 transitions the process to step S1204. On the other hand, if the driver is not gazing at the first captured image, the display control unit 110 transitions the process to step S1205.

When the process proceeds to step S1204, the display control unit 110 reduces or makes the second display area transparent using the display image generation unit 112. For example, the display control unit 110 reduces the display area 1303 of the second captured image displayed within the first captured image 1301 as shown in FIG. 13(B). Alternatively, the display control unit 110 may make the display area 1002 of the second captured image transparent so that the first captured image can be viewed while maintaining the display area 1002 of the second captured image displayed within the first captured image 1001 as shown in FIG. 10.

On the other hand, when proceeding to step S1205, the display control unit 110 sets the second captured image to the default display method that does not reduce or make it transparent.

In step S1206, the display control unit 110 uses the display image generation unit 112 to generate a display image that displays the first captured image 1301 and the display area 1303 of the second captured image. As shown in FIG. 13B, when the display area 1303 of the second captured image is reduced, the display area 1303 of the second captured image displays the second captured image in which the color scheme of the background of the occupant is changed and transparency processing is performed. On the other hand, as shown in FIG. 10, when the display area 1002 of the second captured image is made transparent, the second captured image does not need to be displayed in the display area 1303 of the second captured image (or the transparency of the second captured image may be further increased).

By performing the above processing, the display control device 100 can further reduce the area of the display area of the first captured image where the first captured image is obscured by the display area of the second captured image.

[Fourth embodiment]
In the fourth embodiment, an example is described in which the display control device 100 has a status detection unit that detects the status of the occupants of the vehicle 10, and corrects the display mode of the background of the occupants in the second captured image displayed on the display device 13 depending on the status of the occupants.

Fig. 14 is a diagram showing an example of the configuration of a display control system according to the fourth embodiment. As shown in Fig. 14, the display control system 1 according to the fourth embodiment has a state detection unit 1401 in addition to the configuration of the display control system 1 according to the embodiment described in Fig. 1.

The state detection unit 1401 is realized, for example, by a program executed by the CPU 501, and detects the state of the occupant of the vehicle 10. As an example, the state detection unit 1401 analyzes the second captured image of the occupant in the rear seat of the vehicle 10 generated by the second imaging unit 130 to detect the state of the occupant in the rear seat (e.g., good, car sick, fever, sleeping, etc.). As another example, the state detection unit 1401 may detect the state of the occupant using various sensors such as a temperature sensor, a pulse sensor, a motion sensor, etc. provided on the seat of the vehicle 10. As another example, the state detection unit 1401 may estimate the state of the occupant by inputting the second captured image generated by the second imaging unit 130 into a state estimation model that has been trained by machine learning.

The display mode correction unit 111 according to the third embodiment corrects the display mode of the background of the occupant in the second captured image according to the state of the occupant detected by the state detection unit 1401. Preferably, the state detection unit 1401 detects the state of multiple occupants in the rear seats, and the display mode correction unit 111 corrects the display mode of the background for each occupant according to the state of the multiple occupants. Note that the functional configuration of the display control system 1 other than the above may be the same as the functional configuration of the display control system 1 according to the embodiment described in FIG. 1.

<Processing flow>
Fig. 15 is a flowchart showing an example of processing of a display control device according to the fourth embodiment. Among the processing shown in Fig. 15, the processing of steps S601 to S603, S606, and S607 is the same as the processing of the first embodiment described in Fig. 6, so the description will be omitted here. Also, detailed description of the processing contents similar to those of the first to third embodiments will be omitted here.

In step S1501, the status detection unit 1401 detects the status of the occupants of the vehicle 10. For example, the status detection unit 1401 analyzes the second captured image generated by the second imaging unit 130 to detect the status of multiple occupants sitting in the rear seats of the vehicle 10.

In step S1502, the display control unit 110 uses the display mode correction unit 111 to change the background color of the multiple occupants 1601 to 1603 in the second captured image 1600 according to the state of the occupants detected by the state detection unit 1401, for example as shown in FIG. 16. In the example of FIG. 16, the display mode correction unit 111 sets the background color of the occupant 1601 whose state is "good" to "green." Similarly, the display mode correction unit 111 sets the background color of the occupant 1602 whose state is "fever" to "yellow," and sets the background of the occupant 1603 whose state is "car sick" to "orange." Note that the "color" representing the state of the occupant, or the "state" of the occupant to be detected, may be set by the user (driver, etc.) from a setting screen, etc.

In step S1503, the display control unit 110 uses the display mode correction unit 111 to perform transparency processing on the background of the occupants in the second captured image 1600 in which the color of the background of each occupant has been changed.

In step S1504, the display control unit 110 uses the display image generation unit 112 to generate a display image that simultaneously displays the first captured image and the second captured image.

By the processing of FIG. 15, the display control unit 110 can display, for example, a display image 1604 as shown in FIG. 16 on the display device 13. This allows the user (driver, etc.) to easily grasp the status of the passengers in the rear seats by looking at the second captured image 1600 displayed on the display device 13.

As described above, according to each embodiment of the present invention, in the display control device 100 that displays images outside the vehicle and images inside the vehicle on the display device 13, a user such as a driver can easily distinguish between images outside the vehicle and images inside the vehicle.

The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the gist of the invention described in the claims.

1 Display control system 13 Display device (display unit)
110 Display control unit 120 First imaging unit 402, 702 First captured image 403, 703 Second captured image 1102 Line-of-sight detection unit 1401 Status detection unit

Claims (8)

a first imaging unit that generates a first captured image by capturing an image of a rear area of the vehicle ;
a second imaging unit that generates a second captured image of an occupant in a rear seat ;
a display control unit that displays the first captured image and the second captured image on a display unit;
having
The display control unit is
When the first captured image and the second captured image are simultaneously displayed, the second captured image is displayed within a range of the first captured image displayed on the display unit;
changing a color scheme of a background of the occupant in the second captured image, and making the background of the occupant whose color scheme has been changed transparent so that the first captured image can be visually recognized;
Display control device. A state detection unit that detects a state of the occupant,
The display control unit changes a color scheme of a background of the occupant according to a state of the occupant.
The display control device according to claim 1 . The state detection unit detects states of a plurality of occupants in the rear seats,
The display control device according to claim 2 , wherein the display control unit changes a color scheme of the background for each of the plurality of occupants according to a state of the plurality of occupants. The display control device according to claim 1 , wherein the display control unit is configured to stop displaying the second captured image when the occupant is not captured in the second captured image. A gaze detection unit detects the gaze of a driver of a vehicle,
The display control unit changes a display method of the second captured image in accordance with a line of sight of the driver.
The display control device according to claim 1 . The display control device according to claim 5 , wherein the display control unit reduces or makes transparent a display area of the second captured image when the driver is gazing at the first captured image. The display control device according to claim 5 , wherein the display control unit displays the second captured image on the display unit in a default display method when the driver is not gazing at the first captured image. A display control device mounted on a vehicle,
a first imaging process for capturing an image of a rear area of the vehicle and generating a first captured image;
a second imaging process for imaging a rear seat occupant and generating a second captured image;
a display control process for displaying the first captured image and the second captured image on a display unit;
Run
The display control process includes :
When the first captured image and the second captured image are simultaneously displayed, the second captured image is displayed within a range of the first captured image displayed on the display unit;
changing a color scheme of a background of the occupant in the second captured image, and making the background of the occupant whose color scheme has been changed transparent so that the first captured image can be visually recognized;
Display control method.

Images