JP7640435B2 - Image display device, image display method, and image display program - Google Patents

Description

This disclosure relates to an image display device, an image display method, and an image display program for use in a vehicle.

Patent Document 1 discloses technology for an electronic mirror camera system that allows the electronic mirror display for a vehicle mirror to switch between displaying image information from an electronic mirror camera and image information from an electronic rearview camera. In this technology, the electronic mirror display (electronic inner mirror) switches the display from image information from the electronic mirror camera to image information from the electronic rearview camera.

JP 2020-164115 A

Prior art shows how to switch image information displayed on an electronic inner mirror. However, the prior art does not anticipate a situation in which multiple images are displayed in parallel on an electronic inner mirror. Images displayed on an electronic inner mirror include images of the rear of the vehicle that are displayed when driving forward, and images to assist with parking that are displayed when driving in reverse, and there are cases in which multiple of these images are displayed in parallel. Furthermore, the prior art does not anticipate how multiple images would appear when displayed in parallel.

The present disclosure aims to provide an image display device, an image display method, and an image display program that enable intuitive recognition of the center of a first image captured behind a vehicle and a second image for assisting parking when the first image and the second image are displayed side by side.

The image display device according to claim 1 includes a display unit capable of displaying, in a display range, images captured behind the vehicle when the vehicle is moving forward and backward, and a control unit that, when moving backward, causes a first image captured behind the vehicle and a second image different from the first image for checking the rear of the vehicle to be displayed side by side in the display range, and also causes a mark indicating the center of the first image in the left-right direction to be displayed at least either above or below the display range.

According to the image display device of claim 1, the first image and the second image are displayed in parallel, and a mark indicating the center of the first image is displayed. With such a mark indicating the center, when the first image capturing the rear of the vehicle and the second image for assisting parking are displayed in parallel, the center of the first image can be intuitively grasped. This makes it possible to suppress a decrease in visibility of the first image even when multiple images are displayed in parallel.

The image display device of claim 2 is the image display device of claim 1, in which, when the control unit changes from displaying the first image over the entire display range during forward movement to displaying the first image and the second image side by side in the display range during backward movement, the control unit slides the first image so that one end of the first image approaches the center while trimming the left and right sides while maintaining the center of the first image, displays the second image in a position adjacent to the first image, and moves the mark together with the sliding of the center position of the first image.

According to the image display device described in claim 2, the first image is slid from a state in which the first image is displayed over the entire display range, and the mark is moved along with the sliding of the center position. By controlling the movement of the mark in accordance with such a slide, the driver can more intuitively grasp that the center of the first image is moving.

The image display device according to claim 3 is the image display device according to claim 1 or claim 2, in which the first image is captured by a first imaging unit that captures an image of the rear of the vehicle, the second image is captured by a second imaging unit that captures a rear range from a different viewpoint than the first imaging unit and captures an image to assist parking, and the control unit displays the first image in the entire display range when moving forward, and displays the first image and the second image side by side in the display range when moving backward.

According to the image display device described in claim 3, the first image is displayed when moving forward, and the first image and the second image are displayed side by side when moving backward, so that the display can be switched between moving forward and backward.

The image display device according to claim 4 is the image display device according to any one of claims 1 to 3, in which the control unit does not display the mark when moving forward, and displays the mark when moving backward.

According to the image display device described in claim 4, when multiple images are displayed in parallel, a mark is displayed to help the driver intuitively grasp the center of the first image.

The image display device described in claim 5 is the image display device described in any one of claims 1 to 4, in which the control unit can set the display mode including at least one of whether or not to display the mark and the type of display pattern.

According to the image display device described in claim 5, it is possible to set whether or not to display the mark and the display pattern so that the driver can easily understand it.

The image display device according to claim 6 is the image display device according to claim 5, in which the control unit is capable of setting the display mode of the mark for each driving state including at least one of forward and reverse of the vehicle.

According to the image display device described in claim 6, the display mode of the detailed mark can be set for each driving condition.

The image display method described in claim 7 is a method in which a computer executes a process that enables images captured behind the vehicle to be displayed in the display range of a display unit when the vehicle is moving forward and backward, displays a first image captured behind the vehicle and a second image different from the first image for checking the rear of the vehicle side by side in the display range when the vehicle is moving backward, and displays a mark indicating the center of the first image in the left-right direction at least on the top or bottom of the display range.

The image display program of claim 8 allows a display unit to display, in a display range, images of the rear of the vehicle when the vehicle is moving forward and backward, and causes a processor to control a computer to execute processing to display, in parallel, a first image of the rear of the vehicle and a second image, different from the first image, for checking the rear of the vehicle in the display range of the display unit when the vehicle is moving backward, and to display a mark indicating the center of the width of the first image at least above or below the display range.

According to the technology disclosed herein, when a first image capturing the rear of a vehicle and a second image for assisting parking are displayed side by side, the center of the first image can be intuitively grasped.

1 is a diagram showing a schematic configuration of an in-vehicle system according to an embodiment; FIG. 2 is a block diagram showing a hardware configuration of a vehicle. 11A to 11C are diagrams illustrating examples of displays when a first image is displayed on the electronic rear-view mirror and when the first image and a second image are displayed side by side. 4 is a block diagram showing a functional configuration of a control unit; FIG. 1A and 1B are diagrams showing examples of a "display when driving forward" and a "display when driving backward" of an electronic rear-view mirror. FIG. 13 is a diagram showing an example of a slide when switching from (A) "display when moving forward" to (B) "display when moving backward." 13A and 13B are diagrams showing examples of a case where a depression type and a protrusion type are provided as a pattern of marks representing depressions or protrusions in an image. 5 is a flowchart showing the flow of an image display process executed in a control unit. 6 is a flowchart showing the flow of a setting process executed in a control unit.

An example of an embodiment of the present disclosure will be described in detail below with reference to the drawings. FIG. 1 shows a schematic configuration of an in-vehicle system 10 according to the present embodiment. The vehicle 12 of the in-vehicle system 10 includes an electronic inner mirror 14, an external camera 16A, a back monitor camera 16B, and a control unit 20. The installation positions of the external camera 16A and the back monitor camera 16B on the vehicle 12 in FIG. 1 are for convenience, and the external camera 16A and the back monitor camera 16B are installed anywhere on the vehicle 12. For example, the installation positions can be near the license plate, near the rear window, near the rear spoiler, etc. For example, the external camera 16A may be installed on the door mirror in addition to the rear of the vehicle. In this case, the images captured by the multiple external cameras 16A can be synthesized and displayed on the electronic inner mirror 14. The electronic inner mirror 14 is an example of a display unit according to the present disclosure. The external camera 16A is an example of a first imaging unit according to the present disclosure. The back monitor camera 16B is an example of a second imaging unit according to the present disclosure.

2 shows the hardware configuration of the vehicle 12. As shown in FIG. 2, the vehicle 12 includes a CPU (Central Processing Unit) 36, a ROM (Read Only Memory) 38, a RAM (Random Access Memory) 40, and a storage 42 provided in the control unit 20, the electronic inner mirror 14, an external camera 16A, a rear monitor camera 16B, and a liquid crystal monitor 18. Each component is connected to each other so as to be able to communicate with each other via a bus 44. In this embodiment, the ROM 38 or the storage 42 stores an image display program for performing image processing of an image to be displayed on the electronic inner mirror 14. The control unit 20, together with the electronic inner mirror 14, is an example of an image display device according to the present disclosure.

The electronic inner mirror 14 is provided on the upper side of the front windshield glass of the vehicle 12 and approximately in the center in the vehicle width direction. The electronic inner mirror 14 is provided with a display area 50 for displaying images, and is capable of displaying images captured behind the vehicle 12 when the vehicle 12 is moving forward and backward. The electronic inner mirror 14 is also capable of displaying multiple images. The images displayed on the electronic inner mirror 14 are rendered as continuously captured moving images (video).

The external camera 16A is a camera installed near the rear end of the vehicle 12 for the purpose of capturing an image (first image 52) to be displayed on the electronic inner mirror 14, and captures the area behind the vehicle 12 as the first image 52. The focal length of the lens is set so that the angle of view of the external camera 16A is such that a following vehicle a predetermined distance away from the vehicle 12 fits within the angle of view, in accordance with the field of view of an optical inner mirror. Therefore, the blind spot of the external camera 16A is a relatively short range behind the vehicle 12. The first image 52 captured by the external camera 16A is displayed over the entire display range 50 of the electronic inner mirror 14, for example, when the vehicle 12 is traveling forward.

The rear monitor camera 16B is a camera provided near the rear end of the vehicle 12 for the purpose of capturing an image (second image 54) to be displayed on the electronic inner mirror 14, and is a camera that captures the blind spot of the external camera 16A behind the vehicle 12 and captures an image to assist in parking. This allows the rear monitor camera 16B to capture a second image 54 to assist in parking the vehicle 12. The second image 54 captured by the rear monitor camera 16B is displayed in parallel with the first image 52 in the display range 50 of the electronic inner mirror 14, for example, when the vehicle 12 is traveling in reverse.

Figure 3 shows examples of the display when the first image 52 is displayed on the electronic rearview mirror 14 and when the first image 52 and the second image 54 are displayed in parallel. In Figure 3, (A) shows the display when the first image 52 is displayed, and (B) shows the display when the first image 52 and the second image 54 are displayed in parallel. When switching from the display (A) to the display (B), a problem occurs in that the center 52A of the display in the left-right direction (width direction) of the first image 52 is shifted, making it difficult to identify the center 52A of the first image 52. Therefore, in this embodiment, a mark 56 indicating the center 52A is introduced so that the driver can intuitively grasp the center 52A of the first image 52. The mark 56 will be described later.

As described above, the first image 52 captured by the external camera 16A and the second image 54 captured by the rear monitor camera 16B are images captured from different viewpoints. The viewpoints may include cases where the viewpoint position is the same but the viewing angles are different. In other words, images captured from different viewpoints include images captured by cameras installed in the same position but with different viewing angles. Furthermore, the camera capturing the first image 52 is not limited to the external camera 16A, and similarly, the camera capturing the second image 54 is not limited to the rear monitor camera 16B.

The LCD monitor 18 is a touch panel monitor that can be operated by the driver of the vehicle 12. The LCD monitor 18 can display a setting menu, which will be described later.

As shown in FIG. 4, in the control unit 20, the CPU 36 executes the image display program to function as an acquisition unit 200, a state detection unit 210, a display control unit 220, and a setting unit 230.

The acquisition unit 200 acquires a first image from the external camera 16A and a second image from the rear monitor camera 16B.

The state detection unit 210 detects the driving state of the vehicle 12. The state detection unit 210 detects at least forward or reverse as the driving state, which is necessary for switching the display of the image on the electronic inner mirror 14. The driving state can be detected as forward or reverse using values related to the shift position from among the values of various sensors (not shown) mounted on the vehicle 12.

The display control unit 220 displays an image in the display range 50 of the electronic inner mirror 14. When the driving state is forward, the display control unit 220 displays the first image 52 in the entire display range 50 of the electronic inner mirror 14. When the driving state is reverse, the display control unit 220 displays the first image 52 and the second image 54 side by side in the display range 50 of the electronic inner mirror 14, and displays marks 56 indicating the center 52A of the first image 52 above and below the display range 50.

5 shows an example of (A) "display when moving forward" and (B) "display when moving backward" of the electronic inner mirror 14. As shown in FIG. 5, in (A) display when moving forward, only the first image 52 is displayed, and the marks 56 are displayed above and below the center 52A of the first image 52. In (B) display when moving backward, the first image 52 is trimmed to narrow the left and right sides of the image while sliding, and the second image 54 is displayed in a position adjacent to the first image 52. In (B) display when moving backward, the mark 56 is moved and displayed as the position of the center 52A of the first image 52 slides. In this example, the mark 56 is triangular in shape. The mark 56 is bordered with a color opposite to the color of the mark 56 itself. For example, if the color of the mark 56 itself is black, the border color is white. Note that the color is not limited to the opposite color, and any color scheme that takes into consideration visibility different from the color of the mark 56 itself may be used.

Here, the slide control of the first image 52 performed when switching from the "display when moving forward" to the "display when reversing" will be described. FIG. 6 shows an example of a slide when switching from (A) the "display when moving forward" to (B) the "display when reversing". As shown in FIG. 6, in (A) the display when moving forward, the left end 52B of the first image 52 is near the left end of the display range 50. In (B) the display when reversing, the state in which the first image 52 is displayed in its entirety is switched to a state in which the first image 52 and the second image 54 are displayed side by side. In the switching, the left end 52B of the first image 52 is slid so as to move closer to the center of the display range 50. The slide control is performed by moving the left end 52B of the first image 52 toward the center of the display range 50 while maintaining the display of the original center 52A of the first image 52 and trimming it so that the left and right sides are gradually narrowed. This slide control leaves one side of the first image 52 in the display range 50 empty, so the second image 54 is displayed in a position adjacent to the first image 52. The second image 54 may be displayed by sliding in together with the sliding of the first image 52, or by fading in when the sliding is completed. In this way, with the slide control, the center 52A of the display in the image of the first image 52 is maintained, while the position of the center 52A within the display range 50 slides as indicated by the arrow 52C. Therefore, the display control unit 220 moves the display position of the mark 56 together with the sliding of the position of the center 52A.

The setting unit 230 sets the display mode of the mark 56. As the display mode, whether or not to display the mark 56 and the type of display pattern of the mark 56 can be set. The setting by the setting unit 230 is performed by displaying a setting menu on the liquid crystal monitor 18 mounted on the vehicle and accepting a setting operation. Whether or not to display the mark 56 may be set for each driving state including forward and reverse. For example, the mark 56 may be set to be hidden when moving forward and to be displayed when moving backward. Also, it may be possible to set the mark 56 to be displayed only at the upper part when moving forward and to be displayed at the upper and lower parts when moving backward. Also, it may be possible to set the stages of the driving state when moving backward in detail, and to set it to flash for 10 seconds until the start of reverse and to display constantly after 10 seconds. Also, it may be possible to set it to display only the upper part or only the lower part of the display range 50. Note that instead of the liquid crystal monitor 18, a setting menu may be displayed on a mobile terminal or the like held by the driver to accept the setting operation. As shown in FIG. 7, the display pattern of the mark 56 may be a recessed type or a protruding type, which represents a recess or a protrusion in the image. FIG. 7 shows an example of the display pattern of the mark 56 at the bottom of the display range 50. The recessed type includes a triangular display pattern (a1), a rounded corner type (a2), and a rectangular display pattern (a3), and the protruding type includes a triangular display pattern (b1), a rounded corner type (b2), and a rectangular display pattern (b3). The recessed type can reduce the area covered by the display range 50 compared to the protruding type. The display pattern is not limited to this example, and any display pattern of any shape, such as a polygon, can be used. The display menu is not limited to being displayed on the liquid crystal monitor 18, and the setting menu may be displayed on the electronic inner mirror 14 so that the switch can be operated.

(Processing flow)
Next, the operation of the control unit 20 will be described. Fig. 8 shows the flow of image display processing by the control unit 20. Fig. 9 shows the flow of setting processing by the control unit 20. In the control unit 20, the CPU 36 reads out an image display program from the ROM 38 or storage 42, expands it into the RAM 40, and executes it, thereby performing image display processing and setting processing.

First, the image display process (Figure 8) will be described. In step S100, the CPU 36 detects whether the vehicle is traveling forward or backward.

In step S101, the CPU 36 determines whether the vehicle is traveling forward. If the vehicle is traveling forward, the process proceeds to step S102. If the vehicle is not traveling forward, the process proceeds to step S104.

In step S102, the CPU 36 acquires a first image 52 from the external camera 16A.

In step S103, the CPU 36 displays the first image 52 over the entire display range 50 of the electronic inner mirror 14, and displays marks 56 indicating the center 52A of the first image 52 above and below the display range 50. After this step, the process returns to step S100 and is repeated, maintaining the display of the first image 52 and marks 56 as long as the vehicle is traveling forward.

In step S104, the CPU 36 determines whether the vehicle is in reverse. If the vehicle is in reverse, the process proceeds to step S105. If the vehicle is not in reverse, the process returns to step S100 and the process is repeated.

In step S105, the CPU 36 acquires a first image 52 from the external camera 16A and a second image 54 from the rear monitor camera 16B.

In step S106, the CPU 36 performs control to slide the first image 52 to display the first image 52 and the second image 54 side by side, and to move and display the mark 56. In this control, the first image 52 is slid while being trimmed so as to narrow the left and right sides of the image, the second image 54 is displayed in a position adjacent to the first image 52, and the display position of the mark 56 is moved together with the sliding of the position of the center 52A. After this step, the process returns to step S100 and is repeated, and the side by side display of the first image 52 and the second image 54 and the display of the mark 56 are maintained as long as the driving state is reverse.

Next, the setting process (FIG. 9) will be described. In step S200, the CPU 36 determines whether or not a setting operation for the liquid crystal monitor 18 has been accepted. If accepted, the process proceeds to step S201, and if not accepted, the process of this step is repeated. The setting operation is an operation regarding whether or not to display the mark 56, an operation regarding the type of display pattern of the mark 56, etc.

In step S201, the CPU 36 determines whether the received setting operation is an operation related to whether or not to display. If it is an operation related to whether or not to display, the process proceeds to step S202, and if it is not an operation related to whether or not to display, the process proceeds to step S203.

In step S202, the CPU 36 reflects the display/non-display setting made in the setting operation in the mark 56 to be displayed on the electronic inner mirror 14.

In step S203, the CPU 36 determines whether the received setting operation is an operation related to the type of display pattern. If it is an operation related to the type of display pattern, the process proceeds to step S204, and if it is not an operation related to the type of display pattern, the process ends.

In step S204, the CPU 36 reflects the setting of the type of display pattern set by the setting operation in the mark 56 to be displayed on the electronic inner mirror 14.
(summary)
The electronic inner mirror 14 of this embodiment is provided with a display range 50 for displaying an image, and is capable of displaying multiple images. When the vehicle 12 is traveling forward, the control unit 20 displays the first image 52 in the entire display range 50 of the electronic inner mirror 14. When the vehicle 12 is traveling backward, the control unit 20 displays the first image 52 and the second image 54 in parallel in the display range 50 of the electronic inner mirror 14, and displays a mark 56 indicating the center 52A of the first image 52 above and below the display range 50. By using the mark 56 indicating the center 52A, when the first image captured behind the vehicle and the second image for assisting parking are displayed in parallel, the center of the first image can be intuitively grasped. This makes it possible to suppress a decrease in visibility of the first image even when multiple images are displayed in parallel.

In the above embodiment, the various processes that the CPU 36 reads and executes by reading the software (programs) may be executed by various processors other than the CPU. Examples of processors in this case include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacture, and dedicated electrical circuits such as ASICs (Application Specific Integrated Circuits), which are processors having a circuit configuration designed specifically to execute specific processes. The above-mentioned processes may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, etc.). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

In the above embodiment, each program has been described as being pre-stored (installed) in a non-transitory recording medium that can be read by a computer. For example, the image display program in the control unit 20 is pre-stored in the storage 42. However, this is not limiting, and each program may be provided in a form recorded in a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

The process flow described in the above embodiment is an example, and unnecessary steps may be deleted, new steps may be added, or the process order may be rearranged, without departing from the spirit of the invention.

12 Vehicle 14 Electronic inner mirror 16A External camera 16B Back monitor camera 20 Control unit

Claims (8)

a display unit capable of displaying, in a display range, an image captured behind the vehicle when the vehicle is moving forward and backward;
a control unit that, during the reverse movement, displays in parallel in the display range a first image capturing an image of the rear of the vehicle and a second image for confirming the rear of the vehicle, the second image being different from the first image, and displays a mark indicating a center in the left-right direction of the first image at least either above or below the display range;
An image display device comprising: The control unit is
When the first image and the second image are to be displayed side by side in the display range during the backward movement from a state in which the first image is displayed in the entire display range during the forward movement,
2. The image display device according to claim 1, wherein the first image is slid so that one end of the first image is brought closer to the center while trimming the left and right sides while maintaining the center of the first image, the second image is displayed at a position adjacent to the first image, and the mark moves along with the sliding of the center position of the first image. The first image is captured by a first imaging unit that captures an image of the rear of the vehicle, and the second image is captured by a second imaging unit that captures a rear range having a different viewpoint from that of the first imaging unit and captures an image for assisting parking,
3. The image display device according to claim 1, wherein the control unit causes the first image to be displayed in the entire display range when moving forward, and causes the first image and the second image to be displayed side by side in the display range when moving backward. The image display device according to any one of claims 1 to 3, wherein the control unit does not display the mark when moving forward, and displays the mark when moving backward. The image display device according to any one of claims 1 to 4, wherein the control unit is capable of setting a display mode including at least one of whether or not to display the mark and the type of display pattern. The image display device according to claim 5, wherein the control unit is capable of setting the display mode of the mark for each driving state including at least one of forward and reverse of the vehicle. An image captured behind the vehicle when the vehicle is moving forward and backward can be displayed in a display range of the display unit,
During the reverse movement, a first image capturing an image of the rear of the vehicle and a second image for checking the rear of the vehicle, which is different from the first image, are displayed in parallel in the display range, and a mark indicating the center of the first image in the left-right direction is displayed at least either above or below the display range.
An image display method in which processing is performed by a computer. An image captured behind the vehicle when the vehicle is moving forward and backward can be displayed in a display range of the display unit,
During the reverse movement, a first image capturing an image of the rear of the vehicle and a second image for checking the rear of the vehicle, which is different from the first image, are displayed in parallel in the display range, and a mark indicating the center of the first image in the left-right direction is displayed at least either above or below the display range.
An image display program that causes a computer to carry out processing.