JP6656493B2 - Display control device and display control method - Google Patents

Description

  The present invention relates to a display control device and a display control method for controlling a display device.

  2. Description of the Related Art In recent years, an electronic mirror has been proposed that presents a driver with left and right rear sides of a vehicle instead of a door mirror. In the electronic mirror, the left and right rear sides of the vehicle are photographed by a camera, and an image obtained by the photographing is inverted left and right and displayed on a display device. For example, Patent Literature 1 proposes an electronic mirror that displays a situation behind a vehicle traveling lane and an adjacent lane.

JP 2015-144407 A

  However, the display of the electronic mirror includes a part that is not visually recognized by the driver, and power is consumed for displaying the part. For this reason, there is room for improvement in power consumption.

  Therefore, the present invention has been made in view of the above problems, and has as its object to provide a technique capable of reducing power consumption.

  The display control device according to the present invention is a display control device that controls the display device, an acquisition unit that acquires a captured image around the vehicle, and a captured image and a feature point on the captured image, according to a predetermined rule. Control to determine the end area of the area of the captured image based on the image, and to make the display mode of the end area different from the display mode of the remaining area, which is the area of the captured image other than the end area, on the display device. The photographed image is an inner side that is one of the left side and the right side and corresponds to the side where the vehicle exists on the photographed image, and an outer side that is the other of the left side and the right side , An upper side, and a lower side, and the predetermined rule is that, when the feature point is located on the upper side, the first point located on the upper end of the outer side and the outer side other than the first point And a second point located below the feature point and the feature point. The area is determined as an end area. If the feature point is located near the upper side, the first point, the second point, the third point located at the upper end of the inner side, and the point on the inner side are obtained. A fourth point located below a straight line connecting the second point and the feature point, the same as the third point or below the third point, and located above the feature point; When the feature point is located on the lower side, a fifth point located at the lower end of the outer side and a point on the outer side other than the fifth point are obtained. Determining a region surrounded by the sixth point located above the point and the feature point as an end region, and, when the feature point is located near the lower side, a fifth point, a sixth point, A seventh point located at the lower end of the inner side and a point on the inner side that is located above a straight line connecting the sixth point and the characteristic point, and is the same as or above the seventh point. Located in, and includes a eighth point located below the characteristic points, to obtain the area surrounded by the feature point as an end region, of at least any one.

  According to the present invention, an end region of a region of a captured image is obtained based on a captured image and a feature point on the captured image according to a predetermined rule, and the display of the end region is performed on a display device. Control is performed to change the mode from the display mode of the remaining area. According to such a configuration, the power consumption of the display control device can be reduced.

  Objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description and the accompanying drawings.

FIG. 2 is a block diagram illustrating a configuration of the display control device according to the first embodiment. FIG. 4 is a diagram for explaining a first rule according to the first embodiment. FIG. 4 is a diagram for explaining a second rule according to the first embodiment. FIG. 5 is a diagram for explaining a third rule according to the first embodiment. FIG. 9 is a diagram for explaining a fourth rule according to the first embodiment. FIG. 9 is a block diagram illustrating a configuration of a display control device according to a second embodiment. It is a figure for explaining the example of mirror adjustment. It is a figure for explaining the example of mirror adjustment. FIG. 13 is a diagram illustrating a display example by the display control device according to the second embodiment. FIG. 9 is a diagram schematically illustrating a display by the display control device according to the second embodiment. FIG. 13 is a diagram illustrating a display example by the display control device according to the second embodiment. FIG. 9 is a diagram schematically illustrating a display by the display control device according to the second embodiment. 9 is a flowchart illustrating an operation of the display control device according to the second embodiment. FIG. 15 is a diagram illustrating a display example by a display control device according to a first modification of the second embodiment. FIG. 13 is a diagram schematically illustrating a display by a display control device according to a first modification of the second embodiment. FIG. 15 is a diagram illustrating a display example by a display control device according to a second modification of the second embodiment. FIG. 19 is a diagram schematically illustrating a display by a display control device according to a second modification of the second embodiment. FIG. 15 is a diagram schematically illustrating a display by a display control device according to a third modification of the second embodiment. FIG. 19 is a diagram schematically illustrating a display by a display control device according to a modification 5 of the second embodiment. FIG. 19 is a diagram schematically illustrating a display by a display control device according to a modification 5 of the second embodiment. FIG. 13 is a diagram illustrating a display example by the display control device according to the third embodiment. FIG. 13 is a diagram illustrating a display example by the display control device according to the third embodiment. FIG. 13 is a diagram illustrating a display example by the display control device according to the third embodiment. FIG. 14 is a diagram schematically illustrating a display by the display control device according to the third embodiment. 9 is a flowchart showing an operation of the display control device according to the third embodiment. FIG. 14 is a diagram schematically illustrating a display by the display control device according to the fourth embodiment. FIG. 14 is a diagram schematically illustrating a display by the display control device according to the fourth embodiment. FIG. 15 is a block diagram illustrating a configuration of a display control device according to a fifth embodiment. FIG. 3 is a diagram illustrating shooting areas of a standard camera and a wide-angle camera. 15 is a diagram illustrating a display example by the display control device according to Embodiment 5. FIG. 20 is a diagram illustrating a display example by the display control device according to the first modification of the fifth embodiment. FIG. 20 is a diagram illustrating a display example by the display control device according to the second modification of the fifth embodiment. FIG. 20 is a diagram illustrating a display example by the display control device according to the second modification of the fifth embodiment. FIG. FIG. 15 is a block diagram illustrating a configuration of a display control device according to a sixth embodiment. FIG. 15 is a diagram illustrating a display example by the display control device according to the sixth embodiment. FIG. 15 is a diagram illustrating a display example by the display control device according to the sixth embodiment. FIG. 19 is a diagram illustrating a display example by a display control device according to a third modification of the sixth embodiment. FIG. 19 is a diagram illustrating a display example by a display control device according to a third modification of the sixth embodiment. FIG. 15 is a block diagram illustrating a configuration of a display control device according to a seventh embodiment. FIG. 21 is a diagram illustrating a display example by the display control device according to the seventh embodiment. FIG. 15 is a block diagram illustrating a configuration of a display control device according to an eighth embodiment. FIG. 14 is a block diagram illustrating a hardware configuration of a display control device according to another modification. FIG. 14 is a block diagram illustrating a hardware configuration of a display control device according to another modification. It is a block diagram showing the composition of the server concerning other modifications. FIG. 14 is a block diagram illustrating a configuration of a communication terminal according to another modification.

<First Embodiment>
Hereinafter, a vehicle in which the display control device according to the first embodiment of the present invention is mounted and which is a target of attention will be described as “own vehicle”.

  FIG. 1 is a block diagram showing a configuration of the display control device 1 according to the first embodiment. The display control device 1 in FIG. 1 includes an acquisition unit 11 and a control unit 12, and can control the display device 21.

  The display device 21 is provided on, for example, an instrument panel of the host vehicle, a dashboard, or the like. As the display device 21, various display devices such as a liquid crystal display device, an organic EL (electroluminescence) display device, and an LED (Light Emitting Diode) display device can be used.

  The acquisition unit 11 acquires a captured image. The captured image is an image used for display on the display device 21, and is an image around the host vehicle, such as an image behind the host vehicle, an image of the host vehicle side, or an image of the host vehicle front. The captured image in the case of displaying the door mirror is a mirror display image obtained by inverting the image behind the host vehicle from side to side. The captured image may be an entire image captured by a camera or the like, or may be a part of the image. The acquisition unit 11 may be a camera or various interfaces.

  The control unit 12 obtains an end region among the regions of the captured image based on the captured image acquired by the acquisition unit 11 and the feature points on the captured image according to a predetermined rule. Here, the predetermined rule includes a first rule, a second rule, a third rule, or a fourth rule. Hereinafter, the first to fourth rules will be described.

  FIG. 2 is a diagram for explaining the first rule. Before describing the first rule, a plurality of sides of the captured image 2 will be described. As shown in FIG. 2, the captured image 2 has an inner side Lin, an outer side Lout, an upper side Lup, and a lower side Lud. The inner side Lin is one of the left side and the right side, and is a side corresponding to the side where the host vehicle 3 exists on the captured image 2. The outer side Lout is the other of the left side and the right side.

  In FIG. 2, an image obtained by inverting an image captured by a camera that captures the rear left side of the host vehicle is shown as a captured image 2. In this case, the right side of the captured image 2 is the side where the host vehicle 3 exists, so the right side is the inner side Lin and the left side is the outer side Lout. When the photographed image is an image obtained by inverting the image photographed by the camera that photographs the right rear side of the own vehicle, the left side of the photographed image 2 is the side where the own vehicle 3 exists, so the left side Becomes the inner side Lin, and the right side becomes the outer side Lout.

  As shown in FIG. 2, the first rule is a rule when the feature point FP is located on the upper side Lup. According to the first rule, the control unit 12 determines a region surrounded by the first point P1, the second point P2, and the feature point FP as the end region 2a. Note that the first point P1 is a point located at the upper end of the outer side Lout. The second point P2 is a point on the outer side Lout other than the first point P1 and located below the feature point FP. The position of the second point P2 is a position expected to sufficiently satisfy the condition that the second point P2 is located below the feature point FP, for example, the distance between the second point P2 and the first point P1 is １ ／ of the length of the outer side Lout. The position may be fixed to the degree, or may be appropriately changed as described later.

  FIG. 3 is a diagram for explaining the second rule. As shown in FIG. 3, the second rule is a rule when the feature point FP is located near the upper side Lup, such as when the feature point FP is located in the upper half of the captured image 2.

  According to the second rule, the control unit 12 sets a region surrounded by the first point P1, the second point P2, the third point P3, the fourth point P4, and the feature point FP as an end region 2a. Ask. The third point P3 is a point located at the upper end of the inner side Lin. The fourth point P4 is a point on the inner side Lin and is located below the straight line L1 connecting the second point P2 and the feature point FP, and is the same as the third point P3 or below the third point P3. , And above the feature point FP.

  FIG. 4 is a diagram for explaining the third rule. As shown in FIG. 4, the third rule is a rule in the case where the feature point FP is located on the lower side Lud, and is a rule substantially upside down from the first rule.

  According to the third rule, the control unit 12 obtains an area surrounded by the fifth point P5, the sixth point P6, and the feature point FP as the end area 2a. Note that the fifth point P5 is a point located at the lower end of the outer side Lout. The sixth point P6 is a point on the outer side Lout other than the fifth point P5 and located above the feature point FP. The position of the sixth point P6 is expected to sufficiently satisfy the condition of being located above the feature point FP, for example, the distance between the sixth point P6 and the fifth point P5 is about 1/4 of the length of the outer side Lout. May be fixed, or may be appropriately changed as described later.

  FIG. 5 is a diagram for explaining the fourth rule. As shown in FIG. 5, the fourth rule is a rule when the feature point FP is located in the vicinity of the lower side Lud, for example, when the feature point FP is located in the lower half of the captured image 2, and substantially differs from the second rule in the vertical direction. This is the reversed rule.

  According to the fourth rule, the control unit 12 sets a region surrounded by the fifth point P5, the sixth point P6, the seventh point P7, the eighth point P8, and the feature point FP as an end region 2a. Ask. The seventh point P7 is a point located at the lower end of the inner side Lin. The eighth point P8 is a point on the inner side Lin and is located above the straight line L2 connecting the sixth point P6 and the feature point FP, and is located at the same position as the seventh point P7 or above the seventh point P7. And is located below the feature point FP.

  The first to fourth rules in the case where the captured image 2 is an image captured by the camera that captures the left rear side of the host vehicle have been described. When the photographed image 2 is an image photographed by a camera that photographs the left rear side of the host vehicle, the process of turning left and right in the description of FIGS. It is done as a rule.

  In the above description, the predetermined rule has been described as including the first rule, the second rule, the third rule, or the fourth rule. However, the predetermined rule may include at least one of the first rule, the second rule, the third rule, and the fourth rule. For example, the control unit 12 may obtain a region obtained according to a rule obtained by combining the first rule or the second rule with the third rule or the fourth rule as an end region.

  As shown in FIGS. 2 to 4, the control unit 12 changes the display mode of the end area 2 a on the display of the display device 21 to the display mode of the remaining area 2 b which is the area of the captured image 2 other than the end area 2 a. And control to make it different.

  In the case of the first rule (FIG. 2), the shape of the upper part of the remaining area 2b is defined by a line that inclines in the vertical direction, connects the second point P2 and the characteristic point FP, and the upper side Lup. In the case of the second rule (FIG. 3), the shape of the upper part of the remaining area 2b is relatively large in the vertical direction, and is a line connecting the second point P2 and the characteristic point FP, and relatively small in the vertical direction. It is defined by an inclined line connecting the feature point FP and the fourth point P4. Each line defining the shape of the upper portion of the remaining area 2b may be a straight line or a curve.

  In the case of the third rule (FIG. 4), the shape of the lower part of the remaining area 2 b is defined by a line connecting the sixth point P <b> 6 and the characteristic point FP, which is inclined in the vertical direction, and the lower side Lud. In the case of the fourth rule (FIG. 5), the shape of the lower portion of the remaining area 2b is relatively large in the vertical direction, a line connecting the sixth point P6 and the characteristic point FP, and relatively small in the vertical direction. It is defined by an inclined line connecting the feature point FP and the eighth point P8. Each line defining the shape of the lower part of the remaining area 2b may be a straight line or a curve.

<Summary of Embodiment 1>
In the display control device 1 according to the first embodiment as described above, the end area 2a is obtained based on the captured image 2 and the feature points on the captured image 2 according to a predetermined rule, and the display device 21 In the display, control is performed so that the display mode of the end area 2a is different from the display mode of the remaining area 2b. In such a configuration, for example, if the display mode of the end region 2a is a black display mode, an image in which the upper or lower portion on the outer side Lout side is cut out, that is, an image in the remaining region 2b of the captured image 2 Is displayed. For this reason, it is possible to take out and display an image that is a natural image that gives the driver a sense of depth and that is obtained by cutting out a relatively unimportant image. Further, the shape of the remaining area 2b can be appropriately changed according to the content of the captured image 2.

  In the above configuration, when the display device 21 is a specific display device that reduces power consumption when performing black display, power consumption can also be reduced. The specific display device may be, for example, an organic EL device, an LED display device, or has a plurality of backlights, and can turn off the backlights in correspondence with pixels performing black display. It may be a liquid crystal display device.

<Embodiment 2>
FIG. 6 is a block diagram showing a configuration of the display control device 1 according to Embodiment 2 of the present invention. Hereinafter, among the components according to the second embodiment, the same or similar components as those described above are denoted by the same reference numerals, and different components will be mainly described.

  The display control device 1 in FIG. 6 is connected to a left display device 21a, a right display device 21b, and a camera control device 26c.

  Each of the left display device 21a and the right display device 21b corresponds to the display device 21 of FIG. Note that one display device may be used instead of the left display device 21a and the right display device 21b. Hereinafter, when the left display device 21a and the right display device 21b are not distinguished from each other, they may be described as “display device 21”.

  The camera control device 26c is connected to the left rear side camera 26a and the right rear side camera 26b.

  The rear left side camera 26a is a camera for photographing the rear left side of the host vehicle, and is provided in the host vehicle instead of the left door mirror. The rear right camera 26b is a camera for photographing the rear right side of the host vehicle, and is provided in the host vehicle instead of the right door mirror. In the second embodiment, each of the left rear side camera 26a and the right rear side camera 26b is neither a panoramic camera nor a wide-angle camera, and is a standard camera that captures an image having an angle of view of about 10 degrees. It will be described as.

  In the following description, an image for mirror display based on an image captured by the left rear camera 26a may be referred to as a “left camera image”, and may be referred to as an image captured by the right rear camera 26b. The image for mirror display may be referred to as a “right camera image”, and when the left camera image and the right camera image are not distinguished, they may be referred to as “camera images”.

  The camera control device 26c appropriately controls the left rear side camera 26a and the right rear side camera 26b to change the angle of view and the focus based on an operation received by an operation device (not shown). According to such a configuration, by appropriately performing an operation, it is possible to appropriately adjust a range captured by the left rear side camera 26a and the right rear side camera 26b.

  Note that an electric drive mechanism for driving the left rear side camera 26a and the right rear side camera 26b may be provided in the host vehicle so that the postures of the left rear side camera 26a and the right rear side camera 26b can be changed. Further, the camera control device 26c controls the direction of the left rear side camera 26a and the right rear side camera 26b by controlling the electric driving mechanism based on an operation received by an operation device (not shown). Is also good. According to such a configuration, it is possible to increase the degree of freedom in adjusting the range captured by the left rear side camera 26a and the right rear side camera 26b.

  7 and 8 are diagrams showing an example of adjusting a mirror image of a standard door mirror. Here, a general example of adjustment of the door mirror will be described, but the adjustment of the range captured by the left rear camera 26a and the right rear camera 26b is performed in the same manner as the adjustment of the door mirror.

  As shown in FIG. 7, the door mirror is adjusted so that the driver can visually recognize a range of a width of 5 m from the side of the host vehicle at a position 30 m behind the rear of the host vehicle 3. As shown in FIG. 8, the driver recognizes the body of the vehicle 3 within a range of about 1/4 of the horizontal length of the mirror surface 4a from the left end of the mirror surface 4a of the right door mirror 4, and the mirror surface from the lower end of the mirror surface 4a. The right door mirror 4 is adjusted so that the driver can visually recognize the road 5 in a range of about 1/3 of the vertical length of 4a. The left and right door mirrors are adjusted so that the right and left mirrors are reversed. The adjustments recommended for the left and right door mirrors vary depending on the left and right door mirrors and the recommended values for each country, and the size and type of the left and right door mirrors.

  Returning to FIG. 6, the camera control device 26c not only controls the left rear side camera 26a and the right rear side camera 26b as described above, but also controls the left rear side camera 26a and the right rear side camera 26b. The captured left camera image and right camera image are output to the display control device 1.

  Next, the internal configuration of the display control device 1 of FIG. 6 will be described. The display control device 1 includes an image acquisition unit 11a, an image recognition unit 12a, a video processing unit 12b, and an image output unit 12c. Note that the concept of the image acquisition unit 11a is a concept included in the acquisition unit 11 in FIG. 1, and the concepts of the image recognition unit 12a, the video processing unit 12b, and the image output unit 12c are included in the control unit 12 in FIG. It is a concept.

  The image acquisition unit 11a acquires a camera image from the camera control device 26c as a captured image. In the second embodiment, the image acquisition unit 11a acquires a camera image as a captured image in consideration of that each of the left rear side camera 26a and the right rear side camera 26b is a standard camera. It is configured to be. If each of the left rear side camera 26a and the right rear side camera 26b is, for example, a panoramic camera, a wide-angle camera, a camera with a fisheye lens, or the like, the image acquisition unit 11a uses an electric drive mechanism to operate the camera. An image equivalent to an image obtained by physically moving the direction may be acquired as a captured image. An image equivalent to an image obtained by physically moving the direction of the camera by the electric drive mechanism is, for example, an image obtained by cutting a part of the camera image, or an image obtained by performing a viewpoint conversion process on the camera image. . With such a configuration, hardware such as an electric drive mechanism for changing the direction of the camera is not required, so that cost reduction can be expected.

  The image recognition unit 12a and the video processing unit 12b will be described with reference to FIGS.

  FIG. 9 is a diagram illustrating a display example of the left display device 21a and the right display device 21b according to the second embodiment. In the following description, the captured image corresponding to the left camera image may be referred to as “left captured image”, and the captured image corresponding to the right camera image may be referred to as “right captured image”.

  In the example of FIG. 9, the left display device 21 a and the right display device 21 b are provided with the meter region 7 provided with the meter therebetween, and the remaining region 2 b of the left captured image 2 and the remaining region of the right captured image 2 are provided. 2b are displayed as a left electronic mirror and a right electronic mirror, respectively. In the remaining area 2b of the left captured image 2, the body of the own vehicle 3, the road 5 with the solid white line 5a, and the other vehicle 6 are displayed, and in the remaining area 2b of the right captured image 2, , The body of the own vehicle 3 and the road 5 with the solid white line 5a and the like are displayed.

  FIG. 10 is a diagram schematically showing the end region 2a and the remaining region 2b in the left photographed image 2 in FIG. FIG. 11 is a diagram illustrating a display example of the left display device 21a according to the second embodiment. FIG. 12 is a diagram schematically showing the end region 2a and the remaining region 2b in the left photographed image 2 in FIG. In the following description, the edge region 2a and the remaining region 2b of the left captured image 2 will be mainly described, but the edge region 2a and the remaining region 2b of the right captured image 2 will be the same as the following description. is there.

  The image recognition unit 12a in FIG. 6 recognizes a predetermined object from the captured image 2 acquired by the image acquisition unit 11a, and obtains a feature point based on the object. In the second embodiment, the predetermined object is a lane marking of a road. The lane markings of the road include a solid yellow line corresponding to prohibition of protruding when a vehicle passes, a solid white line corresponding to prohibition of all protruding vehicles, and a dashed white line corresponding to permission of protruding a vehicle. The role of the solid white line and the like is not limited to this.

  In the examples of FIGS. 10 and 12, the image recognition unit 12a obtains the feature point FP based on the position of the vanishing point 5b of the solid white line 5a in the captured image 2. In the example of FIG. 10, the image recognizing unit 12a obtains, as a feature point FP, a point directly above the vanishing point 5b among points on the upper side Lup. Note that the image recognition unit 12a may determine the midpoint between the upper side Lup and the vanishing point 5b as the feature point FP. In the example of FIG. 12, the image recognition unit 12a finds a point directly below the vanishing point 5b among the points on the lower side Lud as the feature point FP. Note that the image recognition unit 12a may determine the midpoint between the lower side Lud and the vanishing point 5b as the feature point FP.

  The video processing unit 12b in FIG. 6 performs the same operation as the control unit 12 described in the first embodiment. That is, the video processing unit 12b obtains the end area 2a based on the captured image 2 obtained by the image obtaining unit 11a and the feature point FP obtained by the image recognition unit 12a according to a predetermined rule.

  When the feature point FP is located on the upper side Lup as in the example of FIG. 10, the video processing unit 12b obtains the end area 2a according to the first rule. When the feature point FP is the midpoint between the upper side Lup and the vanishing point 5b, the video processing unit 12b obtains the end area 2a according to the second rule. When the feature point FP is located on the lower side Lud as in the example of FIG. 12, the video processing unit 12b obtains the end area 2a according to the third rule. When the feature point FP is the midpoint between the lower side Lud and the vanishing point 5b, the video processing unit 12b obtains the end area 2a according to the fourth rule.

  The video processing unit 12b performs image processing for changing the display mode of the end area 2a as shown in FIGS. Note that the portions with dot hatching in FIG. 9 are black. Such a mode of black display can be realized, for example, by hiding the end region 2a.

  The image output unit 12c in FIG. 6 outputs the video signal of the left captured image 2 subjected to the image processing by the video processing unit 12b to the left display device 21a. Further, the image output unit 12c outputs the video signal of the right captured image 2 on which the image processing has been performed by the video processing unit 12b to the right display device 21b.

<Operation>
FIG. 13 is a flowchart showing the operation of the display control device 1 according to the second embodiment. This operation is started, for example, when the accessory power supply of the own vehicle is turned on and when the drive source of the own vehicle is turned on.

  First, in step S1, the image acquisition unit 11a acquires a left camera image and a right camera image as a captured image 2.

  In step S2, the image recognition unit 12a recognizes a solid white line 5a from the captured image 2 acquired by the image acquisition unit 11a.

  In step S3, the image recognition unit 12a recognizes the vanishing point 5b of the recognized solid white line 5a.

  In step S4, the image recognition unit 12a obtains a feature point FP based on the position of the recognized vanishing point 5b.

  In step S5, the video processing unit 12b obtains the end area 2a based on the feature points FP and the like, and performs image processing for changing the display mode of the end area 2a.

  In step S6, the image output unit 12c outputs the video signal of the left captured image 2 subjected to the image processing by the video processing unit 12b to the left display device 21a, and the image processing is performed by the video processing unit 12b. The video signal of the right photographed image 2 is output to the right display device 21b. Then, the process returns to step S1.

<Summary of Embodiment 2>
The display control device 1 according to the second embodiment as described above recognizes a predetermined object such as the solid white line 5a from the captured image 2, and obtains a feature point based on the object. According to such a configuration, it is possible to appropriately change the shape of the remaining area 2b according to the content of the captured image 2.

<Modification 1 of Embodiment 2>
FIG. 14 is a diagram illustrating a display example of the left display device 21a according to the first modification of the second embodiment. FIG. 15 is a diagram schematically showing the end region 2a and the remaining region 2b in the left photographed image 2 in FIG.

  In the example of FIG. 10 of the second embodiment, the video processing unit 12b obtains, as a feature point FP, a point directly above the vanishing point 5b among points on the upper side Lup. However, as shown in FIG. 15, the video processing unit 12b may obtain, as the feature point FP, a point on the upper side Lup that is on an extension beyond the vanishing point 5b of the solid white line 5a.

  Further, in the second embodiment, the predetermined object for obtaining the feature point FP is a division line such as the solid white line 5a. However, the predetermined object may be a specific vanishing point of the road 5, for example, a vanishing point at the center of the road 5, a vanishing point slightly to the right of the center of the road 5, or a slightly vanishing point at the center of the road 5. It may be a vanishing point of the part. Alternatively, the predetermined object may be a road-related object such as a sidewalk, a guardrail, and a street, a feature, or a display such as a road sign.

<Variation 2 of Embodiment 2>
FIG. 16 is a diagram illustrating a display example of the left display device 21a according to the second modification of the second embodiment. FIG. 17 is a diagram schematically showing the end region 2a and the remaining region 2b in the left photographed image 2 in FIG.

  In the second embodiment, the image recognition unit 12a has obtained one feature point FP from the captured image 2. However, as shown in FIG. 17, the image recognition unit 12a may obtain a plurality of feature points FP from the captured image 2. In the examples of FIGS. 16 and 17, the image recognition unit 12a obtains a first feature point FP1 and a second feature point FP2 as a plurality of feature points FP. The first feature point FP1 is a point located between the vanishing point 5b of the solid white line 5a and the upper side Lup, and is a point located near the upper side Lup. The second feature point FP2 is a point located on the inner side Lin side of the first feature point FP1, and is a point on the upper side Lup just above the vanishing point 5d of the dotted white line 5c.

  In such a case, the video processing unit 12b obtains an area surrounded by the first point P1, the second point P2, the first feature point FP1, and the second feature point FP2 as the end area 2a. Is also good. In this case, the shape of the upper portion of the end region 2a has a relatively large inclination in the vertical direction, a line connecting the second point P2 and the first characteristic point FP1, and a relatively small inclination in the vertical direction. It is defined by a line connecting the point FP1 and the second feature point FP2 and the upper side Lup.

  In the example of FIG. 17, one first feature point FP1 is used. Instead of one first feature point FP1, a plurality of first feature points FP1 approaching the upper side Lup from the outer side Lout toward the inner side Lin. One feature point FP1 may be used. Further, the number of feature points such as the first feature point FP1 and the second feature point FP2 is not limited to two, and the number of section lines such as the solid white line 5a and the dotted white line 5c is not limited to two. Absent.

  Further, in the same manner as described above, the video processing unit 12b may obtain the end region 2a that is the upside down of FIGS. 16 and 17. Although not shown, a third feature point, which is a feature point located near the lower side Lud from the captured image 2, and a feature point located on the inner side Lin side of the third feature point and located on the lower side Lud. When the fourth feature point is obtained, the video processing unit 12b obtains an area surrounded by the fifth point, the sixth point, the third feature point, and the fourth feature point as the end area 2a. You may.

<Variation 3 of Embodiment 2>
In the above configuration, the video processing unit 12b sets the remaining area 2b in accordance with a change in the direction in which the left rear camera 26a and the right rear camera 26b shoot (hereinafter, may be referred to as “camera direction”). May be changed. Normally, when the horizontal angle of the door mirror is changed, the length of the entire door mirror as viewed from the driver in the left-right direction is reduced. In view of this, when the camera direction is moved to the left or right, the video processing unit 12b, as shown in FIG. 18, determines the horizontal length Wup above the remaining area 2b and the remaining area Wup. The horizontal length Wud of the lower portion of 2b may be shortened.

  Although not shown, when the camera direction is approached to the body of the host vehicle 3, the video processing unit 12b determines the vertical length of the outer side Lout side portion of the remaining area 2b to the inner side Lin side portion. It may be longer than the length in the vertical direction. Further, when the camera direction is moved away from the body of the vehicle 3, the video processing unit 12 b determines the vertical length of the outer side Lout side portion of the remaining area 2 b in the vertical direction of the inner side Lin side portion. It may be shorter than the length.

  Although not shown, when the camera direction is directed downward, the video processing unit 12b sets the horizontal length Wup of the upper part of the remaining area 2b to be smaller than the horizontal length Wud of the lower part of the remaining area 2b. May also be shortened. Further, when the camera direction is directed upward, the video processing unit 12b sets the upper horizontal length Wup of the remaining area 2b to be longer than the lower horizontal length Wud of the remaining area 2b. You may.

  According to the above configuration, an electronic mirror similar to a real door mirror can be displayed. Note that the video processing unit 12b may perform the reverse of the above-described deformation or a combination of the above-described deformations according to the setting of the user.

<Modification 4 of Embodiment 2>
In the second embodiment, the video processing unit 12b changes the display mode in the end area 2a to a mode in which the end area 2a is displayed in black. However, the video processing unit 12b may change the display mode in the edge area 2a to a mode in which a display object including a warning mark or the like is superimposed on the edge area 2a, or a specific image processing may be performed in the edge area 2a. The mode may be changed to the mode applied to the area 2a.

  Note that the specific image processing is, for example, a semi-transmissive process of making the image in the end region 2a semi-transparent of the captured image 2 acquired by the image acquiring unit 11a, a blurring process of the image, or a process of blurring the image. Color change processing for changing the color of the image. The color change process includes, for example, a process of changing the color of the image in the end region 2a to the same color as the background color around the end region 2a displayed on the display device 21.

  According to the above configuration, it is possible to display an electronic mirror that suits the user's preference.

<Variation 5 of Embodiment 2>
In the second embodiment, the video processing unit 12b changes the display mode in the end area 2a to a black mode. In this case, the driver cannot confirm the image of the end area 2a. Therefore, as shown in FIG. 19, the video processing unit 12b may divide the remaining area 2b into an adjacent part 2b1 vertically adjacent to the end area 2a and another part 2b2. Then, as shown in FIG. 20, the video processing unit 12b controls the edge region 2a and the adjacent portion 2b1 so that the images of the edge region 2a and the adjacent portion 2b1 of the captured image 2 fall within the adjacent portion 2b1. The image may be compressed.

<Embodiment 3>
The block diagram showing the configuration of the display control device 1 according to the third embodiment of the present invention is the same as the block diagram (FIG. 6) of the display control device 1 according to the second embodiment. Hereinafter, among the components according to the third embodiment, the same or similar components as those described above are denoted by the same reference numerals, and different components will be mainly described.

  In the second embodiment, the predetermined object for obtaining the feature point FP is a division line such as the solid white line 5a. In the third embodiment, the predetermined object is a moving object, and the image recognition unit 12a obtains the feature point FP based on the position of the moving object in the captured image acquired by the image acquiring unit 11a. Hereinafter, a description will be given assuming that the moving body is another vehicle traveling in an adjacent lane adjacent to the lane in which the host vehicle 3 is traveling.

  FIGS. 21, 22, and 23 are diagrams illustrating display examples of the left display device 21a according to the third embodiment. FIG. 24 is a diagram schematically showing the end region 2a and the remaining region 2b in the left photographed image 2 in FIG.

  In FIG. 21, no other vehicle exists in the adjacent lane of the captured image 2. In this case, the image recognition unit 12a obtains a predetermined point on the upper side Lup as the feature point FP. Alternatively, the image recognition unit 12a obtains, as a feature point FP, a point directly above the outer end of the body of the vehicle 3 among points on the upper side Lup.

  In FIG. 22, the other vehicle 6 exists relatively far from the host vehicle 3 in the lane adjacent to the captured image 2, and in FIG. 23, the other vehicle 6 exists relatively close to the host vehicle 3 in the lane adjacent to the captured image 2. I do. In these cases, as shown in FIG. 24, the image recognition unit 12a obtains, as a feature point FP, a point on the upper side Lup, which is directly above the outer end of the other vehicle 6. The outer end of the other vehicle 6 is an end of the other vehicle 6 that is closer to the outer side Lout.

  FIG. 25 is a flowchart showing the operation of the display control device 1 according to the third embodiment. This operation is started, for example, when the accessory power supply of the own vehicle is turned on and when the drive source of the own vehicle is turned on.

  First, in step S11, the image acquisition unit 11a acquires a left camera image and a right camera image as a captured image 2.

  In step S12, the image recognition unit 12a recognizes another vehicle 6 traveling in the adjacent lane from the captured image 2 acquired by the image acquisition unit 11a.

  In step S13, the image recognition unit 12a recognizes the recognized outer end of the other vehicle 6.

  In step S14, the image recognition unit 12a obtains a feature point FP based on the recognized position of the outer end.

  In step S15, the video processing unit 12b obtains the end area 2a based on the feature points FP and the like, and performs image processing for changing the display mode in the end area 2a of the captured image 2.

  In step S16, the image output unit 12c outputs the video signal of the left captured image 2 subjected to the image processing by the video processing unit 12b to the left display device 21a, and the image processing is performed by the video processing unit 12b. The video signal of the right photographed image 2 is output to the right display device 21b. Thereafter, the process returns to step S11.

<Summary of Embodiment 3>
In the display control device 1 according to the third embodiment as described above, the feature point FP is obtained based on the position of the moving object such as the other vehicle 6 in the captured image. According to such a configuration, since the feature point FP is obtained based on the apparent position of the moving object, it is possible to present the driver with an image in which the user can easily obtain a perspective of the object to be noted.

<Modification 1 of Embodiment 3>
In the third embodiment, the image recognition unit 12a obtains, as the feature point FP, a point directly above the outer end of the other vehicle 6 among points on the upper side Lup. However, the image recognizing unit 12a may obtain, as the feature point FP, a point on the upper side Lup that is directly above the center of the other vehicle 6 or a portion other than the center of the other vehicle 6. Further, the image recognition unit 12a obtains a point directly above the outer end of the other vehicle 6 as the first feature point FP1 described in the second modification of the second embodiment, and determines the true point of the inner end of the other vehicle 6. The above point may be obtained as the second feature point FP2 described in the second modification of the second embodiment. The inner end of the other vehicle 6 is an end near the inner side Lin of the other vehicle 6.

<Modification 2 of Embodiment 3>
Modification 4 of Embodiment 2 may be applied to Embodiment 3. That is, in the third embodiment, the video processing unit 12b sets the display mode in the end area 2a to a mode in which the end area 2a is displayed in black, or a mode in which a predetermined display object is superimposed on the end area 2a. Alternatively, the image processing may be changed to a mode in which specific image processing is performed on the end region 2a.

  In this case, the display mode of the end region 2a may be changed based on the positional relationship between the own vehicle 3 and another vehicle 6 other than the own vehicle 3. For example, the video processing unit 12b obtains the distance between the host vehicle 3 and the other vehicle 6 in the traveling direction of the host vehicle 3 based on the positional relationship between the host vehicle 3 and the other vehicle 6, and the distance is short. The transmittance of the display object that is superimposed on the end region 2a may be lower. Alternatively, the video processing unit 12b obtains the distance between the host vehicle 3 and the other vehicle 6 in the lateral direction of the host vehicle 3 based on the positional relationship between the host vehicle 3 and the other vehicle 6, and the distance is short. The color of the display object superimposed on the end area 2a may be closer to a color indicating danger, such as red. Similarly, the video processing unit 12b may change specific image processing parameters based on various distances between the host vehicle 3 and the other vehicle 6.

<Variation 3 of Embodiment 3>
The configurations of the third embodiment and the modifications of the third embodiment may be appropriately combined with the configurations of the second embodiment and the modifications of the second embodiment.

  For example, the image recognizing unit 12a may obtain a feature point FP located on the upper side Lup or in the vicinity of the upper side Lup based on the position of the moving object in the captured image 2. In addition, the image recognition unit 12a may obtain a feature point FP located on the lower side Lud or in the vicinity of the lower side Lud based on the position of the division line in the captured image 2. In this case, the shape of the upper portion of the remaining region 2b is, for example, the same as the shape of the upper portion of the remaining region 2b of FIG. 22, and the lower portion of the remaining region 2b is, for example, the shape of the lower portion of the remaining region 2b of FIG. Will be the same as

  For example, when the moving object cannot be recognized from the captured image 2, the image recognition unit 12 a obtains the feature point FP based on the position of the vanishing point of the lane marking in the captured image 2 as in the second embodiment. You may. When the moving object can be recognized from the captured image 2, the image recognizing unit 12 a does not determine the position of the vanishing point of the dividing line in the captured image 2 but the position of the moving object in the captured image 2 as in the third embodiment. The feature point FP may be obtained based on the position.

<Embodiment 4>
The block diagram showing the configuration of the display control device 1 according to Embodiment 4 of the present invention is the same as the block diagram (FIG. 6) of the display control device 1 according to Embodiment 2. Hereinafter, among the components according to the fourth embodiment, the same or similar components as those described above are denoted by the same reference numerals, and different components will be mainly described.

  In the above description, the position of the second point P2 and the position of the sixth point P6 have been described as being fixed. On the other hand, in the fourth embodiment, the position of the second point P2 and the position of the sixth point P6 change appropriately.

  FIGS. 26 and 27 are diagrams schematically showing the end area 2a and the remaining area 2b in the left captured image 2 displayed by the left display device 21a according to the fourth embodiment.

  The image recognition unit 12a according to the fourth embodiment recognizes a predetermined object from the captured image 2 acquired by the image acquisition unit 11a, and determines a second point P2 and a feature point FP based on the shape of the object. Ask. In the fourth embodiment, this object is a lane marking of the road, and the image recognition unit 12a determines that the shape of the line L3 connecting the second point P2 and the feature point FP corresponds to the shape of the solid white line 5a. A second point P2 and a feature point FP are obtained.

  More specifically, first, the image recognition unit 12a obtains a feature point FP from the captured image 2 in the same manner as in the second embodiment. Thereafter, the image recognition unit 12a obtains the second point P2 such that the line L3 is a parallel line of the solid white line 5a or a line corresponding to the solid white line 5a. When the solid white line 5a is a straight line as shown in FIG. 26, the line L3 is also a straight line, and when the solid white line 5a is a curved line as shown in FIG. 27, the line L3 is also a curved line.

<Summary of Embodiment 4>
In the display control device 1 according to the fourth embodiment as described above, the second point P2 and the feature point FP are obtained based on the shape of the object recognized from the captured image 2. According to such a configuration, the boundary line between the end region 2a and the remaining region 2b additionally indicates the solid white line 5a, so that a display is provided that makes it easy for the driver to recognize the solid white line 5a. be able to.

<Modification of Fourth Embodiment>
In the fourth embodiment, the image recognizing unit 12a obtains the characteristic point FP, and then obtains the second point P2 based on the characteristic point FP and the shape of the solid white line 5a. However, the image recognition unit 12a may obtain the feature point FP based on the fixed second point P2 and the shape of the solid white line 5a as in the first embodiment. Alternatively, a parallel line whose distance from the solid white line 5a is set to a fixed distance may be the line L3. In this case, the image recognition unit 12a can obtain the second point P2 and the feature point FP in parallel.

<Embodiment 5>
FIG. 28 is a block diagram showing a configuration of the display control device 1 according to Embodiment 5 of the present invention. Hereinafter, among the components according to the fifth embodiment, the same or similar components as those described above are denoted by the same reference numerals, and different components will be mainly described.

  In the fifth embodiment, instead of the left rear side camera 26a and the right rear side camera 26b in FIG. 6, a left rear side wide angle camera 26e and a right rear side wide angle camera 26f are attached to the vehicle. Is provided. Here, the left rear wide angle camera 26e and the right rear wide angle camera 26f are not standard cameras but wide angle cameras. In accordance with this, each of the left display device 21a and the right display device 21b according to the fifth embodiment is a display device capable of displaying a horizontally long image displayed by a wide-angle camera.

  FIG. 29 is a diagram showing a photographing region R1 that is a region that can be photographed by a standard camera having an angle of view of 10 degrees and a photographing region R2 that can be photographed by a wide-angle camera that has an angle of view of 30 degrees. It is. FIG. 30 is a diagram illustrating a display example of the left display device 21a according to the fifth embodiment. Note that, in the display example of FIG. 30, for convenience of explanation, the vanishing point 5b, the feature point FP, and the broken line between the vanishing point 5b and the feature point FP are shown, but these may not be displayed. . This is the same in the following display examples.

  As shown in FIG. 30, the shooting area R2 of the wide-angle camera is wider than the shooting area R1 of the standard camera, and the display range of the left display device 21a is wider in accordance with the shooting area R2 of the wide-angle camera. ing. As a result, the photographed image 2 is a horizontally long image.

  In the fifth embodiment, the image recognition unit 12a obtains a feature point FP for such a horizontally-long photographed image 2 in the same manner as described above, and the video processing unit 12b sets the end region 2a and the The remaining area 2b is obtained, and control is performed so that the display mode of the end area 2a is different from the display mode of the remaining area 2b.

<Summary of Embodiment 5>
In the display control device 1 according to the fifth embodiment described above, the captured image 2 is a horizontally long image. According to such a configuration, the effects described in the first embodiment and the like can be obtained even for a horizontally long image captured by a wide-angle camera.

<Modification 1 of Embodiment 5>
The configurations of the second to fourth embodiments and their modifications may also be applied to the configuration of the fifth embodiment. FIG. 31 is a diagram illustrating a display example of the left display device 21a in a case where Modification 2 of Embodiment 2 is applied to Embodiment 5.

  As shown in FIG. 31, the image recognition unit 12a determines a point between the vanishing point 5b of the solid white line 5a and the upper side Lup as the first feature point FP1, and among the points on the upper side Lup, the disappearance of the dotted white line 5c. A point directly above the point 5d may be obtained as the second feature point FP2. Then, the video processing unit 12b may determine an area surrounded by the first point P1, the second point P2, the first feature point FP1, and the second feature point FP2 as the end area 2a. Further, the number of feature points such as the first feature point FP1 and the second feature point FP2 is not limited to two, and the number of section lines such as the solid white line 5a and the dotted white line 5c is not limited to two. Absent.

<Modification 2 of Embodiment 5>
In the above configuration, the range of the remaining area 2b displayed on the display device 21 may be changed based on the number of lanes of the road on which the host vehicle 3 is traveling. FIGS. 32 and 33 are diagrams illustrating display examples of the left display device 21a according to the second modification of the fifth embodiment. 32 and 33, the wall 8 provided outside the outermost road is also displayed.

  In the second modification, the image recognition unit 12a extracts the number of lanes from the captured image 2 acquired by the image acquisition unit 11a. The image recognition unit 12a extracts the number of two lanes (one adjacent lane) in the case of FIG. 32, and extracts the number of three lanes (two adjacent lanes) in the case of FIG.

  In the second modification, the video processing unit 12b increases the left-right length of the remaining area 2b on the display of the display device 21 as the number of lanes extracted by the image recognition unit 12a increases. In the examples of FIGS. 32 and 33, the video processing unit 12b is located outside the remaining area 2b at the lower end 5e of the solid white line 5a that is the outermost lane marking of the lane in which the host vehicle 3 is traveling. , The length of the remaining region 2b in the left-right direction is increased.

  According to the above configuration, a relatively important image around the own vehicle 3 can be extracted and displayed.

<Embodiment 6>
FIG. 34 is a block diagram showing a configuration of a display control device 1 according to Embodiment 6 of the present invention. Hereinafter, among the components according to the sixth embodiment, the same or similar components as those described above are denoted by the same reference numerals, and different components will be mainly described.

  The configuration of the sixth embodiment is the same as the configuration of FIG. 28 except that a left / right turn information acquisition unit 11b is added. Note that the concept of the right / left turn information acquisition unit 11b is a concept included in the acquisition unit 11 of FIG.

  The right / left turn information acquisition unit 11b acquires the operation received by the direction indicator of the own vehicle 3 as right / left turn information indicating whether the own vehicle 3 performs a right turn or a left turn. The image acquisition unit 11a acquires a left captured image that is a captured image corresponding to the rear left of the own vehicle 3 and a right captured image that is a captured image corresponding to the right rear of the own vehicle 3 as described above. I do.

  35 and 36 are diagrams illustrating display examples of the left display device 21a and the right display device 21b according to the sixth embodiment.

  The video processing unit 12b according to the sixth embodiment sets the left and right turn information of the remaining area 2b of the left captured image 2 in the display of the left display device 21a based on the right / left turn information acquired by the right / left turn information acquisition unit 11b. The length and the length in the left-right direction of the remaining area 2b of the right photographed image 2 on the display of the right display device 21b are controlled.

  As an example, when the right / left turn information indicates a left turn, the video processing unit 12b switches from the display in FIG. 35 to the display in FIG. That is, when the right / left turn information indicates a left turn, the video processing unit 12b displays the left-right length of the remaining area 2b of the left captured image 2 displayed on the left display device 21a on the right display device 21b. The length of the remaining area 2b of the right photographed image 2 is made larger than the length in the left-right direction. Also, although not shown, when the right / left turn information indicates a right turn, the video processing unit 12b calculates the left-right length of the remaining area 2b of the right captured image 2 displayed on the right display device 21b by using the left display device. The length in the left-right direction of the remaining area 2b of the left photographed image 2 displayed on 21a is made larger.

  In the left display device 21a in the example of FIG. 36, a point directly above the vanishing point 5b among points on the upper side Lup of the left captured image 2 is obtained as the feature point FP, and the end area 2a and the remaining area are obtained. 2b are classified based on the feature points FP and the like. On the other hand, in the right display device 21b in the example of FIG. 36, the feature point FP is not obtained, and the end region 2a and the remaining region 2b are not divided, so that the right photographed image 2 is left as it is, This is displayed as the remaining area 2b of the image 2.

<Summary of Embodiment 6>
In the display control device 1 according to the sixth embodiment described above, the left-right length of the remaining area 2b of the left captured image 2 and the left-right length of the remaining area 2b of the right captured image 2 are determined based on the left / right turn information. Control the length in the horizontal direction. According to such a configuration, it is possible to present to the driver a display in which the situation in the traveling direction can be easily checked.

<Modification 1 of Embodiment 6>
In the sixth embodiment, the video processing unit 12b increases the left-right length of the remaining area 2b for one of the left captured image 2 and the right captured image 2 when the right / left turn information indicates the direction. It is not limited to this. For example, when the right / left turn information indicates the direction, the video processing unit 12b may increase the left-right length of the remaining area 2b for both the left captured image 2 and the right captured image 2.

<Modification 2 of Embodiment 6>
The left / right turn information may be input to the image recognition unit 12a. When the right / left turn information indicates a left turn, the image recognition unit 12a determines the maximum number of feature points that can be extracted from the left captured image 2 and the maximum number of feature points that can be extracted from the right captured image 2. May be increased, or the extraction of the feature points from the right photographed image 2 may be stopped. Similarly, when the right / left turn information indicates a right turn, the image recognition unit 12a determines the maximum number of feature points that can be extracted from the right captured image 2 and the maximum number of feature points that can be extracted from the left captured image 2. Alternatively, the extraction of the feature points from the left captured image 2 may be stopped.

<Variation 3 of Embodiment 6>
FIGS. 37 and 38 are diagrams illustrating display examples of the left display device 21a and the right display device 21b according to the third modification of the sixth embodiment. In the third modification as well, it is assumed that the right / left turn information is input to the image recognition unit 12a.

  The image recognizing unit 12a may recognize a division line such as a solid white line 5a of a road from the left captured image and the right captured image acquired by the image acquiring unit 11a in the same manner as in the second embodiment. Then, similarly to the third embodiment, a moving object such as another vehicle 6 may be recognized from the left captured image and the right captured image acquired by the image acquisition unit 11a.

  Further, when the right / left turn information acquired by the right / left turn information acquisition unit 11b indicates a left turn, the image recognition unit 12a determines the feature point FPL of the left captured image 2 based on the other vehicle 6 in the left captured image 2. Alternatively, the characteristic point FPR of the right photographed image 2 may be obtained based on the solid white line 5a in the right photographed image 2. Further, when the right / left turn information acquired by the right / left turn information acquisition unit 11b indicates a right turn, the image recognition unit 12a determines the characteristic point FPR of the right photographed image 2 based on the other vehicle 6 in the right photographed image 2. Alternatively, the characteristic point FPL of the left captured image 2 may be determined based on the solid white line 5a in the left captured image 2.

  Further, the configuration of the sixth embodiment and each modification of the sixth embodiment may be applied to the second to fifth embodiments.

<Embodiment 7>
FIG. 39 is a block diagram showing a configuration of the display control device 1 according to Embodiment 7 of the present invention. Hereinafter, among the components according to the seventh embodiment, the same or similar components as those described above are denoted by the same reference numerals, and different components will be mainly described.

  In the configuration of the seventh embodiment, in addition to the configuration of FIG. 6, a rear camera 26h capable of capturing a rear image is provided in the host vehicle 3.

  The camera control device 26c combines the left camera image and the right camera image captured by the left rear lateral camera 26a and the right rear lateral camera 26b, and the right image captured by the rear camera 26h, and combines them. Generate an image. Then, the camera control device 26c outputs the generated composite image to the display control device 1. The image acquisition unit 11a acquires a composite image from the camera control device 26c as a captured image. Therefore, in the seventh embodiment, the captured image 2 includes a horizontally long image including the left rear image, the right rear image, and the right rear image between the left rear and the right rear of the vehicle 3. Become.

  FIG. 40 is a diagram illustrating a display example of the display device 21 according to the seventh embodiment. As in the second embodiment, the image recognition unit 12a according to the seventh embodiment recognizes the solid white line 5a from the captured image 2 acquired by the image acquisition unit 11a, and places the solid white line 5a at the position of the vanishing point 5b of the solid white line 5a. Then, a feature point FP is obtained based on this. In the example of FIG. 40, the image recognition unit 12a recognizes the leftmost solid white line 5a and the rightmost solid white line 5a from the captured image 2.

  As in the second embodiment, the video processing unit 12b according to the seventh embodiment includes the captured image 2 acquired by the image acquisition unit 11a and the feature points obtained by the image recognition unit 12a in accordance with a predetermined rule. The end area 2a is obtained based on the FP. In the example of FIG. 40, the video processing unit 12b determines the left end region 2aL by using the left side as the outer side Lout, the right side as the inner side Lin, and using a predetermined rule as the first rule. . Further, the video processing unit 12b obtains the right end region 2aR by using the right side as the outer side Lout, the left side as the inner side Lin, and using a predetermined rule as a first rule.

  Then, the video processing unit 12b makes the display mode of the end areas 2aL and 2aR different from the display mode of the remaining area 2b that is the area of the captured image 2 other than the end areas 2aL and 2aR on the display of the display device 21. Perform control. Note that the display object indicating the host vehicle 3 may be appropriately displayed on the captured image 2 or the position of the display object may be appropriately changed.

<Summary of Embodiment 7>
In the display control device 1 according to the seventh embodiment as described above, the captured image 2 includes the left rear image, the right rear image, and the right rear image between the left rear and the right rear of the vehicle 3. The same effect as in the second embodiment can be obtained even with a horizontally long image composed of.

<Embodiment 8>
FIG. 41 is a block diagram showing a configuration of the display control device 1 according to Embodiment 8 of the present invention. Hereinafter, among the components according to the seventh embodiment, the same or similar components as those described above are denoted by the same reference numerals, and different components will be mainly described.

  The configuration of the eighth embodiment is the same as the configuration of FIG. 6 except that the image recognition unit 12a is deleted, the peripheral information acquisition unit 11c is added, and the peripheral detection device 27 is connected. Note that the concept of the peripheral information acquisition unit 11c is a concept included in the acquisition unit 11 of FIG.

  The peripheral detection device 27 is provided outside the display control device 1. The periphery detection device 27 detects object information on a relative position of a predetermined object around the host vehicle 3 with respect to the host vehicle 3. In the eighth embodiment, the predetermined object is a division line such as a solid white line around the own vehicle 3, and the object information is, for example, the relative position of the division line itself or photographed by each camera. The distance between the host vehicle 3 and the lane marking obtained by performing image recognition on an image.

  The peripheral information acquisition unit 11c acquires the object information detected by the peripheral detection device 27. The video processing unit 12b calculates a feature point based on the object information acquired by the peripheral information acquisition unit 11c. That is, in the second embodiment and the like, the image recognition unit 12a obtains the feature points, but in the eighth embodiment, the surrounding detection device 27 and the video processing unit 12b cooperate to obtain the feature points.

  The video processing unit 12b obtains the end region 2a based on the captured image 2 obtained by the image obtaining unit 11a and the obtained feature points according to a predetermined rule. Then, the video processing unit 12b controls the display of the display device 21 so that the display mode of the end area 2a is different from the display mode of the remaining area 2b which is the area of the captured image 2 other than the end area 2a.

<Summary of Embodiment 8>
In the display control device 1 according to the eighth embodiment described above, a feature point is obtained based on the object information acquired by the peripheral information acquisition unit 11c. According to such a configuration, a part of the process for obtaining the feature points can be performed outside the display control device 1, so that the processing load on the display control device 1 can be reduced.

<Modification 1 of Embodiment 8>
In the eighth embodiment, the predetermined object is described as a lane marking around the own vehicle 3, but is not limited to this. For example, the predetermined object may be a moving object such as another vehicle 6 around the own vehicle 3. In this case, the surrounding detection device 27 may detect the object information by performing image recognition processing on an image captured by each camera, for example, or may include an optical sensor, a millimeter wave radar, a high-precision image recognition device, The object information may be detected by an ultrasonic sensor or the like.

<Modification 2 of Embodiment 8>
Instead of detecting the object information, the peripheral detection device 27 may detect the feature points in the same manner as the image recognition unit 12a of the second embodiment. In this case, the peripheral information acquisition unit 11c acquires a feature point from the peripheral detection device 27. Then, the video processing unit 12b obtains the end area 2a based on the captured image 2 obtained by the image obtaining unit 11a and the feature points obtained by the peripheral information obtaining unit 11c according to a predetermined rule.

  Alternatively, instead of detecting the object information, the periphery detection device 27 may detect the second point and the feature point in the same manner as the image recognition unit 12a of the fourth embodiment, The second point and the feature point may be detected based on the time series information of the body. In this case, the peripheral information acquisition unit 11c acquires the second point and the feature point from the peripheral detection device 27. Then, the video processing unit 12b, based on a predetermined rule, based on the captured image 2 obtained by the image obtaining unit 11a and the second feature point and the feature point obtained by the peripheral information obtaining unit 11c, The area 2a is obtained.

  According to such a configuration, the processing for obtaining the characteristic point or the processing for obtaining the second point and the characteristic point can be performed outside the display control apparatus 1. Can be reduced.

<Other modifications>
The above-described acquisition unit 11 and control unit 12 in FIG. 1 are hereinafter referred to as “acquisition unit 11 and the like”. The acquisition unit 11 and the like are realized by the processing circuit 81 illustrated in FIG. That is, the processing circuit 81 obtains the captured image around the vehicle and, based on the captured image and the characteristic points on the captured image, the end area of the captured image area in accordance with a predetermined rule. A control unit 12 that determines the display mode of the end area 2a in the display of the display device 21 and makes the display mode of the end area 2a different from the display mode of the remaining area 2b that is the area of the captured image other than the end area 2a. Prepare. As the processing circuit 81, dedicated hardware may be applied, or a processor that executes a program stored in a memory may be applied. For example, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), and the like correspond to the processor.

  When the processing circuit 81 is dedicated hardware, the processing circuit 81 includes, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), and an FPGA (Field Programmable Gate). Array), or a combination of these. The function of each unit such as the acquisition unit 11 may be realized by a circuit in which processing circuits are distributed, or the function of each unit may be realized by one processing circuit.

  When the processing circuit 81 is a processor, the functions of the acquisition unit 11 and the like are realized by a combination with software and the like. In addition, software etc. correspond to software, firmware, or software and firmware, for example. Software and the like are described as programs and stored in the memory 83. As shown in FIG. 43, a processor 82 applied to the processing circuit 81 reads out and executes a program stored in a memory 83 to realize the function of each unit. That is, when the display control device 1 is executed by the processing circuit 81, the display control device 1 obtains an image around the vehicle and, based on the captured image and the characteristic points on the captured image, according to a predetermined rule. Control to determine the end area 2a of the area and to make the display mode of the end area 2a different from the display mode of the remaining area 2b, which is the area of the captured image other than the end area 2a, on the display device 21. And a memory 83 for storing a program to be executed as a result. In other words, it can be said that this program causes a computer to execute the procedure and method of the acquisition unit 11 and the like. Here, the memory 83 is, for example, a nonvolatile memory such as a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), and an electrically erasable programmable read only memory (EEPROM). Volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), its drive device, etc., or any storage medium to be used in the future. You may.

  The configuration in which the functions of the acquisition unit 11 and the like are realized by one of hardware and software has been described above. However, the present invention is not limited to this, and a configuration in which a part of the acquisition unit 11 and the like is realized by dedicated hardware and another part is realized by software and the like may be employed. For example, the function of the acquisition unit 11 is realized by a processing circuit 81 as dedicated hardware and a receiver, and the processing circuit 81 as the processor 82 otherwise reads and executes a program stored in the memory 83. By doing so, it is possible to realize that function.

  As described above, the processing circuit 81 can realize each function described above by hardware, software, or the like, or a combination thereof.

  The display control device 1 described above includes a navigation device such as a PND (Portable Navigation Device), a communication terminal including a mobile terminal such as a mobile phone, a smartphone, and a tablet, and at least one of a navigation device and a communication terminal. The present invention can also be applied to a display control system constructed as a system by appropriately combining a function of an application to be installed and a server. In this case, each function or each component of the display control device 1 described above may be dispersedly arranged in each device constituting the system, or may be arranged intensively in any device. Good. For example, the display control device may be incorporated in the display device 21 of FIG.

  FIG. 44 is a block diagram illustrating a configuration of a server 91 according to the present modification. 44 includes a communication unit 91a and a control unit 91b, and can perform wireless communication with a navigation device 93 of a vehicle 92.

  The communication unit 91a, which is an acquisition unit, receives a captured image around the vehicle 92 by performing wireless communication with the navigation device 93.

  The control unit 91b has a function similar to that of the control unit 12 in FIG. 1 when a processor (not shown) of the server 91 executes a program stored in a memory (not shown) of the server 91. That is, the control unit 91b appropriately obtains a feature point of the captured image from the captured image received by the communication unit 91a, and obtains an end area based on the captured image and the characteristic points according to a predetermined rule. Then, the control unit 91b generates a control signal for performing control to make the display mode of the end area different from the display mode of the remaining area in the display of the navigation device 93. The communication unit 91a transmits the control signal to the navigation device 93. According to the server 91 configured as described above, the same effects as those of the display control device 1 described in the first embodiment can be obtained.

  FIG. 45 is a block diagram illustrating a configuration of a communication terminal 96 according to the present modification. The communication terminal 96 in FIG. 45 includes a communication unit 96a similar to the communication unit 91a and a control unit 96b similar to the control unit 91b, and can perform wireless communication with the navigation device 98 of the vehicle 97. ing. The communication terminal 96 is, for example, a mobile terminal such as a mobile phone, a smartphone, and a tablet carried by the driver of the vehicle 97. According to the communication terminal 96 configured as described above, the same effects as those of the display control device 1 described in the first embodiment can be obtained.

  In the present invention, each embodiment and each modified example can be freely combined, and each embodiment and each modified example can be appropriately modified or omitted within the scope of the invention.

  Although the present invention has been described in detail, the above description is illustrative in all aspects and the present invention is not limited thereto. It is understood that innumerable modifications that are not illustrated can be assumed without departing from the scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 Display control apparatus, 2 captured image, 2a end area, 2b remaining area, 3 own vehicle, 5a solid white line, 5b vanishing point, 5c dotted white line, 5d vanishing point, 6 other vehicles, 11 acquisition unit, 12 control unit, 21 display device, FP feature point, FP1 first feature point, FP2 second feature point, Lin inner edge, Lout outer edge, Lud lower edge, Lup upper edge, P1 first point, P2 second point, P3 third point, P4 Fourth point, P5 fifth point, P6 sixth point, P7 seventh point, P8 eighth point.

Claims (21)

A display control device for controlling a display device,
An acquisition unit for acquiring a photographed image around the vehicle,
According to a predetermined rule, an end region of the region of the photographed image is obtained based on the photographed image and a feature point on the photographed image, and a display mode of the end region is displayed on the display device. A control unit that performs control to make the display mode of the remaining area that is the area of the captured image other than the end area different from the display mode,
The captured image is an inner side that is one of the left side and the right side and corresponds to the side where the vehicle is present on the captured image, an outer side that is the other of the left side and the right side, and an upper side. , And the lower side,
The predetermined rule is:
When the feature point is located on the upper side, a first point located at an upper end of the outer side and a point on the outer side other than the first point and located below the feature point. Determining a region surrounded by the second point and the feature point as the end region;
When the feature point is located near the upper side, the first point, the second point, a third point located at an upper end of the inner side, and a point on the inner side, A fourth point located below a straight line connecting the two points and the feature point, the same as the third point or below the third point, and located above the feature point; Obtaining an area surrounded by the feature points as the end area,
When the feature point is located on the lower side, a fifth point located at the lower end of the outer side and a point on the outer side other than the fifth point and located above the feature point Determining a region surrounded by a sixth point and the feature point as the end region; and
When the feature point is located near the lower side, the fifth point, the sixth point, a seventh point located at a lower end of the inner side, and a point on the inner side, An eighth point located above a straight line connecting the six points and the feature point, the same as the seventh point or located above the seventh point, and located below the feature point; Obtaining a region surrounded by a point as the end region,
A display control device comprising at least one of the following. The display control device according to claim 1,
The control unit includes:
A display control device that recognizes a predetermined object from the captured image acquired by the acquisition unit and obtains the feature point based on the object. The display control device according to claim 2, wherein
The display control device, wherein the object includes a lane marking. The display control device according to claim 3, wherein
The control unit includes:
A display control device that obtains the feature point based on a position of a vanishing point of the lane marking in the captured image. The display control device according to claim 2, wherein
The object includes a moving object,
The control unit includes:
A display control device for obtaining the feature point based on a position of the moving body in the captured image. The display control device according to claim 2, wherein
The objects include road markings and moving objects,
The control unit includes:
When the moving body cannot be recognized from the captured image, the feature point is obtained based on a position of a vanishing point of the lane marking in the captured image, and when the moving body can be recognized from the captured image. Is a display control device for obtaining the feature point based on a position of the moving body in the captured image. The display control device according to claim 1,
The acquisition unit,
Further acquiring object information on a relative position of the predetermined object around the vehicle with respect to the vehicle,
The control unit includes:
A display control device that obtains the feature point based on the object information acquired by the acquisition unit. The display control device according to claim 7, wherein
The display control device, wherein the object includes a moving body around the vehicle. The display control device according to claim 1,
The control unit includes:
A display control device that recognizes a predetermined object from the captured image acquired by the acquisition unit and obtains the second point and the feature point based on a shape of the object. The display control device according to claim 9,
The object includes a lane marking of a road,
The control unit includes:
A display control device for determining the second point and the feature point such that a shape of a line connecting the second point and the feature point corresponds to a shape of the division line. The display control device according to claim 1,
The display control device, wherein the acquisition unit further acquires the feature point, or further acquires the second point and the feature point. The display control device according to claim 1,
The control unit includes:
From the captured image, a first feature point that is the feature point located near the upper side, and a first feature point that is located on the inner side of the first feature point and located on the upper side. When two feature points are obtained, a region surrounded by the first point, the second point, the first feature point, and the second feature point is obtained as the end region,
From the captured image, a third feature point that is the feature point located near the lower side, and a third feature point that is located on the inner side of the third feature point and located on the lower side. A display control unit that, when four feature points are obtained, obtains an area surrounded by the fifth point, the sixth point, the third feature point, and the fourth feature point as the end area; apparatus. The display control device according to claim 1,
The display control device, wherein the captured image includes a horizontally long image including a left rear image, a right rear image of the vehicle, and an image between the left rear and the right rear. The display control device according to claim 1,
The display mode of the end region is
A display control device including a mode in which the end region is displayed in black, a mode in which a predetermined display object is superimposed on the end region, or a mode in which specific image processing is performed on the end region . The display control device according to claim 14, wherein
The display control device, wherein the specific image processing includes a translucent processing, a blur processing, or a color change processing. The display control device according to claim 14, wherein
The display control device, wherein a display mode of the end region is changed based on a positional relationship between the vehicle and a vehicle other than the vehicle. The display control device according to claim 1,
The acquisition unit,
A left captured image that is the captured image corresponding to the left rear of the vehicle and a right captured image that is the captured image corresponding to the right rear of the vehicle are obtained, and the vehicle is turned right and left. Get more left / right turn information to indicate which one to do,
The control unit includes:
Based on the right / left turn information acquired by the acquisition unit, the left-right length of the remaining area of the left photographed image on the display of the display device, and the right photographed image of the right on the display of the display device A display control device that controls the length of the remaining area in the left-right direction. The display control device according to claim 17, wherein
The control unit includes:
When the right / left turn information indicates a left turn, the left-right length of the remaining region of the left captured image is larger than the left-right length of the remaining region of the right captured image,
A display control device, wherein when the right / left turn information indicates a right turn, the left-right length of the remaining region of the right captured image is larger than the left-right length of the remaining region of the left captured image. The display control device according to claim 17, wherein
The control unit includes:
Recognizing lane markings and moving objects from the left captured image and the right captured image acquired by the acquiring unit,
The control unit includes:
When the right / left turn information indicates a left turn, the characteristic point of the left captured image is obtained based on the moving body in the left captured image, and the feature point is determined based on the division line in the right captured image. Find the feature points of the right shot image,
When the right / left turn information indicates a right turn, the characteristic point of the right photographed image is obtained based on the moving body in the right photographed image, and the feature point of the left photographed image is obtained. A display control device for obtaining the feature point of the photographed image on the left. The display control device according to claim 1,
The control unit includes:
A display control device, wherein the number of lanes is extracted from the captured image acquired by the acquisition unit, and the longer the number of lanes is, the larger the left-right length of the remaining area in the display of the display device is. A display control method for controlling a display device,
Acquire the captured image around the vehicle,
According to a predetermined rule, an end region of the region of the photographed image is obtained based on the photographed image and a feature point on the photographed image, and a display mode of the end region is displayed on the display device. Perform a control to be different from the display mode of the remaining area that is the area of the captured image other than the end area,
The captured image is an inner side that is one of the left side and the right side and corresponds to the side where the vehicle is present on the captured image, an outer side that is the other of the left side and the right side, and an upper side. , And the lower side,
The predetermined rule is:
When the feature point is located on the upper side, a first point located at an upper end of the outer side and a point on the outer side other than the first point and located below the feature point. Determining a region surrounded by the second point and the feature point as the end region;
When the feature point is located near the upper side, the first point, the second point, a third point located at an upper end of the inner side, and a point on the inner side, A fourth point located below a straight line connecting the two points and the feature point, the same as the third point or below the third point, and located above the feature point; Obtaining an area surrounded by the feature points as the end area,
When the feature point is located on the lower side, a fifth point located at the lower end of the outer side and a point on the outer side other than the fifth point and located above the feature point Determining a region surrounded by a sixth point and the feature point as the end region; and
When the feature point is located near the lower side, the fifth point, the sixth point, a seventh point located at a lower end of the inner side, and a point on the inner side, An eighth point located above a straight line connecting the six points and the feature point, the same as the seventh point or located above the seventh point, and located below the feature point; Obtaining a region surrounded by a point as the end region,
A display control method comprising at least one of the following.

Images