JP5589644B2 - Peripheral image display device and display method thereof - Google Patents

Description

  The present invention relates to a peripheral image display device that displays a peripheral image by capturing an omnidirectional image by an omnidirectional imaging unit and a display method thereof.

  Conventionally, various omnidirectional imaging means for imaging an omnidirectional image have been proposed, and disclosed in, for example, Patent Documents 1 and 2. Such an omnidirectional imaging means 1 reflects ambient light L with a hyperboloidal reflecting mirror 2 and images the surroundings with an imaging element 3 such as a CCD (see FIG. 4). The image sensor 3 is mounted on the circuit board 4, and the circuit board 4 on which the image sensor 3 is mounted is disposed on the bottom wall portion 5 a of the case body 5. A cylindrical tube 6 made of a translucent material is provided on the side wall 5 b of the case body 5, and the reflection mirror 2 is disposed inside the tube 6. A lid body 7 is disposed in the upper opening 6 a of the cylindrical body 6, and the reflection mirror 2 is fixed to the lid body 7. The reflection mirror 2 has a hyperboloid shape, and is arranged so as to hang from the lid body 7 with the apex of the hyperboloid shape down.

  Further, the applicant of the present application has proposed an omnidirectional imaging means 10 that can be manufactured at a low cost with a small number of parts, and such an omnidirectional imaging means 10 is disclosed in Patent Document 3. The omnidirectional imaging unit 10 includes an optical member 11, a support member 12, an imaging element 13, a circuit board 14, and a reflective layer 15 (see FIG. 5). The image sensor 13 is mounted on the circuit board 14 and is arranged so that the central axis of the optical member 11 and the central axis of the image sensor 13 are aligned. The ambient light L is incident on the inside of the base portion 16 from the incident surface 16 a, is reflected by the reflective layer 15, and is guided to the image sensor 13 by the light guide portion 17. The ambient light L reflected downward by the reflective layer 15 exits from the emission surface 17b and is imaged by the imaging element 13. The peripheral image display device that displays a peripheral image using the omnidirectional imaging means uses the image processing circuit to output the captured image S1 (see FIG. 6A) output from the imaging elements 3 and 13 of the omnidirectional imaging means 1 and 10. It is converted into a panoramic image G1 (see FIG. 6B) showing the omnidirectional, a cutout image G2 (see FIG. 6C) obtained by cutting out a part of the omnidirectional, and the like, and displayed on a liquid crystal display, an organic EL display, or the like. It is displayed as a peripheral image on a means (not shown).

Japanese Patent Laid-Open No. 11-207772 JP 2000-347650 A JP 2007-81925 A

  However, in the case where the entire captured image is displayed in the panoramic image G1 and / or in the case where a part of the captured entire image is displayed in the cutout image G2, the objects included in the displayed images G1 and G2 are displayed. It is difficult to determine in which direction it is located, and there is a problem that the information that the user wants to obtain cannot be obtained intuitively. For example, as shown in FIG. 6, in the panoramic image G1, it seems that the persons M1 to M4 are arranged in the same direction, and the person M1 is only displayed in the cut-out image G2. It was difficult to determine which direction each of M1 to M4 exists.

  It is a first object of the present invention to provide a peripheral image display device and a display method therefor that can easily grasp the omnidirectional positional relationship of an imaged object. In addition, a second object of the present invention is to provide a peripheral image display device and a display method thereof that are highly convenient to operate.

In order to solve the above-mentioned problems, the present invention provides an omnidirectional imaging unit that images omnidirectional images, an annular image and panoramic image indicating omnidirectional images, and a part of the omnidirectional image data captured by the omnidirectional imaging unit. Control means for converting to a cutout image, a display means for simultaneously displaying the annular image, the panoramic image, and the cutout image, and the annular image, panoramic image, or cutout image displayed on the display means. What Re also provided with an operation unit capable of performing a selection operation of selecting the cut-out area of the cut-out image with respect,
In accordance with the selection operation, the control unit switches the display form of the annular image, the panoramic image, and the cutout image in conjunction with each other,
The annular image includes a first cutout region display unit indicating the cutout region in the annular image, and the panoramic image includes a second cutout region display unit indicating the cutout region in the panoramic image,
The control means moves the first cutout region display unit and the second cutout region display unit as switching of the display form of the annular image, the panoramic image, and the cutout image according to the selection operation. And switching of the cut-out image are performed in conjunction with each other.

  Further, the operation means is composed of a pointing device.

  The pointing device may be a touch panel arranged on a display screen of the display means.

In order to solve the above problems, the present invention converts imaging data obtained by imaging all directions into a ring image and panorama image showing all directions and a cut-out image obtained by cutting out a part of all directions, and the ring image and the panorama. A display method of a peripheral image display device for simultaneously displaying an image and the cutout image on a display means,
Wherein said annular image displayed on the display unit, is capable of selection operation of selecting the cut-out area of the clipped image to anything Re of the panoramic image or the clipped image, in response to the selection operation, the annular The display mode of the image, the panoramic image and the cutout image is switched in conjunction with each other,
The annular image includes a first cutout region display unit indicating the cutout region in the annular image, and the panoramic image includes a second cutout region display unit indicating the cutout region in the panoramic image,
As switching of the display form of the annular image, the panoramic image, and the cutout image according to the selection operation, the movement of the first cutout region display unit, the movement of the second cutout region display unit, and the cutout image The switching is performed in conjunction with each other.

  The present invention can easily grasp the positional relationship in all directions of an object to be imaged. In addition, the present invention can improve the convenience of operation.

Sectional drawing and block diagram which show embodiment of this invention. Explanatory drawing of the captured image and display image which show embodiment same as the above. Explanatory drawing of the display image which shows embodiment same as the above. Sectional drawing which shows a prior art example. Sectional drawing which shows another prior art example. Explanatory drawing of the captured image and display image which show a prior art example.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. The peripheral image display device includes an omnidirectional imaging unit 20, a display control unit (control unit) 30, a display unit 40, and an operation unit 50.

  The omnidirectional imaging means 20 includes a reflection mirror 21, a case body 22, an image sensor 23, and a lens 24, and images all directions with respect to the central axis of the reflection mirror 21.

  The reflection mirror 21 is an optical member having a hyperboloid shape, and is arranged with the apex of the hyperboloid shape facing down. The reflection mirror 21 reflects the ambient light L downward, and the ambient light L reflected by the reflection mirror 21 is condensed on the image sensor 23 by the lens 24. The image sensor 23 is composed of a CCD or a CMOS, and outputs an omnidirectional captured image as imaging data to the display control unit 30.

  The case body 22 has a substantially pudding shape having a side surface 22 a made up of a conical surface and an upper surface 22 b made up of a plane, and accommodates the lens 24 and the image sensor 23. A circular opening 22c is formed on the upper surface 22b of the case body 22, and the ambient light L reflected by the reflection mirror 21 enters the lens 24 accommodated in the case body 22 through the opening 22c.

  The display control unit 30 converts the imaging data output from the imaging element 23 into a display image and outputs the display image data to the display unit 40. The display means 40 is composed of a liquid crystal display or an organic EL display, and inputs display image data subjected to image processing by the display control unit 30 and displays a display image to be described later. The operation means 50 is composed of, for example, a touch panel arranged on the display screen of the display means 40, detects a touch operation on the display image displayed on the display means 40, and outputs it to the display control unit 30.

  FIG. 2 shows a picked-up image S11 picked up by the image pickup device 23 and an annular image G11, a panoramic image G12, and a cut-out image G13 displayed on the display screen of the display means 40. The captured image S11 includes an annular region S111 in which all directions are imaged, an outer region S112 in which the outer periphery of the reflecting mirror 21 is imaged, and a central region S113 in which the imaging element 23 projected onto the reflecting mirror 21 is imaged (FIG. 2). (See (a)). The display control unit 30 converts the captured image S11 into an annular image G11, further converts the annular image G11 into a panoramic image G2 and a cutout image G13, and simultaneously displays the annular image G11, the panoramic image G12, and the cutout image G13. The information is displayed on the means 40 (see FIG. 2B).

  The annular image G11 is an image indicating all directions, and is an image obtained by removing the outer area S112 and the central area S113 that are invalid areas from the captured image S11. The circular image G11 includes a first cursor G111 (first cut-out area display unit) indicating a cut-out area (area displayed as the cut-out image G13) of the cut-out image G13 in the circular image G11. The first cursor G111 is composed of a fan-shaped colored portion that covers an area corresponding to the cutout image G13 of the annular image G11. Although the annular image G11 has distortion in the object to be imaged (in the present embodiment, the persons M1 to M4), it is possible to intuitively grasp the positional relationship of each person M1 to M4 in all directions.

  The panoramic image G12 is an image indicating all directions, and is a rectangular image obtained by cutting the annular image G11 along a predetermined cutting line. The panoramic image G12 includes a second cursor G121 (second cutout area display unit) indicating a cutout area of the cutout image G13 in the panoramic image G12. The second cursor G121 includes a rectangular colored portion that covers an area corresponding to the cutout image G13 of the panoramic image G12. Although the panoramic image G12 has distortion, it can easily recognize an omnidirectional environment.

  The cut-out image G13 is an image showing a partial region in all directions, and is, for example, a rectangular image formed by part of a rectangular image obtained by cutting the annular image G11 along a predetermined cutting line, that is, a part of the panoramic image G12. is there. The cutout image G13 can be corrected for distortion, and a part of all directions can be grasped in detail with an image without distortion.

  FIG. 3 shows switching of the display form of the annular image G11, the panoramic image G12, and the cutout image G13 when a selection operation for selecting a cutout area of the cutout image G13 is performed by the operation unit 50. FIG. 3 shows a case where the selection operation is performed on the panoramic image G12.

  When the user performs a touch operation on a predetermined location (near the location where the person M4 is displayed in FIG. 3) of the panoramic image G12 on the display screen as the selection operation, the operation means 50 detects the touch position and performs display control. To the unit 30. The display control unit 30 changes the cutout area of the cutout image G13 to a predetermined area calculated from the position touched by the user as a base point, and the second cursor G121 is switched to a new cutout area as a switch of the display form of the panoramic image G12. Move. Further, in conjunction with the selection operation, the display control unit 30 switches the display form of the cutout image G13 to an image corresponding to the new cutout area (an image including the person M4 in FIG. 3), and displays the display form of the annular image G11. As the switching, the first cursor G111 is moved to a position corresponding to the new cutout area (a predetermined area including the person M4 in FIG. 3).

  In the present embodiment, the selection operation for selecting the cutout region of the cutout image G13 performed by the operation unit 50 can be performed on the annular image G11 or the cutout image G13. Specifically, when a predetermined portion of the annular image G11 is touched as the selection operation for the annular image G11, the operation unit 50 detects the touch position and outputs the detected touch position to the display control unit 30, and uses the cutout region of the cutout image G13. The position touched by the person is changed to a predetermined area calculated using the base point, and the first cursor G111 is moved to a new cutout area as the display mode of the annular image G11 is switched. Furthermore, in conjunction with the selection operation, the display control unit 30 switches the display form of the cutout image G13 to an image corresponding to the new cutout area, and sets the second cursor G121 as a new display form for switching the display form of the panoramic image G12. Move to a position corresponding to the cutout area.

  When the clipped image G13 is dragged in a predetermined direction as the selection operation for the clipped image G13, the operation unit 50 detects the drag direction and the drag amount and outputs the drag direction and the drag amount to the display control unit 30. A predetermined area is moved in accordance with the direction and amount dragged by the user, and the display form of the cutout image G13 is switched to an image corresponding to a new cutout area. For example, when the clipped image G13 is dragged to the right in FIG. 3, the clipped region moves to the left by a predetermined region corresponding to the drag amount, and the clipped image G13 is an image corresponding to the new clipped region (panoramic image G12). The image is moved to the left by a predetermined area in the image, and is switched to the image moved counterclockwise by the predetermined area in the annular image G11). Further, in conjunction with the selection operation, the display control unit 30 moves the first cursor G111 to a position corresponding to a new cutout region as a switching of the display form of the annular image G11, and switches the display form of the panoramic image G12. The second cursor G121 is moved to a position corresponding to the new cutout area.

  In the present embodiment, the annular image G11, the panoramic image G12, and the cutout image G13 are displayed on the display unit 40 at the same time, whereby the omnidirectional positional relationship of the images displayed in the panoramic image G12 or the cutout image G13 is represented by the circular image G12. It can be easily grasped and the user can recognize necessary information instantly. In addition, in response to the selection operation for selecting a cutout region of the cutout image G13 performed on any one of the annular image G11, the panoramic image G12, and the cutout image G13, the other two images that are not directly operated are displayed. By switching the forms in conjunction with each other, it is desired to change the orientation with the circular image G11 as a reference and select a cutout region, or to select a range separated from the panoramic image G12 as a cutout region, or slightly move with reference to the cutout image G13 Intuitive operation that can accommodate any of the user's selection desires for the three images G11, G12, and G13, such as wanting to display them, is possible, and the convenience of operation as a peripheral image display device can be enhanced. .

  In the present embodiment, the selection operation for selecting a cutout region of the cutout image G13 includes a touch operation, an operation for selecting a cutout region by a tap operation, and dragging the first and second cursors G111 and G121. This includes an operation of following movement by an operation, an operation of scrolling the cutout region by a flick operation, an operation of expanding or reducing the cutout region by a pinch-out operation or a pinch-in operation, and the like. As the operation means 50, a mouse, a joystick, a touch pad, a pen tablet, a trackball, or the like may be used as another pointing device that can be operated intuitively.

  INDUSTRIAL APPLICABILITY The present invention is suitable for a peripheral image display device that displays a peripheral image by capturing an omnidirectional image by an omnidirectional imaging unit and a display method thereof.

20 omnidirectional imaging means 21 reflecting mirror 23 imaging element 24 lens 30 display control unit (control means)
40 display means 50 operation means G11 annular image G111 first cursor (first cutout area display section)
G12 panoramic image G121 second cursor (second cutout area display section)
G13 Cutout image S11 Captured image

Claims (4)

Omnidirectional imaging means for imaging omnidirectional; control means for converting imaging data captured by the omnidirectional imaging means into a ring image and panoramic image showing omnidirectional and a cutout image obtained by cutting out a part of omnidirectional; display means for displaying said annular image and the panoramic image clipped image simultaneously, the annular image displayed on the display unit, the panoramic image or the cut-out area of the clipped image to anything Re of the cut-out image Operating means capable of performing a selection operation to select,
In accordance with the selection operation, the control unit switches the display form of the annular image, the panoramic image, and the cutout image in conjunction with each other,
The annular image includes a first cutout region display unit indicating the cutout region in the annular image, and the panoramic image includes a second cutout region display unit indicating the cutout region in the panoramic image,
The control means moves the first cutout region display unit and the second cutout region display unit as switching of the display form of the annular image, the panoramic image, and the cutout image according to the selection operation. And a peripheral image display device that perform switching of the cut-out image in conjunction with each other.   The peripheral image display apparatus according to claim 1, wherein the operation unit includes a pointing device.   The peripheral image display apparatus according to claim 2, wherein the pointing device includes a touch panel disposed on a display screen of the display unit. Image data obtained by imaging all directions is converted into a ring image and panorama image showing all directions and a cut image obtained by cutting out a part of all directions, and the ring image, the panorama image, and the cut image are simultaneously displayed on the display means. A display method for a peripheral image display device,
Wherein said annular image displayed on the display unit, is capable of selection operation of selecting the cut-out area of the clipped image to anything Re of the panoramic image or the clipped image, in response to the selection operation, the annular The display mode of the image, the panoramic image and the cutout image is switched in conjunction with each other,
The annular image includes a first cutout region display unit indicating the cutout region in the annular image, and the panoramic image includes a second cutout region display unit indicating the cutout region in the panoramic image,
As switching of the display form of the annular image, the panoramic image, and the cutout image according to the selection operation, the movement of the first cutout region display unit, the movement of the second cutout region display unit, and the cutout image A display method for a peripheral image display device, wherein the switching is performed in conjunction with each other.

Images