JP7127569B2 - Image adjustment system, image adjustment device, and image adjustment method - Google Patents

Description

The present invention relates to an image adjustment system, an image adjustment device, and an image adjustment method.

2. Description of the Related Art In recent years, a head-mounted display has attracted attention as an image display device. A head-mounted display displays an image while being worn on the user's head, thereby allowing the user to feel as if he/she entered a virtual space (immersive feeling). As described in Patent Literature 1, a head mounted display can display an image captured by an external camera via a neckwork.

JP-A-2005-56295

An image captured by a stereo camera capable of capturing a right-eye image and a left-eye image as an omnidirectional camera capable of capturing a 360-degree range is displayed on the head-mounted display so that the user can view the image on the head-mounted display. You can see the image stereoscopically.

However, if the difference between the right-eye image and the left-eye image is large, the user may experience motion sickness called VR (Virtual Reality) sickness. VR sickness tends to occur when there is a large vertical shift between the right-eye image and the left-eye image, a large parallax between the right-eye image and the left-eye image, or a large tilt shift between the right-eye image and the left-eye image.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image adjustment system, an image adjustment device, and an image adjustment method that can easily correct a shift between an image for the right eye and an image for the left eye.

The present invention provides a camera that captures a right-eye image and a left-eye image, an image adjustment device that acquires the right-eye image and the left-eye image as captured images from the cameras, adjusts the captured images, and the an image display device for displaying the captured image adjusted by the image adjustment device; and a controller for outputting instruction information to the image adjustment device, wherein the image adjustment device includes an image generation unit for generating a spherical image; acquiring the spherical image from the image generation unit based on the instruction information, displaying the spherical image on the image display device, rotating the spherical image based on the instruction information, and making the spherical image correspond to the rotation of the spherical image; and an image processing unit that adjusts the image for the right eye or the image for the left eye in the captured image displayed on the image display device.

The present invention includes an image generation unit that generates a spherical image, acquires the spherical image from the image generation unit based on instruction information acquired from a controller, displays the spherical image on an image display device, and displays the spherical image on an image display device based on the instruction information. an image processing unit that rotates the spherical image and adjusts the image for the right eye or the image for the left eye in the captured image captured by the camera and displayed on the image display device in accordance with the rotation of the spherical image; To provide an image adjustment device comprising:

In the present invention, a camera captures an image for the right eye and an image for the left eye, an image processing unit acquires the image for the right eye and the image for the left eye from the cameras as captured images, and an image display device captures the captured images. the image processing unit acquires instruction information from a controller, the image processing unit acquires a spherical image from an image generation unit based on the instruction information, and the image display device displays the spherical image and the image processing unit rotates the spherical image based on the instruction information, and the image processing unit rotates the image for the right eye or the image for the left eye in the captured image displayed on the image display device, Provided is an image adjustment method for adjustment corresponding to the rotation of the spherical image.

According to the image adjustment system, the image adjustment device, and the image adjustment method of the present invention, it is possible to easily correct the deviation between the image for the right eye and the image for the left eye.

1 is a configuration diagram showing an image adjustment system according to a first embodiment; FIG. It is a figure which shows the relationship between a spherical image and a user. 4 is a flow chart showing an example of an image adjustment method according to the first embodiment; 4 is a flow chart showing an example of an image adjustment method according to the first embodiment; 4 is a flow chart showing an example of an image adjustment method according to the first embodiment; 4 is a flow chart showing an example of an image adjustment method according to the first embodiment; FIG. 10 is a diagram showing an example of a state in which the image for the right eye and the image for the left eye are out of alignment in the vertical direction; FIG. 4 is a diagram showing an example of a state in which parallax between a right-eye image and a left-eye image is large; FIG. 10 is a diagram showing an example of a state in which there is a large tilt deviation between the image for the right eye and the image for the left eye. FIG. 11 is a configuration diagram showing an image adjustment system of second and third embodiments; It is a figure which shows an example of a camera and an attitude control apparatus. FIG. 10 is a diagram showing a state in which the attitude control device controls the tilt of the camera in the horizontal direction; FIG. 10 is a diagram showing a state in which the attitude control device controls the tilt of the camera in the front-rear direction; It is a block diagram which shows an example of an attitude control apparatus. 10 is a flow chart showing an example of an image adjustment method according to the third embodiment; 10 is a flow chart showing an example of an image adjustment method according to the third embodiment;

[First embodiment]
A configuration example of the image adjustment system according to the first embodiment will be described with reference to FIG. Image adjustment system 101 includes camera 102 , communication unit 103 , image display device 104 , controller 105 , image adjustment device 110 , and server 106 . The image adjustment device 110 has an image processing section 111 , an image generation section 112 and a communication section 113 .

The camera 102 is an omnidirectional camera (360-degree camera) capable of photographing a range of 360 degrees, and is a stereo camera capable of photographing a right-eye image and a left-eye image. Communication unit 103 and communication unit 113 are connected via a network. The image adjustment device 110 can acquire the right-eye image IMR1 and the left-eye image IML1 captured by the camera 102 over a 360-degree range as the captured image IM1 via the communication units 103 and 113 and the network.

Computer equipment may be used as the image adjustment device 110 . A CPU may be used as the image processing unit 111 and the image generation unit 112 . The image generation unit 112 and the communication unit 113 may be provided outside the image adjustment device 110 .

The server 106 connects with the camera 102 via the network and the communication unit 103 , and connects with the image adjustment apparatus 110 via the network and the communication unit 113 . The server 106 may acquire the captured image IM1 from the camera 102 via the communication unit 103 and the network, and the image adjusting apparatus 110 may acquire the captured image IM1 from the server 106 via the network and the communication unit 113 .

A captured image IM<b>1 acquired by the image adjustment device 110 is input to the image processing unit 111 . The image adjustment device 110 executes image processing such as correction of distortion of the captured image IM1, and outputs the processed captured image IM1 to the image display device 104. FIG. The image display device 104 is, for example, a head mounted display. The controller 105 is, for example, a glove-type controller used for VR or the like.

FIG. 2 schematically shows a state in which the image display device 104 is attached to the head of the user US and the controller 105 is attached to the hand of the user US. Symbol ZE shown in FIG. 2 indicates the zenith. It is desirable that the zenith of the camera 102 and the zenith of the user US match. While the image display device 104 is mounted on the head of the user US, the image adjusting device 110 displays the right-eye image IMR1 in the region corresponding to the right eye of the user US, and corresponds to the left eye of the user US. By displaying the left-eye image IML1 in the area, the user US can view the captured image IM1 as a stereoscopic image.

The server 106 corrects the distortion of the captured image IM1 acquired from the camera 102, executes image processing such as horizontal adjustment of the captured image IM1, and outputs the image-processed captured image IM1 to the image adjustment device 110. good too. By wearing the image display device 104 on the head of the user US, the user US can view the captured image IM1 image-processed by the image adjustment device 110 or the server 106 .

The image display device 104 generates posture information PN1 based on the direction in which the user US is facing and the state such as the posture of the user US when the image display device 104 is worn on the head of the user US. The image processing unit 111 acquires posture information PN1 from the image display device 104 . That is, the image processing unit 111 acquires the posture information PN1 based on the posture of the image display device 104. FIG. Based on the posture information PN1, the image processing unit 111 displays, on the image display device 104, an image of an area corresponding to the direction in which the user US is facing and the state such as the posture of the user US from the photographed image IM1. .

The controller 105 generates the instruction information NN1 based on the movement or posture of the hand of the user US, or the movement or posture of the finger of the user US when the controller 105 is worn on the hand of the user US. . Hereinafter, the hand or fingers will be simply abbreviated as hand. The image processing unit 111 acquires instruction information NN1 from the controller 105 . The image processing unit 111 can change or adjust the captured image IM1 displayed on the image display device 104 based on the instruction information NN1.

The image generator 112 generates a spherical image VSS1, which is a virtual image formed by a spherical surface, which is CG (Computer Graphics), and stores it in an internal memory or an external memory.

The image processing unit 111 acquires the spherical image VSS1 from the image generation unit 112 based on the instruction information NN1 and displays it on the image display device 104 . FIG. 2 shows an image of the user US when the user US views a spherical image VSS1 displayed on the image display device 104 while the image display device 104 is worn on the user's head. Schematically.

When the user US looks at the spherical image VSS1 displayed on the image display device 104 while the image display device 104 is worn on the head of the user US, the spherical image VSS1 is It is arranged around the US and the image display device 104, and is set to be displayed within a range where the hand of the user US can reach the spherical image VSS1. The user US moves the hand on which the controller 105 is worn to a position corresponding to the spherical image VSS1 displayed on the image display device 104 so that the hand of the user US touches the spherical image VSS1. I feel like I am.

The controller 105 may have an actuator arranged at a portion that comes into contact with the hand of the user US. When the image processing unit 111 determines that the hand of the user US has moved to a position corresponding to the spherical image VSS1 based on the instruction information NN1, the image processing unit 111 operates the actuator. When the actuator applies pressure to the hand of the user US, the user US can feel that the hand is in contact with the spherical image VSS1.

When the user US moves the hand on which the controller 105 is worn in an arbitrary direction while the spherical image VSS1 is being displayed on the image display device 104, the image processing unit 111 changes the instruction information NN1 to Based on this, image processing is performed so that the spherical image VSS1 displayed on the image display device 104 and the photographed image IM1 move in accordance with the movement direction, movement speed, and destination position of the hand of the user US. to run.

The user US can rotate the spherical image VSS1 in any direction, at any speed, and to any position by moving his or her hand in any direction, at any speed, and any position. can. That is, the user US can rotate the spherical image VSS1 by moving his or her hand. The image processing unit 111 moves the captured image IM1 in correspondence with the rotation of the spherical image VSS1.

An example of the image adjustment method of the first embodiment will be described using the flowcharts shown in FIGS. 3A to 3D. Specifically, an example of a method for adjusting the deviation between the image for right eye IMR1 and the image for left eye IML1 will be described. The image display device 104 is attached to the head of the user US, and the controller 105 is attached to the hand of the user US. The image display device 104 displays a photographed image IM1 (right-eye image IMR1 and left-eye image IML1).

When the user US feels a sense of incongruity or experiences motion sickness when viewing the photographed image IM1 displayed on the image display device 104, the user US operates the controller 105 in FIG. 111 causes the image display device 104 to display a setting screen in step S101.

The setting screen includes, for example, an item for vertical correction for correcting a shift in the vertical direction (vertical direction) between the right-eye image IMR1 and the left-eye image IML1, and an item for correcting parallax between the right-eye image IMR1 and the left-eye image IML1. and a tilt correction item for correcting the tilt deviation between the right-eye image IMR1 and the left-eye image IML1 are displayed as adjustment items.

As shown in FIG. 4, when the right-eye image IMR1 and the left-eye image IML1 are out of alignment in the vertical direction, the user US operates the controller 105 to select the vertical correction item displayed on the setting screen. do. As shown in FIG. 5, when the parallax between the right-eye image IMR1 and the left-eye image IML1 is large, the user US operates the controller 105 to select the parallax correction item displayed on the setting screen. As shown in FIG. 6, when the tilt deviation between the right-eye image IMR1 and the left-eye image IML1 is large, the user US operates the controller 105 to select the tilt correction item displayed on the setting screen. .

When the user US selects any one of the vertical correction, parallax correction, and tilt correction items displayed on the setting screen, the controller 105 outputs an instruction including the selected item in step S102. Information NN1 is output to image processing unit 111 .

When the vertical correction item is selected in step S101, the controller 105 outputs to the image processing unit 111 instruction information NN1 indicating that the vertical correction item has been selected. In step S111, based on instruction information NN1 output from controller 105, image processing unit 111 selects a processing mode (vertical correction mode) for correcting vertical misalignment between right-eye image IMR1 and left-eye image IML1. ).

In step S112, the image processing unit 111 causes the image display device 104 to display an item for selecting whether the image to be corrected is the image for the right eye IMR1 or the image for the left eye IML1. When the user US selects the right-eye image IMR1, the controller 105 outputs to the image processing unit 111 instruction information NN1 indicating that the right-eye image IMR1 has been selected. In step S113, the image processing unit 111 shifts the processing to a processing mode (right-eye vertical correction mode) for correcting the vertical direction of the right-eye image IMR1 based on the instruction information NN1 output from the controller 105. .

The image processing unit 111 acquires the spherical image VSS1 from the image generation unit 112 and displays it on the image display device 104 in step S114. The image display device 104 displays a mixture of the captured image IM1 (the right-eye image IMR1 and the left-eye image IML1) and the spherical image VSS1.

The user US rotates the spherical image VSS1 vertically upward or downward so that the vertical deviation between the right eye image IMR1 and the left eye image IML1 is reduced. In step S115, the image processing unit 111 moves the right-eye image IMR1 displayed on the image display device 104 upward or downward in correspondence with the rotation of the spherical image VSS1. The user US may perform the operation of rotating the spherical image VSS1 a plurality of times until there is no vertical deviation between the right eye image IMR1 and the left eye image IML1.

When the user US selects the left-eye image IML1 in step S112, the controller 105 outputs to the image processing unit 111 instruction information NN1 indicating that the left-eye image IML1 has been selected. In step S116, the image processing unit 111 shifts the processing to a processing mode (left-eye vertical correction mode) for correcting the vertical direction of the left-eye image IML1 based on the instruction information NN1 output from the controller 105. .

The image processing unit 111 acquires the spherical image VSS1 from the image generation unit 112 and displays it on the image display device 104 in step S117. The image display device 104 displays a mixture of the captured image IM1 (the right-eye image IMR1 and the left-eye image IML1) and the spherical image VSS1.

The user US rotates the spherical image VSS1 upward or downward so that the vertical deviation between the image for the right eye IMR1 and the image for the left eye IML1 is reduced. In step S118, image processing unit 111 moves left-eye image IML1 displayed on image display device 104 upward or downward in correspondence with the rotation of spherical image VSS1. The user US may perform the operation of rotating the spherical image VSS1 a plurality of times until there is no vertical deviation between the right eye image IMR1 and the left eye image IML1.

When the mode is shifted to the vertical correction mode for the right eye in step S111, when the mode is shifted to the vertical correction mode for the right eye in step S113, or when the mode is shifted to the vertical correction mode for the left eye in step S116, the image processing unit 111 performs spherical correction. The rotation direction of the spherical image VSS1 may be restricted so that the image VSS1 rotates only in the vertical direction. Further, the left-eye image IML1 may be moved upward or downward by moving the spherical image VSS1 up and down without rotating the spherical image VSS1.

When the parallax correction item is selected in step S<b>101 , the controller 105 outputs instruction information NN<b>1 indicating that the parallax correction item has been selected to the image processing unit 111 . In FIG. 3B, in step S121, the image processing unit 111 selects a processing mode (parallax correction mode) for correcting the parallax between the right-eye image IMR1 and the left-eye image IML1 based on the instruction information NN1 output from the controller 105. mode).

In step S122, the image processing unit 111 causes the image display device 104 to display an item for selecting whether the image to be corrected is the image for the right eye IMR1 or the image for the left eye IML1. When the user US selects the right-eye image IMR1, the controller 105 outputs to the image processing unit 111 instruction information NN1 indicating that the right-eye image IMR1 has been selected. In step S123, the image processing unit 111 shifts the process to a processing mode (right-eye parallax correction mode) for correcting the parallax of the right-eye image IMR1 based on the instruction information NN1 output from the controller 105. FIG.

The image processing unit 111 acquires the spherical image VSS1 from the image generation unit 112 and displays it on the image display device 104 in step S124. The image display device 104 displays a mixture of the captured image IM1 (the right-eye image IMR1 and the left-eye image IML1) and the spherical image VSS1.

The user US rotates the spherical image VSS1 rightward or leftward, which is the horizontal direction, so that the parallax between the image for the right eye IMR1 and the image for the left eye IML1 becomes the desired parallax. In step S125, the image processing unit 111 corrects the parallax of the right-eye image IMR1 displayed on the image display device 104 in correspondence with the rotation of the spherical image VSS1. The user US may rotate the spherical image VSS1 multiple times until the parallax between the right-eye image IMR1 and the left-eye image IML1 reaches the desired parallax.

When the user US selects the left-eye image IML1 in step S122, the controller 105 outputs instruction information NN1 indicating that the left-eye image IML1 has been selected to the image processing unit 111. FIG. In step S126, based on instruction information NN1 output from controller 105, image processing unit 111 shifts the process to a processing mode (left-eye parallax correction mode) for correcting parallax of left-eye image IML1.

The image processing unit 111 acquires the spherical image VSS1 from the image generation unit 112 and displays it on the image display device 104 in step S127. The image display device 104 displays a mixture of the captured image IM1 (the right-eye image IMR1 and the left-eye image IML1) and the spherical image VSS1.

The user US rotates the spherical image VSS1 rightward or leftward so that the parallax between the right-eye image IMR1 and the left-eye image IML1 becomes the target parallax. In step S128, the image processing unit 111 corrects the parallax of the left-eye image IML1 displayed on the image display device 104 in correspondence with the rotation of the spherical image VSS1. The user US may rotate the spherical image VSS1 multiple times until the parallax between the right-eye image IMR1 and the left-eye image IML1 reaches the desired parallax.

When the mode is shifted to the parallax correction mode in step S121, when the mode is shifted to the right-eye parallax correction mode in step S123, or when the mode is shifted to the left-eye parallax correction mode in step S126, the image processing unit 111 performs spherical The rotation direction of the spherical image VSS1 may be restricted so that the image VSS1 rotates only in the horizontal direction.

When the tilt correction item is selected in step S101, the controller 105 outputs to the image processing unit 111 instruction information NN1 indicating that the tilt correction item has been selected. In FIG. 3C, in step S131, the image processing unit 111 selects a processing mode ( tilt correction mode).

In step S132, the image processing unit 111 causes the image display device 104 to display an item for selecting whether the image to be corrected is the image for the right eye IMR1 or the image for the left eye IML1. When the user US selects the right-eye image IMR1, the controller 105 outputs to the image processing unit 111 instruction information NN1 indicating that the right-eye image IMR1 has been selected. In step S133, the image processing unit 111 shifts the process to a processing mode (right-eye tilt correction mode) for correcting the tilt of the right-eye image IMR1 based on the instruction information NN1 output from the controller 105. FIG.

The image processing unit 111 acquires the spherical image VSS1 from the image generation unit 112 and displays it on the image display device 104 in step S134. The image display device 104 displays a mixture of the captured image IM1 (the right-eye image IMR1 and the left-eye image IML1) and the spherical image VSS1.

The user US rotates the spherical image VSS1 in an arbitrary direction so as to reduce the tilt deviation between the image for the right eye IMR1 and the image for the left eye IML1. In step S135, the image processing unit 111 corrects the tilt of the right-eye image IMR1 displayed on the image display device 104 in accordance with the rotation of the spherical image VSS1. The user US may perform the operation of rotating the spherical image VSS1 a plurality of times until there is no deviation in tilt between the right-eye image IMR1 and the left-eye image IML1.

When the user US selects the left-eye image IML1 in step S132, the controller 105 outputs instruction information NN1 indicating that the left-eye image IML1 has been selected to the image processing unit 111. FIG. In step S136, the image processing unit 111 shifts the processing to a processing mode for correcting the tilt of the left-eye image IML1 (left-eye tilt correction mode) based on the instruction information NN1 output from the controller 105. FIG.

The image processing unit 111 acquires the spherical image VSS1 from the image generation unit 112 and displays it on the image display device 104 in step S137. The image display device 104 displays a mixture of the captured image IM1 (the right-eye image IMR1 and the left-eye image IML1) and the spherical image VSS1.

The user US rotates the spherical image VSS1 in an arbitrary direction so as to reduce the tilt deviation between the image for the right eye IMR1 and the image for the left eye IML1. In step S138, the image processing unit 111 corrects the tilt of the left-eye image IML1 displayed on the image display device 104 in accordance with the rotation of the spherical image VSS1. The user US may perform the operation of rotating the spherical image VSS1 a plurality of times until there is no deviation in tilt between the right-eye image IMR1 and the left-eye image IML1.

When the mode is shifted to the tilt correction mode in step S131, when the mode is shifted to the tilt correction mode for the right eye in step S133, or when the mode is shifted to the tilt correction mode for the left eye in step S136, the image processing unit 111 performs the spherical The direction of rotation of image VSS1 is not restricted. Therefore, the user US can rotate the spherical image VSS1 in any direction.

The image processing unit 111 determines to which position on the coordinates of the spherical image VSS1 the zenith ZE before the user US rotates the spherical image VSS1 has moved as a result of the user US rotating the spherical image VSS1. can be determined. The image processing unit 111 generates spherical images before and after the user US rotates the spherical image VSS1 based on the movement direction and destination position of the zenith ZE on the coordinates of the spherical image VSS1. Calculate the amount of change in VSS1.

The amount of change in the spherical image VSS1 is the amount of rotation (rotation angle) about the X axis, the amount of rotation (rotation angle) about the Y axis, and the amount of rotation (rotation angle) about the Z axis in the spherical image VSS1. It corresponds to the rotation amount (rotation angle) of the combined spherical image VSS1. The image processing unit 111 stores the amount of change in the spherical image VSS1 as the correction value CV1 in association with the image-processed captured image IM1 (right-eye image IMR1 or left-eye image IML1).

That is, the correction value CV1 is calculated based on the rotation direction of the spherical image VSS1 and the movement amount or movement angle of the zenith ZE (the rotation angle of the spherical image VSS1). The image processing unit 111 may store the coordinates of the zenith ZE on the spherical image VSS1 after the user US rotates the spherical image VSS1 as the correction value CV1.

The image processing unit 111 may store the correction value CV1 in an internal memory or an external memory. The server 106 may acquire the correction value CV1 from the image adjustment device 110 via the network and the communication unit 113 in association with the image-processed captured image IM1 (right-eye image IMR1 or left-eye image IML1). Server 106 stores correction value CV1 in an internal memory or an external memory.

When the user US determines that the deviation between the right-eye image IMR1 and the left-eye image IML1 has been corrected, the image processing unit 111 in FIG. The display of VSS1 is ended, and the setting screen is displayed on the image display device 104 .

When the user US operates the controller 105 and selects a predetermined item (for example, an end item) displayed on the setting screen, the controller 105 confirms that the end item has been selected in step S142. to the image processing unit 111. In step S143, the image processing unit 111 shifts the processing to a predetermined processing mode corresponding to the selected item. When the end item is selected, the image processing unit 111 shifts the process to a processing mode (end mode) for ending the image adjustment of the right-eye image IMR1 and the left-eye image IML1.

In step S144, the image processing unit 111 acquires the amount of rotation (rotation angle) before and after the rotation of the spherical image VSS1 as the amount of change in the spherical image VSS1. In step S145, the image processing unit 111 stores the amount of change in the spherical image VSS1 as the correction value CV1 in association with the image-processed captured image IM1 (right-eye image IMR1 or left-eye image IML1), and processes the correction value CV1. exit.

In the image adjustment system 101, the image adjustment device 110, and the image adjustment method of the first embodiment, the image display device 104 is caused to display the spherical image VSS1. According to the image adjustment system 101, the image adjustment apparatus 110, and the image adjustment method of the first embodiment, the user US operates the controller 105 when a shift occurs between the right-eye image IMR1 and the left-eye image IML1. By rotating the spherical image VSS1 in this manner, the deviation between the right-eye image IMR1 and the left-eye image IML1 displayed on the image display device 104 can be easily adjusted.

According to the image adjustment system 101, the image adjustment apparatus 110, and the image adjustment method of the first embodiment, when the correction value CV1 is stored in the image adjustment apparatus 110 or the server 106, the image processing unit 111 Correction value CV1 is read from 110 or server 106, photographed image IM1 (right-eye image IMR1 or left-eye image IML1) captured by camera 102 is adjusted based on correction value CV1, and displayed on image display device 104. can.

[Second embodiment]
A configuration example of the image adjustment system according to the second embodiment will be described with reference to FIG. The image adjustment system 201 includes a camera 202 , an attitude control device 220 , a communication section 203 , an image display device 204 , a controller 205 , an image adjustment device 210 and a server 206 . The image adjustment device 210 has an image processing section 211 , an image generation section 212 and a communication section 213 .

The camera 202, the communication unit 203, the image display device 204, the controller 205, the image adjustment device 210, and the server 206 are the camera 102, the communication unit 103, the image display device 104, the controller 105, and the image adjustment device of the first embodiment, respectively. 110 and server 106 . The image processing unit 211, the image generation unit 212, and the communication unit 213 correspond to the image processing unit 111, the image generation unit 112, and the communication unit 113 of the first embodiment, respectively.

As shown in FIG. 8, the camera 202 has a first surface PL1 and a second surface PL2 opposite to the first surface PL1. The camera 202 has right-eye and left-eye fisheye lenses LR1 and LL1 arranged on the first plane PL1 side, and right-eye and left-eye fisheye lenses LR2 and LL2 arranged on the second plane PL2 side. Hereinafter, the right eye fisheye lens LR1 is the first right eye lens LR1, the left eye fisheye lens LL1 is the first left eye lens LL1, the right eye fisheye lens LR2 is the second right eye lens LR2, and the left eye fisheye lens Let LL2 be the second left-eye lens LL2.

The camera 202 includes a first camera 2021 that captures a 180-degree range on the first plane PL1 side using a first right-eye lens LR1 and a first left-eye lens LL1, and a second plane PL2 side. using a second right-eye lens LR2 and a second left-eye lens LL2. That is, the camera 202 is an omnidirectional camera (360-degree camera) capable of photographing a range of 360 degrees, and is a stereo camera capable of photographing a right-eye image and a left-eye image.

The camera 202 photographs a 360-degree range using first and second right eye lenses LR1 and LR2, which are photographing lenses, and generates a right eye image IMR2. The camera 202 captures a 360-degree range using first and second left-eye lenses LL1 and LL2, which are imaging lenses, and generates a left-eye image IML2.

The attitude control device 220 controls the attitude of the camera 202 . As shown in FIG. 9, the attitude control device 220 controls the tilt of the camera 202 in the horizontal direction. For example, the posture control device 220 can be set to a position midway between the first right eye lens LR1 and the first left eye lens LL1, and a position midway between the second right eye lens LR2 and the second left eye lens LL2. By rotating the camera 202 about a rotation axis CLa (first rotation axis) that passes through the position and is parallel to the optical axes of the lenses LR1, LL1, LR2, and LL2, the camera 202 is tilted in the left-right direction. to control. That is, the tilt of the camera 202 in the left-right direction is the tilt of the camera 202 about the rotation axis CLa.

As shown in FIG. 10, the posture control device 220 controls the tilt of the camera 202 in the front-rear direction. For example, the posture control device 220 can be positioned between the first right-eye lens LR1 and the second left-eye lens LL2, and between the first left-eye lens LL1 and the second right-eye lens LR2. By rotating the camera 202 around a rotation axis CLb (second rotation axis) that passes through the position and is orthogonal to the optical axes of the lenses LR1, LL1, LR2, and LL2, the camera 202 is tilted in the front-rear direction. to control. That is, the tilt of the camera 202 in the front-rear direction is the tilt of the camera 202 about the rotation axis CLb.

The first rotation axis CLa and the second rotation axis CLb are orthogonal to each other. When the camera 202 is horizontally controlled, the first and second rotation axes CLa and CLb are parallel to the horizontal plane.

As shown in FIG. 11 , the attitude control device 220 has a 3-axis acceleration sensor 221 , a 3-axis magnetic sensor 222 , a 3-axis gyro sensor 223 , a drive control section 224 , and drive sections 225 and 226 . Let the three-axis acceleration sensor 221 be the first sensor 221 , the three-axis magnetic sensor 222 be the second sensor 222 , and the three-axis gyro sensor 223 be the third sensor 223 . The drive section 225 is assumed to be a first drive section, and the drive section 226 is assumed to be a second drive section.

The first sensor 221 acquires the inclination of the camera 202 in the left-right direction and the front-rear direction by detecting the direction of gravity. A second sensor 222 detects the magnitude and direction of a magnetic field, and a third sensor 223 detects angular velocity.

The drive control section 224 controls at least one of the drive sections 225 and 226 based on the information detected by the first sensor 221 . When the camera 202 is moving while accelerating or decelerating, the drive control unit 224 receives information detected by the first sensor 221 and information detected by at least one of the second and third sensors 222 and 223. At least one of the drive units 225 and 226 is controlled based on. As shown in FIG. 9, the drive unit 225 rotates the camera 202 in the horizontal direction. As shown in FIG. 10, the drive unit 226 rotates the camera 202 forward and backward.

The image adjusting device 210 can acquire the right-eye image IMR2 and the left-eye image IML2 captured by the camera 202 over a 360-degree range as the captured image IM2 via the communication units 203 and 213 and the network.

The server 206 connects with the camera 202 via the network and the communication unit 203 and connects with the image adjustment apparatus 210 via the network and the communication unit 213 . The server 206 may acquire the captured image IM2 from the camera 202 via the communication unit 203 and the network, and the image adjusting apparatus 210 may acquire the captured image IM2 from the server 206 via the network and the communication unit 213. In FIG. 7, the image adjustment device 210 or the server 206 acquires the captured image IM2 via the attitude control device 220, but may be acquired without the attitude control device 220. FIG.

A captured image IM<b>2 acquired by the image adjustment device 210 is input to the image processing unit 211 . The image adjustment device 210 executes image processing such as correcting the distortion of the captured image IM2, and outputs the processed captured image IM2 to the image display device 204. FIG.

While the image display device 204 is mounted on the head of the user US, the image adjustment device 210 displays the right-eye image IMR2 in the region corresponding to the right eye of the user US, and corresponds to the left eye of the user US. By displaying the left-eye image IML2 in the area, the user US can view the captured image IM2 as a stereoscopic image.

The server 206 corrects the distortion of the captured image IM2 acquired from the camera 202, executes image processing such as horizontal adjustment of the captured image IM2, and outputs the image-processed captured image IM2 to the image adjustment device 210. good too. By wearing the image display device 204 on the head of the user US, the user US can view the captured image IM2 image-processed by the image adjusting device 210 or the server 206 .

The image display device 204 generates posture information PN2 based on the direction in which the user US is facing and the state such as the posture of the user US when the image display device 204 is worn on the head of the user US. The image processing unit 211 acquires posture information PN2 from the image display device 204 . That is, the image processing unit 211 acquires the posture information PN2 based on the posture of the image display device 204. FIG. Based on the posture information PN2, the image processing unit 211 displays, on the image display device 204, an image of an area corresponding to the direction in which the user US is facing and the state such as the posture of the user US from the photographed image IM2. .

The controller 205 generates the instruction information NN2 based on the state such as the movement or posture of the hand of the user US while the controller 205 is worn on the hand of the user US. The image processing unit 211 acquires instruction information NN2 from the controller 205 . The image processing unit 211 can change or adjust the captured image IM2 displayed on the image display device 204 based on the instruction information NN2.

The image generator 212 generates a spherical image VSS2, which is a virtual image formed by a CG spherical surface, and stores it in an internal memory or an external memory. The image processing unit 211 acquires the spherical image VSS2 from the image generation unit 212 based on the instruction information NN2 and displays it on the image display device 204 . The spherical image VSS2 corresponds to the spherical image VSS1 of the first embodiment. The user US moves the hand on which the controller 205 is worn to a position corresponding to the spherical image VSS2 displayed on the image display device 204 so that the hand of the user US touches the spherical image VSS2. I feel like I am.

The controller 205 may have an actuator arranged at a portion that contacts the hand of the user US. When the image processing unit 211 determines that the hand of the user US has moved to the position corresponding to the spherical image VSS2 based on the instruction information NN2, the image processing unit 211 operates the actuator. When the actuator applies pressure to the hand of the user US, the user US can feel that the hand touches the spherical image VSS2.

In a state where the spherical image VSS2 is displayed on the image display device 204, when the user US moves the hand on which the controller 205 is worn in an arbitrary direction, the image processing unit 211 changes the instruction information NN2 to Based on this, image processing is performed so that the spherical image VSS2 displayed on the image display device 204 and the photographed image IM2 move in accordance with the movement direction, movement speed, and destination position of the hand of the user US. to run.

The user US can rotate the spherical image VSS2 in any direction, at any speed, and to any position by moving his or her hand in any direction, at any speed, and to any position. can. That is, the user US can rotate the spherical image VSS2 by moving his or her hand. The image processing unit 211 moves the captured image IM2 in accordance with the rotation of the spherical image VSS2.

In the attitude control device 220, the drive control unit 224 acquires the horizontal plane based on information detected by at least one of the first to third sensors 221 to 223, and tilts the camera 202 in the left-right direction with respect to the horizontal plane. The angle θa (first tilt angle) and the longitudinal tilt angle θb (second tilt angle) are obtained.

The drive control unit 224 controls the drive unit 225 so that the tilt angle θa is equal to or less than a predetermined angle α (first angle) (θa≦α), thereby adjusting the lateral posture of the camera 202 . . The drive control unit 224 adjusts the posture of the camera 202 in the front-rear direction by controlling the drive unit 225 so that the tilt angle θb is equal to or less than a predetermined angle β (second angle) (θb≦β). .

As shown in FIG. 9, when the camera 202 is tilted to the right, for example, when the camera 202 is viewed from the side of the first plane PL1, the first lens LR1 for the right eye is positioned in the first direction on the side of the first plane PL1. 1 is located above the left-eye lens LL1. On the other hand, on the second plane PL2 side, the second right-eye lens LR2 is located below the second left-eye lens LL2.

Therefore, since the parallax between the right-eye image IMR2 and the left-eye image IML2 is reversed between the first plane PL1 side and the second plane PL2 side, the user US cannot see the captured image displayed on the image display device 204. I feel uncomfortable with IM2. On the other hand, when the camera 202 is tilted in the front-rear direction, the parallax between the right-eye image IMR2 and the left-eye image IML2 is not reversed between the first plane PL1 side and the second plane PL2 side. The US is less likely to feel uncomfortable with the captured image IM2 displayed on the image display device 204 .

For the above reasons, the angle α is set to be smaller than the angle β (α<β). For example, the angle α is set to 1 degree and the angle β is set to 5 degrees. Therefore, the drive control unit 224 controls at least one of the drive units 225 and 226 so that the tilt angle θa in the left-right direction is smaller than the tilt angle θb in the front-rear direction (θa<θb). to control the posture of

After the attitude control device 220 controls the attitude of the camera 202, the image adjustment system 201 executes the same processing as steps S101, S111 to S118, S121 to S128, S131 to S138, or S141 to S144 of the first embodiment. do.

In the image adjustment system 201, the image adjustment device 210, and the image adjustment method of the second embodiment, the image display device 204 is caused to display the spherical image VSS2. According to the image adjustment system 201, the image adjustment apparatus 210, and the image adjustment method of the second embodiment, the user US operates the controller 205 when there is a deviation between the right-eye image IMR2 and the left-eye image IML2. By rotating the spherical image VSS2 in this way, the deviation between the right-eye image IMR2 and the left-eye image IML2 displayed on the image display device 204 can be easily adjusted.

According to the image adjustment system 201, the image adjustment apparatus 210, and the image adjustment method of the second embodiment, when the correction value CV1 is stored in the image adjustment apparatus 210 or the server 206, the image processing unit 211 calculates the correction value CV1 is read out from the image adjustment device 210 or the server 206, and the photographed image IM2 (right-eye image IMR2 or left-eye image IML2) photographed by the camera 202 is adjusted based on the correction value CV1 and displayed on the image display device 204. can.

According to the image adjustment system 201, the image adjustment device 210, and the image adjustment method of the second embodiment, the attitude of the camera 202 is controlled to be horizontal or nearly horizontal by the attitude control device 220. FIG. Since the user US adjusts the deviation between the right-eye image IMR2 and the left-eye image IML2 while the camera 202 is posture-controlled, the deviation can be easily adjusted.

[Third embodiment]
A configuration example of the image adjustment system according to the third embodiment will be described with reference to FIG. The image adjustment system 301 includes a camera 302 , an attitude control device 320 , a communication section 303 , an image display device 304 , a controller 305 , an image adjustment device 310 and a server 306 . The image adjustment device 310 has an image processing section 311 , an image generation section 312 and a communication section 313 .

The camera 302, the attitude control device 320, the communication unit 303, the image display device 304, the controller 305, the image adjustment device 310, and the server 306 are the camera 202, the attitude control device 220, the communication unit 203, the image It corresponds to the display device 204 , the controller 205 , the image adjustment device 210 and the server 206 . An image processing unit 311, an image generation unit 312, and a communication unit 313 correspond to the image processing unit 211, the image generation unit 212, and the communication unit 213 of the second embodiment, respectively. In order to make the description easier to understand, the same reference numerals are given to the same components as those of the second embodiment.

As shown in FIG. 8, the camera 302 includes a first camera 3021 that captures a 180-degree range on the first plane PL1 side using a first right-eye lens LR1 and a first left-eye lens LL1. , and a second camera 3022 that captures a 180-degree range on the second plane PL2 side using a second right-eye lens LR2 and a second left-eye lens LL2. The first and second cameras 3021 and 3022 correspond to the first and second cameras 2021 and 2022 of the second embodiment. That is, the camera 302 is an omnidirectional camera (360-degree camera) capable of photographing a range of 360 degrees, and is a stereo camera capable of photographing a right-eye image and a left-eye image.

Camera 302 captures a 360-degree range using first and second right-eye lenses LR1 and LR2 to generate right-eye image IMR3. Camera 302 captures a 360-degree range using first and second left-eye lenses LL1 and LL2 to generate left-eye image IML3.

The attitude control device 320 controls the attitude of the camera 302 . As shown in FIG. 9, the attitude control device 320 controls the tilt of the camera 302 in the horizontal direction. For example, the attitude control device 320 controls the tilt of the camera 302 in the left-right direction by rotating the camera 302 around the rotation axis CLa. That is, the tilt of the camera 302 in the left-right direction is the tilt of the camera 302 about the rotation axis CLa. As shown in FIG. 10, the attitude control device 320 may control the tilt of the camera 302 in the front-back direction. For example, the posture control device 320 controls the tilt of the camera 302 in the front-rear direction by rotating the camera 302 around the rotation axis CLb. That is, the tilt of the camera 302 in the front-rear direction is the tilt of the camera 302 around the rotation axis CLb.

As shown in FIG. 11, the attitude control device 320 includes a three-axis acceleration sensor 221 (first sensor), a three-axis magnetic sensor 222 (second sensor), and a three-axis gyro sensor 223 (third sensor). , a drive control section 224 , a drive section 225 , and a drive section 226 . When the camera 302 is moving while accelerating or decelerating, the drive control unit 224 receives information detected by the first sensor 221 and information detected by at least one of the second and third sensors 222 and 223. At least one of the drive units 225 and 226 is controlled based on.

The image adjusting device 310 can acquire the right-eye image IMR3 and the left-eye image IML3 captured by the camera 302 over a 360-degree range as the captured image IM3 via the communication units 303 and 313 and the network.

The server 306 connects with the camera 302 via the network and the communication unit 303 , and connects with the image adjustment apparatus 310 via the network and the communication unit 313 . The server 306 may acquire the captured image IM3 from the camera 302 via the communication unit 303 and the network, and the image adjustment device 310 may acquire the captured image IM3 from the server 306 via the network and the communication unit 313 . In FIG. 7 , the image adjustment device 310 or the server 306 acquires the captured image IM3 via the posture control device 320 , but it may be acquired without the posture control device 320 .

A captured image IM3 acquired by the image adjustment device 310 is input to the image processing unit 311 . The image adjustment device 310 executes image processing such as correcting distortion of the captured image IM3, and outputs the processed captured image IM3 to the image display device 304. FIG.

While the image display device 304 is mounted on the head of the user US, the image adjusting device 310 displays the right-eye image IMR3 in the region corresponding to the right eye of the user US, and corresponds to the left eye of the user US. By displaying the left-eye image IML3 in the area, the user US can view the captured image IM3 as a stereoscopic image.

The server 306 corrects the distortion of the captured image IM3 acquired from the camera 302, executes image processing such as horizontal adjustment of the captured image IM3, and outputs the image-processed captured image IM3 to the image adjustment device 310. good too. By wearing the image display device 304 on the head of the user US, the user US can view the captured image IM3 image-processed by the image adjusting device 310 or the server 306 .

The image display device 304 generates posture information PN3 based on the direction in which the user US is facing and the state such as the posture of the user US when the image display device 304 is worn on the head of the user US. The image processing unit 311 acquires posture information PN3 from the image display device 304 . That is, the image processing unit 311 acquires the posture information PN3 based on the posture of the image display device 304. FIG. Based on the posture information PN3, the image processing unit 311 displays, on the image display device 304, an image of an area corresponding to the direction in which the user US is facing and the state such as the posture of the user US from the photographed image IM3. .

The controller 305 generates the instruction information NN3 based on the state such as the movement or posture of the hand of the user US when the controller 305 is worn on the hand of the user US. The image processing unit 311 acquires instruction information NN3 from the controller 305 . The image processing unit 311 can change or adjust the captured image IM3 displayed on the image display device 304 based on the instruction information NN3.

The image generator 312 generates a spherical image VSS3, which is a virtual image formed by a CG spherical surface, and stores it in an internal memory or an external memory. The image processing unit 311 acquires the spherical image VSS3 from the image generation unit 312 based on the instruction information NN3 and displays it on the image display device 304 . The spherical image VSS3 corresponds to the spherical images VSS1 and VSS2 of the first and second embodiments. The user US moves the hand on which the controller 305 is worn to a position corresponding to the spherical image VSS3 displayed on the image display device 304 so that the hand of the user US touches the spherical image VSS3. I feel like I am.

The controller 305 may have an actuator arranged at a portion that comes into contact with the hand of the user US. When the image processing unit 311 determines that the hand of the user US has moved to the position corresponding to the spherical image VSS3 based on the instruction information NN3, the image processing unit 311 operates the actuator. By the actuator applying pressure to the hand of the user US, the user US can feel that the hand is in contact with the spherical image VSS3.

When the user US moves the hand on which the controller 305 is worn in an arbitrary direction while the spherical image VSS3 is displayed on the image display device 304, the image processing unit 311 changes the instruction information NN3 to Based on this, image processing is performed so that the spherical image VSS3 displayed on the image display device 304 and the photographed image IM3 move in accordance with the movement direction, movement speed, and destination position of the hand of the user US. to run.

The user US can rotate the spherical image VSS3 in any direction, at any speed, and to any position by moving his or her hand in any direction, at any speed, and any position. can. That is, the user US can rotate the spherical image VSS3 by moving his or her hand. The image processing unit 311 moves the captured image IM3 in accordance with the rotation of the spherical image VSS3.

An example of the image adjustment method of the third embodiment will be described using the flowcharts shown in FIGS. 12A and 12B. Specifically, an example of a method for adjusting the vertical deviation between the image for right eye IMR3 and the image for left eye IML3 will be described. The image display device 304 is attached to the head of the user US, and the controller 305 is attached to the hand of the user US. The image display device 304 displays a photographed image IM3 (right-eye image IMR3 and left-eye image IML3).

When the user US corrects the vertical deviation between the right-eye image IMR3 and the left-eye image IML3 displayed on the image display device 304, the user US operates the controller 305 in FIG. The processing unit 311 causes the image display device 304 to display a setting screen in step S301.

On the setting screen, for example, an item for vertical correction for correcting vertical deviation between the image for right eye IMR3 and the image for left eye IML3 is displayed as an adjustment item. The image processing unit 311 has a parallax correction item for correcting the parallax between the right-eye image IMR3 and the left-eye image IML3, and correction of tilt deviation between the right-eye image IMR3 and the left-eye image IML3 on the setting screen. A tilt correction item for correcting the tilt may be displayed as an adjustment item.

As shown in FIG. 4, when the right-eye image IMR3 and the left-eye image IML3 are vertically misaligned, the user US selects the vertical correction item displayed on the setting screen. In step S302, the controller 305 outputs to the image processing unit 311 instruction information NN3 indicating that the vertical correction item has been selected. In step S311, based on instruction information NN3 output from controller 305, image processing unit 311 selects a processing mode (vertical correction mode) for correcting vertical misalignment between right-eye image IMR3 and left-eye image IML3. ).

In step S312, the image processing unit 311 displays on the image display device 304 an item for selecting whether the image to be corrected is the image for the right eye IMR3 or the image for the left eye IML3. When the user US selects the right-eye image IMR3, the controller 305 outputs to the image processing unit 311 instruction information NN3 indicating that the right-eye image IMR3 has been selected. In step S313, the image processing unit 311 shifts the processing to a processing mode (right-eye vertical correction mode) for correcting the vertical direction of the right-eye image IMR3 based on the instruction information NN3 output from the controller 305. .

The image processing unit 111 acquires the spherical image VSS3 from the image generation unit 312 and displays it on the image display device 204 in step S314. The image display device 204 displays a mixture of the captured image IM3 (the right-eye image IMR3 and the left-eye image IML3) and the spherical image VSS3.

The user US rotates the spherical image VSS3 upward or downward so that the vertical deviation between the right-eye image IMR3 and the left-eye image IML3 is reduced. In step S315, the image processing unit 311 moves the right-eye image IMR3 displayed on the image display device 304 upward or downward in correspondence with the rotation of the spherical image VSS3.

When the user US selects the left-eye image IML3 in step S312, the controller 305 outputs to the image processing unit 311 instruction information NN3 indicating that the left-eye image IML3 has been selected. In step S316, the image processing unit 311 shifts the processing to a processing mode (left-eye vertical correction mode) for correcting the vertical direction of the left-eye image IML3 based on the instruction information NN3 output from the controller 305. .

The image processing unit 311 acquires the spherical image VSS3 from the image generation unit 312 and displays it on the image display device 304 in step S317. The image display device 304 displays a mixture of the captured image IM3 (the right-eye image IMR3 and the left-eye image IML3) and the spherical image VSS3.

The user US rotates the spherical image VSS3 upward or downward so that the vertical deviation between the right-eye image IMR3 and the left-eye image IML3 is reduced. In step S318, image processing unit 311 moves left-eye image IML3 displayed on image display device 304 upward or downward in correspondence with the rotation of spherical image VSS3.

When the mode is shifted to the vertical correction mode for the right eye in step S311, when the mode is shifted to the vertical correction mode for the right eye in step S313, or when the mode is shifted to the vertical correction mode for the left eye in step S316, the image processing unit 311 performs spherical correction. The rotation direction of the spherical image VSS3 may be restricted so that the image VSS3 rotates only in the vertical direction. Further, the left-eye image IML3 may be moved upward or downward by moving the spherical image VSS3 up and down without rotating the spherical image VSS3.

The image processing unit 311 determines to which position on the coordinates of the spherical image VSS3 the zenith ZE before the user US rotates the spherical image VSS3 has moved as a result of the user US rotating the spherical image VSS3. can be determined. The image processing unit 311 generates spherical images before and after the user US rotates the spherical image VSS3 based on the movement direction and the destination position of the zenith ZE on the coordinates of the spherical image VSS3. Calculate the amount of change in VSS3. The amount of change in the spherical image VSS3 corresponds to the amount of change in the spherical image VSS1.

When the camera 302 is tilted in the horizontal direction as shown in FIG. 9, the right eye image IMR3 and the left eye image IML3 are vertically displaced as shown in FIG. Therefore, even if the user US adjusts the photographed image IM3 by rotating the spherical image VSS3, the vertical deviation between the right-eye image IMR3 and the left-eye image IML3 displayed on the image display device 304 can be sufficiently adjusted. Correction may not be possible.

12B, in step S321, the image processing unit 311, based on the posture information PN3 or the captured image IM3 displayed on the image display device 304, changes the captured image IM3 displayed on the image display device 304 into It is determined whether the image is captured by the first camera 3021 or the second camera 3022 .

In step S322, the image processing unit 311 adjusts the determination result (the first camera 3021 or the second camera 3022) in step S321, the instruction information NN3, or the captured image IM3 displayed on the image display device 304. The tilt direction and tilt angle of the camera 302 are estimated from the direction (upward or downward) and the amount of adjustment. Assuming that the distance between the right-eye image IMR3 and the left-eye image IML3 displayed on the image display device 304 is d, and the upward or downward adjustment amount is h, the tilt angle θa of the camera 302 is expressed by the relational expression θa=tan ⁻¹ (h/d).

In step S323, the image processing unit 311 generates correction information CN for correcting the orientation of the camera 302 based on the estimation result in step S322 (specifically, the tilt direction and tilt angle of the camera 302). Furthermore, the image processing unit 311 outputs the correction information CN to the attitude control device 320 via the communication units 313 and 303 and the network. The image processing unit 311 may output the correction information CN to the server 306 via the communication unit 313 and the network, and the server 306 may output the correction information CN to the attitude control device 320 via the network and the communication unit 303. .

The attitude control device 320 controls the attitude of the camera 302 based on the correction information CN in step S324. In step S325, the image processing unit 311 confirms that the attitude control of the camera 302 by the attitude control device 320 has ended. The attitude control device 320 may generate an end signal ES indicating that the attitude control of the camera 302 has ended, and output it to the image processing section 311 via the communication sections 303 and 313 and the network.

In step S326, the image processing unit 311 ends the display of the spherical image VSS3, and performs the correction opposite to that in step S315 or S318. For example, when the right-eye image IMR3 is moved upward in step S315, the image processing unit 311 moves the right-eye image IMR3 downward so that the right-eye image IMR3 is the image before the process of step S315 is executed. Image processing is performed so that the For example, when the left-eye image IML3 is moved downward in step S318, the image processing unit 311 moves the left-eye image IML3 upward so that the left-eye image IML3 is the image before the processing in step S318 is executed. Image processing is performed so that the

That is, the posture control device 320 controls the posture of the camera 302, so that the image processing unit 311 returns the processing to the state before the processing of step S315 is performed on the right-eye image IMR3, or Image processing is performed on the image IML3 so as to return the processing to the state before the processing of step S318 is performed, and the processing ends.

In the image adjustment system 301, the image adjustment device 310, and the image adjustment method of the third embodiment, the image display device 304 is caused to display the spherical image VSS3. According to the image adjustment system 301, the image adjustment apparatus 310, and the image adjustment method of the third embodiment, the user US operates the controller 305 when there is a deviation between the right-eye image IMR3 and the left-eye image IML3. By rotating the spherical image VSS3 in this way, the deviation between the right-eye image IMR3 and the left-eye image IML3 displayed on the image display device 304 can be easily adjusted.

In the image adjustment system 301, the image adjustment device 310, and the image adjustment method of the third embodiment, the tilt direction and the tilt angle of the camera 302 are estimated based on the adjustment result of the captured image IM3 by the image adjustment device 310, and the estimation result The attitude control device 320 controls the attitude of the camera 302 based on . Since the camera 302 is controlled to be horizontal or nearly horizontal by the attitude control device 320, the user US can easily adjust the deviation between the right-eye image IMR3 and the left-eye image IML3. be able to.

The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.

For example, the virtual image VSS generated by CG may be the surface of an ellipsoid (ellipsoid), or any closed surface (closed curved surface) within the reach of the user US. I wish I had. That is, it is sufficient if the user US can obtain the feeling of touching the closed curved surface from the inside and can horizontally adjust the photographed image IM. Since the user US rotates the virtual image VSS, it is desirable that the virtual image VSS has a spherical surface or a spherical shape close to a spherical surface such as an ellipsoidal surface.

101, 201, 301 Image adjustment system 102, 202, 302 Camera 104, 204, 304 Image display device 105, 205, 305 Controller 110, 210, 310 Image adjustment device 111, 211, 311 Image processing section 112, 212, 312 Image Generation unit 220, 320 Attitude control device CN Correction information IM1, IM2, IM3 Photographed image IML1, IML2, IML3 Image for left eye IMR1, IMR2, IMR3 Image for right eye NN1, NN2, NN3 Instruction information VSS1, VSS2, VSS3 Spherical image ( virtual image)
θa, θb Tilt angle

Claims (6)

a camera that captures a right-eye image and a left-eye image;
an image adjustment device that acquires the image for the right eye and the image for the left eye as captured images from the camera and adjusts the captured images;
an image display device that displays the captured image adjusted by the image adjustment device;
a controller that outputs instruction information to the image adjustment device;
with
The image adjustment device is
an image generator that generates a spherical image;
acquiring the spherical image from the image generation unit based on the instruction information, displaying the spherical image on the image display device, rotating the spherical image based on the instruction information, and making the spherical image correspond to the rotation of the spherical image; an image processing unit that adjusts the image for the right eye or the image for the left eye in the captured image displayed on the image display device;
image adjustment system. The camera is an omnidirectional camera that captures a range of 360 degrees,
The image display device is a head-mounted display mounted on the user's head,
The controller is a glove-type controller worn on the user's hand,
With the image display device mounted on the user's head, when the user views the spherical image displayed on the image display device, the spherical image is A virtual image that is arranged around the user and the image display device and is set to be displayed within a range where the user's hand or fingers can reach the spherical image.
The image adjustment system of claim 1. When the user moves his or her hand or fingers in any direction while the controller is attached to the user's hand, the image processing unit controls the user's The spherical image is rotated in accordance with the movement of the hand or fingers, and the image for the right eye or the image for the left eye in the photographed image displayed on the image display device is rotated in correspondence with the rotation of the spherical image. adjust,
3. The image adjustment system of claim 2. The image processing unit acquires the amount of change before and after the spherical image is rotated, and adjusts the captured image based on the amount of change.
4. The image adjustment system of claim 3. an image generator that generates a spherical image;
The spherical image is acquired from the image generating unit based on instruction information acquired from a controller, displayed on an image display device, the spherical image is rotated based on the instruction information, and the image is captured by a camera. an image processing unit that adjusts the image for the right eye or the image for the left eye in the captured image displayed on the display device in accordance with the rotation of the spherical image;
An image adjustment device comprising: a camera captures a right-eye image and a left-eye image;
an image processing unit acquires the image for the right eye and the image for the left eye from the camera as captured images;
an image display device displaying the captured image;
The image processing unit acquires instruction information from the controller,
the image processing unit acquires a spherical image from the image generation unit based on the instruction information;
The image display device displays the spherical image,
The image processing unit rotates the spherical image based on the instruction information,
wherein the image processing unit adjusts the image for the right eye or the image for the left eye in the captured image displayed on the image display device in accordance with the rotation of the spherical image;
Image adjustment method.

Images