JP5906894B2 - 3D input device and 3D input method - Google Patents

Description

  The present invention relates to a 3D input device and a 3D input method.

  2. Description of the Related Art Conventionally, a technology for realizing 3D display by visually recognizing images with a right eye and a left eye is known. There has been proposed a technique for directly reflecting the operation of the operator on the 3D displayed image without using a keyboard or a mouse. For example, the operator's operation motion is imaged with two or more cameras, the position analysis of the position indicated by the operator is relatively performed, and the operator's operation motion in the real space is matched with the virtual space displayed in 3D. Conventional techniques have been proposed.

JP 2006-277666 A JP 2011-90611 A

  However, the conventional technique needs to perform a complicated process of associating the position of the real space with the position of the virtual space, and performing such a process itself decreases accuracy or slows down the reaction process. May be connected.

  The disclosed technology has been made in view of the above, and provides a 3D input device and a 3D input method capable of associating a real space position and a virtual space position without performing complicated processing. For the purpose.

  The 3D input device disclosed in the present application includes a right-eye image capturing unit and a left-eye image capturing unit. The right-eye image capturing unit captures an image of the real space that the operator visually recognizes with the right eye together with the right-eye image displayed on the display unit capable of 3D display. The left-eye image capturing unit captures an image of the real space that the operator visually recognizes with the left eye together with the left-eye image displayed on the display unit.

  According to the 3D input device disclosed in the present application, it is possible to associate the position of the real space and the position of the virtual space without performing complicated processing.

FIG. 1 is a diagram illustrating an overall configuration of a system according to the first embodiment. FIG. 2 is a block diagram illustrating a functional configuration of the 3D input / output device according to the first embodiment. FIG. 3 is a diagram illustrating an example of a displayed image. FIG. 4 is a diagram illustrating an example of an image visually recognized by the right eye. FIG. 5 is a diagram illustrating an example of an image visually recognized by the left eye. FIG. 6 is a diagram schematically showing the position of the visually recognized image. FIG. 7 is a diagram schematically illustrating a state where an instruction is given to a 3D image. FIG. 8 is a diagram illustrating an example of an image captured by the right-eye image capturing unit. FIG. 9 is a diagram illustrating an example of an image captured by the left-eye image capturing unit. FIG. 10 is a diagram illustrating an example of a displayed image. FIG. 11 is a diagram schematically showing the position of the visually recognized image. FIG. 12 is a flowchart showing the procedure of the transmission process. FIG. 13 is a flowchart showing the procedure of the reception process. FIG. 14 is a diagram illustrating an overall configuration of a system according to the second embodiment. FIG. 15 is a block diagram illustrating a functional configuration of the 3D input / output device and the remote control robot according to the second embodiment. FIG. 16 is a diagram illustrating an example of mapping of the right-eye image. FIG. 17 is a diagram illustrating an example of mapping of the left-eye image. FIG. 18 is a diagram showing in detail the coordinates of the shutter of the camera in the right-eye image. FIG. 19 is a diagram showing in detail the coordinates of the shutter of the camera in the left-eye image. FIG. 20 is a flowchart showing the procedure of the detection process. FIG. 21 is a diagram for describing an example of a computer that executes a 3D input program.

  Embodiments of a 3D input device and a 3D input method disclosed in the present application will be described below in detail with reference to the drawings. Note that this embodiment does not limit the disclosed technology. Each embodiment can be appropriately combined within a range in which processing contents are not contradictory.

[Configuration of System 10]
In the first embodiment, a case will be described in which the present invention is applied to a system 10 that performs conversation while sharing images between remote locations. FIG. 1 is a diagram illustrating an overall configuration of a system according to the first embodiment. In the example of FIG. 1, the operator 12a at the base 11a and the operator 12b at the base 11b have a conversation between the two bases 11a and 11b in front of an object displayed as a 3D image while pointing to the 3D image. Shows the case.

  As shown in FIG. 1, the system 10 includes a 3D input / output device 20 at each of the bases 11a and 11b. Hereinafter, when distinguishing between the base 11a side and the base 11b side, the 3D input / output device 20 on the base 11a side is referred to as a 3D input / output device 20a, and the 3D input / output device 20 on the base 11b side is referred to as the 3D input / output device 20b. Called. The 3D input / output devices 20a and 20b according to the first embodiment have basically the same configuration, operate as a transmission side of a 3D image shared by the 3D input / output device 20a, and a 3D image shared by the 3D input / output device 20b. Act as a receiver. The 3D input / output device 20a includes a communication unit 21a, a display unit 22a, a glasses camera 23a, and a control unit 24a. The 3D input / output device 20b includes a communication unit 21b, a display unit 22b, a glasses camera 23b, and a control unit 24b. The communication unit 21a and the communication unit 21b are connected to be communicable via the network 13. As an aspect of the network 13, any communication network such as mobile communication such as a mobile phone, the Internet (Internet), a LAN (Local Area Network), a VPN (Virtual Private Network) or the like can be cited, regardless of whether it is wired or wireless. It is done. The network 13 may select a communication medium according to an application demanded by the system 10 such as performing real-time transmission with as little delay as possible or performing low-cost transmission.

  At the site 11a, the operator 12a wears the glasses camera 23a on the head and visually recognizes the 3D image displayed on the display unit 22a. The display unit 22a displays a 3D image that can be stereoscopically viewed by separately viewing with the right eye and the left eye. For example, the right-eye image and the left-eye image are alternately displayed in a time division manner on the display unit 22a. The operator 12a visually recognizes the object as a 3D image by visually recognizing the right-eye image displayed on the display unit 22a via the glasses camera 23a with the right eye and visually recognizing the left-eye image with the left eye. The example of FIG. 1 illustrates a state where the operator 12a is viewing the camera 14 as a 3D image. The glasses camera 23a incorporates two photographing units and photographs a right-eye image that the operator 12a visually recognizes with the right eye and a right-eye image that the operator 12a visually recognizes with the left eye. The image for the right eye and the image for the left eye photographed by the two photographing units of the glasses camera 23a are transmitted to the base 11b side.

  At the base 11b, the operator 12b wears the glasses camera 23b on the head and visually recognizes the 3D image displayed on the display unit 22b. On the display unit 22b, an image that can be viewed stereoscopically is displayed by separately viewing with the right eye and the left eye. For example, the right-eye image and the left-eye image received from the base 11a are alternately displayed on the display unit 22b in a time division manner. The operator 12b visually recognizes the object as a 3D image by visually recognizing the right-eye image displayed on the display unit 22b via the glasses camera 23b with the right eye and visually recognizing the left-eye image with the left eye. As a result, when the operator 12a points to the camera 14 that is developed as a 3D image in front of the glasses camera 23a, the system 10 expands the image of pointing the camera 14 in front of the operator 12b. Then, a conversation can be performed while pointing to the 3D image.

  In the present embodiment, the case where the image developed in front of the operator 12a is shared with the operator 12b will be described. However, the image developed in front of the operator 12b is displayed by the same mechanism. It is also possible to share with the operator 12a. In addition, for example, the control unit 24a and the control unit 24b communicate with each other to give one side an operation right to the 3D image, and the one to which the operation right is given displays an image displayed on the display unit incorporated in the glasses camera. Is taken and transmitted to the other, and the image transmitted by the other is displayed on the display unit. By alternately giving this operation right, it is also possible to share images with each other.

[Device Configuration of 3D Input / Output Devices 20a and 20b]
Next, the device configuration of the 3D input / output devices 20a and 20b will be described. FIG. 2 is a block diagram illustrating a functional configuration of the 3D input / output device according to the first embodiment. As illustrated in FIG. 2, the 3D input / output device 20a includes a communication unit 21a, a display unit 22a, a glasses camera 23a, and a control unit 24a. The 3D input / output device 20b includes a communication unit 21b, a display unit 22b, a glasses camera 23b, and a control unit 24b. As described above, the 3D input / output devices 20a and 20b have basically the same configuration. Hereinafter, the configuration will be mainly described with respect to the 3D input / output device 20a, and the 3D input / output device 20b will be mainly described with respect to portions different from the 3D input / output device 20a.

  The communication unit 21a is an interface that performs communication control with another device, for example, the 3D input / output device 20 at another base. The communication unit 21 a transmits and receives various types of information to and from the 3D input / output device 20 at another base via the network 13. For example, the communication unit 21a transmits an image signal of image information indicating a 3D image to the 3D input / output device 20b. As an aspect of the communication unit 21a, a network interface card such as a LAN card can be employed.

  The display unit 22a is a display device that displays various types of information. An example of the display unit 22a includes a display device such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube). The display unit 22a displays an image that can be displayed in 3D by visually recognizing the right eye and the left eye individually. For example, the display unit 22a alternately displays the right-eye image and the left-eye image in a time division manner based on the image signal input from the control unit 24a.

  The glasses camera 23a is a device that captures a 3D image that the operator wears on the head and is displayed on the display unit 22a. The eyeglass camera 23a includes a right-eye image capturing unit 30R, a left-eye image capturing unit 30L, a right-eye finder 31R, and a left-eye finder 31L.

  The right-eye image capturing unit 30R and the left-eye image capturing unit 30L are provided with a predetermined interval corresponding to the interval between the left and right eyes of a person so that the capturing direction is the same. The eyeglass camera 23a includes an adjustment mechanism that adjusts the interval between the right-eye image capturing unit 30R and the left-eye image capturing unit 30L, and the operator adjusts the interval between the right-eye image capturing unit 30R and the left-eye image capturing unit 30L. It may be possible. The right-eye image capturing unit 30R captures a right-eye image that the operator 12a visually recognizes with the right eye. The left-eye image capturing unit 30L captures a left-eye image that the operator 12a visually recognizes with the left eye. For example, the right-eye image capturing unit 30R captures the right-eye image displayed on the display unit 22a in synchronization with a signal indicating the capturing timing input from the control unit 24a. The left-eye image capturing unit 30L captures the left-eye image displayed on the display unit 22a in synchronization with a signal indicating the capturing timing input from the control unit 24a. Image information of an image captured by the right-eye image capturing unit 30R is output to the right-eye finder 31R and the control unit 24a. Image information of the image captured by the left-eye image capturing unit 30L is output to the left-eye finder 31L and the control unit 24a.

  The right-eye finder 31R and the left-eye finder 31L face the left and right sides of the person with the display surface facing the direction opposite to the shooting direction on the opposite side of the plane on which the right-eye image shooting unit 30R and the left-eye image shooting unit 30L are provided. Are provided at a predetermined interval corresponding to the interval between the eyes. The eyeglass camera 23a includes an adjustment mechanism that adjusts the interval and position between the right-eye finder 31R and the left-eye finder 31L, and the operator 12a determines the interval and position between the right-eye image capturing unit 30R and the left-eye image capturing unit 30L. It is good also as what can be adjusted. The right-eye finder 31R displays the right-eye image captured by the right-eye image capturing unit 30R. The left-eye finder 31L displays the left-eye image captured by the left-eye image capturing unit 30L. The operator 12a visually recognizes the right-eye image with the right eye via the right-eye finder 31R, and visually recognizes the left-eye image with the left eye via the left-eye finder 31L, thereby visually recognizing the 3D image.

  The control unit 24a has an internal memory for storing programs defining various processing procedures and control data, and executes various processes using these. Various integrated circuits and electronic circuits can be employed for the control unit 24a. For example, an ASIC (Application Specific Integrated Circuit) is an example of the integrated circuit. Examples of the electronic circuit include a central processing unit (CPU) and a micro processing unit (MPU). The control unit 24 a includes an input unit 40, a display control unit 41, an imaging control unit 42, an image signal generation unit 43, and an imaging range adjustment unit 44.

  The input unit 40 is an input interface for inputting an image signal of a 3D image. This image signal includes image information of the right eye image and image information of the left eye image. This image signal may be of any transmission method as long as the image information of the right-eye image and the image information of the left-eye image are included. For example, the image signal may be stored by arranging the image information of the right-eye image and the left-eye image for each frame, such as a frame packing method, a side-by-side method, or a top-and-bottom method. Further, the image signal may be one in which the image information of the right eye image and the image for the left eye are alternately stored for each frame, such as a frame sequential method.

  The display control unit 41 is a processing unit that controls display of an image on the display unit 22a. Based on the image signal input to the input unit 40, the display control unit 41 generates an image signal of an image that can be stereoscopically viewed by separately viewing with the right eye and the left eye, and outputs the image signal to the display unit 22a. For example, the display control unit 41 generates an image signal for alternately displaying the right-eye image and the left-eye image included in the image signal in a time division manner, and outputs the image signal to the display unit 22a. In addition, the display control unit 41 generates a timing signal indicating the timing at which the right-eye image and the left-eye image are displayed, and outputs the generated timing signal to the imaging control unit 42. As a result, the right eye image and the left eye image are alternately displayed in a time-division manner on the display unit 22a.

  In this embodiment, the image signal input to the input unit 40 includes the image information of the right-eye image and the image information of the left-eye image. However, the image information of the right-eye image and the image of the left-eye image are used. Information may be input to the input unit 40 as another image signal. In this embodiment, a signal indicating the identification timing of the left and right images may also be input to the input unit 40 from the outside. In this case, the display control unit 41 alternates the right-eye image shown in the right-eye image signal and the left-eye image shown in the left-eye image signal in a time-division manner based on a signal indicating the identification timing of the left and right images. Display.

  The imaging control unit 42 is a processing unit that controls imaging of the right-eye image capturing unit 30R and the left-eye image capturing unit 30L. In synchronization with the timing signal input from the display control unit 41, the imaging control unit 42 transmits a signal indicating the imaging timing to the right-eye image capturing unit 30R at the timing when the right-eye image is displayed on the display unit 22a. In addition, in synchronization with the timing signal, the imaging control unit 42 transmits a signal indicating the imaging timing to the left-eye image capturing unit 30L at the timing when the left-eye image is displayed on the display unit 22a. Accordingly, the right eye image capturing unit 30R captures the right eye image displayed on the display unit 22a. The left-eye image capturing unit 30L captures the left-eye image displayed on the display unit 22a.

  The image signal generation unit 43 is a processing unit that generates an image signal of a captured 3D image. The image signal generation unit 43 generates an image signal of a 3D image including image information of the right eye image captured by the right eye image capturing unit 30R and image information of the left eye image captured by the left eye image capturing unit 30L. Output. The image signal may be of any transmission method as long as the image information of the right-eye image and the image information of the left-eye image are included. The output image signal is transmitted to the 3D input / output device 20 at the partner site by the communication unit 21a. For example, the image signal of the 3D image captured by the right eye image capturing unit 30R and the left eye image capturing unit 30L of the glasses camera 23a of the 3D input / output device 20a is transmitted to the 3D input / output device 20b.

  The image signal generation unit 43 may generate and output image signals of image information captured by the right-eye image capturing unit 30R and the left-eye image capturing unit 30L as needed. Also, the image signal generation unit 43, for the captured right-eye image and the captured left-eye image, when a difference occurs in the data from the image captured immediately before, respectively, An image signal of an image may be generated and output. In this case, in the 3D input / output device 20 on the receiving side, when the difference image signal is not transmitted, the 3D image received immediately before may be continuously displayed.

  The imaging range adjustment unit 44 is a processing unit that adjusts the angle of view of the display unit and the imaging ranges of the right-eye image capturing unit 30R and the left-eye image capturing unit 30L. The imaging range adjustment unit 44 determines whether or not the display unit 22a is included in the imaging range based on the image information input from the right-eye image capturing unit 30R and the left-eye image capturing unit 30L. For example, the imaging range adjustment unit 44 determines whether the image indicated by the image information input from the right-eye image capturing unit 30R and the left-eye image capturing unit 30L includes a rectangular frame indicating the display unit 22a. The imaging range adjustment unit 44 zooms out the right-eye image capturing unit 30R and the left-eye image capturing unit 30L to display the display unit 22a when the display unit 22a is not included in the capturing range. Adjust to. The imaging range adjusting unit 44 zooms in on the right-eye image capturing unit 30R and the left-eye image capturing unit 30L when the display unit 22a is included in the capturing range but the area of the display unit 22a is equal to or smaller than a predetermined allowable size. Adjustment is made so that the area of the display unit 22a becomes larger. When the glasses camera 23a has a drive mechanism that changes the shooting direction of the right-eye image capturing unit 30R and the left-eye image capturing unit 30L, the imaging range adjustment unit 44 controls the drive mechanism so that the display unit 22a can capture the shooting range. You may adjust so that it may become the center in the inside.

  The communication unit 21b of the 3D input / output device 20b transmits / receives various information to / from the 3D input / output device 20a of another base via the network 13. For example, the communication unit 21b receives an image signal of image information indicating a 3D image from the 3D input / output device 20a.

  The glasses camera 23b includes a right-eye image capturing unit 32R, a left-eye image capturing unit 32L, a right-eye finder 33R, and a left-eye finder 33L. The right-eye image capturing unit 32R, the left-eye image capturing unit 32L, the right-eye finder 33R, and the left-eye finder 33L are a right-eye image capturing unit 30R, a left-eye image capturing unit 30L, a right-eye finder 31R, and a left-eye finder 33R. It is comprised similarly to the viewfinder 31L.

  The control unit 24 b includes an input unit 50, a display control unit 51, an imaging control unit 52, an image signal generation unit 53, and an imaging range adjustment unit 54. The input unit 50, the display control unit 51, the imaging control unit 52, and the image signal generation unit 53 are the input unit 40, the display control unit 41, the imaging control unit 42, the image signal generation unit 43, and the imaging range adjustment of the control unit 24a. It corresponds to each part 44 functionally.

  The image signal received from the 3D input / output device 20a is input to the input unit 50. Based on the image signal input to the input unit 50, the display control unit 51 generates an image signal of an image that can be stereoscopically viewed by separately viewing with the right eye and the left eye, and outputs the image signal to the display unit 22b. As a result, the right-eye image and the left-eye image captured by the 3D input / output device 20a are alternately displayed in a time division manner on the display unit 22b. The display control unit 51 generates a timing signal indicating the timing at which the right-eye image and the left-eye image are displayed. The imaging control unit 52 transmits a signal indicating the imaging timing to the right-eye image capturing unit 32R and the left-eye image capturing unit 32L in synchronization with the timing signal generated by the display control unit 51. Accordingly, the right eye image capturing unit 32R captures the right eye image displayed on the display unit 22b. The left-eye image capturing unit 32L captures the left-eye image displayed on the display unit 22b. The image photographed by the right-eye image photographing unit 32R is displayed on the right-eye finder 33R. The image photographed by the left-eye image photographing unit 32L is displayed on the left-eye finder 33L. The operator 12b visually recognizes the right-eye image with the right eye via the right-eye finder 33R, and visually recognizes the left-eye image with the left eye via the left-eye finder 33L, thereby visually recognizing the 3D image.

  Note that image information of images captured by the right-eye image capturing unit 32R and the left-eye image capturing unit 32L is input to the image signal generating unit 53. In this embodiment, since the 3D input / output device 20b operates as a reception side of the 3D image shared, the image signal generation unit 53 does not generate an image signal of the 3D image. When the 3D input / output device 20b operates as a transmission side of the 3D image shared, the image signal generation unit 53 generates and outputs an image signal of the 3D image.

  Here, an image displayed on the display unit 22a will be described. FIG. 3 is a diagram illustrating an example of a displayed image. In the present embodiment, the positions are exaggerated and shifted in order to easily identify the right-eye image and the left-eye image. Actually, the images of the respective objects are displayed while being shifted by the amount of parallax corresponding to the position in the three-dimensional space. When the operator 12a views the display unit 22a without going through the glasses camera 23a, as shown in FIG. 3, the right-eye image and the left-eye image displayed on the display unit 22a are superimposed and viewed. This superimposed image is an image in which the image for the right eye and the image for the left eye are overlapped, and a 3D effect cannot be obtained even when viewed with both eyes.

  On the other hand, when the operator 12a views the display unit 22a via the glasses camera 23a, the image for the right eye and the image for the left eye displayed on the display unit 22a are visually recognized separately. FIG. 4 is a diagram illustrating an example of an image visually recognized by the right eye. FIG. 5 is a diagram illustrating an example of an image visually recognized by the left eye. In addition, the broken line of FIG. 4, FIG. 5 is a supplement line for demonstrating the position visually recognized as a 3D image. Since the right-eye image capturing unit 30R of the glasses camera 23a captures in synchronization with the timing at which the right-eye image is displayed on the display unit 22a, the right-eye image capturing unit 30R displayed on the display unit 22a is displayed as shown in FIG. Only the image of is taken. Further, the left-eye image capturing unit 30L of the glasses camera 23a captures in synchronization with the timing when the left-eye image is displayed on the display unit 22a, so that the image is displayed on the display unit 22a as shown in FIG. Only the image for the left eye is taken. An image captured by the right-eye image capturing unit 30R is displayed on the right-eye finder 31R. The left-eye finder 31L displays an image captured by the left-eye image capturing unit 30L. In addition, image information of images captured by the right-eye image capturing unit 30R and the left-eye image capturing unit 30L is output to the control unit 24a.

  The operator 12a visually recognizes the right-eye image with the right eye through the right-eye finder 31R of the glasses camera 23a, and visually recognizes the left-eye image with the left eye through the right-eye finder 31R. Thereby, the operator 12a visually recognizes a 3D image having a depth due to the parallax between the right-eye image and the left-eye image. For example, when the image shown in FIG. 4 is viewed with the right eye and the image shown in FIG. 5 is viewed with the left eye, the operator 12a displays the images displayed in A, B, and C in FIGS. As shown in FIG. 6, it is recognized as a 3D image having a 3D depth at positions a, b, and c. FIG. 6 is a diagram schematically showing the position of the visually recognized image.

  Here, for example, a case where an instruction to the virtual space is given by the hand of the operator 12a will be described as an example. The operator 12a gives an instruction to the 3D image that the operator 12a wants to intend toward the 3D image by holding the hand over the 3D image that is virtually viewed with the spatial information of the depth. FIG. 7 is a diagram schematically illustrating a state where an instruction is given to a 3D image. In the example of FIG. 7, the hand 61 is held at a position near the camera 60 in the 3D image.

  When the operator 12a holds the hand 61 over the 3D image, the operator 12a views the real image of the hand 61 together with the 3D image through the glasses camera 23a. The right-eye image capturing unit 30R of the glasses camera 23a captures in synchronization with the timing at which the right-eye image is displayed on the display unit 22a. Therefore, the right-eye image displayed on the display unit 22a and the real image of the hand 61 are displayed. Take a picture of the right eye. Further, the left-eye image photographing unit 30L of the glasses camera 23a performs photographing in synchronization with the timing at which the left-eye image is displayed on the display unit 22a. Therefore, the left-eye image displayed on the display unit 22a and the hand 61 are displayed. Take a picture of the real image of the camera as seen from the left eye. FIG. 8 is a diagram illustrating an example of an image captured by the right-eye image capturing unit. FIG. 9 is a diagram illustrating an example of an image captured by the left-eye image capturing unit. Image information of an image captured by the right eye image capturing unit 30R and the left eye image capturing unit 30L of the glasses camera 23a is converted into an image signal by the image signal generating unit 43, and the generated image signal is transmitted to the communication unit 21a. Is transmitted to the 3D input / output device 20b of the base 11b.

  On the display unit 22b of the 3D input / output device 20b, the right-eye image and the left-eye image are alternately displayed in a time division manner based on the transmitted image information. When the operator 12b views the display unit 22b without using the glasses camera 23b, the right-eye image and the left-eye image displayed on the display unit 22b are visually recognized as shown in FIG. FIG. 10 is a diagram illustrating an example of a displayed image. This superimposed image is an image in which the image for the right eye and the image for the left eye are overlapped, and a 3D effect cannot be obtained even when viewed with both eyes.

  The operator 12b of the base 11b visually recognizes the object as a 3D image by visually recognizing the right-eye image displayed on the display unit 22b via the glasses camera 23b with the right eye and visually recognizing the left-eye image with the left eye. The image visually recognized by the operator 12b of the base 11b is the same as the image visually recognized by the operator 12a of the base 11a. That is, the operator 12b of the base 11b and the operator 12a of the base 11a visually recognize the same image. For example, the operator 12b of the base 11b visually recognizes the image shown in FIG. 8 with the right eye and visually recognizes the image shown in FIG. 9 with the left eye. As a result, the operator 12b can feel the 3D space as shown in FIG. 11 and can visually recognize the 3D image in which the operator 12a holds the hand 61 over the camera 60. The same 3D as the operator 12a feels. Share images. FIG. 11 is a diagram schematically showing the position of the visually recognized image.

  As described above, the system 10 according to the present embodiment can share an image between the operator 12a at the base 11a and the operator 12b at the base 11b. In addition, the 3D input / output device 20a can associate the position of the real space with the position of the virtual space without performing complicated processing.

[Process flow]
Next, a flow of transmission processing for transmitting a 3D image shared by the 3D input / output device 20a according to the present embodiment will be described. FIG. 12 is a flowchart showing the procedure of the transmission process. This transmission process is executed at a timing when a 3D image signal is input to the input unit 40 and a predetermined operation for instructing the start of display is performed.

  As shown in FIG. 12, the display control unit 41 generates an image signal for alternately displaying the right-eye image and the left-eye image in a time division manner based on the image signal input to the input unit 40, and outputs the image signal to the display unit 22a. (S10). As a result, the right eye image and the left eye image are alternately displayed in a time-division manner on the display unit 22a.

  The display control unit 41 generates a timing signal indicating the timing at which the right-eye image and the left-eye image are displayed (S11). The imaging control unit 42 transmits a signal indicating the imaging timing to the right-eye image capturing unit 30R at the timing when the right-eye image is displayed in synchronization with the timing signal, and the left-eye image capturing unit at the timing when the left-eye image is displayed. A signal indicating the photographing timing is transmitted to 30L (S12). As a result, the right-eye image capturing unit 30R performs capturing in synchronization with the timing at which the right-eye image is displayed on the display unit 22a. Further, the left-eye image capturing unit 30L performs capturing in synchronization with the timing at which the left-eye image is displayed on the display unit 22a. An image captured by the right-eye image capturing unit 30R is displayed on the right-eye finder 31R. The left-eye finder 31L displays an image captured by the left-eye image capturing unit 30L.

  The image signal generation unit 43 generates an image signal of a 3D image including image information of the right eye image captured by the right eye image capturing unit 30R and image information of the left eye image captured by the left eye image capturing unit 30L. (S13). The generated image signal of the 3D image is transmitted to the 3D input / output device 20b by the communication unit 21a.

  The display control unit 41 determines whether an instruction to end the process has been given (S14). When the process end is instructed (Yes at S14), the process is ended. On the other hand, if the end of the process is not instructed (No at S14), the process proceeds to step S10.

  Next, a flow of reception processing for receiving a 3D image shared by the 3D input / output device 20b according to the present embodiment will be described. FIG. 13 is a flowchart showing the procedure of the reception process. This reception process is executed at the timing when a 3D image signal is input from the 3D input / output device 20a to the input unit 50 via the communication unit 21b and a predetermined operation for instructing the start of display is performed.

  As illustrated in FIG. 13, the display control unit 51 generates an image signal for alternately displaying the right-eye image and the left-eye image in a time division manner based on the image signal input to the input unit 50 and outputs the image signal to the display unit 22 b. (S20). As a result, the right-eye image and the left-eye image are alternately displayed in a time division manner on the display unit 22b. The image displayed on the display unit 22b is an image visually recognized by the operator 12a at the base 11a.

  The display control unit 51 generates a timing signal indicating the timing at which the right-eye image and the left-eye image are displayed (S21). The imaging control unit 52 transmits a signal indicating the imaging timing to the right-eye image capturing unit 32R at the timing when the right-eye image is displayed in synchronization with the timing signal, and the left-eye image capturing unit at the timing when the left-eye image is displayed. A signal indicating the photographing timing is transmitted to 32L (S22). Accordingly, the right eye image capturing unit 32R performs capturing in synchronization with the timing at which the right eye image is displayed on the display unit 22b. The left-eye image capturing unit 32L performs capturing in synchronization with the timing at which the left-eye image is displayed on the display unit 22b. An image photographed by the right-eye image photographing unit 32R is displayed on the right-eye finder 33R. The left eye finder 33L displays an image photographed by the left eye image photographing unit 32L. Thereby, the operator 12b of the base 11b can share the 3D image visually recognized by the operator 12a of the base 11a.

  The display control unit 51 determines whether or not an instruction to end the process is given (S23). When the process end is instructed (Yes at S23), the process ends. On the other hand, when the end of the process is not instructed (No at S23), the process proceeds to step S20.

[Effect of Example 1]
In the 3D input / output device 20a according to the present embodiment, the right eye image capturing unit 30R captures an image of the real space that the operator 12a visually recognizes with the right eye together with the right eye image displayed on the display unit 22a capable of 3D display. To do. Further, the 3D input / output device 20a captures an image of the real space that the operator 12a visually recognizes with the left eye, together with the left-eye image displayed on the display unit 22a, by the left-eye image capturing unit 30L. Therefore, the 3D input / output device 20a can associate the position of the real space and the position of the virtual space without performing complicated processing.

  Further, the 3D input / output device 20a according to the present embodiment outputs image information indicating images captured by the right-eye image capturing unit 30R and the left-eye image capturing unit 30L to other devices that share the 3D image. Therefore, the 3D input / output device 20a can share a 3D image in which the position of the real space and the position of the virtual space are associated with another device without performing complicated processing.

  In the 3D input / output device 20a according to the present embodiment, the right-eye image capturing unit 30R displays the right-eye image and the left-eye image alternately on the display unit 22a in a time-division manner. Synchronously, an image of the real space that the operator visually recognizes with the right eye is taken together with the right-eye image. In addition, the 3D input / output device 20a is configured so that the left-eye image capturing unit 30L can synchronize with the display timing at which the left-eye image is displayed on the display unit 22a in the real space that the operator visually recognizes with the left-eye. Take a picture. Therefore, the 3D input / output device 20a performs complicated processing by visually recognizing the image captured by the right-eye image capturing unit 30R with the right eye and visually recognizing the image captured by the left-eye image capturing unit 30L. The position of the real space and the position of the virtual space can be associated with each other.

  Further, the 3D input / output device 20a according to the present embodiment adjusts the angle of view of the display unit 22a and the imaging ranges of the right-eye image capturing unit 30R and the left-eye image capturing unit 30L. Therefore, the 3D input / output device 20a can appropriately capture the 3D image displayed on the display unit 22a.

  Further, the 3D input / output device 20a according to the present embodiment displays an image photographed by the right-eye image photographing unit 30R on the right-eye finder 31R of the glasses camera 23a so that the operator 12a can visually recognize with the right eye. In addition, the 3D input / output device 20a displays an image photographed by the left-eye image photographing unit 30L on the left-eye finder 31L of the glasses camera 23a so that the operator 12a can visually recognize with the left eye. Therefore, the 3D input / output device 20a can visually recognize an image visually recognized by the operator 12b at the base 11b.

[Configuration of System 70]
Next, Example 2 will be described. In the second embodiment, a case where the present invention is applied to a system 70 that performs remote operation will be described. FIG. 14 is a diagram illustrating an overall configuration of a system according to the second embodiment. The example of FIG. 14 shows a case where the operator 12a of the base 11a remotely operates the equipment arranged at the base 11b between two remote bases 11a and 11b. The same parts as those of the 3D input / output device 20a according to the first embodiment will be described with the same reference numerals.

  As shown in FIG. 14, the system 70 includes a 3D input / output device 20a at the base 11a. Further, the system 70 has a remote operation robot 80 at the base 11b. Similar to the first embodiment, the 3D input / output device 20a includes a communication unit 21a, a display unit 22a, a glasses camera 23a, and a control unit 24a. The remote operation robot 80 includes a communication unit 81, a camera unit 82, a control unit 83, and an arm unit 84. The communication unit 21a and the communication unit 81 are connected to be communicable via the network 13.

  The camera unit 82 of the remote operation robot 80 includes two imaging units, and each of the operation objects placed at predetermined operation target positions is imaged by the two imaging units. The example of FIG. 14 shows a state where the camera 71 is visually recognized as an operation target. One image captured by the two image capturing units of the camera unit 82 is a right-eye image for viewing with the right eye, and the other image is captured for a right-eye image viewed with the left eye. The right-eye image and the left-eye image captured by the two image capturing units of the camera unit 82 are transmitted to the base 11a side.

  On the display unit 22a of the 3D input / output device 20a, the right-eye image and the left-eye image received from the remote control robot 80 are alternately displayed in a time division manner. The operator 12a visually recognizes the operation target as a 3D image by visually recognizing the right-eye image displayed on the display unit 22a via the glasses camera 23a with the right eye and visually recognizing the left-eye image with the left eye. The example of FIG. 14 shows a state in which the operator 12a is viewing the camera 60 as an operation target by a 3D image. The operator 12a operates the camera 60 visually recognized as a 3D image. The glasses camera 23a photographs the operation of the operator 12a together with the 3D image by two built-in photographing units. The control unit 24a detects the operation content of the operator 12a on the 3D image from the images captured by the two image capturing units incorporated in the glasses camera 23a, and sends operation information indicating the detected operation content to the remote operation robot 80. Send.

  The arm unit 84 of the remote operation robot 80 has two robot arms 85a and 85b. The remote operation robot 80 is similar to the operation performed by the operator 12a on the camera 60 by the controller 83 operating the robot arms 85a and 85b of the arm unit 84 based on the operation information received from the 3D input / output device 20a. Is performed on the camera 71. As a result, the system 10 enables the operator 12a to remotely operate the camera 71 from a remote location.

[Device Configuration of 3D Input / Output Device 20a and Remote Operation Robot 80]
Next, device configurations of the 3D input / output device 20a and the remote control robot 80 will be described. FIG. 15 is a block diagram illustrating a functional configuration of the 3D input / output device and the remote control robot according to the second embodiment. The configuration of the 3D input / output device 20a according to the second embodiment is substantially the same as that of the 3D input / output device 20a according to the first embodiment shown in FIG. The difference will be mainly described.

  As shown in FIG. 15, the remote control robot 80 includes a communication unit 81, a camera unit 82, a control unit 83, and an arm unit 84.

  The communication unit 81 is an interface that performs communication control with another device, for example, the 3D input / output device 20a at the base 11a. The communication unit 81 transmits / receives various information to / from the 3D input / output device 20a via the network 13. For example, the communication unit 81 transmits an image signal of image information indicating a 3D image of the operation target to the 3D input / output device 20a. In addition, the communication unit 81 receives operation information for the operation target from the 3D input / output device 20a. As an aspect of the communication unit 81, a network interface card such as a LAN card can be employed.

  The camera unit 82 is a device that captures an object to be operated by the arm unit 84. The camera unit 82 is arranged at a predetermined distance from a predetermined operation target position that can be operated by the arm unit 84, and includes a right-eye image capturing unit 85R that captures the operation target position, and a left-eye image capturing unit 85L. Have

  The right-eye image capturing unit 85R and the left-eye image capturing unit 85L are provided in the same direction with a predetermined interval corresponding to the interval between the left and right eyes of a person. The camera unit 82 includes an adjustment mechanism that adjusts the interval between the right-eye image capturing unit 85R and the left-eye image capturing unit 85L, and can adjust the interval between the right-eye image capturing unit 85R and the left-eye image capturing unit 85L. Also good. The right-eye image capturing unit 85R captures a right-eye image that the operator 12a visually recognizes with the right eye. The left-eye image capturing unit 85L captures a left-eye image that the operator 12a visually recognizes with the left eye. The image information of the image captured by the right-eye image capturing unit 85R and the image information of the image captured by the left-eye image capturing unit 85L are output to the control unit 83.

  The control unit 83 has an internal memory for storing programs defining various processing procedures and control data, and executes various processes using these. Various kinds of integrated circuits and electronic circuits can be adopted for the control unit 83. The control unit 83 includes an image signal generation unit 90 and an arm control unit 91.

  The image signal generation unit 90 is a processing unit that generates an image signal of a 3D image captured by the camera unit 82. The camera unit 82 generates and outputs an image signal of a 3D image including the image information of the right-eye image captured by the right-eye image capturing unit 85R and the image information of the left-eye image captured by the left-eye image capturing unit 85L. To do. The image signal may be of any transmission method as long as the image information of the right-eye image and the image information of the left-eye image are included. The output image signal is transmitted to the 3D input / output device 20a by the communication unit 81.

  Note that the image signal generation unit 90 may generate and output image signals of image information captured by the right eye image capturing unit 85R and the left eye image capturing unit 85L as needed. Further, the image signal generation unit 90, for the captured right-eye image and the captured left-eye image, when a difference occurs in the data from the image captured immediately before, respectively, An image signal of an image may be generated and output. In this case, in the 3D input / output device 20a on the receiving side, when the difference image signal is not transmitted, the 3D image received immediately before may be continuously displayed.

  The arm control unit 91 is a processing unit that controls the operation of the robot arms 85 a and 85 b of the arm unit 84. The arm control unit 91 drives various actuators 92 such as motors and cylinders that operate the robot arms 85a and 85b provided in the arm unit 84 according to the operation information received from the 3D input / output device 20a. The arms 85a and 85b are operated.

  On the other hand, the control unit 24 a of the 3D input / output device 20 a further includes a separation unit 45, a first difference extraction unit 46, a second difference extraction unit 47, and an operation detection unit 48.

  The image signal received from the remote control robot 80 is input to the input unit 40. The image signal input to the input unit 40 is output to the display control unit 41 and the separation unit 45.

  The separation unit 45 is a processing unit that separates the image information of the right-eye image and the image information of the left-eye image included in the image signal. The separation unit 45 outputs the image information of the separated right-eye image to the first difference extraction unit 46, and outputs the image information of the left-eye image to the second difference extraction unit 47.

  The first difference extraction unit 46 is a processing unit that extracts image changes from image differences. Image information of the right eye image is input from the separation unit 45 to the first difference extraction unit 46. In addition, image information of an image captured from the right-eye image capturing unit 30R is input to the first difference extraction unit 46. The first difference extraction unit 46 compares the portion corresponding to the right-eye image of the image captured by the right-eye image capturing unit 30 </ b> R with the right-eye image separated by the separation unit 45. The first difference extraction unit 46 extracts a change in a portion corresponding to the right-eye image of the captured image as a difference from the separated right-eye image.

  The second difference extraction unit 47 is also a processing unit that extracts a portion where an image has changed due to an operation. Image information of the left-eye image is input from the separation unit 45 to the second difference extraction unit 47. In addition, image information of an image captured from the left-eye image capturing unit 30L is input to the second difference extraction unit 47. The second difference extraction unit 47 compares the portion corresponding to the left-eye image of the image captured by the left-eye image capturing unit 30L with the left-eye image separated by the separation unit 45. The second difference extraction unit 47 extracts, as a difference, a change in a portion corresponding to the left-eye image of the captured image with respect to the separated left-eye image.

  The operation detection unit 48 is a processing unit that detects the operation content of the operator 12a on the 3D image. The operation detection unit 48 detects the operation of the operator 12a based on the difference information extracted by the first difference extraction unit 46 and the second difference extraction unit 47, respectively. For example, the operation detection unit 48 discriminates the target object operated from the position in the image of the changed portion extracted as the difference between the right-eye images and the changed portion extracted as the difference between the left-eye images. To detect.

  Here, operation detection will be described. Here, as illustrated in FIG. 7, the case where the operator 12a operates the shutter of the camera 60 visually recognized as a 3D image will be described as an example. When the operator 12a holds the hand 61 over the shutter of the camera 60, the right-eye image capturing unit 30R captures in synchronization with the timing at which the right-eye image is displayed on the display unit 22a. The displayed image for the right eye and the real image of the hand 61 are taken from the right eye. The left-eye image capturing unit 30L captures the left-eye image displayed on the display unit 22a and the real image of the hand 61 in order to capture in synchronization with the timing at which the left-eye image is displayed on the display unit 22a. Take a picture taken from.

  The first difference extraction unit 46 compares the portion corresponding to the right-eye image of the image photographed by the right-eye image photographing unit 30R with the right-eye image separated by the separation unit 45, and compares the changed portion with the difference. Extract as The second difference extraction unit 47 compares the portion corresponding to the left-eye image of the image photographed by the left-eye image photographing unit 30L with the left-eye image separated by the separation unit 45, and compares the changed portion with the difference. Extract as

  For example, as shown in FIG. 16, the first difference extraction unit 46 maps the right-eye image into an n × n range and uses the parameters β and φ in the right-eye image as shown in the following equation (1). Coordinate the position of. Similarly, as shown in FIG. 17, the second difference extraction unit 47 maps the right-eye image to an n × n range, and uses the parameters α and γ to set the left-eye according to the following equation (2). Coordinate the position in the image. FIG. 16 is a diagram illustrating an example of mapping of the right-eye image. FIG. 17 is a diagram illustrating an example of mapping of the left-eye image.

(X, Y) = (β, φ) (1)
(X, Y) = (α, γ) (2)

  Here, the coordinates of the shutter of the camera 60 in the right-eye image are assumed to be the coordinates shown in the following formula (3), as shown in FIG. Further, the coordinates of the shutter of the camera 60 in the left-eye image are assumed to be coordinates shown in the following formula (4) as shown in FIG. FIG. 18 is a diagram showing in detail the coordinates of the shutter of the camera in the right-eye image. FIG. 19 is a diagram showing in detail the coordinates of the shutter of the camera in the left-eye image.

(X, Y) = (j, k) (3)
(X, Y) = (l, m) (4)

  The first difference extraction unit 46 compares the portion corresponding to the right-eye image of the photographed image with the separated right-eye image, and extracts the coordinates of the portion where the image has changed as a difference. The second difference extraction unit 47 compares the portion corresponding to the left-eye image of the photographed image with the separated left-eye image, and extracts the portion where the image has changed as a difference.

  The operation detection unit 48 determines as follows, and determines that the operator 12a is touching the shutter of the camera 60. That is, the operation detection unit 48 determines whether or not the coordinates of the difference portion extracted by the first difference extraction unit 46 match the coordinates of the shutter shown in Expression (3). In addition, the operation detection unit 48 determines whether or not the coordinates of the difference portion extracted by the second difference extraction unit 47 match the coordinates of the camera shutter shown in Expression (4). The operation detection unit 48, when the coordinates of the difference portion by the first difference extraction unit 46 matches the coordinates of the shutter, and when the coordinates of the difference portion by the second difference extraction unit 47 match the coordinates of the shutter, It is detected that the hand 61 of the operator 12a is touching the shutter of the camera 60. When the operation detection unit 48 detects that the shutter of the camera 60 is touched, the operation detection unit 48 transmits operation information indicating the detected operation content to the remote operation robot 80.

  Further, the operation detection unit 48 detects an operation from a change in coordinates of a portion where the image extracted by the first difference extraction unit 46 and the second difference extraction unit 47 has changed. For example, the operation detection unit 48 moves the coordinates extracted by the first difference extraction unit 46 from (j, k) to the (j, k−1) direction, and is extracted by the second difference extraction unit 47. When the coordinates move from (l, m) to (l, m-1), it is detected that the shutter has been pressed.

[Process flow]
Next, a flow of detection processing for detecting an operation by the 3D input / output device 20a according to the present embodiment will be described. FIG. 20 is a flowchart showing the procedure of the detection process. This detection process is executed at a timing when a 3D image signal is input to the input unit 40 and a predetermined operation instructing the start of operation detection is performed.

  As illustrated in FIG. 20, the first difference extraction unit 46 compares the portion corresponding to the right-eye image of the image captured by the right-eye image capturing unit 30 </ b> R and the right-eye image separated by the separation unit 45. Then, the changed part is extracted as a difference (S30). The second difference extraction unit 47 compares the portion corresponding to the left-eye image of the image photographed by the left-eye image photographing unit 30L with the left-eye image separated by the separation unit 45, and compares the changed portion with the difference. (S31).

  The operation detection unit 48 determines whether or not the changed portion extracted by the first difference extracting unit 46 and the changed portion extracted by the second difference extracting unit 47 are the same object (S32). If they are the same object (Yes at S32), the operation detection unit 48 transmits the operation information indicating the detected operation content to the remote operation robot 80 as an operation on the object (S33). On the other hand, when it is not the same target object (S32 negative), it transfers to S34 mentioned later.

  The operation detection unit 48 determines whether the end of the process is instructed (S34). When the process end is instructed (Yes at S34), the process ends. On the other hand, when the end of the process is not instructed (No at S34), the process proceeds to S30.

[Effect of Example 2]
The 3D input / output device 20a according to the present embodiment inputs image information of the right-eye image and the left-eye image displayed on the display unit 22a. The 3D input / output device 20a extracts the difference between the portion corresponding to the right eye image of the image captured by the right eye image capturing unit 30R and the right eye image indicated by the input image information. Further, the 3D input / output device 20a extracts a difference between a portion corresponding to the left-eye image of the image captured by the left-eye image capturing unit 30L and the left-eye image indicated by the input image information. Then, the 3D input / output device 20a detects the operation of the operator based on the extracted difference information. Therefore, the 3D input / output device 20a can accurately detect the operation of the operator 12a. Further, the 3D input / output device 20a can detect an operation on an object of a 3D image.

  Although the embodiments related to the disclosed apparatus have been described so far, the present invention may be implemented in various different forms other than the embodiments described above. Therefore, another embodiment included in the present invention will be described below.

  For example, in the first embodiment described above, a case where an image signal is transmitted to a communication partner has been described, but the disclosed apparatus is not limited to this. For example, operation information of the operator 12a may be detected and operation information may be further transmitted as in the second embodiment.

  Further, in the above-described embodiment, the case has been described in which the right-eye image capturing unit 30R and the left-eye image capturing unit 30L perform shooting in accordance with the display timing of the right-eye image and the left-eye image of the display unit 22a. It is not limited to this. For example, a shutter is provided in front of the lenses of the right-eye image capturing unit 30R and the left-eye image capturing unit 30L, and the shutters are alternately opened and closed to transmit the right-eye image to the right-eye image capturing unit 30R. The left-eye image may be transmitted through 30L. In this case, the right-eye image capturing unit 30R and the left-eye image capturing unit 30L may perform capturing at the opening timing of the shutter. Alternatively, the right eye image capturing unit 30R and the left eye image capturing unit 30L may perform capturing at a predetermined frame rate, and the control unit 24a may extract an image during the shutter opening period. Further, the display unit 22a displays the right-eye image and the left-eye image with polarized light orthogonal to each other, and the right-eye image capturing unit 30R and the left-eye image capturing unit 30L are attached with polarizing lenses orthogonal to each other. You may permeate | transmit the image for left eyes separately.

  In the above-described embodiment, the case where the operator 12a visually recognizes the images captured by the right-eye finder 31R and the left-eye finder 31L after being captured by the right-eye image capturing unit 30R and the left-eye image capturing unit 30L has been described. However, the disclosed apparatus is not limited to this. For example, it is assumed that the operator 12a wears shutter glasses with shutters built into the left and right lenses to view a 3D image, and the shutter glasses are provided with the right-eye image capturing unit 30R and the left-eye image capturing unit 30L. Good.

[Distribution and integration]
In addition, each component of each illustrated apparatus does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the input unit 40, display control unit 41, imaging control unit 42, image signal generation unit 43, separation unit 45, first difference extraction unit 46, second difference extraction unit 47, operation detection of the 3D input / output device 20a Each processing unit of the unit 48 may be appropriately integrated. Further, the processing of each processing unit may be appropriately separated into a plurality of processing units. Further, the functions of the 3D input / output device 20 described above may be realized by having each processing unit in a separate device and being connected via a network to cooperate.

[3D input program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. Therefore, in the following, an example of a computer that executes a 3D input program having the same function as that of the above embodiment will be described with reference to FIG.

  FIG. 21 is a diagram for describing an example of a computer that executes a 3D input program. As illustrated in FIG. 21, the computer 200 includes an operation unit 210, a display 220, and a communication unit 230. The computer 200 further includes a CPU 250, a ROM 260, an HDD 270, and a RAM 280. These units 210 to 280 are connected via a bus 240.

  The HDD 270 includes the input unit 40, the display control unit 41, the imaging control unit 42, the image signal generation unit 43, the separation unit 45, the first difference extraction unit 46, the second difference extraction unit 47, and the operation detection unit 48. A 3D input program 270a that exhibits the above functions is stored in advance. The 3D input program 270a may be appropriately integrated or separated as in the case of each component shown in each embodiment. In other words, all the data stored in the HDD 270 need not always be stored in the HDD 270, and only the data necessary for processing may be stored in the HDD 270.

  Then, the CPU 250 reads the 3D input program 270a from the HDD 270 and develops it in the RAM 280. Thus, as shown in FIG. 21, the 3D input program 270a functions as a 3D input process 280a. The 3D input process 280a expands various data read from the HDD 270 in an area allocated to itself on the RAM 280 as appropriate, and executes various processes based on the expanded various data. The 3D input process 280a is executed by the display control unit 41, the imaging control unit 42, the image signal generation unit 43, the separation unit 45, the first difference extraction unit 46, the second difference extraction unit 47, and the operation detection unit 48. For example, the processes shown in FIGS. 12, 13, and 20. That is, the 3D input process 280a is the same as the display control unit 41, the imaging control unit 42, the image signal generation unit 43, the separation unit 45, the first difference extraction unit 46, the second difference extraction unit 47, and the operation detection unit 48. Execute the operation. It should be noted that all the processing units virtually realized on the CPU 250 do not always have to operate on the CPU 250, and only the processing units necessary for processing need only be virtually realized.

  Note that the 3D input program 270a is not necessarily stored in the HDD 270 or the ROM 260 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk inserted into the computer 200, so-called FD, CD-ROM, DVD disk, magneto-optical disk, or IC card. Then, the computer 200 may acquire and execute each program from these portable physical media. Each program is stored in another computer or server device connected to the computer 200 via a public line, the Internet, a LAN, a WAN, etc., and the computer 200 acquires and executes each program from these. It may be.

10, 70 System 12a, 12b Operator 20, 20a, 20b 3D input / output device 22a, 22b Display unit 24a, 24b Control unit 30R, 32R Right eye image capturing unit 30L, 32L Left eye image capturing unit 31R, 33R Right eye viewfinder 31L, 33L Left eye finder 40, 50 Input unit 41, 51 Display control unit 42, 52 Imaging control unit 43, 53 Image signal generation unit 44, 54 Imaging range adjustment unit 45 Separation unit 46 First difference extraction unit 47 Second Difference extraction unit 48 Operation detection unit

Claims (7)

A right-eye image capturing unit that captures an image of the real space that the operator visually recognizes with the right eye together with the right-eye image displayed on the display unit capable of 3D display;
A left-eye image capturing unit that captures an image of real space that the operator visually recognizes with the left eye together with the left-eye image displayed on the display unit;
An input unit for inputting image information of a right-eye image and a left-eye image displayed on the display unit;
A first difference extraction unit that extracts a difference between a portion corresponding to a right-eye image of an image photographed by the right-eye image photographing unit and a right-eye image indicated by the image information input to the input unit;
A second difference extraction unit that extracts a difference between a portion corresponding to a left-eye image of an image photographed by the left-eye image photographing unit and a left-eye image indicated by the image information input to the input unit;
The position of the change portion of the right-eye image obtained from the difference extracted by the first difference extraction unit and the position of the change portion of the left-eye image obtained from the difference extracted by the second difference extraction unit are the same object. If it is the same object, the operation detection unit that detects the operation of the operator on the object,
A 3D input device comprising: To claim 1, further comprising an output unit for outputting the image information indicating the image taken respectively by the right eye image capturing unit and the left eye image capturing unit to the other devices sharing the 3D image 3D input device of description. The right-eye image photographing unit synchronizes with the display timing of the right-eye image in which the right-eye image and the left-eye image are alternately displayed on the display unit in a time-sharing manner, and the operator uses the right-eye image together with the right-eye image. Take a picture of the real space to see
The left-eye image capturing unit captures an image of a real space that the operator visually recognizes with the left eye together with the left-eye image in synchronization with a display timing at which the left-eye image is displayed on the display unit. The 3D input device according to claim 1 or 2 . The right-eye image capturing unit is incorporated in a right-eye portion of shutter-type glasses that individually transmits a right-eye image and a left-eye image using a shutter,
The 3D input device according to claim 3 , wherein the left-eye image photographing unit is built in a left-eye portion of the shutter-type glasses. To any one of claims 1-4, characterized in that it further comprises an imaging range adjustment unit that adjusts the imaging range of the angle between the right eye image capturing unit and the left eye image capturing unit of the display unit 3D input device of description. A first display unit that displays an image captured by the right-eye image capturing unit as an image that is visually recognized by the operator with the right eye;
A second display unit that displays an image captured by the left-eye image capturing unit as an image visually recognized by the operator with the left eye;
3D input device according to any one of claims 1-5, characterized in that it further comprises a. Computer
The image of the real space that the operator visually recognizes with the right eye together with the image for the right eye displayed on the display unit capable of 3D display is captured by the image capturing unit for the right eye,
The image of the real space that the operator visually recognizes with the left eye together with the image for the left eye displayed on the display unit is photographed by the image photographing unit for the left eye ,
Extracting the difference between the portion corresponding to the right eye image of the image photographed by the right eye image photographing unit and the right eye image indicated by the image information of the right eye image and the left eye image input to the input unit; Extracting a difference between a portion corresponding to the left-eye image of the image photographed by the left-eye image photographing unit and the left-eye image indicated by the image information input to the input unit;
It is determined whether the position of the changed portion of the right-eye image and the position of the changed portion of the left-eye image obtained from the extracted difference are the same object, and if the same object, the operation of the operator on the object A 3D input method characterized by executing each processing to be detected .

Images