JP4829924B2 - Stereoscopic image conversion apparatus and program thereof - Google Patents

Description

  The present invention relates to a stereoscopic image conversion apparatus that converts stereoscopic information acquired by photographing a subject through a lens array or a spatial filter array into image information used for displaying a reproduced image, and particularly displays as a stereoscopic image. The present invention relates to a three-dimensional image conversion apparatus that converts image information to a reproduced image that is the same as the projections and depressions of a subject when the image size is changed.

  Conventionally, among methods for acquiring stereoscopic information of a subject through a lens array (micro optical element array) or a spatial filter (micro aperture array), as a device that captures and displays a stereoscopic image using a micro optical element array, An integral photography (IP) method is known.

  Here, normal stereoscopic image shooting based on the IP method will be described with reference to FIGS. 6A and 6B. The stereoscopic image capturing apparatus illustrated in FIG. 6A captures an object 111 from a capturing direction 114 indicated by an arrow through a lens group 112 configured by, for example, a convex lens. Here, the photographing direction 114 is a direction in which the stereoscopic image photographing device photographs the subject 111 disposed in front of the lens group 112 (leftward in FIG. 6A). At this time, an image of the object 111, for example, an image 115, is formed on the imaging plate 113 behind the lens group 112 (right side in FIG. 6A) as many as the number of convex lenses constituting the lens group 112. To do. Here, the imaging plate 113 is an information acquisition device configured to include a plurality of imaging elements arranged on a substrate. Each image sensor is, for example, a CCD (Charge Coupled Device) image sensor.

  FIG. 6B is a diagram showing a stereoscopic image display device including a lens group 122 in which convex lenses are arranged in one plane and a display element 123, a stereoscopic image 121, an observation direction 124, and an image 125 of the lens group. . Here, the display element 123 displays an image 125 corresponding to the image 115 photographed by the photographing plate 113 of the stereoscopic image photographing device shown in FIG. The display element 123 is composed of an information display device such as a liquid crystal panel. As a result of the operation of the stereoscopic image display device, as shown in FIG. 6B, a stereoscopic image 121 is generated at the same position as the place where the subject 111 exists. However, the stereoscopic image 121 is generated as a reverse view image. That is, as shown in FIG. 6A, when viewed from the shooting direction 114, the cylinder of the subject 111 is present in front of the prism. However, as illustrated in FIG. 6B, in the stereoscopic image 121 corresponding to the subject 111, a prism is generated in front of the cylinder as viewed from the observation direction 124. That is, a reverse view image in which the depth is inverted compared to the subject is generated.

  In FIGS. 6A and 6B, the lens group is displayed as a minute optical element array, and the operation is described on the assumption that the three-dimensional information of the subject is obtained and displayed using this lens group. As the minute optical element array, a minute aperture array (spatial filter) may be used.

In FIG. 6A, the size (height) of the prism 111 of the subject 111 is x _c , the distance from the lens group 112 to the prism of the subject 111 is z _c , and the distance from the lens group 112 to the plane on which the image 115 is taken. distance d _c, the convex lens pitch p _c of the lens group 112, the size of the image of the subject generated by the convex lens constituting the lens group 112 (height) and k _c. In FIG. 6B, the size (height) of the prism of the stereoscopic image 121 is x _r , the distance from the lens group 122 to the prism of the stereoscopic image 121 is z _r , and the image 125 can be displayed from the lens group 122. the distance to the surface d _r, the convex lens pitch p _r of the lens group 122, the size of each image displayed on the display device 123 (height) and k _r. In this case, the relationship between x _c and x _r is expressed by Equation (101) according to Non-Patent Document 1.

From the relationship of the above-mentioned formulas (102a) to (102c), the pitch pc of the convex lens constituting the lens group 112 of the stereoscopic image photographing apparatus shown in FIG. _6A and the stereoscopic shown in FIG. of the convex lens constituting the lens group 122 of the image display device and a pitch p _r with different guided to be able to control the size ratio x _r / x _c of the three-dimensional image 121 with respect to the subject 111. Similarly, the ratio x _r / x of the size of the stereoscopic image 121 with respect to the subject 111 is changed by changing the size k _c of the photographed subject image 115 and the size k _r of the displayed image 125. It is derived that _c can be controlled.

  Conventionally, in order to solve the problem of generating a reverse-view image in which the depth is inverted compared to the subject, the information acquired by the stereoscopic image capturing apparatus shown in FIG. A stereoscopic image depth conversion device is disclosed that inputs the information after being input to the stereoscopic image display device shown in FIG. 6B and finally generates a stereoscopic image with the correct depth (see Patent Document 1).

  A stereoscopic image depth conversion device disclosed in Patent Document 1 will be described with reference to FIGS. 7 (a) and 7 (b). 7A shows the image 131 acquired in FIG. 6A, the first virtual lens array 132, the virtually generated stereoscopic image 133, the second virtual lens array 134, and the second. It is a figure which shows the image 135 (image of the three-dimensional image 133 produced | generated virtually) produced | generated by the virtual lens array 134 of FIG. Here, the image 135 is not optically generated, but is generated by calculation processing of the stereoscopic image depth conversion device.

FIG. 7B shows an image 141 generated by the arithmetic processing in FIG. 7A (that is, the image 135 in FIG. 7A), a lens group 142 in which convex lenses are arranged in one plane, a stereoscopic image 143, and an observation. It is a figure which shows the direction 144. FIG. As shown in FIG. 7B, for example, as a result of the operation of the stereoscopic image display apparatus in FIG. 6B, the cylinder is observed in front of the prism as viewed from the observation direction 144. The stereoscopic image 143 is equivalent to the depth relationship of the subject in FIG.
Japanese Patent Laying-Open No. 2007-114484 (paragraph 0056, FIG. 4) J. Opt. Soc. Am. A, Vol. 21, pp.951-958, 2004

If the method described in Patent Document 1 is combined with a method for controlling the ratio of the size of a stereoscopic image derived from the relationship of the above-described formulas (102a) to (102c), a problem of generating a reverse-view image It is possible to generate a stereoscopic image with the correct depth while solving the above. However, in this case, when the ratio x _r / x _c of the size of the stereoscopic image 121 with respect to the subject 111 is to be changed, the lens group 122 (or the lens group 112) according to the ratio x _r / x _c. ) Will need to be changed. Thus, if the optical system apparatus must be changed in accordance with the ratio x _r / x _c , for example, in a stereoscopic television system, a reproduced image in which the size with respect to the subject is changed to a desired size is displayed. It is difficult to improve the production effect.

  The present invention has been made in view of the above-described problems, and does not depend on an optical device used for photographing and displaying a subject, and uses a calculation process to obtain the same reproduced image size as the unevenness of the subject. An object of the present invention is to provide a stereoscopic image conversion apparatus capable of controlling the ratio and a program thereof.

  The present invention was created to achieve the above object, and first, the stereoscopic image conversion apparatus according to claim 1 becomes a reproduced image in which the unevenness of the subject is inverted when displayed as a stereoscopic image. A stereoscopic image conversion apparatus that converts a light wave of a first element image group and generates a second element image group that is a reproduced image in which the size of the image is the same as the unevenness of the subject when displayed as a stereoscopic image. Thus, a distribution unit, a first element image conversion unit, an addition unit, a redistribution unit, a second element image conversion unit, and a combination unit are provided.

  According to such a configuration, the stereoscopic image conversion apparatus distributes the light waves of the first element image group for each element image by the distribution unit. Here, the light wave is a complex number representing the amplitude and phase when a video signal is handled as a wave of light. Then, the stereoscopic image conversion apparatus passes the first virtual aperture group through the first virtual aperture group and transmits the light wave of the element image distributed by the distribution unit by the first element image conversion unit. It converts into the light wave which propagates to the 2nd virtual opening group from which the pitch of opening differs. Here, the aperture refers to an optical element such as a lens or a spatial filter such as a pinhole. Then, the stereoscopic image conversion apparatus adds the light waves of the respective element images converted by the first element image conversion means by the adding means at the incident surface of the second virtual aperture group. To do. Therefore, the element image group is in an intermediate generation stage as a result of the change in the element images of the first element image group on the incident surface of the second virtual aperture group. Then, the stereoscopic image conversion apparatus redistributes the light waves added by the adding means by the redistributing means for each element image constituting the second element image group. Then, the stereoscopic image conversion apparatus propagates the light wave of the element image distributed by the redistribution means by the second element image conversion means by a distance until it forms an image through the second virtual aperture group. Converted into light waves. Therefore, as a result of each element image being changed by passing through the second virtual aperture group, the state of the re-photographed element image is obtained. Then, the stereoscopic image conversion apparatus generates the second element image group by combining the redistributed number of light waves converted by the second element image conversion unit by the combining unit. Therefore, the three-dimensional image conversion apparatus executes arithmetic processing as if the light of the elemental image group photographed by the photographing apparatus is optically displayed and re-photographed. At this time, since the light wave passes through two virtual aperture groups with different aperture pitches, the length of the region of the first element image group is different from the length of the region of the second element image group. . Therefore, when this second element image group is displayed as a stereoscopic image, it is possible to display a reproduced image having a different image size from the case where the first element image group is displayed as a stereoscopic image. As a result, the stereoscopic image conversion device changes the ratio of the pitches of the two virtual aperture groups without changing the optical system of the stereoscopic image display device that displays the second element image group as a stereoscopic image. It is possible to generate the second element image group for displaying a reproduction image having a free size changed from the size of the subject.

The stereoscopic image conversion apparatus according to claim 2 converts the light wave of the first element image group, which is a reproduced image in which the unevenness of the subject is inverted when displayed as a stereoscopic image, and displays the stereoscopic image as a stereoscopic image. A stereoscopic image conversion apparatus for generating a second element image group that is a reproduced image having the same size as the projections and depressions of the subject and having a changed image size, the inversion information generation means, the distribution means, and the first element image conversion Means, adding means, redistributing means, second element image converting means, and combining means.

According to such a configuration, in the stereoscopic image conversion apparatus, individual element images constituting an element image group that is the same reproduced image as the unevenness of the subject when displayed as a stereoscopic image are input by the inversion information generation unit , The first element image group is generated by inverting the input individual element images in a point-symmetric manner. Then, the stereoscopic image conversion device distributes the light wave of the first element image group generated by the inversion information generation unit for each element image by the distribution unit. Then, the stereoscopic image conversion apparatus passes the first virtual aperture group through the first virtual aperture group and transmits the light wave of the element image distributed by the distribution unit by the first element image conversion unit. It converts into the light wave which propagates to the 2nd virtual opening group from which the pitch of opening differs. Then, the stereoscopic image conversion apparatus adds the light waves of the respective element images converted by the first element image conversion means by the adding means at the incident surface of the second virtual aperture group. To do. Then, the stereoscopic image conversion apparatus redistributes the light waves added by the adding means by the redistributing means for each element image constituting the second element image group. Then, the stereoscopic image conversion apparatus propagates the light wave of the element image distributed by the redistribution means by the second element image conversion means by a distance until it forms an image through the second virtual aperture group. Converted into light waves. Then, the stereoscopic image conversion apparatus generates the second element image group by combining the redistributed number of light waves converted by the second element image conversion unit by the combining unit.

  The stereoscopic image conversion program according to claim 3 converts the light wave of the first element image group, which is a reproduced image in which the unevenness of the subject is inverted when displayed as a stereoscopic image, and displays the stereoscopic image as a stereoscopic image. In order to generate a second element image group that is a reconstructed image having the same image size as the concavo-convex portion of the subject, a computer is provided with a distribution unit, a first element image conversion unit, an addition unit, The second element image converting unit and the combining unit are allowed to function.

  According to such a configuration, the stereoscopic image conversion program distributes the light wave of the first element image group for each element image by the distribution unit, and the light wave of the element image distributed by the distribution unit by the first element image conversion unit. Is converted into a light wave that passes through the first virtual aperture group and propagates to the second virtual aperture group having a different aperture pitch from that of the first virtual aperture group. Then, the stereoscopic image conversion program adds the light waves of the respective element images converted by the first element image conversion unit by the adding unit by the distributed number on the incident surface of the second virtual aperture group. To do. The stereoscopic image conversion program redistributes the light wave added by the adding means by the redistributing means for each element image constituting the second element image group, and redistributing means by the second element image converting means. The light wave of the element image distributed in (1) is converted into a light wave propagated by a distance until it passes through the second virtual aperture group and forms an image. Then, the stereoscopic image conversion program generates the second element image group by combining the redistributed number of light waves converted by the second element image converting means by the combining means.

The present invention has the following effects.
According to the first aspect of the present invention, the stereoscopic image conversion apparatus uses the first element image group that forms a reproduced image in which the unevenness of the subject is reversed when displayed, and the light wave of the element image has an opening pitch. A second element image group that reproduces an image in which the size of the image is changed to a desired size by performing arithmetic processing assuming that the two aperture groups differing in size are virtually passed Can be converted to Therefore, it is possible to control the ratio of the size of the reproduced image of the subject without depending on the optical device used for photographing and displaying the subject. Therefore, there is an advantage that it is not necessary to change a lens, a pinhole, or the like when changing the size of the stereoscopic image.

  According to the first aspect of the present invention, the stereoscopic image conversion apparatus can convert the first element image group into a second element image group corresponding to a stereoscopic image whose size with respect to the subject is arbitrarily controlled. Therefore, when displaying a reproduced image whose size with respect to the subject is arbitrarily controlled, the effect of rendering a stereoscopic image can be enhanced.

  According to the second aspect of the present invention, the stereoscopic image conversion apparatus is configured such that, even if the element image group input for conversion reproduces an image in which the unevenness of the subject is reversed, the unevenness of the subject Any one of those that reproduces the same image can be converted into a second element image group that reproduces the same image as the unevenness of the subject.

  According to the invention described in claim 3, the computer in which the stereoscopic image conversion program is installed can achieve the same effects as the stereoscopic image conversion device by realizing each function based on this program. .

The best mode for carrying out a stereoscopic image conversion apparatus of the present invention (hereinafter referred to as “embodiment”) will be described in detail below with reference to the drawings.
The

[Outline of configuration of stereoscopic image conversion apparatus]
First, an outline of the configuration of the stereoscopic image conversion apparatus according to the present embodiment will be described. FIG. 1 is a configuration diagram schematically illustrating an example of a stereoscopic image conversion apparatus according to an embodiment of the present invention.

  The stereoscopic image conversion apparatus 10 converts the light wave of the first element image group that is a reproduced image in which the unevenness of the subject is reversed when displayed as a stereoscopic image, and is the same as the unevenness of the subject when displayed as a stereoscopic image. A second element image group that is a reproduced image in which the size of the image is changed is generated. The first element image group input to the stereoscopic image conversion apparatus 10 is a video signal captured by an imaging element such as a CCD in the stereoscopic image capturing apparatus 20. The second element image group generated by the stereoscopic image conversion apparatus 10 is displayed on an information display device such as a liquid crystal panel in the stereoscopic image display apparatus 30. As the stereoscopic image capturing device 20 and the stereoscopic image display device 30, for example, the configurations illustrated in FIGS. 6A and 6B can be employed.

  The stereoscopic image conversion apparatus 10 includes, for example, a calculation device such as a CPU (Central Processing Unit), a storage device (storage means) such as a memory and a hard disk, and an input device that detects input of information from the outside such as a mouse and a keyboard. An interface device that transmits and receives various types of information via a communication line such as a LAN (Local Area Network), a computer having a display device such as an LCD (Liquid Crystal Display), and a program installed in the computer Is done. The stereoscopic image conversion apparatus 10 is configured such that the hardware device and software cooperate with each other, and the hardware resources described above are controlled by a program, whereby the distribution unit 11 and the first element image conversion unit shown in FIG. 12, adding means 13, redistributing means 14, second element image converting means 15, and combining means 16 are realized.

  The distribution unit 11 distributes the light wave of the first element image group for each element image. Here, the distributed element image is identified by m (−M ≦ m ≦ M). The first element image conversion unit 12 functions for each of the distributed (2M + 1) element images. The distributed element image m is then added by the adding means 13 and distributed again by the redistributing means 14. At this time, the redistributed element images are identified by n (−N ≦ n ≦ N). The second element image conversion means 15 functions for each (2N + 1) element images that have been redistributed. Then, the redistributed element image n is then combined by the combining means 16 to generate a second element image group.

[Outline of virtual aperture group assumed in calculation processing of stereoscopic image conversion apparatus]
Next, an outline of a virtual aperture group assumed in the arithmetic processing of the stereoscopic image conversion apparatus of the present embodiment will be described with reference to FIG. 2 (refer to FIG. 1 as appropriate). FIG. 2 is an explanatory diagram schematically illustrating an example of a virtual aperture group assumed in the arithmetic processing of the stereoscopic image conversion apparatus. The first element image conversion means 12 shown in FIG. 1 passes the first virtual aperture group 41 (see FIG. 2) through the first virtual aperture group 41 (see FIG. 2) and transmits the light waves of the element image distributed by the distribution means 11. It converts into the light wave which propagates to the opening group 42 (refer FIG. 2). Here, the light wave is a complex number representing the amplitude and phase when a video signal is handled as a wave of light. In addition, the elements (openings) constituting the aperture group mean optical elements such as lenses and spatial filters such as pinholes. Hereinafter, the aperture is represented by a convex lens (element lens) as shown in FIG. Further, as shown in FIG. 2, the first virtual opening group 41 and the second virtual opening group 42 are separated by a predetermined distance L. If the first virtual aperture group 41 is a virtual lens array as shown in FIG. 2, the predetermined distance L can be the focal length of the element lens. Note that the first and second virtual aperture groups 41 and 42 do not actually exist, and are assumed for the stereoscopic image conversion apparatus 10 to perform arithmetic processing.

In FIG. 2, the light waves of the first element image group are emitted from the first element image surface 43, and the planar first and second virtual opening groups 41 and 42 whose thickness direction is shown in FIG. , And enters the second element image plane 44 from the left to the right. K ₁ represents the area of the m-th element image in the first element image group, and d ₁ represents the distance from the first element image group (first element image plane 43) to the first virtual aperture group 41. Indicates distance. Similarly, k ₂ represents the areas of the n th element image in the second elemental image group, to d ₂ and the second element image group from the second virtual aperture group 41 (second element image plane 44) Indicates the distance.

In FIG. 2, p ₁ indicates the pitch of the element lenses that form the first virtual aperture group 41, and p ₂ indicates the pitch of the element lenses that form the second virtual aperture group 42. In the present embodiment, the pitch ratio Φ represented by the following formula (1) is set as Φ ≠ 1. That is, the pitches of the element lenses (openings) of the first and second virtual aperture groups 41 and 42 are different. In FIG. 2, the state of Φ <1 is shown, but Φ> 1 may be set.

[Detailed Configuration of Stereoscopic Image Conversion Device]
<Distributing means>
The distribution means 11 divides the input video signal (light wave of the first element image group) into element images. The input video signal (the light wave emitted from the first element image plane 43) corresponds to a bundle of light waves of each element image constituting the first element image group, and this is expressed as a first element image group t1. I will do it. The distribution unit 11 converts the light wave (gs _{, m} ( _{xs, m} , ys _{, m} )) of the mth element image of the first element image group t1 in the input video signal into the mth element. It outputs to the 1st element image conversion means 12 matched previously with the image. Here, x _{s, m} represents the x coordinate when the center of the m-th element image in the entire input image (first element image group) is the origin. Similarly, y _{s, m} represents the y coordinate when the center of the m-th element image in the entire input image (first element image group) is the origin.

<First Element Image Conversion Means>
As shown in FIG. 1, the first element image conversion means 12 includes a light wave calculation means 12a, a phase shift means 22b, and a light wave calculation means 12c.

<< Light wave calculation means 12a >>
The light wave calculation means 12a calculates a light wave incident on the element lens constituting the first virtual aperture group 41 (see FIG. 2) from the light wave of the element image based on Fresnel approximation. is there. That is, the light wave calculation means 12a uses a general Fresnel approximation as a signal corresponding to the light wave reaching the m-th element lens of the first virtual aperture group 41 and uses the following equation (2) to calculate the element. A light wave (R _{i, m} (x _{o, m} , y _{o, m} )) for each image is calculated.

Here, x _{o, m} is the x coordinate when the optical axis center of the m-th element lens of the first virtual aperture group 41 is the origin. Similarly, y _{o, m} is the y coordinate when the optical axis center of the mth aperture of the element lens group of the first virtual aperture group 41 is the origin. F ₁ represents the focal length of the element lens of the first virtual aperture group 41. K is the wave number 2π / λ (λ is the wavelength of the light wave). The light wave (R _{i, m} (x _{o, m} , y _{o, m} )) for each element image is output to the phase shift means 22b.

<< Phase shift means 22b >>
The phase shift unit 22b corresponds to the element lens of the first virtual aperture group 41 with the phase from the light wave (R _{i, m} (x _{o, m} , _{yo, m} )) of the input element image. The light wave shifted by the phase is calculated. That is, the phase shift unit 22b converts the light wave (R _{i, m} (x _{o, m} , y _{o, m} )) into an element of the first virtual aperture group 41 as shown in the following equation (3). By shifting the phase corresponding to the lens, a signal (R _{o, m} (x _{o, m} , y _{o, m} )) corresponding to the light wave emitted from the element lens of the first virtual aperture group 41 is obtained. Calculate. This light wave (R _{o, m} (x _{o, m} , y _{o, m} )) is output to the light wave calculation means 12c.

<< Light wave calculating means 12c >>
Optical wave calculating means 12c are light waves emitted from the first virtual element lens aperture group 41, i.e., the input element image from the phase shift means 22b lightwave _{(R o, m (x o} , m, y o _{, m 2} )) is subjected to Fresnel approximation to calculate the light wave incident on the element lens of the second virtual aperture group 42 (see FIG. 2). That is, the light wave calculation unit 12c uses a general Fresnel approximation as a signal corresponding to the light wave reaching the element lens of the second virtual aperture group 42, and uses the following formula (4) to calculate each element image. The light wave (R _{d, m} (x _{d, m} , y _{d, m} )) is calculated. The light wave (R _{d, m} (x _{d, m} , y _{d, m} )) for each element image is output to the adding means 13.

In Expression (4), x _{d, m} is the incident surface of the second virtual aperture group 42 when the optical axis center of the m-th element lens of the first virtual aperture group 41 is the origin. Is the x coordinate. Similarly, y _{d, m} is the y coordinate on the incident surface of the second virtual aperture group 42 when the optical axis center of the m-th element lens of the first virtual aperture group 41 is the origin. Indicates. L is the distance between the first virtual opening group 41 and the second virtual opening group 42. Further, the range in which the integral calculation is performed is set to be equivalent to the range in which the m-th element image (element image (m)) expands. An example of a range w in which the m-th element image in the first element image group extends is shown in FIG. In this case, a range w in which the m-th element image in the first element image group extends is expressed by Expression (5). Incidentally, k ₁ represents a region of the m-th element image in the first elemental image group, d ₁ is from the first element image group (first element image plane 43) to the first virtual opening group 41 Indicates distance.

<Adding means>
The adding means 13 adds the light waves of the respective element images converted by the first element image converting means 12 by the number distributed by the distributing means 11 at the entrance surface of the second virtual aperture group 42. It is. The adding means 13 adds light waves corresponding to the field angles of the element lenses of the first virtual aperture group 41. This adding means 13 emits a light wave (R _{d, m} (x _d, x ₎ emitted from the element lens of the first virtual aperture group 42 and reaching the entrance surface of the element lens of the second virtual aperture group 42 _{. m} , y _{d, m} )) are added within the region of the second virtual aperture group 42. That is, as a signal corresponding to the light wave on the incident surface of the second virtual aperture group 42, the light wave (R _p ( x _p , y _p )) is calculated.

Here, x _p is the x coordinate when the center of the second virtual aperture group 42 is the origin, and similarly, y _p is the center of the second virtual aperture group 42 as the origin. This is the y coordinate. This light wave (R _p (x _p , y _p )) is output to the redistribution means 14.

<Redistribution means>
The redistribution unit 14 redistributes the light wave added by the addition unit 13 for each element image constituting the second element image group. The redistribution unit 14 divides the light wave (R _p (x _p , y _p )) input from the addition unit 13 into element image units. As a light wave for each element image obtained by redistributing the light wave (R _p (x _p , y _p )), a light wave corresponding to the n-th element lens constituting the second virtual aperture group 42 is represented by (R _{p, n} (x _{p, n} , y _{p, n} )). Here, the redistribution unit 14 converts the light wave (R _{p, n} (x _{p, n} , y _{p, n} )) input to each element lens constituting the second virtual aperture group 42 into the second Are output to the second element image conversion means 15 previously associated with the nth element lens that lies across the virtual aperture group 42.

<Second element image conversion means>
The second element image conversion means 15 converts the light wave of the element image distributed by the redistribution means 14 into a light wave propagated by a distance until it passes through the second virtual aperture group 42 and forms an image. It is. The second element image conversion means 15 redistributes the light wave input from the redistribution means 14 to the aperture region of the element lens (focal length f ₂ ) of the second virtual aperture group 42, and then the element. It converts into a light wave that passes through the lens (focal length f ₂ ), and further propagates the light wave by the focal length f _{2 of the} element lens. The second element image conversion means 15 functions for every (2N + 1) element images, for example, to propagate a bundle of light waves of each element image. The second element image group is composed of a bundle of light waves of the element images that propagate. This is expressed as a second element image group t2. That is, the second element image conversion means 15 calculates the second element image group t2. As shown in FIG. 1, the second element image conversion means 15 includes a phase shift means 15a and a light wave calculation means 15b.

<< Phase shift means 15a >>
The phase shift unit 15 a converts the phase from the light wave (R _{p, n} (x _{p, n} , y _{p, n} )) input from the redistribution unit 14 to the element lens of the second virtual aperture group 42. The light wave shifted by the corresponding phase is calculated. That is, the phase shift means 15a converts the light wave (R _{p, n} (x _{p, n} , y _{p, n} )) into an element of the second virtual aperture group 42 as shown in the following equation (7). By shifting the phase corresponding to the lens, a signal (R _{r, n} (x _{p, n} , y _{p, n} )) corresponding to the light wave emitted from the element lens of the second virtual aperture group 42 is obtained. Calculate.

Here, x _{p, n} is the x coordinate when the optical axis center of the nth element lens of the second virtual aperture group 42 is the origin. Similarly, y _{p, n} is the y coordinate when the center of the optical axis of the nth element lens of the second virtual aperture group 42 is the origin. F ₂ is the focal length of the element lens of the second virtual aperture group 42. The light wave (R _{r, n} (x _{p, n} , y _{p, n} )) for each element image is output to the light wave calculation means 15b.

<< Light wave calculating means 15b >>
The light wave calculation means 15b emits the light wave of each element lens of the second virtual aperture group 42 from the element lens of the second virtual aperture group 42 by performing Fresnel approximation, and the second element image plane The light wave reaching 44 (see FIG. 2) is calculated. That is, the light wave calculation means 15b calculates the light wave (R _{e, E)} emitted from the nth element lens of the second virtual aperture group 42 and reaching the second element image plane 44 by the following equation (8) _{. n} (x _{e, n} , y _{e, n} )) is calculated.

Here, x _{e, n} is the x coordinate from the center of the n-th element image in the entire output image (second element image group). Similarly, y _{e, n} is the y coordinate from the center of the nth element image in the entire output image. The light wave (R _{e, n} (x _{e, n} , y _{e, n} )) calculated by the light wave calculating means 15 b is output to the coupling means 16.

<Coupling means>
The combining unit 16 generates the second element image group by combining the light waves converted by the second element image converting unit 15 by the number redistributed by the redistributing unit 14. This combining means 16 calculates the sum of the powers of the light waves from the light wave for each element image output from the second element image conversion means 15, that is, a video signal in which the size of the stereoscopic image is converted, that is, Then, a video signal to be the second element image group t2 is generated. The second element image group t2 obtained by the combining unit 16 is output to the stereoscopic image display device 30 (see FIG. 1).

Here, the power of the light wave (R _{e, n} (x _{e, n} , y _{e, n} )) reaching the second element image plane 44 output from the second element image conversion means 15 is the light amplitude. It can be expressed as a square. Moreover, since the light wave emitted as each element image of the first element image group t1 is incoherent (the wavelength and phase are not constant), the phase of the light wave can be regarded as uncorrelated. Therefore, as shown in the following formula (9), the combining unit 16 transmits the light wave of each element image that reaches the second element image plane 44 (for example, if it is the n-th element image, the light wave is ( _{Re, n} (x _{e, n} , y _{e, n} )) is calculated, and the sum thereof is calculated to obtain a video signal of the entire second element image group t2.

[Reproduction image size ratio control]
The stereoscopic image conversion apparatus 10 has the configuration described in detail as described above, and the ratio of the size of the reproduced image is set in advance by setting the pitch ratio Φ represented by the above-described formula (1). Can be controlled to a desired value. If the processing of the stereoscopic image conversion apparatus 10 of the present embodiment is not performed, that is, if the first and second virtual aperture groups 41 and 42 are not used, the subject size x _c (FIG. 6). The relationship between the (a) reference) and the size of the stereoscopic image to be reproduced x _r (see FIG. 6B) is as in the above-described equation (101). When the stereoscopic image conversion apparatus 10 is not used in this way, if α = γ in the above-described equation (101), the size x _{c of the} subject (see FIG. 6A) and the size of the reproduced stereoscopic image The relationship with the _length x _r (see FIG. 6B) is expressed by the following equation (10).

That is, in FIG. 1, the stereoscopic image conversion apparatus 10 is ignored, and the element lens pitch _pc (see FIG. 6A) in the stereoscopic image photographing apparatus 20 (see FIG. 1) and the stereoscopic image display apparatus 30 (see FIG. 1). 1 pitch p _r of the element lenses in the reference) when are equal (FIG. 6 (b) reference), the subject of the magnitude x _c reference (FIG. 6 (a)), the three-dimensional image to be reproduced image size It becomes equal to x _r (see FIG. 6B). On the other hand, when the stereoscopic image conversion apparatus 10 of the present embodiment is used, the size x _{c of the} subject (see FIG. 6A) and the size x _r of the reproduced stereoscopic image (see FIG. 6B). The relationship is expressed by the following formula (11).

That is, in FIG. 1, the pitch p _c of the element lens in the stereoscopic image capturing device 20 (see FIG. 1) (see FIG. 6A) and the pitch p _r of the element lens in the stereoscopic image display device 30 (see FIG. 1). Even if (see FIG. 6B) is equal, the element lens pitch p ₁ (see FIG. 2) of the first virtual aperture group 41 and the elements of the second virtual aperture group 42 By controlling the lens pitch p ₂ (see FIG. 2), the size x _r of the reproduced stereoscopic image with respect to the subject size x _c (see FIG. 6 (a)) (see FIG. 6 (b)). Can be changed.

For convenience of explanation, the case where α and γ are equal in the above-described equation (101) has been described in the above-described equations (10) and (11). However, in the present invention, the above-described equation (101) is described. ) May be α ≠ γ. Similarly, the pitch p _c of the element lenses in the three-dimensional image photographing apparatus 20, but the pitch p _r of the element lenses in the stereoscopic image display apparatus 30 has been described equal, it may be p _c ≠ p _r.

  According to the present embodiment, the stereoscopic image conversion apparatus 10 calculates the first element image group t1 as a light wave of each element image m passing through the first and second virtual opening groups 41 and 42. By processing, the second element that reproduces an image in which the size of the image is changed to a desired size is the same as the unevenness of the subject without depending on the optical device used for photographing and displaying the subject. It can be converted into an image group t2. Therefore, since the size of the stereoscopic image can be changed by the arithmetic processing in this way, it is not necessary to change the optical device used for photographing and displaying such as a lens as in the conventional case. In addition, according to the present embodiment, the stereoscopic image conversion apparatus 10 generates an image equivalent to a case where light emitted from an image as an element image group obtained by capturing an object as a plurality of element images is captured again by arithmetic processing. can do. Furthermore, when displaying a reproduced image whose size with respect to the subject is arbitrarily controlled, it is possible to enhance the effect of rendering a stereoscopic image.

  Although the present embodiment has been described above, the present invention is not limited to this, and can be implemented in various ways without changing the gist thereof. For example, in the present embodiment, the stereoscopic image conversion apparatus 10 uses the first element image as it is as an input video signal that is a reproduced image in which the unevenness of the subject is reversed when the stereoscopic image capturing apparatus 20 captures and displays the stereoscopic image. Although described as handling as a group, in the present invention, an element image group in which the unevenness of the subject becomes the same reproduced image when displayed as a stereoscopic image may be input.

  A configuration example of the stereoscopic image conversion apparatus in this case is shown in FIG. The stereoscopic image conversion apparatus 10A shown in FIG. 4 is further provided with the inversion information generation means 19 in the configuration shown in FIG. Here, the inversion information generating means 19 inverts the individual element images constituting the element image group that is the same reconstructed image as the unevenness of the subject when displayed as a stereoscopic image, by inverting each of the element images in a point-symmetric manner. An element image group is generated. In the example shown in FIG. 4, an input video signal that is a reproduced image in which the unevenness of the subject is reversed when captured by the stereoscopic image capturing device 20 and displayed as a stereoscopic image is input to the inversion information generation unit 19 to generate inversion information. Each element image is inverted to point symmetry by means 19 and output to distribution means 11. The distribution unit 11 distributes the light wave of the first element image group generated by the inversion information generation unit 19 for each element image. Thereby, there can exist an effect equivalent to the stereo image conversion apparatus 10 shown in FIG. Note that the captured stereoscopic information may be stored in a storage unit (not shown) of the stereoscopic image conversion apparatus 10A, and the stored stereoscopic information may be input to the inversion information generation unit 19.

  Further, in the present embodiment, the formula using the Fresnel approximation is shown in the calculations of the formulas (2), (4), and (8) described above. The calculation may be performed using integration or integration by Fraunhofer diffraction.

In the present embodiment, the focal lengths f ₁ of the element lenses of the first virtual aperture group 41 in the calculations of the expressions (2), (3), (7), and (8) described above, the focal length f ₂ of the second virtual element lens opening group 42 were used, respectively, but is not limited thereto. In other words, the focal length may be different for each element lens constituting the first virtual aperture group 41 or may be different for each element lens constituting the second virtual aperture group 42. .

In this embodiment, the relationship between the number and density of the element lenses constituting the first virtual aperture group 41 and the number and density of the element lenses constituting the second virtual aperture group 42 is described. Although not, these relationships are arbitrary. For example, as shown in FIG. 5, the element lenses constituting the first virtual aperture group 51 are roughly arranged in a small number, and the element lenses constituting the second virtual aperture group 42 are densely arranged in large numbers. May be arranged. In this case, the pitch of the element lenses constituting the first virtual aperture group 51 is p ₃ , and what is the pitch p _{1 of the} element lenses constituting the first virtual aperture group 41 shown in FIG. Is different.

  In addition, the stereoscopic image conversion apparatus 10 (10A) is realized by controlling the hardware resources described above by the cooperation of the hardware apparatus and software as described above. This program (stereoscopic image conversion program) can be provided via a communication line, or can be written on a recording medium such as a CD-ROM and distributed. In addition, the stereoscopic image conversion apparatus 10 (10A) can also realize each of the above-described means by an arithmetic circuit.

In the present embodiment, the apertures constituting the first and second virtual aperture groups 41 and 42 have been described as element lenses, but may be replaced with a spatial filter such as a pinhole. In this case, the distances corresponding to the focal lengths f ₁ and f ₂ of the element lenses can be arbitrary distances.

  Further, the stereoscopic image conversion apparatus 10 (10A) can be configured by being incorporated in the stereoscopic image photographing apparatus 20 or the stereoscopic image display apparatus 30.

It is a block diagram which shows typically an example of the stereo image conversion apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows typically an example of the virtual opening group assumed by the arithmetic processing of the stereo image conversion apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows typically an example of the integration range used by the arithmetic processing of the stereo image conversion apparatus which concerns on embodiment of this invention. It is a block diagram which shows typically the other example of the stereo image conversion apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows typically the other example of the virtual aperture group assumed by the arithmetic processing of the three-dimensional image converter which concerns on embodiment of this invention. It is explanatory drawing which shows typically imaging | photography and reproduction | regeneration of the stereo image by the conventional IP system, (a) has shown the stereo image imaging device, (b) has each shown the stereo image display apparatus. It is explanatory drawing which shows the method of avoiding the conventional reverse vision typically, (a) is the stereo image depth conversion apparatus which converts the information acquired by imaging | photography, (b) is the stereo image which displays the converted image. Each of the display devices is shown.

Explanation of symbols

10 (10A) stereoscopic image conversion apparatus 11 distributing means 12 first element image converting means 12a light wave calculating means 12b phase shift means 12c light wave calculating means 13 adding means 14 redistributing means 15 second element image converting means 15a phase shift means 15b light waves Calculation means 16 Coupling means 19 Inversion information generation means 20 Stereoscopic image capturing apparatus 30 Stereoscopic image display apparatus

Claims (3)

The light wave of the first element image group, which is a reproduced image in which the unevenness of the subject is inverted when displayed as a stereoscopic image, is converted, and the image size changes in the same manner as the unevenness of the subject when displayed as a stereoscopic image A stereoscopic image conversion apparatus for generating a second element image group to be a reproduced image,
Distributing means for distributing the light wave of the first element image group for each element image;
The light wave of the element image distributed by this distribution means passes through the first virtual aperture group and propagates to the second virtual aperture group having a pitch different from that of the first virtual aperture group. First element image conversion means for converting into a light wave to
Adding means for adding the light waves of the respective element images converted by the first element image conversion means by the distributed number at the entrance surface of the second virtual aperture group;
Redistribution means for redistributing the light waves added by the addition means for each element image constituting the second element image group;
A second element image conversion means for converting the light wave of the element image distributed by the redistribution means into a light wave propagated by a distance until it passes through the second virtual aperture group and forms an image;
A combining unit that generates the second element image group by combining the redistributed number of light waves converted by the second element image converting unit;
A stereoscopic image conversion device comprising: The light wave of the first element image group, which is a reproduced image in which the unevenness of the subject is inverted when displayed as a stereoscopic image, is converted, and the image size changes in the same manner as the unevenness of the subject when displayed as a stereoscopic image A stereoscopic image conversion apparatus for generating a second element image group to be a reproduced image,
Individual element images constituting an element image group that is the same reproduced image as the unevenness of the subject when displayed as a three-dimensional image are input, and the input individual element images are inverted in a point-symmetric manner. Inversion information generation means for generating a one-element image group ;
Distributing means for distributing the light wave of the first element image group generated by the inversion information generating means for each element image ;
The light wave of the element image distributed by this distribution means passes through the first virtual aperture group and propagates to the second virtual aperture group having a pitch different from that of the first virtual aperture group. First element image conversion means for converting into a light wave to
Adding means for adding the light waves of the respective element images converted by the first element image conversion means by the distributed number at the entrance surface of the second virtual aperture group;
Redistribution means for redistributing the light waves added by the addition means for each element image constituting the second element image group;
A second element image conversion means for converting the light wave of the element image distributed by the redistribution means into a light wave propagated by a distance until it passes through the second virtual aperture group and forms an image;
A combining unit that generates the second element image group by combining the redistributed number of light waves converted by the second element image converting unit;
Steric image converter, characterized in that it comprises. The light wave of the first element image group, which is a reproduced image in which the unevenness of the subject is inverted when displayed as a stereoscopic image, is converted, and the image size changes in the same manner as the unevenness of the subject when displayed as a stereoscopic image In order to generate a second element image group to be a reproduced image,
Distributing means for distributing the light wave of the first element image group for each element image;
The light wave of the element image distributed by the distributing means passes through the first virtual aperture group to the second virtual aperture group having a pitch of the aperture different from that of the first virtual aperture group. First element image conversion means for converting into a propagating light wave;
Adding means for adding the light waves of the respective element images converted by the first element image converting means by the distributed number at the entrance surface of the second virtual aperture group;
Redistribution means for redistributing the light waves added by the addition means for each element image constituting the second element image group;
Second element image conversion means for converting the light wave of the element image distributed by the redistribution means into a light wave propagated by a distance until it passes through the second virtual aperture group and forms an image;
Combining means for generating the second element image group by combining the redistributed number of light waves converted by the second element image converting means;
3D image conversion program characterized in that it functions as

Images