JP3774983B2 - Imaging device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a high-resolution image input device in which a mechanism capable of angular displacement of incident light is provided between an imaging device using a two-dimensional solid-state imaging device and a photographing optical system.
[0002]
[Prior art]
In an image input device that uses a two-dimensional image sensor such as a CCD, the image is input after each change in the relative position between the image and the image sensor and then the images are combined to produce an apparent number of pixels. It is known to increase the resolution by increasing. In order to improve the resolution by such a method, a flat transparent member is disposed in parallel between the photographing optical system and the two-dimensional image sensor as described in Japanese Patent Application Laid-Open No. 7-236086. Using one of the three tilting means not on the straight line arranged above as a support part and using the other two points as operating parts for operating the transparent member, imaging is performed by displacing incident light from the imaging optical system. The image on the element is slightly displaced.
[0003]
FIG. 22 is a block diagram of an image input apparatus for optically increasing the number of pixels disclosed in, for example, Japanese Patent Application Laid-Open No. 7-236086, and FIG. 23 is a diagram illustrating a mechanism of the imaging unit. 22 and 23, 1 is an image pickup lens for forming an image, 3 is an image pickup device such as a CCD arranged in two dimensions for photoelectrically converting the image, and 103 is an image pickup lens 1 and image pickup device 3 A transparent flat plate member that is arranged substantially in parallel and brings a slight displacement in the incident angle of incident light from the imaging lens 1 to the image pickup device 3, and 104 is a base unit that supports the lens 1 and the transparent flat plate member 103, 105a, 105b, Reference numeral 105c denotes a compression spring 106a, 106b, 106c for fixing the transparent flat plate member 103 to the base unit 104 and selectively actuating two points to incline the transparent flat plate unit 106a, 106b, 106c. Each of the spring pressing plates 107a and 107b (not shown) to be pressed is provided together with a screw penetrating the transparent flat plate member 103 and the compression springs 105a and 105b, and is driven to displace the vicinity of the transparent flat plate member 103 in the optical axis direction. Tilt flat plate member 103 Dripping motor, 108a, 108b are first and second actuating unit includes a transparent flat plate member 103 and the compression spring 105a, 105b, spring pressing plate 106a, the 106b. Reference numeral 109 denotes a support portion including the transparent flat plate member 103, the compression spring 105c, and the spring pressing plate 106c, and supports the transparent flat plate member 103 when the operation portions 106a and 106b are operated. These are integrally fixed to a housing (not shown), and, as shown in FIG. 22, a predetermined image processing circuit, an image synthesis memory 110, and an image buffer memory for processing a photoelectrically converted image signal as shown in FIG. 7 etc. are connected.
[0004]
Next, the operation will be described. First, imaging is performed in a state where neither of the two operation units 106a and 106b is operated, and an image is stored in the image buffer memory 7 at the subsequent stage. Next, when one of the two actuating parts 106a is actuated, the transparent flat plate member 103 is rotated with a line connecting the other actuating part 106b and the support part as a rotation axis. Therefore, the image transmitted through the transparent flat plate member 103 is moved by the inclination of the transparent flat plate member 103 to form an image on the image pickup device 2, and an image slightly shifted in the image buffer memory 7 is stored. Further, if the same single operation unit 106a is operated, images are sequentially moved in the same direction, and images are sequentially formed on the image sensor 3 and stored. Further, when the actuating portion 106b is driven, the transparent flat plate member 103 is inclined with the line connecting the actuating portion 106a and the support portion 109 as the rotation axis, and the image is moved in a direction different from that described above.
[0005]
By appropriately combining these movements in these two directions, each of the two-dimensional pixel shifts to an arbitrary position is performed, and then each of the plurality of captured images stored in the image buffer memory 7 is considered in consideration of the pixel shift execution direction. An image in which the number of pixels is optically increased by interpolation for each pixel is obtained in the image synthesis memory 110.
[Problems to be solved by the invention]
[0006]
However, the invention disclosed in Japanese Patent Application Laid-Open No. 7-236086 realizes a highly accurate pixel shift amount because the transparent flat plate member is mechanically driven and controlled by two motors in order to change the optical path two-dimensionally. It was difficult. In particular, the pixel pitch of recent solid-state imaging devices is mainly several microns, and a complicated control system is required to obtain a mechanical accuracy that is a fraction of that.
[0007]
In order to solve the problem of requiring such a complicated control system, the present applicant has filed a method for shifting a small pixel without mechanically vibrating the imaging optical system in Japanese Patent Application Laid-Open No. 62-26984. ing.
[0008]
However, since a plurality of images obtained by optical pixel shifting are stored in the image buffer memory as they are, a large-capacity storage device is necessary. There was a problem of increasing the size.
[0009]
In addition, since a plurality of images obtained by optical pixel shifting are simply interpolated and synthesized according to the shifting position in the image processing circuit, for example, the impulse image shown in FIG. When the four images shown in FIGS. 24 (c), 24 (d), 24 (e), and 24 (a) obtained by shifting in two steps are combined, the integration of light energy in each image sensor Since the synthesized image has the result of degradation of the MTF shown in FIG. 24B, there is a problem in that the sharpness of the edge component of the image is lost without sufficiently improving the resolution with respect to the increase in the number of pixels. .
[0010]
It is an object of the present invention to obtain an imaging device that does not require a mechanical pixel shifting mechanism and can realize a high-resolution image using a small-capacity storage device.
[0011]
[Means for Solving the Problems]
An image pickup apparatus according to a first aspect of the present invention guides reflected light from an object to be picked up to an image pickup element composed of a plurality of pixels arranged two-dimensionally by an image pickup optical system, and performs photoelectric conversion by the image pickup element. In an imaging apparatus that generates an image signal, a pixel shifting unit that shifts the incident position of incident light from the imaging optical system to the imaging element by a minute pixel, and an imaging control unit that controls a pixel displacement amount of the pixel shifting unit And an image compression means for compressing at least N−1 images out of N images (N ≧ 2, N: natural number) taken continuously within a predetermined time into a fixed-length code, and by this image compression means Image storage means for storing compressed images, and image composition means for forming an image in which the number of pixels is increased M times from M (N ≧ M ≧ 2, M: natural number) of the N images , Images formed by this image composition means Those having an image correction means for performing a predetermined sharpness enhancing process.
[0012]
An image pickup apparatus according to a second aspect of the invention is characterized in that the pixel shifting means generates a linearly polarized light, a first optical element having a magneto-optical effect disposed between the polarizer and the image pickup element, A second optical element disposed between the first optical element and the imaging element and configured to divide the light into independent polarization components, and the imaging control means supplies the first optical element to the first optical element. The magnetic field application condition is changed, and the image correction unit is configured to perform predetermined filtering on the composite image.
[0013]
According to a third aspect of the present invention, there is provided an imaging apparatus in which the pixel shifting means generates a linearly polarized light, and a first optical element having a magneto-optical effect disposed between the polarizer and the imaging optical system; A second optical element that divides the light disposed between the first optical element and the imaging optical system into independent polarization components, and the imaging control means supplies the first optical element to the first optical element. The magnetic field application conditions can be changed, and the image correction means is configured to perform predetermined filtering on the composite image.
[0014]
In the imaging apparatus according to a fourth aspect of the invention, the pixel shifting means includes an electro-optic element and a polarizer that change a refraction phenomenon of transmitted light according to the strength of the electric field, and the imaging control means applies to the electro-optic element. The electric field application condition can be changed, and the image correction unit is configured to perform predetermined filtering on the composite image.
[0015]
In the imaging apparatus according to a fifth aspect of the invention, the image correcting unit includes a region extracting unit that extracts a small region composed of K × L pixels (K: natural number, L: natural number) centered on the pixel of interest from the composite image; It comprises edge vector detection means for detecting whether or not the central pixel in the region is an edge component and detecting a line segment direction vector, and pixel correction means for performing predetermined edge pixel correction according to the edge vector information.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Embodiment 1
The first embodiment of the first invention will be described below with reference to the drawings. FIG. 1 is an overall block diagram showing Embodiment 1 of the present invention, and describes a case where it is realized as a digital still camera capable of photographing a still image. In FIG. 1, reference numeral 1 denotes an imaging lens for forming an image, 3 denotes a two-dimensionally arranged imaging element such as a CCD for photoelectrically converting an image, and 2 denotes an area between the imaging lens 1 and the imaging element 3. Pixel shifting means that is arranged in parallel and brings a small displacement in the incident angle of incident light from the imaging lens 1 to the imaging device 3, 4 is an imaging unit control circuit that controls the operation of the pixel shifting means 2 and the imaging device 3, and 5 The imaging unit includes the lens 1, the pixel shifting unit 2, the imaging device 3, and the imaging unit control circuit 4.
[0017]
6 is an A / D conversion circuit that sequentially converts an analog image signal from the image sensor 3 into a digital signal, and 7 is an image compression / decompression that compresses a digital signal from the A / D conversion circuit 6 or expands an encoded signal. Circuit, 8 is an image memory for storing the encoded signal compressed by the image compression / expansion circuit 7, and 9 is an image memory for calling a plurality of encoded signals slightly shifted in the imaging position on the image sensor 3, and compressing the image. An image composition circuit for forming a high pixel density image by interpolating the image signal decompressed by the decompression circuit 7, and an image correction process for sequentially storing several lines of the high pixel density image formed by the image composition circuit 9 Memory 11, an image correction circuit that sequentially applies an MTF correction filter to the contents of the image correction processing memory 10 to generate a high-resolution image, 12 an image compression / decompression circuit 7, an image synthesis circuit 9, an image correction circuit 11, and an imaging Control circuit 4 An image processing control circuit 13 for controlling the operation, 13 is an image processing unit comprising an image compression / expansion circuit 7, an image memory 8, an image composition circuit 9, an image correction processing memory 10, an image correction circuit 11, and an image processing control circuit 12. . Reference numeral 14 denotes an image storage means for storing the high-resolution image generated by the image processing unit 13 for subsequent processing or an interface means for combining the characteristics of signals.
[0018]
FIG. 2 shows a detailed configuration of the image pickup unit 5 in FIG. 1. As a pixel shifting means, for example, a magnetism that exhibits anisotropy by a magnetic field disclosed in Japanese Patent Application Laid-Open No. 62-26984 filed by the present applicant. A case where an optical element is used will be described as an example. 15 is a magnetic field generating circuit, and 16 is its coil. Reference numeral 17 denotes a Faraday element capable of changing the deflection angle by a magnetic field applied by the coil 16 as a first optical element having a magneto-optical effect. Reference numeral 18 denotes a linearly polarized light installed between the Faraday element 17 and the lens 1. A polarizing element 19 is a birefringent plate such as a quartz plate that decomposes light into independent deflection components as a second optical element disposed between the Faraday element 17 and the imaging element 3.
[0019]
Next, the operation will be described. In the image processing unit 13 in FIG. 1, the image compression / decompression circuit 7 uses a fixed-length block coding method, and the image correction circuit 11 uses, for example, a 5 × 5 MTF correction filter. Furthermore, a procedure for obtaining two high-definition images in the horizontal direction having two × 2 times the pixel density will be described.
[0020]
In FIG. 1, as a high-definition mode according to the present invention, a process of forming a single high-resolution image from two photographed images that have been shifted by 1/2 pixel in two horizontal steps will be described. When shooting in the high-definition mode is set by the photographer, a release switch (not shown) is pushed down, and transmitted to the image processing control circuit 12 that controls the entire camera from a man-machine interface (not shown), the imaging unit 5 Then, the imaging unit control circuit 4 operates to perform an imaging operation under the first magnetic field application condition (for example, no applied magnetic field) in the magnetic field generation circuit 15.
[0021]
The image thus captured is transmitted from the imaging unit 5 to the image processing unit 13 via a transmission path. In the image processing unit 13, the input image signal is sequentially converted into a digital signal by the A / D conversion circuit 6. In the image compression / decompression circuit 7, the image data is quantized into blocks of 4 × 4 pixels, compressed to 1/4 capacity, and stored in the image memory 8. Similarly, the photographing operation is performed by shifting pixels by 1/2 pixel in the horizontal direction under the second magnetic field application condition in the magnetic field generation circuit 15.
[0022]
Next, the image processing unit 13 expands the first and second photographed images stored in the image memory 8, and the image composition circuit 9 shifts the image formation on the image sensor 3 due to the second applied magnetic field. Interpolation processing to a position, that is, a pixel shift position is performed, and a high pixel density image in the horizontal direction formed thereby is sequentially stored in the image correction processing memory 10 for several lines. At this time, in order to maintain the aspect ratio of the image, the data of the same line is written into the image correction processing memory 10 every two lines. The image correction circuit 11 sequentially applies an MTF correction filter to the contents of the image correction processing memory 10 and sequentially transfers the high resolution image generated thereby to the subsequent image storage or interface 14.
[0023]
Next, the operation of the imaging unit 5 in FIG. 2 will be described in detail with reference to FIG. In FIG. 3, the views of the polarizer 18, the Faraday element 17, and the birefringent plate 19 viewed from the lines AA, BB, and CC, respectively, are shown in FIGS. 4, 5, and 6, respectively. ing. In FIG. 3, when light from the lens 1 for forming an image enters the polarizer 18, linearly polarized light in the amplitude direction Wa shown in FIG. 4 is obtained. Ha shown in FIG. 3 is the direction of the magnetic field applied to the Faraday element 17. When the linearly polarized light in the vibration direction Wa shown in FIG. 4 enters the Faraday element 17 such as lead glass, the polarization plane of the linearly polarized light is rotated by the applied magnetic field in the magnetic field direction Ha. When linearly polarized light traveling in the magnetic field direction Ha is incident on the Faraday element 17, the plane of polarization of the transmitted light rotates, and the rotation angle θ is obtained by the following equation.
[0024]
θ = RIH ・ ・ ・ ・ ・ ・ Equation 1
[0025]
Here, I is the thickness of the Faraday element 17, H is the strength of the magnetic field, and R is a Verdet constant. Note that the above Equation 1 is described in, for example, “Optical Measurement Handbook” published by Asakura Shoten Co., Ltd. In FIG. 3, assuming that the magnetic field intensity at which the rotation angle θ is 0 degree is Hθ0, linearly polarized light having the vibration direction Wa shown in FIG. 4 is obtained when H = Hθ, and when H = Hθ0, the linearly polarized light is shown in FIG. Linearly polarized light in the vibration direction Wθ0 is obtained. When Q shown in FIG. 6 is the optical axis of the birefringent plate 19 and linearly polarized light in the vibration direction Wa shown in FIG. 4 is incident on the birefringent plate 19, an ordinary ray Lo shown in FIG. 6 is obtained. When linearly polarized light having the vibration direction Wθ0 is incident on the birefringent plate 19, an extraordinary ray LE shown in FIG. 6 is obtained. The distance between the ordinary ray Lo and the extraordinary ray LE is P, and the above P in the birefringent plate 19 shown in FIGS. 2 and 3 is selected as P = PH / 2 (PH indicates the horizontal pixel pitch of the image sensor 3). . The phase of the change in the magnetic field strength H applied to the Faraday element 17 is matched with the field shift pulse shown in FIG.
[0026]
With the above-described operation, in the imaging apparatus according to the present invention, signal charge accumulation in each of the A and B fields can be performed at a position separated by PH / 2 with respect to the relative position between the incident image and the pixel of the imaging device 3. it can. That is, the spatial sampling region can be increased by temporally changing the strength H of the magnetic field applied to the Faraday element 17 and temporally changing the relative positional relationship between the incident optical image and the imaging element 3. . Correspondingly, the timing of the signal readout pulse shown in FIG. 7 (3) is also shifted by a time T corresponding to PH / 2. As a result, as shown in FIG. 7 (5), the imaging apparatus according to the present invention can obtain a high pixel density image in one cycle with each A and B field as one frame.
[0027]
In this embodiment, the horizontal direction, that is, the coordinates with respect to the one-dimensional space has been described as the pixel shift direction for the sake of simplification. However, the pixel shift position can be two-dimensionally performed according to the same principle. In this case, n sets of optical element groups each including a polarizer 18, a first optical element having a magneto-optical effect, and a birefringent plate 19 are arranged in an imaging system, and linear polarization is provided between the optical element groups. It is only necessary to insert n-1 quarter-wave plates for converting the light into circularly polarized light. By setting binary voltages to be applied to n optical elements having the magneto-optical effect in the n sets of optical element groups, it is possible to obtain an image shifted by 2 n pixels.
[0028]
With reference to the drawings, a detailed description will be given of the resolution enhancement processing in the case where the same target pixel as in the conventional example is increased to 2 × 2 pixels by using this imaging principle. That is, the imaging unit 5 has a configuration in which n = 2 based on the above principle, and when an applied magnetic field is applied to each of the two optical element groups exclusively to the first optical element, one is in FIG. (x, y) = (1/2, 0) Pixel shift to the relative position to the ○ mark and the other pixel shift to the relative position to the (x, y) = (0, 1/2) ○ mark Arrange to do. As a result, when the above-mentioned exclusive magnetic field is applied and when there is no applied magnetic field (marked with (x, y) = (0, 0) ●) and when any magnetic field is applied ((x, y) = (1 / The shooting operation on the premise of 2 × 2 times higher resolution can be performed by the four shooting operations (circles 2) and 1/2).
[0029]
The captured images of these four screens are generated as images for each signal plane in a pixel interpolation circuit (not shown) through A / D conversion of the transfer data from the imaging unit 5 sequentially in the A / D circuit 6 and a color processing circuit (not shown). Is done.
[0030]
In the image compression / decompression circuit 7, the data for each signal plane is divided into blocks each having 4 × 4 pixels, converted into a fixed-length block code, and stored in the image memory 7.
[0031]
FIG. 9 is a diagram showing an encoding algorithm. An image is divided into 4 × 4 blocks as shown in FIG. 9A and encoding is performed. FIG. 9B shows an encoding algorithm. 9B and 9C show a reference level LA for designating a representative signal level that represents the signal level of each pixel in each block of the original image, and a distribution range of representative levels in the block. The difference value LD is calculated, and a two-stage quantized value indicating the resolution component of each pixel in the block is obtained based on the difference value LD. By giving the quantization level corresponding to each pixel by a 1-bit code, the corresponding block is compressed to a fixed data capacity of 1/4 as compared with the input data as shown in FIG. 9 (d).
[0032]
The captured images up to the fourth screen continuously captured are stored in the image memory 8. When the encoding of the image data of the fourth screen is completed, the images up to the fourth screen stored in the image memory 8 are expanded and read out by the image compression / expansion circuit 7 and arranged in the order of the corresponding pixels in FIG. . At this time, the image compression / decompression circuit 7 performs decompression by using the decoding algorithm of FIG. 9 (e).
[0033]
The high pixel density image synthesized by the image synthesis circuit 9 is subjected to MTF correction in the image correction circuit 11 by, for example, an MTF correction filter shown in FIG. FIG. 10B shows output values obtained by the imaging apparatus of the present embodiment with respect to the impulse input shown in FIG. 24A, and is expensive without providing a mechanical drive portion or a large-capacity image memory 8. High-pixel density and high-resolution imaging is possible without using a high-density imaging element 3.
[0034]
In the present embodiment, the fixed-length block code is used as the image compression / decompression circuit 7. However, the present invention is not limited to this, and the same effect can be obtained by using another fixed-length image compression encoding method. can get.
[0035]
In the present embodiment, an example in which the MTF correction filter having a matrix of 5 × 5 pixels shown in FIG. 10A is used as the image correction circuit 11 is shown, but the input characteristics of the imaging system are considered. It is desirable to change the coefficient and the matrix size.
[0036]
In the present embodiment, a configuration example of a digital still camera capable of continuously capturing still images has been described. However, the present embodiment can be configured as a high-definition still image shooting mode in a digital movie capable of capturing moving images. Needless to say.
[0037]
Further, in the present embodiment, all the captured images under the n types of applied magnetic field conditions in the magnetic field generation circuit 15 are fixed length compression encoded and stored in the image memory 8, but the captured images and images under the nth applied magnetic field conditions are stored. By synchronizing the decompression process by the image compression / decompression circuit 7 of the n-1st compressed photograph data stored in the image memory 8 with the image data transfer to the processing unit 13 and transferring it to the image composition circuit 9 The processing time can be shortened by adopting a configuration and control that does not compress and expand the nth captured image.
[0038]
Embodiment 2
The second embodiment of the present invention will be described below with reference to the drawings. The overall block diagram of the second embodiment is the same as that of FIG. 1, and the imaging unit 5 has the same configuration and operation as the first embodiment. Further, the configuration and operation of the image compression / decompression circuit 7 in the image processing unit 13 are the same as those in the first embodiment, and therefore will be omitted.
[0039]
FIG. 11 is a diagram showing a detailed configuration of the image correction processing memory 10 and the image correction circuit 11 of FIG. In FIG. 11, 21 is an input terminal for inputting RGB dot sequential data from the image synthesis circuit 9 in raster scanning order, and 10 is a line buffer as an image correction processing memory for accumulating image data for three lines. 10a, line buffer 10b, and line buffer 10c. 22 is a luminance extraction circuit that calculates the luminance value for each pixel, 23 is an average value calculation circuit that calculates the average luminance value in the window 27 by inputting luminance data from the luminance extraction circuit 22, and 24 is from the luminance extraction circuit 22 A maximum brightness difference extraction circuit 25 for calculating the maximum brightness value Imax and the minimum brightness value Imin when the brightness data is input, and 25 constitutes the window 27 using the average brightness value in the window 27 and the input window 27 as a threshold value. An edge detection circuit comprising a binarization processing circuit 26 for binarizing each pixel, 28 is a pattern memory in which an edge pattern is stored in advance, 29 is a maximum luminance difference value in the window 27 from the maximum luminance difference extraction circuit 24 Is input from the edge detection circuit 25 to determine whether or not the central pixel of the window 27 corresponds to the edge component of the line segment. The tone interpolation circuit selectively compensates the resolution of the edge component of the line segment from the information and the original pixel value input from the line buffer 10.
[0040]
Next, the operation of the image correction circuit 11 will be described. RGB dot sequential data is input from the image composition circuit 9 in FIG. 1 to the input terminal 21 in FIG. 11 in the order of raster scanning. The input image data is sequentially stored in the line buffer 10c, line buffer 10b, and line buffer 10a from the first one line of data, and the image data for three lines is accumulated. At this time, each line buffer 10 behaves like a FIFO memory. Each line buffer 10 outputs data for three pixels in the main scanning direction to form a window 27 of 3 × 3 pixels, and the luminance extraction circuit 22 calculates a luminance value for each pixel. The calculation method at this time can be simply obtained, for example, in Equation 2.
[0041]
I (x, y) = (R (x, y) + 2 × G (x, y) + B (x, y)) / 4 Equation 2
[0042]
Here, R (x, y), G (x, y), and B (x, y) are signal level values of the R component, G component, and B component at coordinates (x, y) in the window 27, I (x, y) is a luminance value. The luminance data for the nine pixels calculated here is input to the average value calculation circuit 23 on the one hand, and the average luminance value in the window 27 is calculated according to, for example, Equation 3.
[0043]
Iav = (ΣI (i, j)) / 9 Equation 3
[0044]
Here, Iav is an average luminance value, and Σ is the sum of I (x, y) over {(i, j) | (i = 0, 1, 2; j = 0, 1, 2)}. The luminance value for each pixel and the average luminance value in the window 27 are output to the edge detecting means 25.
[0045]
On the other hand, the luminance data for nine pixels calculated in the luminance extraction circuit 22 is input to the maximum luminance difference extraction circuit 24 to calculate the maximum luminance value Imax and the minimum luminance value Imin.
[0046]
Next, the edge detection circuit 25 will be described. In the edge detection circuit 25, the binarization processing circuit 26 binarizes each pixel constituting the window 27 using the average luminance value of the input window 27 as a threshold value. The binarized signal is compared with a plurality of bit patterns corresponding to edge patterns stored in advance in the pattern memory 28, and it is determined whether or not the central pixel of the window 27 is an edge. FIG. 11 shows an example in which an edge direction vector exists in the upper right direction, for example, and an edge pattern defines a case where an edge region traverses the window 27, except for those corresponding to isolated points. Define. In this way, only the line segment including the edge in the window 27 can be detected.
[0047]
Next, the gradation interpolation circuit 29 will be described. In the gradation interpolation circuit 29, the maximum luminance difference value in the window 27 is input from the maximum luminance difference extraction circuit 24. Further, the edge detection circuit 25 receives a determination result as to whether or not the center pixel of the window 27 corresponds to the edge component of the line segment. From these information and the original pixel value input from the line buffer 10, the edge component of the line segment is selectively subjected to resolution compensation. That is, when the center pixel of the window 27 is an edge component of the line segment and the maximum luminance difference value satisfies Expression 4, for example, the result of performing the high frequency emphasis filtering shown below is output to the interface 14. Conversely, when these conditions are not satisfied, the original pixel value is output to the interface 14 as it is.
[0048]
Imax _- min ≥ TH1 Equation 4 where TH1 is a predetermined threshold
[0049]
In the present embodiment, the luminance value calculated from the R component, G component, and B component of the input data is used for the edge determination means of the line segment, but the luminance extraction circuit 22 is removed from the block diagram of FIG. Three sets may be prepared and implemented separately for each color plane. Further, although the pattern memory 28 is used for the edge judgment of the line segment, it may be realized by a logic circuit using AND and OR instead of using this. In the present embodiment, the high definition processing is performed on the window 27 composed of 3 × 3 pixels, but the window size is not limited to this.
[0050]
Embodiment 3
Hereinafter, a third embodiment of the present invention will be described with reference to the drawings. The overall block diagram of the third embodiment is the same as FIG.
[0051]
In the third embodiment, the configuration of the imaging unit 5 is different from that of the first embodiment. FIG. 12 shows a detailed configuration of the imaging unit 5 in the third embodiment, 15 is a magnetic field generation circuit, and 16 is a coil thereof. Reference numeral 17 denotes a Faraday element capable of changing a deflection angle by a magnetic field applied by the coil 16 as a first optical element having a magneto-optical effect. Reference numeral 18 denotes a straight line disposed between the Faraday element 17 and the image pickup element 3. A polarizing element 19 for making polarized light is a birefringent plate such as a quartz plate that decomposes light into independent deflection components as a second optical element disposed between the Faraday element 17 and the lens 1. That is, the installation positions of the polarizing element 18 and the birefringent plate 19 are opposite to those in the first embodiment.
[0052]
Next, the operation will be described. The imaging unit 5 will be described. In FIG. 13, when light from the lens 1 for forming an image is incident on the birefringent plate 19, the incident light is divided into an ordinary ray Lo and an extraordinary ray LE as shown in FIG. In FIG. 14, the vibration directions of the ordinary ray Lo and the extraordinary ray LE are indicated by arrows, respectively. Q shown in FIGS. 13 and 14 indicates the optical axis of the birefringent plate 19. Ha shown in FIG. 13 is the direction of the magnetic field applied to the Faraday element 17. When the ordinary ray Lo and the extraordinary ray LE shown in FIG. 14 are incident on the Faraday element 17 such as lead glass, the plane of polarization of the linearly polarized light is rotated by the applied magnetic field in the direction of the magnetic field. When linearly polarized light traveling in the direction Ha of the magnetic field is incident on the Faraday element 17, the plane of polarization of the transmitted light rotates, and the rotation angle θ is obtained by Equation 1.
[0053]
In Equation 1, when the magnetic field intensity at which the rotation angle θ is 0 degrees is H0 and the magnetic field intensity at 90 degrees is H90, when H = H0, the above-mentioned ordinary ray Lo and extraordinary ray LE are supported. The vibration direction of the optical axis is represented by arrows such as Lo0 and LE0 shown in FIG. When H = H90, the vibration direction of the optical axis corresponding to the aforementioned Lo and LE is represented by arrows as indicated by Lo90 and LE90 in FIG.
[0054]
Here, when the polarizer 18 that transmits only the component of light that vibrates in the direction indicated by Wa in FIG. 16 is used as the polarizer 18, the light beam indicated by LE0 is transmitted when H = Ho, and when H = H90. The light beam indicated by Lo90 is transmitted. The distance between the ordinary ray Lo and the extraordinary ray LE shown in FIG. 13 is P, and the above P in the birefringent plate 19 is selected as P = PH / 2 (PH indicates the horizontal pixel pitch of the image sensor 3). The phase of the change in the magnetic field strength H applied to the Faraday element 17 is made to coincide with the field shift pulse shown in FIG. With the above-described operation, in the imaging apparatus according to the present invention, signal charge accumulation in each of the A and B fields can be performed at a position separated by PH / 2 with respect to the relative position between the incident image and the pixel of the imaging element 3. it can. That is, the spatial sampling area can be increased by temporally changing the strength H of the magnetic field applied to the Faraday element 17 and temporally changing the relative positional relationship between the incident optical image and the imaging element 3. . Correspondingly, the timing of the signal readout pulse shown in FIG. 7 (3) is also shifted by a time T corresponding to PH / 2. As a result, as shown in FIG. 7 (5), the imaging apparatus according to the present invention can obtain a high pixel density image in one cycle with each A and B field as one frame.
[0055]
Regarding the imaging unit 5 in the present embodiment, the description has been made on the increase in pixel density by one-dimensional pixel shifting in the horizontal direction as in the first embodiment. Expansion is possible. In this case, n sets of optical element groups each including a polarizer 18, a first optical element having a magneto-optical effect, and a birefringent plate 19 are arranged in an imaging system, and linear polarization is provided between the optical element groups. It is only necessary to insert n-1 quarter-wave plates for converting the light into circularly polarized light. By setting binary voltages to be applied to n optical elements having the magneto-optical effect in the n sets of optical element groups, it is possible to obtain an image shifted by 2 n pixels.
[0056]
Embodiment 4
Embodiment 4 of the present invention will be described below with reference to the drawings. The overall block diagram of the fourth embodiment is the same as that of FIG. 1, and the configuration of the imaging unit 5 is different from that of the first embodiment. FIG. 17 shows a detailed configuration of the imaging unit 5 in the fourth embodiment. In this figure, a lens 1, an image pickup device 3, an image pickup unit control circuit 4, and a polarizer 18 are the same as those in the third embodiment shown in FIG. 30 is an electric field generating circuit for generating an electric field, and 31 is an electro-optical element in which the refraction phenomenon of transmitted light changes according to the strength of the electric field, and is installed between the lens 1 and the polarizer 18.
[0057]
Next, the operation will be described. The imaging unit 5 will be described. The electro-optic element 31 and the polarizer 18 in FIG. 18 viewed from the directions of AA, BB, and CC are shown in FIGS. 19, 20, and 21, respectively. In FIG. 18, natural light incident on the electro-optic element 31 from the lens 1 for forming an image can be represented by two polarization components LX and LY orthogonal to each other as shown in FIG. When no electric field is applied to the electro-optic element 31, the polarization components indicated by LX0 and LY0 in FIG. 20 are observed at positions where the LX and LY have traveled straight. When a predetermined electric field E1 is applied to the electro-optical element 31, the electro-optical element 31 exhibits a birefringence phenomenon, and polarization components indicated by LX1 and LY1 in FIG. 20 are observed. Here, LX1 is observed at a position where LX has traveled straight as an ordinary ray in the birefringence phenomenon, and LY1 is observed as an extraordinary ray in the birefringence phenomenon with LY being shifted by a distance P. However, an electric field E1 such that P = PH / 2 (PH indicates the horizontal pixel pitch of the image sensor 3) is applied to the electro-optical element 31 by the electric field generation circuit 30.
[0058]
FIG. 7 is an operation timing chart in the imaging apparatus. 7 (1) is a field shift pulse, FIG. 7 (2) is a diagram showing the intensity of the electric field E applied to the electro-optic element 31, FIG. 7 (3) is a signal readout pulse, and FIG. FIG. 7 (5) is a diagram showing the output signal of the element 3, and FIG. 7 (5) is a diagram when the A and B fields are viewed as one frame. The phase of the change in the intensity of the electric field E is matched with the field shift pulse in FIG.
[0059]
With the above-described operation, the image pickup apparatus of this embodiment can perform signal charge accumulation in the A and B fields at a position separated by PH / 2 with respect to the relative position between the incident image and the pixel of the image pickup device 3. it can. Correspondingly, the timing of the signal readout pulse shown in FIG. 7 (3) is also shifted by a time T corresponding to PH / 2. As a result, as shown in FIG. 7 (5), the imaging apparatus can obtain an image with a high pixel density in one cycle with the A and B fields as one frame. The configuration and operation of the image processing unit 13 are the same as those in the first embodiment.
[0060]
Further, regarding the imaging unit 5 in the present embodiment, as in the first embodiment, a description has been given of increasing the pixel density by one-dimensional pixel shifting in the direction of the extraordinary ray LY1 in FIG. Two-dimensional expansion is possible with the same concept as in FIG. In this case, a set of n optical elements each including a polarizer 18 and an electro-optical element 31 is arranged in the imaging system, and for example, a quarter-wave plate for converting linearly polarized light into circularly polarized light between each optical element group. N-1 may be inserted. By setting binary voltages to be applied to n electro-optical elements in the n sets of optical element groups, it is possible to obtain an image shifted by 2 n pixels.
[0061]
【The invention's effect】
According to the image pickup apparatus of the first aspect of the present invention, since a plurality of images obtained by pixel shifting are compressed to a fixed-length code by the image compression / expansion means and stored in the image storage means, the high pixel density processing Therefore, there is no need for a large-capacity storage device, and there is an effect that it can be configured at a low price.
[0062]
In addition, since the frequency correction of the edge component of the image is performed by the image correction means against the decrease in MTF caused by image synthesis, a high-resolution captured image with high resolution that follows the increase in pixel density can be obtained. There is an effect.
[0063]
As described above, it has an excellent effect that a high-resolution imaging device can be obtained using an inexpensive low-density imaging device.
[0064]
In addition, according to the image pickup apparatus according to the second to fourth aspects of the present invention, since mechanical drive control by a motor or the like is not required as a pixel shifting means, a pixel shifted image with a high power and a simple device configuration and high shifting accuracy. The effect is that can be input.
[0065]
In the fifth aspect of the invention, since the correction processing in the pixel correction means is selectively performed only on the edge component of the line segment, resolution correction can be performed without enhancing high-frequency noise mixed from the imaging unit. Therefore, there is an effect that the image quality can be improved.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment.
FIG. 2 is a configuration diagram of an imaging unit according to the first embodiment.
FIG. 3 is a principle diagram of an imaging unit according to the first embodiment.
FIG. 4 is a diagram showing an operation according to the first embodiment.
FIG. 5 is a diagram showing an operation according to the first embodiment.
FIG. 6 is a diagram showing an operation according to the first embodiment.
7 is a timing chart showing the operation of the imaging unit according to Embodiment 1. FIG.
FIG. 8 is a diagram illustrating a pixel shift position of the imaging unit according to the first embodiment.
FIG. 9 is a diagram showing an image compression algorithm using a fixed-length code according to the first embodiment.
10 is a diagram showing an example of MTF correction filtering according to Embodiment 1. FIG.
FIG. 11 is a configuration block diagram of an image processing unit according to the second embodiment.
12 is a configuration diagram of an imaging unit according to Embodiment 3. FIG.
13 is a principle diagram of an imaging unit according to Embodiment 3. FIG.
FIG. 14 is a diagram showing an operation according to the third embodiment.
FIG. 15 is a diagram showing an operation according to the third embodiment.
FIG. 16 is a diagram showing an operation according to the third embodiment.
17 is a configuration diagram of an imaging unit according to Embodiment 4. FIG.
FIG. 18 is a principle diagram of an imaging unit according to the fourth embodiment.
FIG. 19 is a diagram illustrating an operation according to the fourth embodiment.
FIG. 20 shows an operation according to the fourth embodiment.
FIG. 21 is a diagram illustrating an operation according to the fourth embodiment.
FIG. 22 is a block diagram illustrating a configuration of a conventional imaging device.
FIG. 23 is a configuration diagram of an imaging unit of a conventional imaging apparatus.
FIG. 24 is an explanatory diagram of the principle of increasing the pixel density taken by a conventional imaging device.
[Explanation of symbols]
1: Lens 2: Pixel shifting means 3: Image sensor 4: Imaging unit control circuit
5: Imaging unit 7: Image compression / decompression circuit 8: Image memory 9: Image composition circuit
10a: Line buffer 1 10b: Line buffer 2 10c: Line buffer 3
11: Image correction circuit 12: Image processing control circuit 13: Image processing unit
14: Interface 15: Magnetic field generator 16: Coil
17: Magneto-optical element 18: Polarizer 19: Birefringent plate 22: Luminance extraction circuit
23: Average value calculation circuit 24: Maximum luminance difference extraction circuit 25: Edge detection means
26: Binary processing circuit 27: Window 28: Pattern memory
29: Gradation interpolation circuit 30: Electric field generation circuit 31: Electro-optic element
103: Transparent flat member 104: Base unit 105a: Compression spring
105b: Compression spring 105c: Compression spring 106a: Spring retainer plate
106b: Spring retainer plate 106c: Spring retainer plate 107a: Motor
107b: Motor 108: Actuator 108a: First actuator
108b: Second operation unit 110: Image composition memory

Claims (5)

In an imaging apparatus that guides reflected light from an object to be picked up to an imaging device composed of a plurality of pixels arranged two-dimensionally by an imaging optical system, and generates an image signal by photoelectric conversion by the imaging device. A pixel shifting unit that shifts the incident position of incident light from the imaging optical system to the image sensor by a minute pixel, an imaging control unit that controls the amount of pixel displacement of the pixel shifting unit, and continuous imaging within a predetermined time Image compression means for compressing at least N−1 images out of N (N ≧ 2, N: natural number) images to a fixed-length code, and image storage means for storing images compressed by the image compression means An image composition unit for forming an image in which the number of pixels is increased M times from an image of M (N ≧ M ≧ 2, M: natural number) of the N images, and an image formed by the image composition unit A predetermined sharpness improving process is performed on Imaging apparatus characterized by comprising an image correcting unit. The pixel shifting means includes a polarizer for generating linearly polarized light, a first optical element having a magneto-optical effect disposed between the polarizer and the image sensor, and the first optical element and the image sensor. And a second optical element that divides the light into independent polarization components, and the imaging control means is configured to change a magnetic field application condition to the first optical element. 2. The imaging apparatus according to claim 1, wherein the image correction means is configured to perform predetermined filtering on the composite image. The pixel shifting means includes a polarizer that generates linearly polarized light, a first optical element having a magneto-optical effect disposed between the polarizer and the imaging optical system, the first optical element, and the imaging. A second optical element that is disposed between the optical system and divides the light into independent polarization components, and the imaging control means is configured to change a magnetic field application condition to the first optical element. 2. The imaging apparatus according to claim 1, wherein the image correction means is configured to perform predetermined filtering on the composite image. The pixel shifting means includes an electro-optic element and a polarizer that change the refraction phenomenon of transmitted light according to the strength of the electric field, and the imaging control means is configured to change an electric field application condition to the electro-optic element. 2. The imaging apparatus according to claim 1, wherein the image correction means is configured to perform predetermined filtering on the composite image. The image correction means is an area extraction means for extracting a small area composed of K × L pixels (K: natural number, L: natural number) centered on the target pixel from the composite image, and whether or not the central pixel in the small area is an edge component. 5. The method according to claim 1, further comprising: edge vector detection means for determining the line segment direction vector and pixel correction means for performing predetermined edge pixel correction according to the edge vector information. The imaging device described.

Images