JPH0245873B2 - - Google Patents

Description

Detailed Description of the Invention [Technical Field of the Invention] The present invention relates to solid-state imaging that obtains high-resolution images using a solid-state imaging device having a limited number of pixels and having photosensitive parts arranged two-dimensionally on a semiconductor substrate. Regarding equipment.

[Technical background of the invention and its problems] Solid-state image sensors such as CCDs have the characteristics of being smaller, lighter, and more reliable than conventional image pickup tubes.In terms of characteristics, they also have no shape distortion, small afterimages, and It has many advantages such as no burn-in. Therefore, its applications are wide, including industrial television cameras, home video cameras, and electronic cameras that do not use silver halide film, and are expected to expand further in the future.

Figure 1 shows a typical interline transfer type CCD.
The schematic configuration of the system is shown. Pij (j=1, 2,...,
M, j=1, 2, . . . , N) are two-dimensionally arranged photosensitive parts, Ci is a vertical readout CCD register, and H is a horizontal readout CCD register. When applying such a solid-state imaging device to the wide range of applications described above, a major problem is how to achieve high resolution changes with a limited number of pixels.

Therefore, the present inventors previously proposed in Japanese Patent Application No. 56-209381 an apparatus in which a solid-state image sensor with a limited number of pixels was used to achieve high resolution changes. As shown in the principle diagram of this device in FIG.
In the horizontal direction (X direction), 1/2 of the horizontal pixel pitch P _H
It is oscillated relative to the incident optical image with an amplitude of P _H /2, which is equivalent. Here, as shown in the figure, the vibration changes over time into a trapezoidal shape in synchronization with the imaging operation in which the first (A) field and the second (B) field of the solid-state imaging device have one frame period. As a result, the pixel aperture shown in the figure is located at the solid line 2 in the A field, and at the dashed line 3 in the B field.
position. Then, by turning on the field shift pulse at the time of switching between the A and B fields or the B and A fields, the optical signal accumulated charge at the position of the aperture 2 is read out at the time of switching between the A and B fields. Then, at the time of switching between the B and A fields, the optical signal accumulated charge at the position of the aperture 3 is read out. Then, the readout signal accumulated charge is displayed on the reproduced image by shifting the driving timing or by signal processing so that it becomes an image corresponding to the position of the spatial sampling point of the A and B fields. By adding , the number of spatial sampling points in the horizontal direction that the solid-state image sensor itself has is doubled, and the horizontal resolution can be doubled. Furthermore, since the invalid portion of the solid-state image sensor with respect to the incident optical image is reduced, moire inherent to the solid-state image sensor is improved. FIG. 3 is a diagram showing spatial sampling points 4 on the reproduced image. In the A field, this becomes the spatial sampling point 4a on the 1H _A , 2H _A , and 3H _A scanning lines. Then, in the B field, it becomes the spatial sampling point 4b in the scanning lines 1H _B , 2H _B . . . . As a result, the phase of the spatial sampling point in the A field and the spatial sampling point in the B field is 1/2fcp, which is equivalent to 1/2 of the horizontal readout frequency fcp.
become. Therefore, the number of spatial sampling points in the horizontal direction on the reproduced image is doubled.

However, two of the spatial sampling points in this device
Doubling is the scanning line in the A field, for example 1H _A
For example, 1 scan line in the B field adjacent to
It is being held at _HB . Therefore, the spatial sampling points on the same scanning line are not doubled,
For fine incident optics, errors occur because the spatial sampling points span the A and B fields. This error causes a zigzag pattern on the black-and-white boundary line on the reproduced image, resulting in a problem of deterioration of image quality. Also,
The spatial sampling points in the vertical direction are A and B.
Since the fields are not on the same vertical line, errors occur as described above, resulting in a zigzag-like image quality deterioration at the black-and-white boundary in the vertical direction on the reproduced image.

[Object of the Invention] The present invention has been made in view of the above points, and an object of the present invention is to provide a high-resolution solid-state imaging device in which the vibration mode is improved so as to double the spatial sampling points on one scanning line. do.

[Summary of the Invention] The present invention is an interline transfer type CCD as shown in FIG. 1, for example. It has an imaging operation in which the signal charges accumulated in the photosensitive section are simultaneously moved to the vertical readout register during the vertical blanking period and read out during the valid period of the next field, and the A field and B field are treated as one frame period. Thereafter, the solid-state image sensor chip substrate, which has a motion leading to the next frame period, is set to have a vibration center at every other vertical blanking period, which is a valid period between fields, and is then turned into an incident optical image in a certain invalid period. 1/2 horizontal image pitch relative to the incident optical image, leave it as it is during the next invalidation period, and then move it in the opposite direction between the next invalidation periods by 1/2 horizontal pixel pitch relative to the incident optical image. A high-resolution change is achieved by applying a pulse-like vibration with one period of two frame periods to the solid-state image sensor chip substrate.

[Effects of the Invention] The solid-state imaging device according to the present invention can achieve substantially higher resolution changes than conventional solid-state imaging devices. Further, even when compared with a solid-state imaging device having the same number of pixels as the present invention, by increasing the density of the chip substrate itself, it is possible to obtain images with better characteristics such as dynamic range and generation of false signals. Furthermore, manufacturing of drive circuits for operating these devices is also easier than in high-density solid-state imaging devices. Further, the zigzag-like image quality deterioration at the black and white boundary line on the reproduced image, which was observed in the above-mentioned Japanese Patent Application No. 56-209381, does not appear in the present invention, and a good high-resolution image can be obtained.

[Embodiment of the Invention] FIG. 4 is a diagram for explaining an embodiment of the present invention. This figure shows the relationship between the solid-state imaging device, the time relationship in which the solid-state imaging device is vibrated, and the timing relationship in the field shift pulse for reading signal charges from the photosensitive portion of the solid-state imaging device.

_The interline transfer type CCD _chip substrate 1 shown in FIG. let The time change of this vibration is determined by the interlace operation in which the A and B fields are one frame period in the imaging operation of the interline transfer type CCD in the field time accumulation mode, with two frame periods as one cycle.
It is shaped into a trapezoid so that the vibration center is within the invalid period between B fields. The amount of vibration from X ₁ to X ₂ is set to P _H /2, which is equivalent to 1/2 of the horizontal pixel pitch P _H.

Next, specific operations will be explained. First, the first
At _t1 of the _A1 field of the frame, optical signal accumulation is performed with the center of the opening 2 of the CCD chip substrate 1 at the position _X1 . Then, at t ₂ of B ₁ field of the next first frame, CCD chip board 1
vibrate relatively to the right in the figure, and bring the opening center of opening 2 to position 3 so that it is at position X ₂ . Then, optical signals are accumulated at this position. The amount of vibration at this time is 1/2 of the horizontal pixel pitch P _H
The equivalent value is P _H /2. Then, at t ₃ of the A ₂ field of the next second frame, optical signal accumulation is performed at the position X ₂ with the aperture center position unchanged.
And in the next second frame's B ₂ field t ₄
The CCD chip board 1 is vibrated to the left in the figure and brought from the position X ₂ to the position X ₁ A. Then, optical signals are accumulated at this position. The amount of vibration at this time is set to P _H /2 as described above. Thereafter, the third frame, the fourth frame...the nth frame is the first frame, the second frame, etc.
Repeat the same action as the frame.

In the above operation, if the period during which the center of the opening 2 moves from the position X ₁ to the position X ₂ and the period during which it moves from the position X ₂ to the position X ₁ is sufficiently short, A ₁ , B ₁ , A ₂ and B _2, it can be considered that the centers of the openings are stationary at the positions of X ₁ and X ₂ , respectively. As a result, the number of spatial sampling points in the horizontal pixel arrangement direction is doubled. The drive timing is then shifted so that the image corresponds to the position of the spatial sampling point of each field on the reproduced image, and each field is added and displayed, doubling the horizontal resolution of the solid-state image sensor itself. can.

FIG. 5 is a diagram showing the movement of spatial sampling points with respect to the incident optical image when the interline type CCD chip substrate of FIG. 1 is vibrated in the vibration mode shown in FIG. 4. a is A ₁ field,
b is B ₂ field, c is A ₂ field, d is B ₂ field
1H _A , 2H _A , 3H _A ,..., 1H _A ′, 2H _A ′, which are spatial sampling points in the field and are shown as solid lines
3H _A ′, ... indicate the scanning line position in the A field in the imaging operation of the interline transfer type CCD in the field time accumulation mode, and 1H _B ′, 2H _B ′,
3H _B ′, 1H _B ′, 2H _B , 3H _B , . . . indicate scanning line positions in the B field. The arrow displayed at each spatial sampling point indicates the direction of movement of the spatial sampling point when passing over the next field.

In the next field _B1 , the spatial sampling point in field _A1 moves diagonally to the lower right in the figure due to a combination of interlaced imaging and P _H /2 vibration in the horizontal pixel arrangement direction. and. next
In the _A2 field, no vibration is performed and only interlaced imaging is performed, so the spatial sampling point moves upward in the figure. In field _B2 , interlaced imaging is combined with vibration, and the spatial sampling point moves toward the lower left in the figure. As a result of the above operations, the spatial sampling points will be at the positions shown in FIG. a, b, c shown in Figure 6,
The spatial sampling points of d are a, b, c, and
It corresponds to d. As a result, the sampling points in the horizontal direction on the scanning lines of the A and B fields, which are displayed on the reproduced image in accordance with the actual sampling points, are twice the horizontal pixel pitch _PH . The sampling point in the vertical direction becomes the vertical image pitch P _V in interlaced imaging of the A and B fields.
Here, the time width of the horizontal image pitch P _H is determined by the reciprocal of the drive frequency fcp of the horizontal readout register, so
The pitch of spatial sampling points in the horizontal direction is 1/2fcp
become. Therefore, the horizontal limit image resolution can be improved to twice the Nyquist limit determined by the drive frequency of the horizontal readout register.

Furthermore, the zigzag-like image quality deterioration at the black-and-white boundary on the reproduced image, which was a problem in the conventional operation explained in FIG. 3, is greatly improved in this embodiment.

Furthermore, in this embodiment, the spatial sampling points in the vertical interlaced imaging of the A and B fields have an inverse phase relationship, so that the distortion of moiré can be improved.
That is, in addition to improving moire by increasing the number of spatial sampling points, further reduction of moire can be achieved by changing the combination of adding and reading pixels in the vertical direction for each field.

Further, in this embodiment, the vibration period is halved compared to the conventional example shown in FIG. This becomes effective when a bimorph piezoelectric element using ceramic, for example, is used as the vibration mechanism. When a bimorph piezoelectric element is used as a vibration mechanism, it is vibrated at a frequency lower than the resonance frequency of the element itself. As a result, ringing is likely to occur when the vibration pulse changes. This is because if the vibration period is reduced, the amount of ringing can also be reduced.

FIG. 7 is a configuration diagram of this embodiment, and FIG. 8 is an operating waveform diagram thereof. A solid-state image sensor chip 10 is fixed on a vibration table 11, and input light passes through an imaging lens 12 and is imaged on the solid-state image sensor chip 10. A vibration pulse shown in FIG. 8 is applied to the vibration table 11 from a vibration pulse generation circuit 13 with an amplitude equivalent to P _H /2. This vibration pulse is synchronized with the field shift pulse operating within the vertical blanking period, as shown in FIG. This is a pulse whose period is two frame periods. The timing generation circuit 14 generates a timing signal of a P _H /2 delay circuit 15 for delaying the timing of the horizontal read register by P _H /2 in accordance with the period of the vibration pulse, and other necessary synchronization pulses such as a timing signal of the vertical read register. occurs. The output signal obtained from the solid-state image sensor chip 10 driven by the clock driver 16 is outputted through the preamplifier 17 and the signal processing circuit 18. That is, in this example, in order to shift the output signal in accordance with the spatial sampling point caused by the vibration of the solid-state image sensor chip 10, the timing of the horizontal clock pulse is adjusted by 1/2 clock (1/2fcp) in accordance with the vibration period as shown in FIG. ), thereby obtaining an output signal that matches the actual spatial sampling point. By adding the first frame and second frame composed of the A ₁ , B ₁ , A ₂ , and B ₂ fields on the reproduced image, the resolution in the horizontal direction can be doubled.

As explained above, in this embodiment, like an interline transfer type CCD, a solid state device having an imaging operation that simultaneously transfers signal charges accumulated in a photosensitive area for each field to a vertical readout register, which is a readout unit, during a vertical blanking period is used. The image sensor chip board is divided into 2 frames (4 frames) in the horizontal pixel arrangement direction so that the vibration center is located during the planking period.
To double the spatial sampling points of the solid-state image pickup device itself by vibrating the solid-state image sensor with a periodicity (field), and to obtain a reproduced image with twice the horizontal resolution without deterioration of the vertical resolution. Can be done.

Even if a device with double the number of pixels in the horizontal direction is realized by high-density technology, the solid-state imaging device according to this embodiment has the following characteristics advantages compared to this device.

(1) In a high-density device, the horizontal pixel pitch is halved, so the saturation signal level decreases and the dynamic range becomes smaller, but in this embodiment, it is equivalent to a device that does not vibrate. Therefore, using the same manufacturing technology, the dynamic range can be increased compared to essentially higher density devices.

(2) In high-density devices, the clock frequency of the horizontal read register is doubled, which increases the power consumption of the drive circuit and signal processing circuit, and increases the difficulty of circuit fabrication, but this is not the case with the device of this embodiment. do not have.

(3) In a high-density device, the ratio of the aperture area of the photosensitive section may decrease but never increase, so even if the density is increased, the invalid area for optical information in the entire photosensitive area will not decrease. On the other hand, in this embodiment, since optical information is obtained even from a region that was conventionally ineffective, the effective region for collecting input optical information is essentially wide.

FIG. 9 is a comparative example in which different vibration waveforms are provided. In this case, during the first frame period consisting of fields A ₁ (t ₁ ) and B ₁ (t ₂ ), the opening 2 of the solid-state imaging chip substrate 1 is set at the position X ₁ . and a second field consisting of A ₂ (t ₃ ), B ₂ (t ₄ D) fields.
During the frame period, the aperture is moved by P _H /2 and brought to the position X ₂ as shown by the broken line 3. That is,
Bring the vibration center within the dead period between frames. As a result, the positions of the spatial sampling points explained in the embodiment of FIG. 5 are A ₁ →B ₁ →A ₂ in FIG.
→B The spatial sampling points are located in the order of a → b → c → d in the field order _{of 2} , and in Figure 9 the spatial sampling points are in the order of a → b → c → d in the same field order as in Figure 5. Moving. As a result, on the reproduced image, twice as many sampling points are obtained in the horizontal pixel arrangement direction as in the case explained in FIG. An image of
The moiré improvement effect due to the zigzag combination of pixel addition in the vertical direction as in the previous embodiment cannot be obtained.

In the example explained above, the present invention was applied to a video camera that is applied to the television standard system, but for example, an electronic camera that does not use silver halide film, etc.
The present invention can be applied to systems such as OCR. 1st
FIG. 0 is a diagram showing the operation when the present invention is applied to an electronic camera. In this case, it is necessary to synchronize the opening/closing timing of the optical shutter with the operation of the image sensor, as in a normal optical camera.
The first frame consisting of A ₁ and B ₁ fields and A ₂ ,
B In the imaging operation of _{the second} frame consisting of two fields, a field shift pulse is applied at the time of switching each field, and the signal charge accumulated in each field is read out. The vibration pulse has a waveform similar to that shown in FIG. 4, in which the first and second frames are one period. In the shutter pulse, the shutter is ON during the first and second frames, and the spatial sampling points of the incident optical image are doubled during this period.
In addition, in the case of shutter pulse, turn on the shutter
This is an operation with a shortened period. In addition, in the case of a shutter pulse, the shutter is turned OFF during the period of change of the vibration pulse to provide a more reliable operation for doubling the spatial sampling points.

Furthermore, various modifications and applications of the present invention are possible as listed below.

a The image sensor is not limited to an interline transfer type CCD image sensor, and for example, a frame transfer type image sensor may be used. What these image sensors have in common is that signal charges accumulated in the pixel region are simultaneously read out during the vertical blanking period. Therefore, the present invention can be applied to any image sensor that operates in a similar manner.

b The shape of the aperture of the pixel of the image sensor is not limited to a rectangle, nor is the size particularly limited.

c The pixel arrangement of the image sensor is not limited to being arranged in a line in the vertical direction, but may be arranged in a zigzag manner, which is more advantageous for reducing moiré and improving resolution.

d The image sensor is a normal CCD as a photoelectric conversion section.
A so-called two-story sensor structure can be used in which a photoelectric conversion film such as amorphous silicon is layered on the image sensor.

e In the case of a two-story sensor using a photoelectric conversion film, it is also possible to obtain an ultraviolet image or an infrared image depending on the properties of the photoconductor. The present invention can also be applied to the case where an X-ray image is obtained using a phosphor film instead of a photoconductor film. Furthermore, it can be used as a device to superimpose visible and invisible images and observe them on the reproduced image by arranging a normal visible conversion film and an invisible conversion film such as infrared, ultraviolet, and X-ray for every other pixel in a two-story sensor. The present invention can be applied. In this case, by setting the amount of vibration pulses to the horizontal pixel pitch, both visible and invisible images can be reproduced without deterioration in horizontal resolution.

f In the present invention, the image sensor chip is vibrated relative to the optical image, but the same idea can be applied to an electron beam impact type solid-state image sensor. That is, the same effect can be obtained by utilizing the fact that the electron beam image can be deflected and changing it in a predetermined direction for each of the fields A ₁ , B ₁ , A ₂ , and B ₂ .

g The present invention can also be applied to a color camera that performs color imaging using one, two, or three imaging munctures. In a two-panel or three-chip color camera, the resolution can be further improved by using the present invention and the pixel shifting method. In addition, in a two-chip color camera that uses primary colors, the image sensor that obtains the G signal is set to 1/1/2 of the horizontal pixel pitch.
The resolution can be further improved by vibrating the image sensor that obtains the R and B signals by a horizontal pixel pitch.

h The means of vibration in the present invention is not limited to the vibration of the solid-state image sensor itself as explained in FIG. 7. For example, an optical deflection plate may be provided on the light incident side of the solid-state image sensor, and this deflection plate may be periodically vibrated, or the incident light may be deflected by an optical mirror. In short, it is sufficient to vibrate the solid-state image sensor relative to the incident light image.

i The vibration pulse of the present invention was explained using a trapezoidal wave, but
The present invention is applicable to this shape even if the shape is not a trapezoid, but a rectangle, a triangle, etc., as the number of sampling points within a unit pixel increases.

j Most of the description of the present invention has been made using a TV imaging system based on the NTSC standard system, but the invention is not limited to this system. That is, the present invention is applicable to any system that performs interlaced imaging. For example, it can be applied to SECAM format, PAL format, low-speed imaging method, high-speed imaging method, etc.

[Brief explanation of the drawing]

Fig. 1 is a diagram showing the schematic configuration of an interline transfer type CCD image sensor, Fig. 2 is a diagram explaining the principle of the solid-state imaging device proposed earlier by the present inventors, and Fig. 3 is a diagram showing the operation of Fig. 2. FIG. 4 is a diagram showing the state of vibration of the image sensor in an embodiment of the present invention, and FIG. 5 is a diagram showing each field of the spatial sampling points in the same embodiment. 6 is a diagram showing the spatial sampling points of FIG. 5 added on the reproduced image, FIG. 7 is a diagram showing the overall configuration of the device of the same embodiment, and FIG. 8 is an explanation of its operation. FIG. 9 is a diagram showing the state of vibration in another embodiment of the present invention, and FIG.
The figure is a diagram for explaining the operation when the present invention is applied to an electronic camera. DESCRIPTION OF SYMBOLS 1... Solid-state imaging device chip board, 2... Pixel aperture, 10... Solid-state imaging device chip, 11... Vibration table, 12... Imaging lens, 13... Vibration pulse generation circuit, 14... Timing generation circuit , 15...
... _PH /2 delay circuit, 16...clock driver,
17... preamplifier, 18... signal processing circuit.

Claims (1)

[Claims] 1. Using a solid-state image sensor having photosensitive parts arranged two-dimensionally on a substrate, each of the photosensitive parts is The signal charges accumulated through photoelectric conversion are transferred simultaneously to the readout section every field period, and while the signal charges in the readout section are sequentially transferred to the output section and read out, each photosensitive section receives the signal charges of the next field. A solid-state imaging device that performs an operation of accumulating images is provided with means for vibrating the solid-state imaging element chip substrate in a horizontal pixel arrangement direction relative to an incident optical image, and the vibration waveform is set such that two consecutive frames constitute one cycle. The vibration amplitude is 1/2 of the horizontal pixel array pitch or its vicinity,
The vibration center of the vibration waveform is set between the first field and the second field of the n-th frame and between the first field and the second field of the n+1-th frame, and the spatial sampling point of the incident optical image due to this vibration is A solid-state imaging device characterized in that a process is performed to correct a shift on a reproduced image.