JP4399693B2 - Imaging device - Google Patents

Description

[0001]
[Technical field to which the invention belongs]
The present invention relates to a technique for correcting various problems when a moving subject is photographed, particularly in an imaging apparatus capable of obtaining a wide dynamic range image by combining a plurality of image data.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, in an imaging apparatus such as a video camera or a still video camera, in order to obtain an image with a wide dynamic range, images having different exposure times are obtained by time division and these images are combined. .
[0003]
For example, when shooting a subject with a large difference in brightness, images with exposure adjusted to a portion with a high brightness and a portion with a low brightness are captured and widened based on these images. Creating a composite image with a dynamic range.
[0004]
The above composite image creation method calculates the ratio of the exposure time between a short-time exposure image in which exposure is adjusted to a portion with high luminance and a long-time exposure image in which exposure is adjusted to a portion with low luminance, and the ratio is reduced. After multiplying the time exposure image, the pixel data of these two images are partially switched according to the level of the long exposure image in each pixel of the image sensor.
[0005]
However, when the subject of the two types of images is misaligned, for example, the level of the long exposure image at a certain pixel of the image sensor is switched from the pixel data of the long exposure image to the pixel data of the short exposure image. When necessary, if the pixel data of the short-exposure image to be switched is not a sufficient level due to the deviation of the subject, there is a problem that the continuity of the image is disturbed and the image is defective. On the other hand, if the level of the long-exposure image at a certain pixel is not a level to be switched but the pixel data of the short-exposure image at the switching destination is already at a sufficient level, an afterimage occurs in the subject. There was a problem that.
[0006]
[Problems to be solved by the invention]
In view of the above problems, the present invention provides an image composition method that obtains images with different exposure times by time division and combines them to obtain a wide dynamic range image. It is an object of the present invention to correct a defect in a composite image that occurs due to image shift.
[0007]
[Means for Solving the Problems]
In order to solve the above problems, the present invention includes an image sensor, a signal processing circuit, an image switching unit that appropriately switches data corresponding to a short-exposure image and a long-exposure image according to the polarity of the difference, Multiplying means that multiplies the exposure ratio between the time exposure image and long exposure image by the short exposure image, and calculates the data difference between the short exposure image multiplied by the exposure ratio by the multiplication means and the long exposure image. Subtracting means for detecting, and a first comparing means for detecting long-exposure image data at a switching level or higher, and a second detecting means for detecting a pixel having a level higher than a preset reference level from data from the subtractor. Comparison means, third comparison means for detecting data with a positive subtraction result from data from the subtractor, logic of output data of the first comparison means and the second comparison means And a second logic circuit that outputs a logical product of output data of the first logic circuit and the third comparator, and outputs the output of the second logic circuit. It is used as a control signal for the image switching means to create a composite image by selectively using the data of the short exposure image and the long exposure image for each pixel .
[0008]
Therefore, it is possible to eliminate the occurrence of afterimages and discontinuous points that occur when a moving subject is imaged.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
Details of the imaging apparatus of the present invention will be described below with reference to the accompanying drawings.
[0010]
First, a configuration and problems of an example of a conventional imaging device will be described for comparison with the present invention.
[0011]
As shown in the block diagram of FIG. 8, the conventional imaging device 1 includes an imaging device 2, a signal processing circuit 3, a multiplier 4, a comparator 5, an image changeover switch 6, and the like. The readout pulse shown in FIG. 2B is applied by a timing generator (not shown) so that the readout is performed twice within one field period.
[0012]
The interval between the readout pulses in FIG. 2B is divided into two different length regions, that is, a long exposure period and a short exposure period, within one field period. Of these two regions having different lengths, the accumulated voltage output during the long exposure period corresponds to the long exposure image, and the accumulated voltage output during the short exposure period corresponds to the short exposure image.
[0013]
Therefore, as for the output from the image sensor 2, as shown in FIG. 2D, two types of images having different exposure times are output in time series within one horizontal period. This time series output is output by the signal processing circuit 3 in a synchronized manner such as the long exposure image output of FIG. 2E and the short exposure image output of FIG. Depending on the configuration of the image sensor 2, it is also possible to obtain the long-time exposure image output of FIG. 2E and the short-time exposure image output of FIG.
[0014]
Since the two types of images have different levels as they are, the multiplier 4 first multiplies the short exposure image by the ratio (exposure ratio) between the long exposure time and the short exposure time. In order to replace the saturated pixel portion in the long-exposure image with the corresponding portion of the short-exposure image, the saturated pixel is detected using the comparator 5, and the predetermined switching level is used. The large pixel portion is replaced by the image changeover switch 6 with a pixel obtained by multiplying the short-time exposure image by an integer. The input / output relationship is as shown in FIG. Note that the image synthesizing method in the imaging apparatus 1 can also be performed by adding an offset to the short exposure image side and adding it to the long exposure image.
[0015]
In the image composition method in the imaging apparatus 1, the long exposure image in FIG. 2E and the short exposure image in FIG. 2F are outputs of the same pixel of the image sensor 2, but the exposure time is shown in FIG. 2 (B), which is controlled by applying the readout pulse, there is no problem when imaging a stationary subject as shown in FIG. 4, but when a moving subject is imaged. As shown in FIG. 5, there is a deviation between the long exposure image and the output of the short exposure image. In the case of FIG. 5, the subject is moving from right to left.
[0016]
Looking at the image output for one horizontal period at the position 1-1 ′ in FIG. 5, the output waveforms are as shown in FIGS. 3 (A) and 3 (B). In addition, the comparative output of the image when the long-time exposure image is equal to or higher than a predetermined switching level has an output waveform shown in FIG.
[0017]
When the long-time exposure image and the short-time exposure image are switched by the image changeover switch 6 based on the comparison output of the long-time exposure image and the short-time exposure image, an image output shown in FIG. In the image output of FIG. 3 (D), the deviation of the image generated in the moving subject due to the difference in exposure time is shown as a discontinuous point on the left side of the subject as shown in the composite image in FIG. Then it will appear as an afterimage.
[0018]
Therefore, in the composite image by the conventional imaging device 1, a white edge due to the discontinuous point occurs on the left side of the subject, and the size of the subject becomes larger than the original size due to the afterimage on the right side. Note that where each defect occurs depends on the brightness of the subject and the direction of movement.
[0019]
Next, an image composition method that has been performed in order to eliminate the problems in the imaging apparatus 1 will be described.
[0022]
Further , as shown in FIG. 1, the imaging device 10 includes an imaging device 11, a timing generator 12 that applies a readout pulse to the imaging device 11, a signal processing circuit 13, and a short-time exposure image according to the polarity of the difference. And an image changeover switch (image switching means) 14 for appropriately switching data corresponding to the long exposure image and a multiplier (multiplication means) for multiplying the short exposure image by an exposure ratio between the short exposure image and the long exposure image. ) 15, a subtractor (subtraction means) 16 for calculating the difference between the data of the short-time exposure image multiplied by the exposure ratio by the multiplier 15 and the long-time exposure image, and the long-time exposure image data equal to or higher than the switching level. First comparator (first comparison means) 17 that performs detection, detects pixels having a level higher than a preset reference level from data from the subtractor A second comparator (second comparison means) 18, a third comparator (third comparison means) 19 for detecting positive data from the data from the subtractor, and a first comparator. 17 and the first logical circuit (OR circuit) 20 that outputs the logical sum of the output data of the second comparator 18 and the logical product of the output data of the first logical circuit 20 and the third comparator 19 A second logic circuit (AND circuit) 21 that uses the output of the second logic circuit 21 as the control signal 22 of the image changeover switch 14 and uses a short exposure image and a long exposure image. This data is selectively used for each pixel to create a composite image.
[0023]
As in the case of the imaging device 1, the output of the imaging device 11 is applied twice within one field period by applying a readout pulse 23 as shown in FIG. 2B from the timing generator 12. It is designed to be read out.
[0024]
Similarly, the interval between the readout pulses 23 is divided into two different length regions, ie, a long exposure period and a short exposure period, within one field period. Of these two regions having different lengths, the accumulated voltage output during the long exposure period corresponds to the long exposure image data 24, and the accumulated voltage output during the short exposure period corresponds to the short exposure image data 25.
[0025]
Accordingly, as shown in FIG. 2D, the output from the image sensor 11 outputs two types of image data having different exposure times within one horizontal period in time series. This time series output is synchronized and output by the signal processing circuit 13 as the long exposure image data 24 in FIG. 2E and the short exposure image data 25 in FIG. Depending on the configuration of the image sensor 11, it is also possible to obtain the long exposure image data 24 and the short exposure image output data 25 that are synchronized from the beginning.
[0026]
Next, the exposure ratio of each data of the long-time exposure image and the short-time exposure image is calculated in advance by appropriate means (not shown), and the data 15 of the short-time exposure image is multiplied by the exposure ratio by the multiplier 15. The subtractor 16 subtracts the long exposure image data 24 from the output data 26 from the multiplier 15 multiplied by the exposure ratio. The output data 27 from the subtracter 15 is subtraction result data shown in FIG.
[0027]
In addition, the first comparator 17 detects only long-exposure image data 24 that is higher than a preset switching level. This detected data is output data 28 from the first comparator 17 shown in FIGS. 1 and 3F. The switching level is an output level of a specific long exposure image that is a point for switching data between the long exposure image and the short exposure image.
[0028]
Then, a pixel having a level larger than a reference level set in advance by the second comparator 18 is detected. As the reference level, for example, a switching level used by the first comparator 16 is multiplied by the exposure ratio. As a result, the difference between the output data 29 from the second comparator 18 shown in FIG. 1, that is, the short-time exposure image multiplied by the exposure ratio shown in FIG. It is a waveform which shows the output of the pixel which is.
[0029]
By the way, the output data 29 of the second comparator 18, that is, the pixel whose comparison output 29 is 1, has a sufficient output level for the short-time exposure image. A logical sum of the output data 29 and the output data 28 of the pixel equal to or higher than the switching level of the long exposure image shown in FIG. 3F obtained by the first comparator 16 is obtained by the OR circuit 20 shown in FIG. calculate. The output data 30 of the OR circuit 20 has a waveform indicating the logical sum output shown in FIG. By using this OR output data 30, even if the long-exposure image data 24 is below the switching level, the short-exposure image data 25 is stored if the short-exposure image data 25 has sufficient output. As a result, the afterimage phenomenon is corrected.
[0030]
The third comparator 19 detects a pixel for which the subtraction result in the subtracter 16 is positive. The output data 31 of the third comparator 19 has a waveform shown in FIG. A pixel whose comparison output 31 is 0 is a pixel whose short-time exposure image data 25 is not at a sufficient level even when the long-time exposure image data 24 is equal to or higher than the switching level.
[0031]
Then, a logical product of the output data 31 of the third comparator 19 and the output data 30 of the OR circuit 20 shown in FIG. 3H is calculated by the AND circuit 21 shown in FIG. The logical product output data of the AND circuit 21 is the waveform data shown in FIG. 3 (J), that is, the output waveform of the control signal 22 for controlling the switching operation of the image switch 14 in the present invention.
[0032]
By using the control signal 22, the long-exposure image is used for pixels whose short-exposure image data 25 is not at a sufficient level even if the long-exposure image data 24 is higher than the switching level. The discontinuous point of the image is corrected.
[0033]
Using the control signal 22 obtained by the above method as an input of the image changeover switch 14 shown in FIG. 1, the synthesized image output is as shown in FIG. The composite image shown in FIG. 3 (E) has a waveform in which discontinuous points and afterimage components are removed (corrected) as compared with the output waveform of the composite image in the prior art shown in FIG. 3 (D). I understand. Accordingly, the composite image according to the present invention is an image in which the edge which is a discontinuous point on the left side of the subject and the afterimage on the right side are eliminated as shown in FIG.
[0034]
Further, instead of the method of switching the image at a specific level using the image switching switch 19 or the like, the composition ratio of the long exposure image and the short exposure image is gradually changed according to the image level. It is possible to change the method. Also in this method, the same method as the above method can be applied.
[0035]
Furthermore, the present invention is effective not only for a moving subject as described above, but also for a shift between a long-exposure image and a short-exposure image caused by engineering reasons such as flare. is there.
[0036]
As described above, the present invention takes the data difference between the short-time exposure image and the long-time exposure image multiplied by the exposure ratio of the pixels at the same position, and the difference exceeds the preset reference level. In this case, it is determined that there is a deviation in the image, and depending on the polarity of the difference, a long exposure image is used instead of the short exposure image, or vice versa. Is used to correct the image discontinuity and the afterimage of the subject caused by the misalignment of the subjects of the two images, so as to suppress the occurrence of image defects. Thus, discontinuous points and afterimages are eliminated, and the output quality of the composite image can be improved.
[0037]
Therefore, when the imaging apparatus of the present invention is applied to, for example, a surveillance camera, it is possible to discriminate images at discontinuous points, which occurs when an imaging apparatus that performs conventional image composition is used for a surveillance camera. The difficulty and the problem of missing images in the afterimage portion are less likely to occur, and the monitoring system can be improved.
[0038]
In addition, since continuity occurs in the image, when data processing is performed later using an image processing circuit, discontinuous points may cause erroneous output in image detection or the like, and the correct subject size can be obtained by an afterimage. It becomes possible to solve the problem of disappearing.
[0039]
Furthermore, compared to the method of correcting two types of images using motion detection, etc., it is possible to suppress an increase in unnecessary circuits, such as the need for field memory, and a remarkable effect can be achieved with a remarkably small circuit. Can be obtained.
[0040]
Furthermore, since it is possible to correct two images that are close in exposure time, by applying the image composition method of the present invention in multiple stages, it is possible to obtain an image output having a plurality of exposure times within one field. Even in such a system, it is possible to easily obtain a composite image in which defects such as discontinuities and afterimages are corrected.
[0041]
It should be noted that the specific shapes and structures of the respective parts shown in the above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention is thereby limited. It should not be interpreted in a limited way.
[0043]
The image pickup apparatus of the present invention includes an image pickup device, a signal processing circuit, an image switching unit that appropriately switches corresponding data between a short-exposure image and a long-exposure image according to the polarity of the difference, a short-exposure image, and a long-exposure image. Multiplying means for multiplying the short-exposure image by the exposure ratio with the exposure image, and subtracting means for calculating a data difference between the short-exposure image multiplied by the exposure ratio by the multiplication means and the long-exposure image, and switching From first comparison means for detecting long-exposure image data above the level, second comparison means for detecting a pixel having a level higher than a preset reference level from data from the subtractor, from the subtractor The third comparison means for detecting data having a positive subtraction result from the first data, and the first output of the logical sum of the output data of the first comparison means and the second comparison means And a second logic circuit that outputs a logical product of output data of the first logic circuit and the third comparator, and the output of the second logic circuit is used as a control signal for the image switching means. Since the composite image is created by selectively using the data of the short-exposure image and the long-exposure image on a pixel-by-pixel basis, the problem caused by the misalignment of the composite image is solved. It is possible to eliminate the occurrence of afterimages and discontinuous points that occur when the image is taken, and the output quality of the composite image can be improved.
[Brief description of the drawings]
FIG. 1 shows an embodiment of the present invention together with FIGS. 2 to 6, and is a block diagram schematically showing a configuration of a main part of an imaging apparatus.
FIG. 2 is a diagram illustrating waveforms of various outputs from the image sensor, readout pulses for controlling image output from the image sensor, long-time exposure image data, short-time image data, and the like.
FIG. 3 is a diagram illustrating a waveform of data obtained by performing various processes based on long-time exposure image data and short-time exposure image data.
FIG. 4 is a diagram illustrating an image obtained by photographing a subject in a stationary state.
FIG. 5 is a diagram in which a long-time exposure image and a short-time exposure image obtained by imaging a subject moving from right to left with different exposure times in a time-sharing manner are superimposed.
FIG. 6 is a diagram showing a composite image in the present invention.
FIG. 7 shows a conventional image composition method and image pickup apparatus shown in FIG. 8 and FIG. 9 for comparison with the present invention, and this figure shows a composite image.
FIG. 8 is a diagram schematically illustrating a configuration of an example of an imaging apparatus.
FIG. 9 is a diagram illustrating a relationship between input and output of a long-time exposure image and a short-time exposure image during image composition.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Imaging device, 11 ... Imaging device, 13 ... Signal processing circuit, 14 ... Image switching means, 15 ... Multiplication means, 16 ... Subtraction means, 17 ... 1st comparison means, 18 ... 2nd comparison means, 19 ... 3rd comparison means 20 ... 1st logic circuit, 21 ... 2nd logic circuit, 22 ... Control signal, 24 ... Long exposure image data, 25 ... Short exposure image data

Claims (1)

An image sensor, a signal processing circuit, image switching means for appropriately switching data corresponding to the short-exposure image and the long-exposure image according to the polarity of the difference, and an exposure ratio between the short-exposure image and the long-exposure image Multiplying means for multiplying the short-time exposure image, subtracting means for calculating the data difference between the short-time exposure image multiplied by the exposure ratio by the multiplication means and the long-time exposure image, and long-time exposure equal to or higher than the switching level First comparison means for detecting image data, second comparison means for detecting a pixel having a level higher than a preset reference level from the data from the subtractor, and a subtraction result from the data from the subtractor Third comparison means for detecting positive data, a first logic circuit for outputting a logical sum of output data of the first comparison means and the second comparison means, and the first logic circuit And a second logic circuit for outputting a logical product of the output data of the logic circuit and the third comparator,
The output of the second logic circuit is used as a control signal for the image switching means, and the composite image is created by selectively using the data of the short exposure image and the long exposure image for each pixel. An imaging apparatus characterized by the above.

Images