JP5084473B2 - Infrared imaging device - Google Patents

Description

  The present invention relates to an infrared imaging device, and in particular, performs A / D conversion that effectively uses a dynamic range in A / D conversion for converting an analog signal output from an imaging device into a digital signal.

The infrared imaging device visualizes infrared rays emitted from a subject, and the difference in temperature rise caused by absorbing infrared rays becomes the shading of an image. The infrared rays emitted from the subject are collected by the lens and formed on the image sensor. Elements that detect infrared rays are arranged on a two-dimensional plane in the imaging element, and each element outputs a voltage corresponding to the infrared intensity.
In a conventional infrared imaging device, the voltage output from the imaging device is amplified by a preamplifier circuit and converted into a digital signal, and then fixed pattern noise is removed, and the analog image signal is output after being converted into an analog video signal ( Patent Document 1).

JP 2000-88644 (paragraphs 0004 to 0005)

  In an infrared imaging device, a temperature change that occurs according to the intensity of infrared rays emitted from a subject is minute with respect to variations in output of the imaging device (fixed pattern noise (FPN)). Therefore, if the fixed pattern noise is removed after digital conversion of the signal including fixed pattern noise as in the conventional infrared imaging device, the effective dynamic range of the signal is reduced due to the fixed pattern noise, and the quantization noise is reduced. There is a problem that the image quality is degraded due to an increase in the image quality. Also, because of the processing required for updating the fixed pattern noise, it has not been possible to efficiently perform imaging.

The infrared imaging device of the present invention is
Imaging means having effective pixels sensitive to a predetermined wavelength range and ineffective pixels having no sensitivity;
Imaging means for forming an image of a predetermined wavelength region component on the imaging surface of the imaging means;
Shielding means for shielding incidence of the predetermined wavelength region component to the imaging means;
Difference means for removing analog fixed pattern noise from the output signal of the imaging means;
Digital conversion means for converting the video signal output from the difference means into a digital signal and outputting the digital video signal;
FPN acquisition means for acquiring fixed pattern noise based on the digital video signal;
Analog conversion means for converting the fixed pattern noise acquired by the FPN acquisition means into an analog signal and outputting the analog fixed pattern noise;
In an infrared imaging device comprising the shielding means and a control means for controlling the operation of the FPN acquisition means,
The FPN acquisition means
A frame memory for storing the fixed pattern noise;
A mixing means for calculating a new fixed pattern noise by mixing the digital video signal and the fixed pattern noise stored in the frame memory;
Updating means for updating the fixed pattern noise stored in the frame memory with a new fixed pattern noise calculated by the mixing means;
The control means includes
Shielding control means for making the shielding means open or shielded at a predetermined timing;
The fixed pattern noise acquisition start timing is determined based on the invalid pixel digital video signal, and the fixed pattern noise acquisition completion timing is determined based on the effective pixel digital video signal and the invalid pixel digital video signal. Detecting means for detecting
Based on a detection signal output from the detection means, a timing for generating a shielding operation timing signal of the shielding control means, and an update operation timing signal for setting the update circuit to an update execution state or an update stop state are generated. And a timing generation means.

According to the present invention, by converting the difference between the signal output from the image sensor and the output signal of the frame memory in which the component corresponding to the fixed pattern noise is recorded, a more dominant fixed pattern noise is generated. The subtracted signal will be digitally converted, and the dynamic range of the analog-to-digital (A / D) converter can be used effectively. This reduces the quantization noise due to digital conversion and improves image quality. To do.
In addition, by automatically detecting the start and end timing of fixed pattern noise acquisition, the number of frames (operation time) required to acquire fixed pattern noise is minimized, and the calibration process is executed efficiently. be able to.

Embodiments of the present invention will be described below with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of the infrared imaging apparatus according to Embodiment 1 of the present invention.

The illustrated infrared imaging device includes a lens 1, a shutter 2, an imaging device 3, a difference circuit 4, an amplifier circuit 5, an A / D converter 6, a mixing circuit 7, an update circuit 8, and a frame memory. 9, a D / A converter 10, a shutter control circuit 11, a timing generation circuit 12, a detection circuit 13, an integration circuit 21, and a gain control circuit 22.
The mixing circuit 7, the update circuit 8, and the frame memory 9 constitute an FPN acquisition unit 30.
The shutter control circuit 11, the timing generation circuit 12, and the detection circuit 13 constitute a control means 40.

The image sensor 3 has sensitivity in a predetermined wavelength range, for example, a wavelength range of approximately 8 to 14 micrometers.
The lens 1 is used as image forming means for forming an image of the predetermined wavelength band component on the image pickup surface of the image pickup device 3.
The amplifier circuit 5 amplifies the imaging signal output from the imaging element 3.
The A / D converter 6 is used as digital conversion means for converting the analog output signal of the amplifier circuit 5 into a digital signal and outputting a digital image pickup signal.
First, an operation in a state where the shutter 2 is opened as a standard operation state will be described.

  Infrared light emitted from the subject is collected by the lens 1 and forms an image on the image sensor 3 when the shutter 2 is open. Pixels that detect infrared rays (effective pixels) are arranged on the two-dimensional plane in the imaging device, and a signal that changes in accordance with the infrared intensity is obtained from each pixel.

  The image sensor 3 includes pixels having no sensitivity (invalid pixels) in addition to pixels having sensitivity to infrared rays (effective pixels). FIG. 2 shows an example of the waveform of the output signal from the image sensor during the horizontal period. One horizontal period Ph includes a period Pe in which an output signal from an effective pixel appears, a period Pf in which an output signal from an invalid pixel appears, and a part of the period Pf in which an output signal from an invalid pixel appears is a blanking period Pb. . In the illustrated waveform, there is an output from the invalid pixel before and after the output from the effective pixel. The invalid pixel has no sensitivity, but the level of the output signal from the invalid pixel fluctuates due to a temperature change such as the surrounding environment or self-heating of the element.

  The shutter control circuit 11 performs opening / closing control of the shutter 2. The shutter control circuit 11 opens (opens) or closes (shields) the shutter 2 in accordance with the shielding operation timing signal ST sent from the timing generation circuit 12. When the shutter 2 is opened and opened, infrared rays emitted from a subject in front of the lens 1 enter the image sensor 3. When the shutter 2 is closed and blocked, the infrared rays collected by the lens 1 are blocked from being incident on the image pickup device 3, so that the image pickup device 3 picks up an image corresponding to fixed pattern noise that does not include a signal component. A signal is output.

  The difference circuit 4 obtains a difference between the image signal IN that is an output signal of the image sensor 3 and the analog fixed pattern noise O obtained by analog conversion of the fixed pattern noise FPN stored in the frame memory 9 to obtain a video signal. S is generated. The video signal S generated by the difference circuit 4 is amplified by the amplifier circuit 5 and then converted into a digital video signal AD by the A / D converter 6.

  The mixing circuit 7 mixes the fixed pattern noise FPN stored in the frame memory 9 and the digital video signal AD obtained from the A / D converter 6 to generate a new fixed pattern noise D.

FIG. 3 shows an example of the mixing circuit. The illustrated mixing circuit 7 includes coefficient generation circuits 7a and 7b, multipliers 7c and 7d, and an adder 7e. The coefficient generation circuit 7a sets the first coefficient 1 / K (K is a number greater than 1). The second coefficient generation circuit 7b sets the second coefficient (K-1) / K.
The first multiplier 7c multiplies the digital video signal AD by the first coefficient 1 / K and outputs AD / K. The second multiplier 7d multiplies the fixed pattern noise FPN by the second coefficient (K-1) / K and outputs FPN · (K-1) / K. The adder 7e adds the output AD / K of the first multiplier 7c and the output (FPN · (K−1) / K) of the second multiplier 7d and outputs the result, as shown in FIG. The mixing circuit 7 calculates a new fixed pattern noise by mixing the digital video signal AD and the fixed pattern noise stored in the frame memory at a predetermined ratio.

  As described above, the fixed pattern noise FPN attenuated by (K−1) / K times may be supplied to the adder 7e as it is instead of being supplied to the adder 7e. That is, the digital image signal AD attenuated 1 / K times and the fixed pattern noise FPN may be added.

  FIG. 4 shows another example of the mixing circuit. The mixing circuit 7 shown is similar to the mixing circuit of FIG. 3, but a reference value generation circuit 7f and a subtractor 7g are added. The reference value generation circuit 7f outputs a reference value RF. The subtraction circuit 7g subtracts the reference value RF from the digital video signal AD and outputs a subtraction result AE. The first multiplier 7c multiplies the output AE of the subtraction circuit 7g by the first coefficient 1 / K from the coefficient generation circuit 7a, and outputs AE / K. The second multiplier 7d multiplies the fixed pattern noise FPN by the second coefficient (K-1) / K from the coefficient generation circuit 7b, and outputs FPN · (K-1) / K. The adder 7e adds and outputs the output AE / K of the first multiplier 7c and the output (FPN · (K−1) / K) of the second multiplier 7d. The coefficient generation circuits 7a and 7b The mixing circuit 7 shown in Fig. 4 subtracts the reference value RF from the digital video signal AD and then stores the fixed pattern noise FPN stored in the frame memory. Are mixed at a predetermined ratio to calculate new fixed pattern noise.

  As described above, the fixed pattern noise FPN attenuated by (K−1) / K times may be supplied to the adder 7e as it is instead of being supplied to the adder 7e. That is, the signal AE attenuated 1 / K times and the fixed pattern noise FPN may be added.

  The fixed pattern noise FPN used for addition in the mixing circuit 7 is equal to the signal before analog conversion of the analog fixed pattern noise O subtracted from the output signal IN of the image sensor 3 in the difference circuit 4 one frame period before.

The new fixed pattern noise D calculated by the mixing circuit 7 is sent to the update circuit 8.
The shielding operation timing signal ST output from the timing generation circuit 12 is also used as an update control signal.
The update circuit 8 updates the frame memory 9 in accordance with the shielding operation timing signal (update control signal) ST output from the timing generation circuit 12. When the shutter 2 is in the shielding state, the data in the frame memory 9 is updated with a new fixed pattern noise D generated every frame period. When the shutter 2 is in the open state, the data in the frame memory 9 is not updated.
The new fixed pattern noise D stored in the frame memory 9 is used for the mixing operation with the digital video signal AD in the mixing circuit 7 of the next frame. Also, after being converted into an analog signal by the D / A converter 10, it is sent to the difference circuit 4 and used for a difference calculation with the imaging signal of the next frame.

  In the present embodiment, the amplification factor of the amplifier circuit 5 is controlled so that the integrated value obtained by integrating the video signals of the pixels output from the A / D converter 6 for each frame is substantially constant.

  For this reason, an amplifier circuit 5 that amplifies the video signal S in accordance with the gain setting value (variable gain amplification means) is used, and the video signal of each effective pixel output from the A / D converter 6 is converted into a frame. An integration circuit 21 that integrates every time, and a gain control circuit 22 that controls the gain setting value of the amplifier circuit 5 so that the integration value output from the integration circuit 21 substantially matches a predetermined reference value (shown in FIG. 2). And are provided. For example, the difference between the integral value and the reference value is obtained in the gain control circuit 22 and the gain of the amplifier circuit 5 is controlled so that the difference becomes zero.

  When the mixing circuit 7 shown in FIG. 4 is used, the reference value used in the gain control circuit 22 is preferably the same as the reference value used in the mixing circuit in FIG. It may be a value.

  The digital video signal AD output from the A / D converter 6 is also sent to the detection circuit 13. The detection circuit 13 constantly monitors the change in the state of the digital video signal of the effective pixel portion and the digital video signal of the invalid pixel portion, and determines the timing for starting the acquisition process of the fixed pattern noise (detects the timing to start) In addition, during the process of acquiring the fixed pattern noise, it is detected that the acquisition of the fixed pattern noise has been completed, and the detection signal TD indicating that the timing to start and the timing to end is detected in this way This is sent to the generation circuit 12. For example, when the fixed pattern noise acquisition process is started, the detection signal TD is set to the first value (first content), and when the fixed pattern noise acquisition process is completed, the detection signal TD is set to the first signal. It is assumed that the second value (content) is different from the value (content).

Hereinafter, a method for determining the timing for starting the acquisition process of fixed pattern noise will be described.
Since effective pixels of the image sensor 3 are sensitive to infrared rays and are affected by infrared rays from the subject, it is difficult to separate and extract only changes in fixed pattern noise of the image sensor 3. On the other hand, since the invalid pixel has no sensitivity to infrared rays and is not affected by the subject, when the fixed pattern noise changes due to some temperature change, the change appears as unevenness in the video signal of the invalid pixel.
Therefore, it is necessary to always observe values that show variations such as dispersion and standard deviation of the video signal level of invalid pixels, maximum values, minimum values, etc., and reacquire fixed pattern noise when the level reaches a certain value or higher. It is only necessary to generate a signal that determines and prompts the start of the reacquisition process. In other words, the acquisition start timing of the fixed pattern noise may be determined based on the digital video signal of the invalid pixel.

For example, the level of the digital video signal of the invalid pixel at the time when the acquisition operation of the fixed pattern noise is completed is stored as a reference signal, and is continuously compared with the digital video signal of the invalid pixel being captured. When a difference greater than the value occurs, it may be determined that it is necessary to reacquire fixed pattern noise.
The reference signal to be stored may not be the digital image signal itself of the invalid pixel, but instead may be an average value, a maximum value, a minimum value, a standard deviation or variance indicating variation, and the like. In other words, the average value, maximum value, minimum value, and standard deviation and variance indicating the dispersion of digital video signals of invalid pixels at the time when the acquisition operation of fixed pattern noise is completed are stored as reference signals, and invalid during imaging. Continue to compare with the standard deviation or variance indicating the average value, maximum value, minimum value, and dispersion of the digital video signal of the pixel. If the result of the comparison shows that a difference greater than a certain value occurs, it is determined that it is necessary to reacquire fixed pattern noise. It is also good to do.

  Re-acquisition of fixed pattern noise may be performed at regular time intervals, but the required timing varies depending on the surrounding environment, so there is a problem that re-acquisition is executed when it is unnecessary or re-acquisition is not executed when necessary. is there. In addition, there is a method in which the user views the image and executes reacquisition as necessary, but the convenience is inferior. It is possible to efficiently update the fixed pattern noise by automatically detecting the necessity of reacquiring the fixed pattern noise and automatically starting the execution of the reacquisition.

Next, a method for detecting completion of acquisition of fixed pattern noise during detection processing of fixed pattern noise (detecting timing at which acquisition processing is completed) will be described.
FIGS. 5A to 5E show the video signal S at times T0 to T4, respectively. FIGS. 5F to 5J show the output signal AD of the A / D converter at times T0 to T4, respectively. Show. Time elapses in the order of times T0, T1, T2, T3, and T4.

  When the fixed pattern noise acquisition process is started, the fixed pattern noise component is gradually removed from the video signal S as shown in FIGS. During this time, the fixed pattern noise is gradually accumulated in the frame memory 9 and eventually the fixed pattern noise stored in the frame memory 9 converges. This state is referred to as a “fixed pattern noise acquisition has been completed” state.

The time (number of frame periods) required to complete acquisition of fixed pattern noise depends on the difference in resolution between the A / D converter 6 and the D / A converter 10 and the calculation method (mixing ratio, etc.) in the mixing circuit 7. Can be predicted to some extent.
However, depending on the element, there are differences in the amplitude of fixed pattern noise, differences in characteristics of random noise, and the like, and the time required to capture the fixed pattern noise into the frame memory 9 (the number of necessary frame periods) differs. In other words, the capture may end earlier than expected (convergence converges faster), and conversely, it may take longer than predicted.

Therefore, during the acquisition process of the fixed pattern noise, the digital video signal is monitored, and it is determined that the acquisition of the fixed pattern noise is completed based on the result, and the acquisition process is terminated.
For example, the average value of the effective pixels and the average value of the invalid pixels may be constantly compared, and it may be determined that the acquisition of the fixed pattern noise is completed when the difference becomes less than a certain value.
Instead, it may be determined that the acquisition of the fixed pattern noise is completed when the difference between the maximum value and the minimum value of the video signal becomes less than a certain value regardless of whether the pixel is valid or invalid.
Furthermore, when acquiring fixed pattern noise based on the level of invalid pixels, the maximum and minimum values of the digital video signal of valid pixels are compared with the average of the digital video signals of invalid pixels, and the difference is It may be determined that acquisition of the fixed pattern noise is completed when the value becomes less than a certain value.

  If there is no need to match the signal levels of the effective and invalid pixels, only the maximum and minimum values of the effective pixels are observed and fixed when the difference between the maximum and minimum values is less than a certain value. It may be determined that the pattern noise acquisition has been completed.

Furthermore, the maximum value and the minimum value of the digital video signal of the effective pixel may be acquired, and it may be determined that the acquisition of the fixed pattern noise is completed when the difference becomes less than a certain value.
Furthermore, the average value of the digital video signal of the effective pixel is not the maximum value or the minimum value of the digital video signal of the effective pixel, and the difference is less than a certain value when compared with the average of the digital video signal of the invalid pixel. Sometimes, it may be determined that acquisition of fixed pattern noise has been completed.

  Further, it may be determined that the acquisition of the fixed pattern noise is completed by detecting that the variation in the video signal of the effective pixel is less than a certain value.

  The detection circuit 13 transmits a detection signal TD indicating that the acquisition processing end timing has been detected to the timing generation circuit 12 when the completion of acquisition of the fixed pattern noise is detected by any of the above methods (detection). The signal TD is set to a value (second value) indicating that the acquisition process is completed).

The timing generation circuit 12 determines the value (contents) of the shielding operation timing signal ST according to the detection signal TD received from the detection circuit 13.
When the detection signal TD has a value (first value) indicating that the acquisition process of fixed pattern noise is started, the shielding operation timing signal ST is a value (first value) for putting the shutter in a shielding state. 1 value).
When the detection signal TD is a value (second value) indicating that the acquisition process of the fixed pattern noise is completed, the shielding operation timing signal ST is a value (second value) for opening the shutter. Value).

  The timing generation circuit 12 also transmits the shielding operation timing signal ST to the update circuit 8 as an update control signal. The update circuit 8 updates the data stored in the frame memory 9 in accordance with the shielding operation timing signal ST sent from the timing generation circuit 12. When the shutter 2 is controlled to the shielding state (when the shielding operation timing signal ST is the first value), the data in the frame memory 9 is updated with the new fixed pattern noise D sent from the mixing circuit 7. To do. When the shutter is controlled to the open state (when the shielding operation timing signal ST is the second value), the data in the frame memory 9 is not updated.

  The difference circuit 4 uses the fixed pattern noise (analog fixed pattern noise O obtained by D / A conversion of FPN) accumulated in the frame memory 9 when the shutter 2 is in the shielding state, from the video signal IN when the shutter 2 is in the open state. By subtracting, the fixed pattern noise is removed from the imaging signal output from the imaging device 3.

  In this way, by digitally converting the signal obtained by subtracting the analog fixed pattern noise O from the image pickup signal IN output from the image pickup device 3, the image pickup signal IN output from the image pickup device 3 is all digitally converted. The dynamic range of the / D converter 6 can be used effectively. In particular, this is effective in an image sensor in which the amplitude of a signal indicating the contrast of an image is small with respect to the amplitude (variation) of fixed pattern noise.

  In such a configuration, the time required for accumulating the fixed pattern noise in the frame memory 9 differs depending on the degree of the fixed pattern noise. Furthermore, since the change in the fixed pattern noise is affected by the surrounding environment, the frequency at which the fixed pattern noise re-acquisition process is required is not constant. By automatically detecting the start and end of fixed pattern noise acquisition processing from the levels of the video signals of valid pixels and invalid pixels, efficient fixed pattern noise acquisition processing can be performed.

Next, the resolution of the A / D converter 6 and the D / A converter 10 will be described.
When the acquisition process of the fixed pattern noise is completed, the video signal S that is the difference between the imaging signal IN and the analog fixed pattern noise O is input to the A / D converter 6, but the D / A converter 9 The fixed pattern noise FPN restored by the FPN acquisition means 30 is input. Therefore, by making the resolution of the D / A converter 10 higher than the resolution of the A / D converter 6, the dynamic range of the A / D converter 6 can be utilized to the maximum extent.

  At this time, when the mixing circuit 7 of FIG. 3 is used, the fixed pattern noise is acquired by adding the captured digital video signal AD and the data FPN stored in the frame memory 9 at a predetermined ratio. . Although only a part of the fixed pattern noise can be captured with only one frame, the fixed pattern noise can be restored by performing addition processing repeatedly for several frames.

  The difference in resolution between the D / A converter 10 and the A / D converter 6 only needs to be the magnitude of the amplitude difference between the fixed pattern noise and the signal component. For example, when the amplitude (variation) of the fixed pattern noise is twice the amplitude of the signal indicating the contrast of the video, the resolution of the A / D converter 6 may be set to half that of the D / A converter 10. . That is, when the A / D converter 6 is 8 bits, a video signal in which the FPN component is accurately removed from the image pickup signal can be obtained by using a 9-bit or more D / A converter, and the A / D converter 6 Can be applied to the amplitude of the signal indicating the contrast of the image.

  Further, since the A / D converter 6 and the D / A converter 10 are different circuits even if the bit precision is the same, there is a difference in the amplitude and linearity of the output signal. Therefore, if the digital video signal AD is added to the fixed pattern noise FPN as it is without being attenuated, the signal diverges depending on the characteristic difference between the A / D converter 6 and the D / A converter 10 and is difficult to converge.

  On the other hand, as shown in FIG. 3, if the mixing circuit 7 attenuates the amplitudes of the digital video signal AD and the fixed pattern noise FPN and then adds them to the fixed pattern noise FPN, the A / D converter 6 and D The characteristic difference of the / A converter 10 can be absorbed, and signal divergence can be prevented. Further, the fixed pattern noise signal at the time of shielding can be converged with high accuracy.

  In the detection of the start and end timings of the fixed pattern noise acquisition process in the detection circuit 13, the digital video signal of the pixel with an abnormal operation may not be used. An abnormally operated pixel (defective pixel) refers to a pixel that is an effective pixel but has no sensitivity, a pixel whose sensitivity variation is significantly different from other pixels, and a blinking pixel. At that time, a pixel whose fixed pattern noise value is extremely large or extremely small compared to other pixels (for example, from a reference value) is determined as a defective pixel, and the video signal from the defective pixel is It is not used for calculating the average value, maximum value, minimum value, variation, etc. used for detection of acquisition start and end timings. For example, when the reference value used in the gain control means 22 is deviated by a certain value or more, it may be determined that the pixel is extremely large or extremely small compared to other pixels. In this case, on the condition that the deviation from the reference value of the fixed pattern noise for each pixel stored in the frame memory is less than a certain value, the detection circuit 13 converts the digital video signal of the pixel into the fixed pattern noise. This is used to detect the timing of the start and end of acquisition. Although it is necessary to determine whether or not the pixel is a defective pixel, it is possible to prevent erroneous detection of starting and ending acquisition of fixed pattern noise and unauthorized operation.

  Next, the effective utilization of the dynamic range of the A / D converter 6 will be described with reference to the drawings, with respect to the relationship between the magnitude of each signal level and the dynamic range of digital conversion.

  FIGS. 6A to 6C show imaging signals output from the imaging device 3 when an image having a high-temperature subject HS in a low-temperature background LB is taken, as shown in FIG. 6A. A part of IN (FIG. 6B) and a part of the waveform of the video signal S after removing the fixed pattern noise (FIG. 6C) are shown. As can be seen from the figure, the infrared imaging element has a feature that the output variation of each pixel is large with respect to the temperature change caused according to the infrared intensity emitted from the subject.

7, 8 and 9 show signals during a part of one horizontal period Ph shown in FIG.
FIG. 7 shows the difference between the imaging signal IN output from the imaging device 3, the analog fixed pattern noise O obtained by analog conversion of the fixed pattern noise FPN read from the frame memory 9, and the difference between the imaging signal IN and the analog fixed pattern noise O. 3 is a schematic diagram showing a relationship of an analog video signal S. FIG. The horizontal axis represents the position of each pixel on the image sensor, and the vertical axis represents the output level of each signal.

When the image pickup signal IN output from the image pickup device 3 is directly converted into a digital signal, the level in the range indicated by the symbol Ra in FIG. 7 is assigned to the effective dynamic range of the digital conversion.
However, in the infrared imaging device, the output variation for each element, that is, the proportion of fixed pattern noise is large in the output signal level, and the fluctuation amount of the output signal due to the difference in the temperature of the subject is small. There is a problem that the range cannot be used effectively.

  Therefore, when the difference between the output signal IN and the output signal FPN (analog-converted signal OPN) of the frame memory 9 is taken and the signal S excluding the component corresponding to the fixed pattern noise FPN is digitally converted, the code in FIG. The range indicated by Rc is assigned to the dynamic range of the A / D converter 6, the dynamic range can be used effectively, and a new pattern noise FPN with reduced quantization noise can be obtained. .

  FIG. 8 is a schematic diagram showing the relationship between the output signal IN from the image sensor and the new fixed pattern noise D ′ obtained when the signal IN is directly converted to digital. Since the wide range indicated by the symbol Ra in FIG. 7 is assigned to the dynamic range of the A / D converter 6, the quantization error becomes large.

  FIG. 9 shows a video signal S, a digital signal AD obtained by digitally converting the video signal S, a fixed pattern noise FPN before analog conversion of the analog fixed pattern noise O used for calculating the video signal S, and the present embodiment. It is the schematic which showed the relationship of the new fixed pattern noise D obtained by the form.

  When the video signal S is digitally converted, the narrow range indicated by the symbol Rc in FIG. 7 is assigned to the dynamic range of the digital conversion, so that the quantization error of the obtained digital signal AD is reduced. As a result, the quantization error of the new fixed pattern noise D obtained by adding the fixed pattern noise FPN to the signal obtained by attenuating the digital signal AD to 1 / K by the attenuation circuit 40 is also reduced.

  Next, an example of the processing procedure will be described with reference to FIGS. 10 (a) to 10 (l) and FIGS. 11 (a) to 11 (l). FIGS. 10A to 10L and FIGS. 11A to 11L illustrate images captured by the image sensor 3 in the frame periods of times T0, T1, T2, and Tn (FIG. 10A and FIG. 10A). (B), (c), (d)), an imaging signal IN (FIGS. 10 (e), (f), (g), (h)) output from the imaging device 3 is stored in the frame memory 9. Fixed pattern noise FPN (FIGS. 10 (i), (j), (k), (l)), a video signal S that is a difference signal between the imaging signal IN and the fixed pattern noise O, (FIG. 11 (a), (B), (c), (d))), a video signal AD obtained by digitally converting the video signal S by the A / D converter 6 (FIGS. 11E, 11F, 11G, 11H). The new fixed pattern noise D (FIGS. 11 (i), (j), (k), (l)) calculated by the mixing circuit 7 is illustrated. The waveform of the fixed pattern noise O analog-converted by the D / A converter 10 is the fixed pattern noise FPN stored in the frame memory 9 (FIGS. 10 (i), (j), (k), (l )).

  10 (a), (e), (i), FIG. 11 (a), (e), and (i) show captured images and signals at time T0, and FIGS. 10 (b), (f), (J), FIGS. 11 (b), (f), and (j) show captured images, signals, and the like at time T1, and FIGS. 10 (c), (g), (k), FIG. 11 (c), (G) and (k) show captured images and signals at time T1, and FIGS. 10 (d), (h) and (l), and FIGS. 11 (d), (h) and (l) show time. A captured image, a signal, and the like at Tn are shown. Time elapses in the order of times T0, T1, T2, ... Tn.

First, when the image pickup apparatus is powered on, it starts up with the shutter 2 closed.
When a signal corresponding to the infrared ray absorbed by the image pickup device 3 is output as the image pickup signal IN (T0), the difference circuit 4 holds the image pickup signal IN (T0) acquired in the shielding state and the frame memory 9. The video signal S (T0) is generated by taking the difference of the FPN fixed pattern noise O (T0) obtained by converting the fixed pattern noise FPN (T0) being converted into analog.

  The video signal S (T0), which is an analog signal, is converted into a digital signal output by the A / D converter 6. In the mixing circuit 7, the signal AD (T0) / K attenuated to the signal level of 1 / K is added to the fixed pattern noise FPN (T0) held in the frame memory 9, and the new pattern noise D (T0) is added. Generate. In the example shown in FIGS. 10 and 11, the fixed pattern noise FPN is added as it is without being attenuated, and the attenuation coefficient K for the signal AD is also 1 for convenience.

  Further, since the shutter is in a shielded state, the update circuit 8 is allowed to write control signals to the frame memory, and the update pattern noise D (T0) is held as update data in the frame memory 9, and the next time At T1, the fixed pattern noise FPN (T1) is supplied to the difference circuit 4 and used to generate a new pattern noise D (T1). Thereafter, the same processing is repeated until time T2.

At time Tn, the shutter is opened.
In the figure, for the sake of convenience, the coefficient K used in the mixing circuit 7 is set to 1 (no attenuation) as described above, and the resolution difference between the A / D converter 6 and the D / A converter 10 is doubled (the difference in the number of bits). 1) Turn on the power with the shutter shielded, and after 2 frames have passed since the acquisition of fixed pattern noise, the acquisition of the updated pattern noise D is completed, and the shutter is opened to start normal imaging. However, in an actual circuit, a period of two frames or more is required until the integrated signal level of all fixed pattern noises is stabilized.

  Since the imaging signal output at time Tn is a signal acquired with the shutter opened, the update circuit 8 is in a write-inhibited state, and the data in the frame memory 9 is not updated.

  Although the case where the shielding update pattern noise D is acquired when the power is turned on has been described here, it is not limited to the time when the imaging apparatus is activated, and may be appropriately acquired during the operation. Rather, since the variation in sensitivity of the image sensor 3 changes depending on the external environment, the temperature of the apparatus, etc., it is better to update the shielding update pattern noise D as appropriate during operation.

  Regarding the determination of the completion of acquisition of the fixed pattern noise in the detection circuit 13, the relationship between the video signal S, the range of the A / D converter 6, and the digitally converted digital video signal AD is referred to in FIGS. This will be described with reference to j).

  FIGS. 5A to 5J show the video signal S and the digital video signal AD at times T0 to T4 (time at which one frame of data is processed). 5A to 5E show the video signal S output from the difference circuit 4, and FIGS. 5F to 5J show the digital video signal AD output from the A / D converter 6. FIG. Of the video signal S, the range from the upper limit UL to the lower limit LL corresponds to the dynamic range of the A / D converter 6, and the video signal S and the value of the digital video signal AD have a proportional relationship only within this range. Have. In FIG. 5, the A / D converter 6 is 6 bits, the D / A converter 10 is 8 bits, the difference in resolution between the A / D converter 6 and the D / A converter 10 is quadrupled, and the mixing circuit 7 It is assumed that the digital video signal AD and the fixed pattern noise FPN are added as they are (without being multiplied by an attenuation coefficient).

  First, at the time T0 when the fixed pattern noise acquisition process is started, since the frame memory 9 is in the initial state, the imaging signal IN in the shielding state, that is, the fixed pattern noise held by this element is output from the difference circuit 4 as it is. The As a result, a signal corresponding to 1/4 of the fixed pattern noise is converted into a digital signal.

  The mixing circuit 7 adds the digitally converted video signal AD and the data FPN stored in the frame memory 9 at a predetermined ratio (here, 1: 1) and sends the result to the update circuit 7. The update circuit 7 stores the new fixed pattern noise in the frame memory 9. The signal shown in FIG. 5F is taken into the frame memory 9.

At time T1, as shown in FIG. 5B, the difference between the imaging signal similar to that shown in FIG. 5A and the fixed pattern noise shown in FIG. . The video signal S shown in FIG. 5B does not have a shading component that increases from left to right.
By repeating the same processing as at times T2 and T3, the peak-shaped fixed pattern noise gradually disappears from the video signal S.

  At the stage of time T3, the difference between the maximum value and the minimum value of the video signal of the effective pixel is sufficiently small. If it is not necessary to remove the fixed pattern noise until the signal level of the effective pixel becomes equal to the level of the invalid pixel, the acquisition of the fixed pattern noise may be terminated at this stage. That is, when the difference between the maximum value and the minimum value of the video signal of the effective pixel becomes less than a certain value, it is determined that the fixed pattern noise acquisition process may be terminated.

  When it is necessary to remove the fixed pattern noise until the signal level of the effective pixel becomes equal to the level of the invalid pixel, the acquisition process of the fixed pattern noise is further continued, and as shown at time T4, the video signal of the effective pixel and the invalid pixel noise are invalidated. The process ends when there is no difference between the video signals of the pixels.

  As described above, according to the first embodiment, the difference between the signal output from the image sensor and the output signal of the frame memory in which the component corresponding to the fixed pattern noise is recorded is converted into a digital signal. Since the signal from which the dominant fixed pattern noise is subtracted is converted into a digital signal and the dynamic range of the analog-to-digital (A / D) converter can be used effectively, the quantization noise caused by the digital conversion is reduced. Reduce and improve image quality.

  Also, the attenuation circuit 40 multiplies the amplitude of the digital signal AD by 1 / K and then adds it to the fixed pattern noise FPN, thereby absorbing the characteristic difference between the A / D converter and the D / A converter. In order to prevent divergence, the update pattern noise D at the time of shielding can be accurately converged.

  Furthermore, by automatically detecting the start and end timings of the fixed pattern noise acquisition process, it is possible to efficiently acquire the fixed pattern noise and always maintain a low noise state.

It is a block diagram which shows the structure of the infrared imaging device which concerns on Embodiment 1 of this invention. It is the schematic which shows the effective pixel, invalid pixel, and blanking period which are included in a horizontal period. 3 is a block diagram illustrating an example of a mixing circuit 7. FIG. 6 is a block diagram showing another example of the mixing circuit 7. FIG. (A)-(j) is the schematic which shows the method of detecting the completion of acquisition of fixed pattern noise. (A)-(c) is the schematic which shows the video signal which removed the picked-up image, the imaging signal containing fixed pattern noise, and fixed pattern noise. FIG. 3 is a schematic diagram illustrating a relationship among an imaging signal IN, an analog fixed pattern noise O, and a video signal S. It is the schematic which shows the relationship between imaging signal IN and new fixed pattern noise D '. It is the schematic which shows the relationship between the video signal S and the new fixed pattern noise D. (A)-(l) is the schematic which shows the acquisition procedure of the fixed pattern noise D. FIG. (A)-(l) is the schematic which shows the acquisition procedure of the fixed pattern noise D. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens, 2 Shutter, 3 Image pick-up element, 4 Difference circuit, 5 Amplifier circuit, 6 A / D converter, 7 Mixer circuit, 8 Update circuit, 9 Frame memory, 10 D / A converter, 11 Shutter control circuit, 12 Timing generation circuit, 13 detection circuit, 21 integration circuit, 22 gain control circuit, 30 FPN acquisition means, 40 control means.

Claims (12)

Imaging means having effective pixels sensitive to a predetermined wavelength range and ineffective pixels having no sensitivity;
Imaging means for forming an image of a predetermined wavelength region component on the imaging surface of the imaging means;
Shielding means for shielding incidence of the predetermined wavelength region component to the imaging means;
Difference means for removing analog fixed pattern noise from the output signal of the imaging means;
Digital conversion means for converting the video signal output from the difference means into a digital signal and outputting the digital video signal;
FPN acquisition means for acquiring fixed pattern noise based on the digital video signal;
Analog conversion means for converting the fixed pattern noise acquired by the FPN acquisition means into an analog signal and outputting the analog fixed pattern noise;
In an infrared imaging device comprising the shielding means and a control means for controlling the operation of the FPN acquisition means,
The FPN acquisition means
A frame memory for storing the fixed pattern noise;
A mixing means for calculating a new fixed pattern noise by mixing the digital video signal and the fixed pattern noise stored in the frame memory;
Updating means for updating the fixed pattern noise stored in the frame memory with a new fixed pattern noise calculated by the mixing means;
The control means includes
Shielding control means for making the shielding means open or shielded at a predetermined timing;
The fixed pattern noise acquisition start timing is determined based on the invalid pixel digital video signal, and the fixed pattern noise acquisition completion timing is determined based on the effective pixel digital video signal and the invalid pixel digital video signal. Detecting means for detecting
Based on a detection signal output from the detection means, a timing for generating a shielding operation timing signal of the shielding control means, and an update operation timing signal for setting the update circuit to an update execution state or an update stop state are generated. An infrared imaging device comprising: timing generation means.   During the process of acquiring the fixed pattern noise, the detection means detects that the deviation between the average value of the digital video signal of the effective pixel and the average value of the digital video signal of the invalid pixel is less than a certain value, The infrared imaging device according to claim 1, wherein it is detected that the acquisition of has been completed.   The detection means detects that the acquisition of the fixed pattern noise is completed when the difference between the maximum value and the minimum value of the digital image signal of the effective pixel becomes less than a certain value during the acquisition process of the fixed pattern noise. The infrared imaging device according to claim 1.   2. The infrared imaging apparatus according to claim 1, wherein the detection unit detects that the acquisition of the fixed pattern noise is completed when the variation of the digital video signal of the effective pixel becomes less than a predetermined value.   The detection means stores an average value of the digital video signal of the invalid pixel as a reference signal when the acquisition of fixed pattern noise is completed, and a deviation between the reference signal and the average value of the digital video signal of the invalid pixel is detected. The infrared imaging apparatus according to claim 1, wherein the timing of starting acquisition of fixed pattern noise is detected when the value exceeds a certain value. The detecting device, the condition that the deviation from the standard values of the fixed pattern noise for each pixel stored in the frame memory is less than a predetermined value, the digital video signal of the pixel, the acquisition of fixed pattern noise The infrared imaging device according to claim 1, wherein the infrared imaging device is used for detecting a start timing and an end timing.   2. The mixing unit according to claim 1, wherein the mixing unit calculates the new fixed pattern noise by mixing the digital video signal and the fixed pattern noise stored in the frame memory at a predetermined ratio. Infrared imaging device.   The mixing unit subtracts a reference value from the digital video signal and then mixes the fixed pattern noise stored in the frame memory with a predetermined ratio to calculate the new fixed pattern noise. The infrared imaging device according to claim 1.   9. The infrared imaging device according to claim 7, further comprising variable gain amplification means for amplifying the difference signal output from the difference means according to a gain setting value and supplying the amplified signal to the digital conversion means. . Integration means for integrating the digital video signal of the effective pixels output from the digital conversion means in units of frames;
The infrared imaging device according to claim 9, further comprising: gain control means for controlling the gain setting value so that an integral value output from the integrating means substantially matches a predetermined reference value.   11. The infrared imaging apparatus according to claim 1, wherein the number of input signal bits of the analog conversion unit is larger than the resolution of the digital conversion unit.   The infrared imaging device according to any one of claims 1 to 11, wherein the predetermined wavelength band component is approximately a wavelength band of 8 to 14 micrometers.

Images