JP3511073B2 - Solid-state imaging device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a solid-state image pickup device used in a video camera or the like.

[0002]

2. Description of the Related Art Conventionally, as a solid-state image pickup device for synthesizing images having different exposure amounts to obtain a video signal having a high dynamic range, a device described in Japanese Patent Application Laid-Open No. 7-322147 has been proposed.

A conventional solid-state image pickup device of this type includes an image pickup means 50 and an image synthesizing means 51, as shown in FIG. The image pickup means 50 is provided with an all-pixel readout type solid-state image pickup device (not shown) capable of outputting a signal with twice as many scanning lines as that of an ordinary solid-state image pickup device, and is provided with a pair of adjacent scanning lines. By performing the electronic shutter operation so that the pixel corresponding to one of the scanning lines has a normal exposure amount, and performing the electronic shutter operation so that the pixel corresponding to the other scanning line has a smaller exposure amount than the normal exposure amount. , A long-time exposure signal Slong (having a large exposure amount) and a short-time exposure signal Sshort (having a small exposure amount), which are output signals having different exposure amounts.

The image synthesizing means 51 simply adds the output signals Slong and Sshort output from the image pickup means 50 to obtain a synthesized signal Smix.

FIG. 13 shows a solid-state image sensor (imaging means) on the horizontal axis.
Long-time exposure signal S in the case where the vertical axis represents the amount of incident light with respect to 50 and the signal level of the output signal of the image pickup means 50 is taken.
The relationship between long, short-time exposure signal Sshort, and composite signal Smix is shown.

Here, the long-time exposure signal Slong is saturated with the amount of incident light indicated by the symbol A because the exposure amount is large,
If the amount of incident light is less than that, the signal level changes greatly, so the S / N ratio is good. On the other hand, the short-time exposure signal Sshort
Has a small signal amount and a poor S / N ratio, but has a wide dynamic range, and therefore has good signal reproducibility when the incident light amount is equal to or higher than the saturated light amount A.

In the conventional solid-state image pickup device, the image synthesizing means 5 is used.
By combining these two signals Slong and Sshort by 1, a combined signal Smix having a good S / N ratio and a wide dynamic range is obtained.

[0008]

However, the conventional solid-state image pickup device has a problem that the smear component increases and the screen becomes difficult to see. This will be described below.

Since the conventional solid-state image pickup device is intended to expand the dynamic range, it is often used for the purpose of photographing a brighter scene. However, the solid-state imaging device is characterized in that smearing is likely to occur when a high-luminance subject is photographed. The smear refers to a phenomenon in which a white line appears in the vertical direction on the display screen under the influence of a signal component (hereinafter referred to as a smear component SM) generated by light leaking into the vertical transfer unit that constitutes the solid-state imaging unit 50. The higher the brightness of a subject, the more often it occurs. In the above-described conventional solid-state imaging device, smear components are generated on both adjacent scanning lines, and the smear components SM are increased by being added to each other by the image synthesizing means 51 to increase smear components on the display screen. Was getting harder and the screen was hard to see.

Therefore, an object of the present invention is to suppress smear while expanding the dynamic range.

[0011]

In order to solve this problem, the present invention provides a smear removing means after extracting a smear component from a plurality of image signals having different exposures output from an image pickup means by a smear extracting means. The smear component thus extracted is removed from the image signals having different exposure amounts, and a composition process is performed.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The invention according to claim 1 of the present invention is a solid-state image pickup device provided with an image pickup means for outputting a plurality of image signals having different exposure amounts within a period for picking up an image for one screen. In, the signal level adjusting means for adjusting the signal level of the image signal having a different exposure amount, the smear extracting means for extracting a smear component from the image signal having a different level-adjusted exposure amount, and the extracted smear component of the exposure amount It is characterized in that it has a smear removing means for removing different image signals from each other and an image combining means for combining the smear-removed image signals having different exposure amounts, which has the following operation. That is, since smear removal processing is performed when a smear component is present, smear can be suppressed while expanding the dynamic range.

According to a second aspect of the present invention, in the solid-state image pickup device according to the first aspect, the signal level adjusting means changes the signal level of an image signal having a large exposure amount to that of an image signal having a small exposure amount. It is characterized in that it has an adjusting unit for adjusting so as to substantially match the signal level, and thus has the following operation. That is, the signal level of the image signal having a large exposure amount can be adjusted accurately.

According to a third aspect of the present invention, in the solid-state image pickup device according to the first or second aspect, the adjusting section includes an attenuator for attenuating a signal level of an image signal having a large exposure amount. It is characterized by the fact that it has the following effects. That is, the signal level can be easily adjusted.

According to a fourth aspect of the present invention, in the solid-state image pickup device according to any one of the first to third aspects, the smear extracting means is the signal level adjusting means from the image signal having a small exposure amount. The present invention is characterized in that it has a first subtraction unit for performing a subtraction process on a level-adjusted image signal having a large exposure amount, and thereby has the following operation. That is, smear extraction can be performed accurately and easily.

According to a fifth aspect of the present invention, in the solid-state imaging device according to any one of the first to fourth aspects, the smear removing means extracts the image signal having a large exposure amount by the smear extracting means. The feature is that the second subtraction unit that performs the subtraction processing of the smeared component is provided, and thus the following operation is performed. That is, the smear component can be removed accurately and easily.

According to a sixth aspect of the present invention, in the solid-state image pickup device according to any one of the first to fifth aspects, the image synthesizing means has a linear change in signal level of an image signal having a large exposure amount. When determining that the signal level has linearity in the first determining unit that determines whether or not the image signal has a linearity, the smear component-removed image signal having a large exposure amount is output as a combined signal. On the other hand, when the first determination unit determines that the signal level does not have linearity, the gradation continuity of the image signal having a small exposure amount and the image signal having a large exposure amount with the smear component removed is continuous. And a signal selection unit that outputs a combined signal after performing signal processing for maintaining the above. That is, it becomes possible to accurately and easily synthesize images.

According to a seventh aspect of the present invention, in the solid-state image pickup device according to any one of the first to sixth aspects, the smear extracting means includes a level detecting section for detecting a signal level of the extracted smear component. And a first smear suppressing unit that suppresses the smear component when the signal level of the smear component detected by the level detecting unit does not reach a predetermined level. It has the following action. That is, since the smear component is suppressed based on the detection result by the level detection unit, it is possible to suppress a noticeable noise component even when the signal level of the smear component does not reach a predetermined level, and as a result, the image synthesis accuracy is reduced. Will increase.

According to an eighth aspect of the present invention, in the solid-state image pickup device according to any one of the first to seventh aspects, the smear extracting means has a linear change in signal level of an image signal having a large exposure amount. A second judgment unit for judging whether or not the second smear component is present, and a second smear suppression unit for suppressing and outputting the extracted smear component when the second judgment unit judges that the signal level change has no linearity. It is characterized in that it has the following features. That is, the image level of an image signal with a large exposure amount becomes unstable near the point where the signal level change has no linearity. Therefore, the extraction accuracy of the smear component extracted near the point where the signal level change of the image signal having a large exposure amount does not have linearity becomes poor. In the present invention, the signal level change of the image signal having a large exposure amount is output in a state in which the smear component is suppressed in the vicinity of a point where the signal level change does not have linearity, whereby the extraction accuracy of the smear component in this region is improved. It is possible to eliminate the adverse effect caused by the defect.

According to a ninth aspect of the present invention, in the invention according to any one of the first to eighth aspects, the image pickup means has a solid-state image pickup element in which color filters having different spectral characteristics are arranged for each pixel. In addition, it is characterized in that the smear extraction performed by the smear extraction means and the smear removal performed by the smear removal means are performed for each pixel, which has the following operation. That is, the smear component can be removed even in a solid-state imaging device that performs color display.

Embodiments of the present invention will be described below with reference to the drawings.

First Embodiment FIG. 1 is a block diagram showing the structure of a solid-state image pickup device according to the first embodiment of the present invention. The solid-state imaging device extracts a smear component from a solid-state imaging device 1 as an imaging unit, a signal level adjusting unit 2 that adjusts a signal level of an output signal of the solid-state imaging device 1, and a level-adjusted output signal of the solid-state imaging device 1. The smear extracting means 3A, the smear removing means 4 for removing smear components from the output signal of the solid-state image sensor 1, and the image synthesizing means 5A for synthesizing the smear-removed solid-state image sensor 1 are provided.

The solid-state image pickup device 1 uses an all-pixel readout type solid-state image pickup device capable of outputting a signal having twice the number of scanning lines as that of a normal solid-state image pickup device. FIG. 2 shows a block diagram of an example of such an all-pixel readout type solid-state imaging device 1. The solid-state imaging device 1 includes a photoelectric conversion unit 6 arranged two-dimensionally, a vertical transfer unit 7 capable of vertically transferring charges read by the photoelectric conversion unit 6, and a vertical transfer unit 7 reading the charges. A pair of horizontal transfer units 8a and 8b that serially output the generated charges within the next field period, and each horizontal transfer unit 8
Output amplifiers 9a and 9b for amplifying the outputs of a and 8b are provided. In FIG. 2, the number of pixels is 4 horizontal × 6 vertical, but in reality, for example, VGA (Video
In the case of Graphics Array) image format, etc., width 64
The structure of 0 × vertical 480 pixels is adopted. In the solid-state imaging device 1 configured as described above, the electronic shutter operation is performed so that the pixels corresponding to one scanning line of the pair of adjacent scanning lines have the normal exposure amount, and the pixels corresponding to the other scanning line are operated. By performing the electronic shutter operation so that the pixels to be exposed become smaller than the normal exposure amount, a long-time exposure signal Slong (a large exposure amount), which is an output signal having a different exposure amount,
The short-time exposure signal Sshort (the exposure amount is small) is obtained.

The signal level adjusting means 2 is a solid-state image pickup device 1.
The short-time exposure signal Sshort output from the solid-state imaging device 1 is not adjusted while the signal level of the long-time exposure signal Slong output from the solid-state imaging device 1 is adjusted (specifically, it is attenuated at a constant attenuation rate (1 / N)). An adjustment unit 11 for adjusting the level of the long-time exposure signal Slong is provided for passing through the processing. The adjusting unit 11 is composed of, for example, an attenuator.

The smear extracting means 3A uses the short-time exposure signal S
Attenuation signal Slong / N, which is a long-time exposure signal that has been attenuated, is subtracted from short [Sshort- (Slong /
N)] to generate the pseudo smear signal SM ′, and includes the first subtraction unit 12. The first subtraction unit 12 is composed of, for example, a subtractor.

The smear removing means 4 has a long exposure signal Slo.
Pseudo smear signal SM 'is subtracted from ng (Slong-S
M ′) to generate the pseudo original long-time exposure signal SlongO ′, while the short-time exposure signal Sshort is passed through unprocessed, and the second subtraction unit 1 for performing the above-mentioned subtraction processing.
Equipped with 3. The second subtraction unit 13 is composed of, for example, a subtractor.

The image synthesizing means 5A uses the short-time exposure signal Ssh.
The signal level of the long-time exposure signal Slong is compared by comparing the signal level of the adder 14 that adds an offset to ort with the signal level of the offset short-time exposure signal Sshort ′, which is the short-time exposure signal after the offset addition, and the threshold value Ssat ′. The comparator 15A that determines whether or not the change has linearity, and the pseudo original long-time exposure signal SlongO ′ and the offset short-time exposure signal Sshor based on the determination result of the comparator 15A.
The selector 16 which selects one signal from t ′ and outputs it as a combined signal Smix. In the present embodiment, the comparator 15A constitutes the first determination unit, and the adder 14 and the selector 16 constitute the signal selection unit.

Next, the operation of this solid-state image pickup device will be described. First, the operation of the solid-state image sensor 1 will be described.

The long-time exposure signal Slong (denoted by ● in FIG. 2) obtained by photoelectric conversion in the vertical effective period of one field period in the television system by the photoelectric conversion unit 6 is vertically transferred in the vertical blanking period. It is read by the unit 7. In addition, a short-time exposure signal Sshort (O in FIG. 2) obtained by photoelectric conversion during the vertical blanking period.
Are also read out to the vertical transfer unit 7 independently of the long-time exposure signal Slong during the vertical blanking period. The long-time exposure signal Slong and the short-time exposure signal Sshort thus obtained are sequentially transferred from the vertical transfer unit 7 to the horizontal transfer units 8a and 8b independently of each other, and are output to the output amplifier 9a during the effective period of the next field. And 9b.

FIG. 3 shows the relationship between the long-time exposure signal Slong and the short-time exposure signal Sshort when the horizontal axis represents the amount of incident light and the vertical axis represents the output signal level of the solid-state image pickup device 1. Here, it is known that the smear component SM maintains a constant level regardless of the output signal level of the solid-state image sensor 1. Correlation occurs in the S / N ratio between the output signal level and the noise component, and when the signal level of the output signal is small, the noise component becomes conspicuous. Therefore, the long-time exposure signal Slong is characterized in that it is saturated with the saturated light amount A, but at a saturated light amount A or less, a signal having a good S / N ratio can be obtained. On the contrary, the short-time exposure signal Ssh
Although the ort has a small signal amount and a poor S / N ratio when the saturated light amount is A or less, it has a feature that a signal can be reproduced even when the saturated light amount is A or more.

In consideration of such characteristics, in the present embodiment, the long-time exposure signal Slong and the short-time exposure signal Sshort are combined after the signal level is adjusted, whereby the S / N ratio is good and the dynamics are obtained. You are getting a high range signal. Hereinafter, this will be described in detail based on a specific operation.

The long-time exposure signal Slong and the short-time exposure signal Sshort output from the solid-state image pickup device 1 include a constant smear component SM regardless of the signal level. Therefore, the long-time exposure signal Slong and the short-time exposure signal Sshort are obtained by adding the smear component SM to the original long-time exposure signal Slong0 and the original short-time exposure signal Sshort0, as shown in the following equation.

Slong = Slong0 + SM ... Sshort = Sshort0 + SM ... Thus, the long-time exposure signal Slong and the short-time exposure signal Sshort containing the constant smear component SM regardless of the signal level are output from the solid-state image pickup device 1 at the signal level. It is input to the adjusting means 2 and the smear removing means 4.

In the signal level adjusting means 2, the adjusting unit 11 attenuates the level of the long-time exposure signal Slong to 1 / N,
While outputting the attenuation signal Slong / N, the short-time exposure signal Sshort is output in an unprocessed state. The attenuation rate (1 / N) of the adjusting unit 11 is set as follows. That is, when the exposure amount of the short-time exposure signal Sshort is 1, the attenuation ratio (1/1 is calculated based on the ratio 1: N between the exposure amount of the short-time exposure signal Sshort and the exposure amount of the long-time exposure signal Slong.
N) is set.

The adjustment signal Slong / N and the unprocessed short-time exposure signal Sshort are input to the smear extraction means 3A.
In the smear extraction means 3A, the first subtraction unit 12 performs a process (Sshort-Slong / N) of subtracting the attenuation signal Slong / N from the short-time exposure signal Sshort to create the pseudo smear signal SM ', and the smear signal is generated. Output to the removing means 4.

The pseudo smear signal SM 'thus created is expressed by the following equation.

SM '= Sshort-Slong / N ... By substituting the above equations and equations into this equation, the following equation is obtained.

SM ′ = Sshort0 + SM−Slong0 / N−SM / N ... Since the ratio of the exposure amount between the original long-time exposure signal Slong0 and the original short-time exposure signal Sshort0 is N: 1, the formula is as follows. Can be rewritten as

SM '= SM × (N-1) / N Here, if N is sufficiently large, the pseudo smear signal SM'
Can be regarded as equivalent to the smear component SM. For example, letting N be 16 which is an approximate ratio between the vertical effective period and the vertical blanking period, it can be seen that SM ′ = SM × 15/16 and SM′≈SM.

In the smear removing means 4, the second subtraction unit 13 converts the long exposure signal Slong into the pseudo smear signal S.
A process (Slong-SM ') for subtracting (removing) M'is performed to generate and output the pseudo original long-time exposure signal Slong0', while the short-time exposure signal Sshort is passed through unprocessed.

Pseudo original long exposure signal SlongO '
And the short-time exposure signal Sshort are input from the smear removing means 4 to the image synthesizing means 5A. In the image composition means 5A,
The adder 14 adds a specific offset Soff to the input short-time exposure signal Sshort (Ss
(hort + Soff) to offset short-time exposure signal Ss
hort 'is created and output to the selector 16 and the comparator 15A. On the other hand, the input pseudo original long-time exposure signal SlongO ′ is output to the selector 16 without being processed.

Offset Soff added by adder 14
Is set as follows. That is, as shown in FIG. 3, the signal level obtained by subtracting the smear component SM from the saturated signal level Ssat which is the signal level of the long-time exposure signal Slong at the saturated light amount A is the threshold level Ssat '(= Ssat-S
M). Then, at the saturated light amount A, the offset short-time exposure signal Sshort ′ is equal to the threshold level Ssat ′.
The offset Soff is added to the short-time exposure signal Sshort so that Therefore, Soff = Ssat'-Ss
It becomes hort.

In the comparator 15A, the input offset short-time exposure signal Sshort 'and the threshold level Ssat' are input.
And the result of the determination is output to the selector 16.
In the selector 16, when the comparator 15A determines that the offset short-time exposure signal Sshort 'is larger than the threshold level Ssat' (Sshort '>Ssat'), the signal level change of the long-time exposure signal Slong is linear. If not, the selector 16 selects the offset short-time exposure signal Sshort ′ as the combined signal Smix and outputs it. On the other hand, in the comparator 15A, the offset short-time exposure signal Sshort 'is smaller than the threshold value Ssat' (Sshort '<Ssa
If it is determined to be t ′), it is determined that the signal level change of the long-time exposure signal Slong has linearity, and the pseudo original long-time exposure signal Slong0 ′ is selected and output as the composite signal Smix.

FIG. 3 shows the photoelectric characteristics showing the relationship between the signal level of each part and the incident light quantity at this time. In FIG. 3, the composite signal Smix is a long-time exposure signal Slong and a short-time exposure signal Ss.
It can be seen that the smear SM contained in hort is suppressed. That is, it is possible to perform signal processing equivalent to the synthesis processing after removing the smear component SM from the output signals (Slong, Sshort) output from the solid-state imaging device 1 having different exposure amounts, and thereby the dynamic range is obtained. Can be suppressed and the smear component SM can be suppressed.

In the above description, the smear extracting means 3
Although A was composed of only the first subtraction unit 12,
An amplifier is configured on the output side of the subtraction unit 12, and the first subtraction unit 12
If the output (pseudo smear signal SM ') is multiplied by N / (N-1), the smear component SM can be completely suppressed.

In the above description, the signal level obtained by subtracting the smear component SM from the saturated signal level Ssat which is the signal level of the long-time exposure signal Slong at the saturated light amount A is set as the threshold level Ssat '(= Ssat-SM). Thus, it is determined whether the signal level change of the long-time exposure signal Slong has reached the saturated light amount A, and the presence or absence of linearity is checked. However, the long-time exposure signal Slong is equal to the saturated light amount A
It is known that the linearity of the signal level is lost by the above. Therefore, a value that is slightly smaller than the above-mentioned threshold value Ssat 'may be set as the threshold value.
It is possible to determine that the signal level change becomes non-linear due to the long-time exposure signal Slong becoming unstable at the position before.

Further, in the present embodiment, the all-pixel readout type solid-state image pickup device 1 is used as the image pickup means. However, by using a normal solid-state image pickup device, the solid-state image pickup device is driven at twice the frequency of the conventional one. Can obtain signals of different exposure amounts, and can obtain the same effect as that of the present embodiment.
However, in this case, there is a time difference in the time required for the signal components corresponding to the same pixel to be transferred in the image sensor, and even if the pixel is the same pixel, the smear component differs due to the change of the subject. The effect of smear suppression may be weakened.

Further, in the present embodiment, the black and white solid-state image pickup device 1 is used as the image pickup means, but a multi-plate type image pickup device using an optical prism and color filters having different spectral characteristics for each pixel are arranged. The same effect can be obtained by using a single-plate image pickup device.

Further, in the present embodiment, the solid-state image pickup device 1 is used as the image pickup means and smear removal processing is applied to the output signal of the solid-state image pickup device 1. The same effect can be obtained by performing the smear removal processing after performing the signal processing of.

In this embodiment, the case where the ratio of the exposure amount between the long-time exposure signal Slong and the short-time exposure signal Sshort (N: 1) is 16: 1 has been described. If the attenuator 11 in the adjusting means 2 is configured by a digital circuit and then N is adjusted to be a power of 2, the attenuator 11 can be easily configured by a bit shift circuit.

Second Embodiment In the above-described first embodiment, the second subtractor 13 constituting the smear removing means 4 forcibly subtracts the pseudo smear signal SM 'from the long-time exposure signal Slong (Slong- S
M '). This is the solid-state image sensor 1
This configuration is based on the assumption that the smear component SM is always present in the output of. However, the solid-state image sensor 1
In some cases, the smear component SM does not exist in the output of the solid-state element 1 depending on the illumination state of the subject. In such a case, if the pseudo subtraction signal SM ′ is forcibly subtracted by the second subtractor 13, a noise component is added by the subtraction processing, and the pseudo original long-time exposure that is the image signal after the subtraction processing. Signal Slo
The S / N ratio in ngO 'may deteriorate.

Therefore, in this embodiment, the smear extraction means (pseudo smear signal S
The signal level of M ') is detected, and the output of the pseudo smear signal SM' is allowed only when the signal level reaches an arbitrary level, thereby suppressing the deterioration of the S / N ratio. ing.

FIG. 4 is a block diagram showing the structure of the smear extracting means of the solid-state image pickup device of this embodiment. In the present embodiment, the configuration and operation of the solid-state imaging device other than the smear extracting means are the same as those in the first embodiment, and therefore the description thereof is omitted here.

The smear signal extraction means 3B included in the solid-state image pickup device of this embodiment includes a base clip circuit 20. The base clip circuit 20 detects the signal level of the pseudo smear signal SM ′ output from the first subtractor 12 and blocks the output of the pseudo smear signal SM ′ until the signal level reaches an arbitrary lower limit level SMO ₁ . When the signal level of the extracted pseudo smear signal SM ′ reaches the arbitrary lower limit level SMO ₁ , the pseudo smear signal SM ′ is allowed to be output. In the present embodiment, the base clip circuit 20 constitutes a level detecting section and a first smear suppressing section.

The operation of this embodiment will be described below. In the smear extraction means 3B, the attenuation signal Slong / N and the short-time exposure signal Sshort are input as in the first embodiment, and the first subtraction unit 12 extracts the pseudo smear signal SM '. Pseudo smear signal SM 'is input to the base clip circuit 20, where the pseudo smear signal SM' signal level pseudo smear signal if any lower level SM0 ₁ below S the
The output of M'is blocked, that is, the pseudo smear signal S
'While the output level forcibly zero, pseudo smear signal SM' M if the signal level of the lower limit level SM0 ₁ or more, suppresses a signal obtained by subtracting the lower limit level SM0 ₁ from the pseudo smear signal SM 'pseudo smear signal rSM Output as'. The lower limit level SMO from the pseudo smear signal SM '
_The reason that ₁ is subtracted is to maintain the continuity of gradation. FIG. 5 shows the input / output characteristics of the base clip circuit 20.

The suppression pseudo smear signal rSM 'in which the noise component is suppressed as described above is input to the second subtraction section 13 of the smear removing means 4 and is subtracted from the long exposure signal Slong here. The same as 1.

By removing the smear component SM as described above, an image signal in which the noise component is suppressed and the dynamic range is expanded can be obtained.

In the present embodiment, the signal level of the noise component included in the smear component SM is set as the lower limit level SM0 ₁ . This lower limit level SMO ₁
Is dependent on the noise level of the image signal output from the solid-state image sensor 1. Therefore, it is possible to change this value according to the imaging conditions. However, since the lower-limit level SM0 ₁ following the smear component SM is not suppressed, it is necessary to adjust the level of the lower limit level SMO ₁ against the smear reduction effect and noise reduction effect.

Further, the long-time exposure signal Slong and the short-time exposure signal Ssh in the solid-state image pickup device 1 may be selected depending on the photographing conditions.
In some cases, the ratio (1: N) of the exposure amount to ort and the attenuation amount in the adjusting unit 11 of the signal level adjusting means 2 cannot be completely matched, and in such a case, the smear component SM There is a possibility that an error will occur in the removal of. However, in the present embodiment, the base clip circuit 20
Since the signal level of the pseudo smear signal SM 'can be suppressed by the above, it is possible to suppress the error that occurs.

In the present embodiment, the level detecting section and the first smear suppressing section are formed from the base clip circuit 20, but the level detecting section and the first smear suppressing section are separately formed. It goes without saying that it is good.

Embodiment 3 FIG. 6 is a block diagram showing the structure of the smear extracting means of the solid-state image pickup device of this embodiment. In the present embodiment, the configuration and operation of the solid-state imaging device other than the smear extracting means are the same as those in the first embodiment, and therefore the description thereof is omitted here.

In the present embodiment, the smear extraction means detects the signal level of the extracted smear component (pseudo smear signal SM '), and the signal level is the lower limit level SMO.
Until reaching ₂ By configured to attenuation processing pseudo smear signal SM ', S which has been a problem in the second embodiment
The deterioration of the / N ratio is suppressed by another structure.

The smear extraction means 3C of this embodiment attenuates the pseudo smear signal SM 'based on the level detection section 21 for detecting the signal level of the pseudo smear signal SM' and the detection result of the level detection section 21. And a smear attenuator 22. The level detector 21 is composed of, for example, a comparator. The smear attenuator 22 is composed of, for example, an attenuator.
In the present embodiment, the level detecting section 21 constitutes the level detecting section, and the smear attenuating section 22 constitutes the first smear suppressing section.

The operation of this embodiment will be described below. In the smear extraction means 3, as in the first embodiment, the attenuation signal S
The long / N and short-time exposure signal Sshort are input, and the first attenuator 12 extracts the pseudo smear signal SM ′. The pseudo smear signal SM ′ is input to the level detecting section 21 and the smear attenuating section 22. Then, the level detecting section 21 compares the signal level of the pseudo smear signal SM ′ with an arbitrary lower limit level SM0 _2, and the comparison result is the smear attenuating section 2
Output to 2. In the smear attenuator 22, the pseudo smear signal SM ′ is converted into the lower limit level SM based on the input comparison result.
If it is O ₂ or less, the signal obtained by attenuating the signal level of the pseudo smear signal SM ′ at an arbitrary attenuation rate is used as the attenuated pseudo smear signal dS.
Output as M '. On the other hand, if the pseudo smear signal SM 'is equal to or higher than the lower limit level SMO ₂ , the attenuated pseudo smear signal dS is not attenuated without attenuating the signal level of the pseudo smear signal SM'.
Output as M '. The input / output characteristics of the smear attenuator 22 are shown in FIG.

In the present embodiment, the pseudo smear signal SM '
Set as a lower limit level SM0 ₂ levels of noise components included in the same manner as the second embodiment, the pseudo smear signal S
Pseudo smear signal S in which M ′ does not reach the lower limit level SMO ₂
It is determined that M ′ contains a lot of noise components, and the attenuation process is performed to prevent the noise components from being mixed into the pseudo smear signal SM ′. This makes it possible to obtain an image signal in which the noise component is suppressed and the dynamic range is expanded.

Fourth Embodiment In the first to third embodiments described above, in the comparator 15A,
Offset short-time exposure signal Sshort 'and threshold level Ssa
By comparing and judging with t ′, it was judged whether or not the incident light amount reached the saturated light amount A. Here, the short-time exposure signal S
short has a relatively wide dynamic range, but S / N
The long-time exposure signal Slong has a relatively narrow dynamic range but has a good S / N ratio. Therefore, in order to further improve the determination accuracy of the saturated light amount A, the long-time exposure signal Slo having a good S / N ratio is obtained.
It is better to use a signal based on ng as a criterion. In this embodiment, the position of the saturated light amount A is determined based on the pseudo original long-time exposure signal SlongO ′ from such a point of interest.

However, when the position of the saturated light amount A is determined by the signal based on the long-time exposure signal Slong, there are the following inconveniences. That is, the long-time exposure signal Slong is saturated at a saturated light amount A or more, and the signal level change has no linearity near the saturated light amount A. The signal SM 'cannot be obtained. For this reason, the pseudo original long-time exposure signal SlongO 'created based on the pseudo smear signal SM' does not have high creation accuracy, and the position determination accuracy of the saturated light amount A is not good.

Therefore, in this embodiment, a high-luminance region where the long-time exposure signal Slong is saturated (a region where the signal level change of the long-time exposure signal Slong becomes non-linear)
In the above, the pseudo smear signal SM ′ is suppressed to suppress the malfunction of the smear suppression process (error of position determination of the saturated light amount A) due to the false extraction of the pseudo smear signal SM ′ in the high luminance region.

FIG. 8 is a block diagram showing the structure of the solid-state image pickup device according to the fourth embodiment of the present invention. In the fourth embodiment,
Since the configuration and operation of the solid-state imaging device other than the smear extracting means and the image synthesizing means are the same as those in the first embodiment, the same or equivalent parts are designated by the same reference numerals, and the description thereof will be omitted. To do.

The image synthesizing means 5B of this solid-state image pickup device is
Threshold Ssat 'and pseudo-original long-time exposure signal Slong
It has a comparator 15B as a first judging unit for making a comparative judgment on the signal level with O '. Also, the smear extraction means 3D
Clip the short-time exposure signal Sshort with Ssat '/ N, and then clip the clipped signal cSshort with the first subtraction unit 12
The clip circuit 23 for outputting to. In addition, in the present embodiment, the second judgment unit and the second judgment unit from the clip circuit 23.
A smear suppressing unit is configured.

The operation of the solid-state image pickup device of this embodiment will be described below. In the smear extracting means 3D, the attenuation signal Slong / N and the short-time exposure signal Sshort are input as in the first embodiment. In the clip circuit 23, the short-time exposure signal S
By determining whether the short has reached the threshold level Ssat '/ N, it is determined whether the signal level change of the long-time exposure signal Slong has linearity. If it does not reach, the short-time exposure signal Sshort is output as it is from the clipping circuit 23 as a clip short-time exposure signal cSshort without any processing because it is considered that the signal level change of the linear exposure signal Slong has linearity. On the other hand, when the threshold level Ssat '/ N is exceeded, it is determined that the signal level change of the linearity exposure signal has no linearity, and the signal clipped at the signal level of Ssat' / N is clipped for the short-time exposure signal. It is output from the clipping circuit 23 as cSshort.

The clip short-time exposure signal cSshort created by the clip circuit 23 is input to the first subtraction unit 12,
Here, the attenuation signal Slong / N is subtracted from the clip short-time exposure signal cSshort 'to create a clip pseudo smear signal cSM'. Clip pseudo smear signal cS
M ′ is input to the smear removing means 4.

In the smear removing means 4, the second subtraction unit 13 subtracts (removes) the clip pseudo smear signal cSM 'from the long-time exposure signal Slong (Slong-cS).
M ') is performed to generate and output the clip pseudo original long-time exposure signal cSlong0', while the short-time exposure signal Sshort is passed through unprocessed.

Clip pseudo original long-time exposure signal c
The SlongO 'and the short-time exposure signal Sshort are input from the smear removing means 4 to the image synthesizing means 5B. In the image synthesizing means 5B, with respect to the input short-time exposure signal Sshort, the adder 14 adds a specific offset Soff (Sshort + Soff) to create an offset short-time exposure signal Sshort ′, and the selector 16 Output to. On the other hand, the input clip pseudo-original long-time exposure signal cSlongO 'is output to the comparator 15B and the selector 16 without being processed.

The comparator 15B compares the input clip pseudo-original long-time exposure signal cSlongO 'with the above-mentioned threshold level Ssat', and outputs the judgment result to the selector 16. In the selector 16, in the comparator 15B, the clip pseudo original long-time exposure signal cSlong
O'is larger than the threshold level Ssat '(cSlongO'> S
Sat '), it is determined that the signal level change of the long-time exposure signal Slong has no linearity, and the selector 16 selects and outputs the offset short-time exposure signal Sshort' as the composite signal Smix. On the other hand, the comparator 15A
At clip pseudo original long-time exposure signal cSl
ongO 'is smaller than the threshold Ssat' (cSlongO '<Ssa
t ′), it is assumed that the signal level change of the long-time exposure signal Slong has linearity, and the clip pseudo-original long-time exposure signal cSlong0 ′ is synthesized signal Smix.
And output it.

FIG. 9 shows the clip short-time exposure signal cSshor.
The signal level changes of t ′, the attenuation signal Slong / N, and the clip pseudo smear signal cSM ′ are shown.

The threshold level Ssat '/ N is the same as in the first embodiment.
The values obtained by multiplying the threshold level Ssat ′ in 3 to 3 by the attenuation rate 1 / N in the attenuation unit 11 are used. Threshold level Ssa
t'is set to be smaller than the signal level Ssat of the long-time exposure signal Slong at the saturated light amount A. for that reason,
The threshold level Ssat '/ N becomes smaller than the signal level (≈saturated signal level Ssat / N) when the short-time exposure signal Sshort reaches the saturated light amount A. As a result, the short-time exposure signal Sshort has an incident light amount B (B <B <that is lower than the saturated light amount A).
It can be seen that the threshold level Ssat '/ N is reached in A).

Like the attenuation signal Slong / N, the clip short-time exposure signal cSshort does not rise above the threshold level Ssat '/ N, and when the clip pseudo-smear signal cSM' becomes saturated light intensity A. , It can be seen that the signal level becomes zero. Moreover, as described above, the short-time exposure signal Sshort reaches the threshold level Ssat '/ N at the incident light amount B lower than the saturated light amount A. Therefore, the clip pseudo smear signal _C SM 'is the incident light amount B
Up to the smear component SM, the signal level is gradually decreased from the incident light amount B to the saturated light amount A, and finally reaches the level zero when the saturated light amount A is reached.

The incident light amount region from the incident light amount B to the saturated light amount A is a region where the signal level change of the long-time exposure signal Slong becomes non-linear, and is created based on such a long-time exposure signal Slong. Pseudo smear signal S
The signal level of M'is incorrect. On the contrary,
In this embodiment, the clip pseudo smear signal in this area is
'By reducing the level gradually, the pseudo smear signal _C SM' _C SM in a state to eliminate the instability of the signal level at, have created a clip pseudo original long exposure signal slongo '. Thus, even if the position of the saturated light amount A is determined based on the pseudo original long-time exposure signal SlongO ′, the position can be accurately determined.

As described above, in the present embodiment, the high-luminance region where the long-time exposure signal Slong is saturated (the region where the signal level change of the long-time exposure signal does not have linearity)
Outputs the suppressed clip pseudo smear signal cSM '. Therefore, the pseudo smear signal cSM 'whose signal level is suppressed in the high-luminance region is the long-time exposure signal Slon.
will be removed from g. As a result, the image synthesizing unit 5B can suppress the smear component without causing an erroneous operation in the high-luminance region (without causing an erroneous determination of the position of the saturated light amount A), and correspondingly increase the dynamic range. be able to.

In the present embodiment, the clipping circuit 2
The second judging unit and the second smear suppressing unit are composed of 3
It goes without saying that the second determination unit and the second smear suppression unit may be separated and separately configured.

Embodiment 5 When a solid-state image pickup device in which color filters having different spectral characteristics are arranged for each pixel is used as the image pickup means, when the output signal thereof is band-limited and smear is removed as a luminance signal,
Since the difference between the smear levels occurring in each row cannot be removed, the area where smear occurs has an unnatural color.

Therefore, in the present embodiment, the smear extraction operation in the smear extraction means and the smear removal operation in the smear removal means are performed for each pixel, so that the smear signal for each column resulting from the color filter array of the solid-state image pickup device is generated. It suppresses the occurrence of false color due to the difference.

The present embodiment will be described below with reference to the drawings. The configuration of the solid-state image pickup device of the present embodiment is the same as that of the first embodiment except for the solid-state image pickup device 1 as the image pickup means and the image synthesizing means 5, and therefore the same or similar portions are denoted by the same reference numerals, Is omitted.

First, the structure of the solid-state image pickup device 30 as the image pickup means of the present embodiment will be described with reference to FIG. The solid-state image sensor 30 that constitutes this solid-state image pickup device is a photoelectric sensor in which color filters having four different spectral characteristics of magenta (Mg), green (Gr), yellow (Ye), and cyan (Cy) are arranged for each pixel. Although the conversion unit 31 is provided, its operation is exactly the same as that of the solid-state imaging device 1 described in the first embodiment. However, as the output signal, four kinds of signals of magenta + yellow (MY), magenta + cyan (MC), green + yellow (GY), and green + cyan (GC) are generated by the pixel mixing operation of the solid-state image sensor 30. The long-time exposure signal Slong and the short-time exposure signal Sshort are output.

Due to the arrangement of the color filters, a signal in which MY and GC are in one set and MC and GY are in one set is
It is alternately output for each line, and a group of MY and GC and a group of MC and GY are alternately output for each pixel. The spectral characteristics of each color filter are designed such that the sum of MY and GC and the sum of MC and GY become the spectral characteristics of the luminance signal Y. Therefore, the signal array of MY and GC and the signal sequence of MC and GY are output as a signal in which the color signal components Cr and Cb are modulated and superimposed on the luminance signal component Y.

In the color filter array as shown in FIG. 11, when viewed from the left end, only three types of color filters of Mg, Ye, and Gr are arranged in the odd columns, and M is arranged in the even columns.
Only three types of color filters of g, Cy, and Gr are arranged. The smear component SM is generated due to the leakage of charges from the photoelectric conversion unit 31 to the vertical transfer unit 7, and the smear component SM signal level varies from column to column in such a color filter array. On the other hand, output signals of the solid-state image sensor 30 (long-time exposure signal Slong, short-time exposure signal Sshor
It is conceivable to remove smear as a luminance signal by band-limiting t), but this cannot remove the difference in smear level for each pixel row, so that an unnatural color appears in the part where smear occurs. Will be attached.

On the other hand, in the solid-state imaging device of the present embodiment, smear extraction by the smear extraction means 3 and smear removal by the smear removal means 4 are performed for each pixel to remove the difference in smear level for each pixel column. By this, the smear component SM can be completely suppressed,
It is possible to obtain an output signal in which false color does not occur. The reason why the difference in smear level for each pixel column can be removed is as follows.

Generally, when processing the output signals of the national image pickup device in which such color filters are arranged, the color signal is obtained from the difference signal level between the pixels of the signals output from the pixels of different filters. . Therefore, it is possible to perform a process in which the timing of each pixel column is strictly adjusted, or a process in which A / D conversion is performed for each pixel and there is no deviation in each pixel column.

Next, a specific structure of this embodiment will be described. The image synthesizing unit 5C included in the solid-state imaging device according to the present embodiment stores the long-time exposure signal Slo for storing the gradation of the color signal.
ng and an amplification unit 32 that amplifies the short-time exposure signal Sshort with a gain corresponding to the light amount ratio of the short-time exposure signal Sshort and outputs an amplified short-time exposure signal Sshort ″, an amplified short-time exposure signal Sshort ″, and a threshold value Ssat ′. The comparator 15C serving as a first judgment unit for making a comparison judgment with the above, and the pseudo original long-time exposure signal SlongO ′ and the amplified short-time exposure signal Sshort ″ are selected based on the judgment result of the comparator 15C and the combined signal Smix is selected. And a selector 16 for outputting as.

Next, the operation of the image synthesizing means 5C will be described. Short-time exposure signal Ssh input to image synthesizing means 5C
The ort is amplified by the amplifier 32 with a gain corresponding to the light amount ratio of the long-time exposure signal Slong and the short-time exposure signal Sshort. The exposure time of the long-time exposure signal Slong is 1/60 second,
When the exposure time of the short-time exposure signal Sshort is 1/1000 second, the gain becomes 16 times.

The amplified short-time exposure signal Sshort ″ amplified by the amplifier 32 is compared with the threshold value Ssat ′ in the comparator 15C, and the comparison judgment result is output to the selector 16. In the selector 16, the input comparison judgment result is Ssh.
If ort ''> Ssat ', amplified short-time exposure signal Sshor
t ″ is selected and output as a composite image signal Smix. On the other hand, when the input comparison determination result is Sshort '<Ssat', the pseudo original long-time exposure signal Slong0 'is selected and output as the combined image signal Smix. As a result, the smear extraction in the smear extraction means 3 and the smear removal in the smear removal means 4 can be performed for each pixel, and the smear signal can be completely suppressed to obtain an output signal in which no false color is generated.

The configuration of the image synthesizing means 5C in the present embodiment is different from that in the first embodiment (short-time exposure signal S
The reason that the short is amplified by the amplifier 32) is that the output signals (Slong, Sshort) of the solid-state image pickup device 30 have a modulation component due to a color signal, and the smear suppression which is the object of the present invention is suppressed. There is no direct relationship with them.

By the way, in each of the above-mentioned embodiments, the image synthesizing means 5A, 5B and 5C have the long exposure signal Slong.
Of the pseudo original long-time exposure signal SlongO 'and the offset short-time exposure signal Sshort' (amplified short-time exposure signal Ss
hort '') was switched and output as a composite signal Smix. This is because it is effective to use the presence or absence of the linearity of the signal level change of the long-time exposure signal Slong as a criterion for removing the smear component. However, the pseudo original long-time exposure signal SlongO 'and the offset short-time exposure signal Sshort' (amplified short-time exposure signal Ss
The criterion for switching to (hort '') may be finely adjusted based on other requirements for image adjustment. In other words, depending on other image adjustment requirements, the long exposure signal Slo
Based on the judgment standard (signal level) slightly adjusted from the signal level (threshold value Ssat '), which is the judgment standard of whether the signal level change of ng has linearity, the pseudo original long-time exposure signal SlongO' and the offset The short-time exposure signal Sshort ′ (amplified short-time exposure signal Sshort ″) may be switched.

[0095]

As described above, according to the present invention, smear components are removed from output signals having different exposure amounts output from the image pickup means, and then the synthesis processing is performed, whereby smear is increased while expanding the dynamic range. The effect that it can be suppressed is obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a solid-state imaging device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of an image pickup element as an image pickup unit of the solid-state image pickup device according to the first embodiment.

FIG. 3 is a photoelectric characteristic diagram showing a relationship between a signal level of each part of the solid-state imaging device according to the first embodiment and an incident light amount.

FIG. 4 is a block diagram showing a configuration of smear extraction means of a solid-state imaging device according to a second embodiment of the present invention.

FIG. 5 is an input / output characteristic diagram of a smear extracting unit of the solid-state imaging device according to the second embodiment.

FIG. 6 is a block diagram showing a configuration of a smear extracting means 3 of a solid-state imaging device according to a third embodiment of the present invention.

FIG. 7 is an input / output characteristic diagram of a smear extraction unit of the solid-state imaging device according to the third embodiment.

FIG. 8 is a configuration diagram showing a configuration of smear extraction means of a solid-state imaging device according to a fourth embodiment of the present invention.

FIG. 9 is a photoelectric characteristic diagram showing a relationship between a signal level of each part of the solid-state imaging device according to the fourth embodiment and an incident light amount.

FIG. 10 is a block diagram showing a configuration of a solid-state imaging device according to a fifth embodiment of the present invention.

FIG. 11 is a diagram showing a configuration of a solid-state image pickup element as an image pickup means according to a fifth embodiment.

FIG. 12 is a block diagram showing a configuration of a conventional solid-state imaging device.

FIG. 13 is a photoelectric characteristic diagram showing a relationship between a signal level of each part of a conventional solid-state imaging device and an incident light amount.

[Explanation of symbols]

1 solid-state image sensor 2 signal level adjusting means 3A smear extracting means 3B smear extracting means 3C smear extracting means 3D smear extracting means 4 smear removing means 5A image combining means 5B image combining means 5C pixel combining means 6 photoelectric conversion section 11 adjusting section 12th 1 Subtraction unit 13
Second subtraction unit 14 Adder 15A
Comparator 15B Comparator 16
Selector 20 Base clip circuit 21
Level detector 22 Smear attenuator 23
Clip circuit 30 Solid-state image sensor

─────────────────────────────────────────────────── ─── Continuation of front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) H04N 5/235 H04N 5/335

Claims (9)

(57) [Claims] 1. A solid-state image pickup device comprising image pickup means for outputting a plurality of image signals having different exposure amounts during a period for picking up an image for one screen, and adjusting the signal level of the image signals having different exposure amounts. A signal level adjusting means, a smear extracting means for extracting a smear component from the image signal having a different exposure amount adjusted in level, a smear removing means for removing the extracted smear component from the image signal having a different exposure amount, and a smear removed An image synthesizing unit for synthesizing image signals having different exposure amounts. 2. The solid-state imaging device according to claim 1, wherein the signal level adjusting means causes the signal level of the image signal with a large exposure amount to substantially match the signal level of the image signal with a small exposure amount. A solid-state imaging device comprising an adjusting unit for adjusting. 3. The solid-state imaging device according to claim 2, wherein the adjustment unit includes an attenuator that attenuates a signal level of an image signal having a large exposure amount. 4. The solid-state image pickup device according to claim 1, wherein the smear extraction unit starts from an image signal with a small exposure amount,
A solid-state imaging device comprising: a first subtraction unit that performs a subtraction process on an image signal having a large exposure amount, the level of which has been adjusted by the signal level adjusting unit. 5. The solid-state image pickup device according to claim 1, wherein the smear removing unit subtracts the smear component extracted by the smear extracting unit from an image signal having a large exposure amount. A solid-state imaging device comprising a 2 subtraction unit. 6. The solid-state imaging device according to claim 1, wherein the image synthesizing unit determines whether or not a signal level change of an image signal having a large exposure amount has linearity. When the first determination unit and the first determination unit determine that the signal level change has linearity, the smear component-removed image signal having a large exposure amount is output as a composite signal while the first determination unit
When the determination unit determines that the signal level change does not have linearity, the image signal with a small exposure amount maintains the gradation continuity with respect to the image signal with a large exposure amount with the smear component removed. A solid-state imaging device, comprising: a signal selection unit that performs processing and outputs a combined signal. 7. The solid-state image pickup device according to claim 1, wherein the smear extraction means detects a signal level of the extracted smear component, and the level detection part detects the level. The solid-state imaging device further comprising: a first smear suppressing unit that suppresses the smear component when the signal level of the smear component does not reach a predetermined level. 8. The solid-state imaging device according to claim 1, wherein the smear extraction unit determines whether or not a signal level change of an image signal having a large exposure amount has linearity. And a second smear suppressing unit for suppressing and outputting the extracted smear component when the second judging unit judges that the signal level change has no linearity. Solid-state imaging device. 9. The solid-state imaging device according to claim 1, wherein the imaging unit has a solid-state imaging device in which color filters having different spectral characteristics are arranged for each pixel, and A solid-state imaging device, wherein the smear extraction performed by the smear extraction unit and the smear removal performed by the smear removal unit are performed for each pixel.