JP5747238B2 - Solid-state imaging device for motion detection and motion detection system - Google Patents

Description

The present invention relates to a solid-state imaging device for motion detection that detects a motion of an object on an imaging screen, and a motion detection system.

In the solid-state imaging device market, CMOS sensor types for mobile phone cameras have been rapidly increasing in recent years. Each pixel cell of the CMOS sensor type solid-state imaging device has a photoelectric conversion unit (hereinafter referred to as a pixel) that generates a signal charge corresponding to incident light, and an amplification that converts the signal charge of the photoelectric conversion unit into a signal voltage and amplifies it. Department.

The solid-state imaging device may not only capture an object (subject) but also detect the movement of the object on the imaging screen of the solid-state imaging device. For example, the same field of view is continuously imaged, and it is detected that a person has entered the field of view, and various devices are controlled in response to the person's intrusion, or the intrusion of this person is notified. Record.

As a method of detecting the movement of an object on such an imaging screen, there is a method of detecting a movement by comparing signal outputs in the same pixel unit between previous and subsequent frames (method 1). In addition, by changing the photoelectric conversion time for a pair of adjacent pixels and adjusting the gain of the amplifier circuit with the photoelectric conversion time ratio so that the signal amount is equivalent, the signal difference processing is performed, so that the output signal is zero for a stationary object In the case of a moving object, there is a method (method 2) for detecting motion by generating an output signal, which is disclosed in Patent Document 1.

Since the former method 1 requires a frame memory, the apparatus becomes larger due to an increase in power consumption and an increase in circuit scale. In addition, costs will increase.

In the latter method 2, the first and second pixels that are spatially adjacent to each other are treated as one set, and these pixel sets are treated as one sensing unit. For example, the photoelectric conversion time of the second pixel is set as the photoelectric conversion time of the first pixel. The conversion time is doubled, and the motion of the object on the imaging screen is detected based on the difference between the signal of the first pixel and the signal of 1/2 of the second pixel. At this time, if a change occurs in the signals of the first and second pixels during the second photoelectric conversion time that is longer than the first photoelectric conversion time, it can be assumed that the object on the imaging screen has moved. . That is, when the object on the imaging screen is stationary, the difference is 0, and if there is movement, the difference occurs. Method 2 can perform motion detection without requiring a frame memory.

However, the difference does not become 0 for a normal stationary object as described above. This is because the difference signal does not become 0 because the signal amount of the two pixels in the pair is different at the end of the photographed object, and the object is erroneously recognized as moving. In the case of color imaging, when the Bayer method, which is a typical color filter arrangement, is applied, the same color pixel at the position where one pixel is skipped is made into two pixels in a pair, so that the difference signal amount and the frequency of occurrence further increase. Therefore, the recognition rate of moving objects further deteriorates.

JP 10-290400 A

The present invention is a solid-state imaging device for motion detection that detects the motion of an object on an imaging screen, and provides a pixel configuration and a calculation system that suppress an error signal (noise signal) generated in the contour of a subject. Accordingly, it is an object of the present invention to provide a motion detection solid-state imaging device that not only simplifies subsequent signal processing but also improves the motion detection capability of a moving object.

The solid-state imaging device for motion detection according to the present invention includes two 2 × 2 pixels arranged in the vertical and horizontal directions as one set, and two adjacent pixels in the diagonal direction as one set, and an oblique direction perpendicular thereto. The remaining two pixels are set as one set, and the photoelectric conversion time between the two sets is changed. Next, the output signal amount of the set on one side is adjusted by the ratio of the two photoelectric conversion times, and the difference between the two is detected to detect the motion, and the contour of the subject is averaged. Suppresses generated noise.
Further, the adjustment of the signal amount due to the difference in photoelectric conversion time may be performed by adjusting the pixel size of each of the two sets.

 ADVANTAGE OF THE INVENTION According to this invention, the solid-state imaging device which suppresses the noise generation | occurrence | production in an outline and performs a motion detection can be provided. Since noise at the contour can be suppressed, the detection capability of a moving object can be improved, and a motion detection imaging system with low power consumption can be provided.

These are figures which show the whole structure of the solid-state imaging device for motion detection of this invention. FIG. 3 is a diagram for explaining a cell layout, a motion detection circuit configuration, and a driving method according to the present invention. FIG. 6 is a diagram illustrating a photoelectric conversion time change of an output level of a pair of pixels. These are diagrams explaining the driving method of the present invention, and are diagrams showing timing charts generated in each signal line and pulse line. These are the figures explaining the circuit structure for obtaining a normal captured image, performing the motion detection of this invention. These are figures explaining another motion detection circuit configuration and driving method of the present invention. FIG. 6 is a diagram illustrating a photoelectric conversion time change of an output level of a pair of pixels. FIG. 8 is a diagram for explaining a driving method, and is a diagram illustrating a timing chart generated in each signal line and pulse line. FIG. 4 is a specific pixel cell configuration of the present invention, and is a diagram showing the positional relationship of the pattern edge with respect to the pixel when the pattern edge position moves in the vertical direction of the pixel arrangement from the vertical pixel gap. is there. FIG. 5 is a diagram showing a signal output pattern generated in a pixel and a motion detection signal pattern after the pixel signal processing of the present invention at the timing shown in FIG. Is a specific pixel cell configuration of the present invention, and in the three cases where the pattern edge position and the pixel are displaced in the vertical direction while the pattern edge position exists in the vertical pixel gap. It is a figure which shows these positional relationships. FIG. 6 is a diagram showing a signal output pattern generated in a pixel and a motion detection signal pattern after undergoing the pixel signal processing of the present invention in the case of the positional relationship shown in FIG. These are figures which show the whole structure which has arrange | positioned the color filter of the solid-state imaging device for motion detection of this invention. Is a configuration of a pixel cell in which a specific color filter of the present invention is arranged, and the pattern edge portion relative to the pixel when the pattern edge position moves in the vertical direction of the pixel arrangement from the vertical pixel center portion. It is a figure which shows a positional relationship. FIG. 8 is a diagram showing a signal output pattern generated in a pixel and a motion detection signal pattern after the pixel signal processing of the present invention at the timing shown in FIG. Is a configuration of a pixel cell in which the specific color filter of the present invention is arranged, and the positional relationship between the pattern end and the pixel is shifted in the vertical direction in a state where the position of the pattern end exists in the center of the pixel in the vertical direction. It is the figure which shows the positional relationship in three cases. FIG. 9 is a diagram showing a signal output pattern generated in a pixel and a motion detection signal pattern after the pixel signal processing according to the present invention in the case of the positional relationship shown in FIG. These are another Examples of this invention, and are figures which show another whole structure of the solid-state imaging device for motion detection which made the readout part common in 4 pixels 1 cell. FIG. 10 is a diagram showing a specific cell configuration of a 4-pixel 1-cell in the embodiment of FIG. 9; These are still another embodiments of the present invention, and are a diagram illustrating a configuration of a 4-pixel 1 cell sharing an output circuit and a configuration of an output circuit at the end of a vertical signal line. FIG. 4 is a diagram for explaining a driving method, and is a diagram showing a photoelectric conversion time change of an output level of a pair of pixels. These are diagrams explaining the driving method of the present invention, and are diagrams showing timing charts generated in each signal line and pulse line. FIG. 5 is still another embodiment of the present invention, and is a diagram showing a configuration diagram of a 4-pixel 1-cell in which the size of a pixel in a cell is changed and sensitivity is changed, and a configuration of an output circuit at the end of a vertical signal line. FIG. 4 is a diagram for explaining a driving method, and is a diagram showing a photoelectric conversion time change of an output level of a pair of pixels. These are diagrams explaining the driving method of the present invention, and are diagrams showing timing charts generated in each signal line and pulse line. FIG. 13 is a diagram showing a repetitive cell arrangement of pixels and output circuits when color filters are arranged in the cell having the configuration shown in FIG. FIG. 13 is a diagram showing an embodiment in which pixels and output cell repetitive cells are arranged efficiently when the color filter is arranged in the cell having the configuration shown in FIG. Is a specific pixel cell configuration of the present invention, and is a configuration diagram of a 4-pixel 1-cell in which the size of the pixel in the cell is changed and the sensitivity is changed, corresponding to FIG. It is a figure which shows the positional relationship of the pattern edge part with respect to a pixel in three timings shown in below-mentioned FIG.15 (b) at the time of moving to. FIG. 4 is a diagram for explaining a driving method, and is a diagram showing a photoelectric conversion time change of an output level of a pair of pixels. FIG. 15B is a diagram showing a signal output pattern generated in a pixel at two timings shown in FIGS. 15B and 15C and a motion detection signal pattern after the pixel signal processing of the present invention. is there. These are figures which show the whole structure of the motion detection system to which this invention is applied. These are the whole block diagrams which show the modification of the motion detection system to which this invention is applied. These are figures which show the whole structure explaining changing the whole drive method based on the motion detection information of the solid-state imaging device for motion detection of this invention.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. It should be noted that the scale of the drawings is emphasized for easy understanding of the features of the invention and is not necessarily the same as the scale of an actual device.

 FIG. 1 is a diagram showing the overall configuration of a solid-state imaging device for motion detection according to the present invention. The solid-state imaging device for motion detection includes a timing generation circuit 24 for controlling the overall timing, an accumulation time control circuit 25 for controlling the photoelectric conversion time (accumulation time) of a pixel with two different accumulation times, and vertical scanning. The circuit 14 includes a motion detection circuit 7, an ADC circuit 8, a horizontal scanning circuit 13, and a pixel group 10.

The pixel arrangement for the motion detection operation is 2 × 2 4 pixels to form one cell. In the figure, pixel 1 (a11), pixel 1 (a'11), pixel 1 (b'11), and pixel 1 (b11) are one cell. Different read wirings (READ1, READ1 ') are applied to the gates of the read transistors 2, 2' of the pixel 1 (a11) and the pixel 1 (a'11) which are paired in the horizontal direction. The electric charge photoelectrically converted and accumulated in the pixel for a certain period of time is transferred under the gates of the read transistors 2 and 2 ′ and flows into the floating junction 3. The potential of the floating junction 3 modulates the gate of the amplification transistor 5. The pixel 1 (a11) and the pixel 1 (a'11) are connected to different vertical signal lines 6 and 6 ', respectively, and are read out as potential changes via the vertical signal lines 6 and 6'.

The floating junction 3 is reset (RESET) to a constant voltage via the gate of the reset transistor 4 before the signal charge is transferred. The pixel 1 (b′11) and the pixel 1 (b11) which are paired in the horizontal direction of the second row of one cell have different readout wirings (READ2 ′ and READ2) in the readout gates 2 ′ and 2 of the pixel 1 (b11), respectively. Applied. Similarly to the pixel 1 (a11) and the pixel 1 (a′11), the pixel 1 (b′11) and the pixel 1 (b11) are connected to different vertical signal lines 6 and 6 ′, respectively. It is read out as a potential change through lines 6 and 6 '.

The vertical signal lines 6, 6 ′ are connected to the motion detection circuit 7 at their ends, and further to an ADC (Analog to Digital Converter) circuit 8. From the ADC circuit 8, the ADC output corresponding to the vertical signal lines 6 and 6 ′ is sequentially read out by the horizontal scanning circuit 13 as a 10-bit digital signal. The motion detection circuit 7 outputs one motion detection signal from the pair of vertical signal lines 6 and 6 ′. The motion detection signal is output to the horizontal signal line 12 via the horizontal readout gate 11.

Pulses applied to the horizontal readout gate 11 are generated by the horizontal scanning circuit 13 and sequentially read out cell by cell in the horizontal direction. Similarly, a pulse (for example, READ1, 1 ′) applied to the gates of the read transistors 2 and 2 ′ and a pulse (RESET) applied to the gate of the reset transistor 4 for reading the cells in the vertical direction are: Generated by the vertical scanning circuit 14 and sequentially read out line by line in the vertical direction.

FIG. 2A is a view for explaining the layout of the unit cell of the present invention shown in FIG. 1, the configuration of the motion detection circuit, and the driving method. In the following drawings, components having the same function are denoted by the same number. As shown in FIG. 1, the arrangement of the pixels is 2 × 2 four pixels (pixel 1 (a11), pixel 1 (a′11), pixel 1 (b′11), pixel 1 (b11)). The cell is configured. Different readout wirings (READ1, READ1 ′) are applied to the readout gates 2, 2 ′ of the pixel 1 (a11) and the pixel 1 (a′11) which are paired in the horizontal direction, respectively, and the remaining horizontal direction Separate read wirings (READ2 ′, READ2) are applied to the gates of the read transistors 2 ′, 2 of the pixel 1 (b′11) and pixel 1 (b11) which are paired with each other. Further, the pixel 1 (a11) and the pixel 1 (b′11) adjacent in the vertical direction are connected to the vertical signal line 6, and the other pixel 1 (a′11) and the pixel 1 (b11) adjacent to each other in the vertical direction are connected. Reading is performed by being connected to another vertical signal line 6 ′.

In FIG. 2A, LOAD Tr (load transistor) is provided at the ends of the vertical signal lines 6 and 6 '.
The potential of Tr is connected to the positive side of the capacitor and the differential amplifier circuit 15 via the switch circuit (SW1). The potential of LOAD Tr is also connected to the negative side of the differential amplifier circuit 15 through the amplifier 16 via the switch circuit (SW2). The output of the differential amplifier 15 is referred to as a differential output 17. If the amplifier 16 is removed from this circuit configuration, it is a CDS (Correlated Double Sampling) circuit configuration generally employed in conventional CMOS sensors. The same applies to the vertical signal line 6 ', and the corresponding parts are a differential amplifier circuit 15', an amplifier 16 ', and a differential output 17', respectively.

Two adjacent differential outputs 17 and 17 ′ are input to the adder circuit 18. The addition output 19 enters the noise removal circuit 20, and the noise removal addition output 21 is connected to the input side of the horizontal readout gate 11 connected to the horizontal signal line 12. The noise removal circuit 20 can be replaced by the above-described CDS circuit configuration. Although not shown in the figure, the noise removal addition output 21 may be connected to the input side of the horizontal readout gate 11 after passing through the ADC circuit 8 in the same manner as the vertical signal lines 6 and 6 ′. On the other hand, LOAD
The potential of Tr extends to the vertical signal lines 6 and 6 ′ and is connected to the ADC circuit 8 through the switch circuit (SW 3). Although not shown, a normal CDS circuit configuration is inserted between the switch circuit (SW 3) and the ADC circuit 8. Alternatively, the CDS operation can be performed by ADC digital processing.

FIG. 2B shows the photoelectric conversion time change of the output level of the pair of pixel 1 (a11), pixel 1 (a′11), and another pair of pixel 1 (b′11) and pixel 1 (b11). FIG. In this case, the photoelectric conversion efficiency (referred to as sensitivity in the following description and corresponding to the slope of the graph in the figure) of the pixel is different from that of the pair of pixel 1 (a11) and pixel 1 (a'11). In the following description, the same pixel 4 (b'11) and pixel 1 (b11) have the same four pixels. Further, the photoelectric conversion times (referred to as first photoelectric conversion times) of the first pixel 1 (a11) and the pixel 1 (b11) adjacent in the oblique direction are made the same. The remaining photoelectric conversion times (referred to as second photoelectric conversion times) of the second pixel (a11 ′) and pixel 1 (b′11) adjacent in the diagonal direction are also set to be the same, and the second photoelectric conversion time is set to the first photoelectric conversion time. It is longer than the time t1 (in the figure, the photoelectric conversion time is doubled). In order to double the ratio of the photoelectric conversion time between the first and second photoelectric conversion times and to substantially match the end points of the two photoelectric conversion times (first photoelectric conversion time, second photoelectric conversion time), READ1 and 2 pulses for discarding the pixel signal are applied to the gate of the read transistor 2 for controlling the photoelectric conversion time of the first pixel 1 (a11) and the pixel 1 (b11), and the potential of the floating junction 3 is applied. Is reset, a reset pulse (indicated by RS in the figure) is applied to the gate of the reset transistor 4.

FIG. 2C is a diagram for explaining the driving method of the present invention, and in particular, a diagram for explaining a sequence of reading out signal output from the pixels at the end of two substantially identical photoelectric conversion times. It is. A timing chart generated in each signal line and pulse line is shown. First, the floating junction 3 is reset (RESET) to a constant voltage via the gate of the reset transistor 4 before the signal charge is transferred (timing T1; hereinafter abbreviated as T1). At this time, the induction of the reset pulse is superimposed on the floating junction 3 as noise, and noise components appear on the vertical signal lines 6 and 6 '(T2). Next, the charges that are photoelectrically converted and accumulated in the first pixel 1 (a11) and the pixel 1 (b11) in the first photoelectric conversion time t1 are applied to the readout gate 2 by applying pulses of the readout wirings (READ1, 2). Thus, it flows into the floating junction 3 (T3).

The signal components of the pixel 1 (a11) and the pixel 1 (b11) are superimposed on the vertical signal lines 6 and 6 'where the noise component is output. Before the floating junction 3 is reset for the next reading, SW1 is closed (ON), and the output levels of the vertical signal lines 6 and 6 'are held in the capacitors. Next, after SW1 is opened (OFF), the floating junction 3 is reset (T4). In this case as well, the induction of the reset pulse is superimposed as noise, and noise components appear on the vertical signal lines 6 and 6 '(T5). Next, the electric charges photoelectrically converted and accumulated in the second pixel (a11 ′) and the pixel 1 (b′11) in the second photoelectric conversion time t2 (2 × t1) are read out wirings (READ1 ′, 2 ′). Is applied to the read gate 2 ′, and flows into the floating junction 3 (T6).

The signal components of the pixel 1 (a'11) and the pixel 1 (b'11) are superimposed on the vertical signal lines 6 and 6 'where noise components are output. Before the floating junction 3 is reset for the next reading, SW2 is closed (ON), the output level of the vertical signal lines 6 and 6 ′ is amplified by the amplifier 16, and the differential amplifier circuit 15 is brought to the negative side. input. The gain of this amplifier is ½, which is the ratio of the first photoelectric conversion time and the second photoelectric conversion time in the above case.

The signal components of the pixel 1 (a11) and the pixel 1 (b11), which are stored in the capacitors, are input to the plus side of the differential amplifier circuits 15 and 15 ', respectively. The signal components of the pixel 1 (a′11) and the pixel 1 (b′11) multiplied by the gain are input to the minus side of the differential amplifier circuits 15 and 15 ′. Therefore, the differential outputs 17 and 17 ′, which are the outputs of the differential amplifiers 15 and 15 ′, are the difference between the pixel 1 (a11) and the pixel 1 (b′11), the pixel 1 (b11), and the pixel, respectively. A difference of 1 (a′11) is output. Further, since there is an adder circuit 18 ahead of the differential outputs 17 and 17 ′, the pixel 1 (a′11) added to the multiplication from the pixel 1 (a11) + pixel 1 (b11) added to the multiplication. ) + The difference from ½ of the pixel 1 (b′11) is the summed output 19 of the summing circuit 18. An output obtained by processing this with the noise removal circuit 20 having the CDS circuit configuration is output as a noise removal addition output 21.

When the subject is stationary, a signal obtained by doubling the photoelectric conversion time is multiplied by a gain of 1/2 by the amplifier 16 to obtain a difference, so that a motion detection output (noise removal difference output 21 ) Is zero. On the other hand, when there is motion, the brightness of the subject changes within the photoelectric conversion time, so the motion detection output (noise removal difference output 21) does not become zero.

Thus, in the present invention, a plurality of pixels are arranged in a matrix to form an imaging screen, and the first and second pixels that are spatially close to each other are taken as one set, and the first pixel 1 photoelectric conversion time t1 is made shorter than the second photoelectric conversion time t2 of the second pixel, the end points of the first and second photoelectric conversion times are substantially matched, and after the first and second photoelectric conversion times, Both the signals of the first and second pixels are read out, the product of the ratio of the first photoelectric conversion time to the second photoelectric conversion time and the signal of the second pixel is obtained, and based on the difference between this product and the signal of the first pixel The movement of the object on the imaging screen can be detected.

The outputs of the vertical signal lines 6 and 6 ′ are input to the ADC circuit 8 through the CDS circuit (not shown) by closing (ON) the switch circuit (SW3), and converted into, for example, a 10-bit digital signal. Output. However, in FIGS. 2A, 2B, and 2C, pixel 1 (a11) and pixel 1 (b11) are combined as the first pixel, and pixel 1 (a′11) is combined as the second pixel. Since the pixels 1 (b′11) are combined, the horizontal scanning lines come out in a zigzag pattern shifted by one line even if they are said sequentially. Moreover, it is output as a composite image of images that are two times different from the output. This is an image output in a mode with an expanded dynamic range. In order to output in the same way as usual, it is necessary to slightly change the circuit as shown below.

First, the latter method will be described with reference to FIG. 2 (d), which is related to a method for preventing the output from being output as a composite image of images that differ by twice the output. In FIG. 2 (d), the vertical signal lines 6 and 6 'are provided with a switch circuit (SW3). In parallel therewith, a switch circuit (SW4) is installed, and an amplifier 22 is installed at the end. Connect the tip to the tip of SW3. When reading the second pixel (for example, pixel 1 (a′11), pixel 1 (b′11)) of the second photoelectric conversion time, SW3 is opened (OFF), SW4 is closed (ON), and SW4 By setting the gain of the previous amplifier to ½, in the case of imaging a subject with the same brightness, the output of each pixel can be made the same.

The former method for preventing the horizontal scanning line from becoming zigzag will be described with reference to FIGS. 3 (a), 3 (b), and 3 (c). 3 (a) is similar to FIG. 2 (d), the difference being the LOAD at the end of the vertical signal line 6 ′.
This is a circuit after Tr, and an amplifier 16 'is inserted between the switch circuit (SW1) and the positive side of the capacitor and the differential amplifier circuit 15, and is connected to the negative side of the differential amplifier 15', and is connected to the switch circuit SW2 side. Is connected to the positive side of the differential amplifier 15 'without being inserted.

FIG. 3B shows the photoelectric conversion time change of the output level of the pair of pixel 1 (a11) and pixel 1 (a′11) and another pair of pixel 1 (b′11) and pixel 1 (b11). It is a figure and is the same as FIG.2 (b).

In FIG. 3C, READ1 and 1 'are read at the same timing, and the pixels 1 (a11) and 1 (a'11) adjacent in the horizontal direction are simultaneously transferred to the vertical signal lines 6 and 6'. In the motion detection circuit 7, SW2 is opened (OFF), SW1 is closed (ON), and the capacitor holds the potential of the vertical signal line. READ2 and 2 'are read out at the same timing at the next timing, and pixel 1 (b'11) and pixel 1 (b11) adjacent in the horizontal direction are simultaneously transferred to the vertical signal lines 6 and 6'. In the motion detection circuit 7, SW1 is opened (OFF), SW2 is closed (ON), and the differential amplifiers 15 and 15 'take the difference.

The signal component of the pixel 1 (a11) stored in the capacitor is on the plus side of the differential amplifier circuit 15, and the pixel 1 (a′11) stored in the capacitor is on the minus side of the differential amplifier circuit 15 ′. ) Is multiplied by the gain of the amplifier 16 '. On the negative side of the differential amplifier circuit 15, the signal component of the pixel 1 (b'11) multiplied by the gain is provided. On the positive side of the differential amplifier circuit 15 ', the signal component of the pixel 1 (b11) is provided. Have been entered. Therefore, the differential outputs 17 and 17 ′, which are the outputs of the differential amplifiers 15 and 15 ′, are the difference between the pixel 1 (a11) and the pixel 1 (b′11), the pixel 1 (b11), and the pixel, respectively. A difference of 1 (a′11) is output. Further, since there is an adder circuit 18 ahead of the differential outputs 17 and 17 ′, the pixel 1 (a′11) added to the multiplication from the pixel 1 (a11) + pixel 1 (b11) added to the multiplication. ) + The difference from ½ of the pixel 1 (b′11) is the summed output 19 of the summing circuit 18. An output obtained by processing this with the noise removal circuit 20 having the CDS circuit configuration is output as a noise removal addition output 21. This is the same as the description of FIG.

As already described with reference to FIG. 2D, in the case of imaging an object with the same brightness, the second pixel (for example, pixel 1 (a′11), pixel 1 (b′11)) in the second photoelectric conversion time. Is gained by the amplifier 22, and is input to the ADC circuit with the same output as the first pixel (for example, the pixel 1 (a11) and the pixel 1 (b11)) in the first photoelectric conversion time. Reading from the pair of pixels 1 (a11) and 1 (a'11) on the same horizontal scanning line simultaneously with READ1 and 1 'is performed in the ADC circuit, which is the same as the line sequential reading in a normal CMOS sensor. You can do it. In this manner, the problem that the output is output in a zigzag shape, which is a problem in FIG. 2A, can be solved. Further, as described above, since the difference in photoelectric conversion time can be adjusted by adjusting the gain, an output difference can be eliminated, so that a motion detection solid-state imaging device capable of outputting an image with a normal CMOS sensor can be provided. .

In the configuration of the present invention shown in FIG. 2D and FIG. 3A, since the switch circuit (SW1, SW2, SW3, SW4) is provided, it is determined that the level of the motion output 21 is small. At this time, the switch circuits (SW3, SW4) are opened (OFF) so that the imaging output of the pixels does not go to the ADC circuit. Therefore, the drive waveform applied to the ADC and the circuit power supply voltage can be controlled by the movement output level. Further, the horizontal scanning circuit can be turned off. This is useful for reducing power consumption.

FIG. 4A is a diagram showing the repetitive arrangement of unit cells in the motion detection imaging apparatus according to the present invention shown in FIGS. 2A and 3A. FIG. 4A shows coordinate positions of A, B, C, and D in the horizontal direction and 1, 2, 3, 4, 5, 6, 7, and 8 in the vertical direction as pixel coordinates. A diagram in which the positions of the object end portion of the subject and the pixel arrangement are changed is also attached. In FIG. 4A, the object end of the black pattern subject extends in the horizontal direction, and the end position in the vertical direction is at the boundary between coordinates 2 and 3 on the upper side and coordinates 5 and coordinates on the lower side. The state at the boundary of 6 is defined as an initial state. A case in which the subject image moves downward by one pixel during the first photoelectric conversion time t1 shown in FIGS. 2B and 3B is indicated by Timing1. Timing 1 is a timing immediately after resetting of pixels (for example, a11, a12, b11, and b12) to which movement for one pixel is completed and '(dash) is not added. A case in which the object of the subject moves further downward by one pixel during the next first photoelectric conversion time t1 is indicated by Timing2. As shown in FIG. 4A, the movement for two pixels is completed in the movement period of 2 × t1, and the timing immediately before the reading from the pixel is Timing2. It is assumed that the movement of the subject is performed at a constant speed.

FIG. 4 (b) shows what kind of output is obtained at each pixel at each coordinate at the three timings (Initial, Timing1, Timing2) shown in FIG. 4 (a), and motion detection is performed by calculating them. The table shows how the signal looks.
In calculating this output, the second photoelectric conversion time t2 is twice as long as the first photoelectric conversion time t1, the pixel output is zero in the black pattern portion, and the second photoelectric conversion time corresponds to the second photoelectric conversion time in the portion other than the black pattern. The output of all the pixels (pixels marked with ') is 1, and the output of the pixels corresponding to the first photoelectric conversion time (pixels not marked with') is 0.5. At the pattern edge, the light intensity changed to a step shape, and crosstalk (leakage) to adjacent pixels was zero. In addition, it was assumed that all black patterns were extended with the same strength in the vertical direction. In the following description of the figure, all of these same preconditions are used.

Assuming that the movement of the subject is performed at a constant speed, when the black pattern end moves from the upper end to the lower end of the pixel (coordinates) during the first photoelectric conversion time, or when the entire surface is exposed during the first photoelectric conversion time The corresponding pixel output is set to half of 0.5. Further, the incident intensity to the pixel when the end of the black pattern comes to the center of the pixel (coordinates) is half that when the entire surface is exposed. The same premise is used in the following description of the figure.

In FIG. 4B, the initial timing is immediately after the start of photoelectric conversion, so the output of all the pixels is zero.

In Timing1, since the pixels (a11, a12, b11, b12, a21, a22, b21, b22,...) Corresponding to the first photoelectric conversion time are read and reset, the output is 0. On the other hand, in the pixels corresponding to the second photoelectric conversion time (a′11, a′12, b′11, b′12, a′21, a′22, b′21, b′22,. It is 0.5 for the selected pixel and 0 for the pixel corresponding to the black pattern and not exposed. In the pixel where the end of the black pattern has moved, the output is 0.25.

In Timing 2, the pixel corresponding to the first photoelectric conversion time is 0.5 for a pixel that is exposed on the entire surface, and is 0 for a pixel that corresponds to a black pattern and is not exposed. In the pixel (b21, b22, a41, a42) in which the end of the black pattern has moved, the output is 0.25. On the other hand, in the pixel corresponding to the second photoelectric conversion time, it is 1 for the pixel that is exposed to the whole surface during the second photoelectric conversion time, and 0 for the pixel that corresponds to the black pattern and is not exposed. For the pixels (a′21, a′22) in which the end of the black pattern is moved by Timing 1 and becomes bright, the output is 0.75. On the other hand, the edge of the black pattern is moved at Timing 1 and the output becomes 0.25 in the darkened pixels (b′31 and b′32). The edge of the black pattern is moved and brightened at Timing 2. The pixel (b′21, b′22) is 0.25, and the darkened pixel (a′41, a′42) is 0.75.

A motion detection signal is calculated from this Timing2. As described with reference to FIGS. 2 and 3, the calculation method is a method of calculating a difference after converting the sum of the multiplying pixels and the sum of the remaining multiplying pixels into a photoelectric conversion time. As an example, when the 2 × 2 pixel block at the upper left of Timing 2 is calculated, (0.5 + 0.5) − (1 + 1) / 2 = 0. When the 2 × 2 pixel block under Timing 2 is calculated, (0.5 + 0.25) − (0.25 + 0.75) /2=0.25. Similarly, when the calculation is performed with the lower 2 × 2 pixel block, (0 + 0) − (0.25 + 0) /2=−0.125. Similarly, when calculating with the bottom 2 × 2 pixel block, (0.25 + 0.5) − (1 + 0.75) /2=−0.125. When calculating with a 2 × 2 pixel block in a portion without a black pattern, (0.5 + 0.5) − (1 + 1) / 2 = 0. In the pixel block of the part covered with the black pattern, (0 + 0) − (0 + 0) / 2 = 0, and like the 2 × 2 pixel block in the upper left which is always bright, it becomes 0 in the part without motion. That is, a motion detection signal is generated at the contour portion of a pattern having motion. The output pattern table of the motion detection signal is halved in both the horizontal direction and the vertical direction, and is a 2 × 4 table in FIG.

FIG. 5A is a diagram showing the repetitive arrangement of unit cells in the same motion detection imaging apparatus according to the present invention as in FIG. FIG. 5A also shows the coordinate positions of A, B, C, and D in the horizontal direction and 1, 2, 3, 4, 5, 6, 7, and 8 in the vertical direction as pixel coordinates.
Using this, the output position of the motion detection signal when the object of the subject does not move is determined, and the object position of the subject is set one pixel up and down around the initial position in FIG. Estimate the 3 cases that have moved.

FIG. 5A also includes a diagram in which the relative position between the object end of the subject and the pixel arrangement is changed. In FIG. 5A, as Case 1, the end of the black pattern extends in the horizontal direction, and the position of the end of the vertical direction is the boundary between the coordinates 1 and 2 on the upper side and the boundary between the coordinates 4 and 5 on the lower side. If you come to. Cases 2 and 3 were obtained when the black pattern was sequentially shifted downward by one pixel from Case 1. In Cases 1, 2 and 3, the black object remains stationary.

FIG. 5 (b) is a table showing the output of each pixel at each coordinate in the case of Cases 1, 2, and 3, and the motion detection signal obtained by calculating them. Yes. The preconditions for calculating this output are the same as those in FIG.

FIG. 5B shows a table showing output values generated in pixels of each coordinate for each case (Case1, Case2, Case3) of the black pattern position of the subject. A motion detection signal is calculated from Cases 1, 2, and 3. As an example, when calculating with the upper left pixel block of Case1, (0.5 + 0) − (0 + 1) / 2 = 0. Similarly, when calculating with the lower left pixel block of Case 1, (0.5 + 0.5) − (1 + 1) / 2 = 0. When this calculation is applied to all the pixel blocks, all are 0. Similarly, other Cases are 0, and these results are shown in Cases 1, 2, and 3 at the bottom of FIG. Since only one motion signal is output to each pixel block (2 × 2 pixels), the table of motion signal output is halved both in the horizontal direction and in the vertical direction. It becomes a table | surface of x4.

When performing color imaging, a Bayer arrangement method that is usually used is a basic arrangement of 2 (horizontal) × 2 (vertical) pixels, and the color filter has two adjacent green (G) pixels in the diagonal direction and the remaining red (R ) And blue (B) pixels. For convenience, the two G are distinguished by changing the horizontal line of R with Gr and the horizontal line of B with Gb. If an attempt is made to configure the motion detection imaging device of Patent Document 1 using the normal Bayer method, it is necessary to change the photoelectric conversion time with the same color pixel at the position where one pixel is skipped. When a pixel pair of the same color whose photoelectric conversion time for motion detection is changed is separated, a noise signal as a motion detection signal is generated by signal processing at the edge of the imaging pattern even for a stationary object. Further, the same color signal on the same vertical signal line skips one pixel, and there is a problem that signal processing becomes complicated.

FIG. 6 is a diagram showing the overall configuration of a motion detection color imaging apparatus according to the present invention. The feature of the arrangement of the solid-state imaging device pixels for motion detection according to the present invention in which 2 × 2 4 pixels form one cell is also advantageous in color imaging.
Corresponding to one pixel cell in FIG. 1 (pixel 1 (a11), pixel 1 (a′11), pixel 1 (b′11), pixel 1 (b11)), color filters of the same color are arranged. In FIG. 6, Green (hereinafter abbreviated as G, g, Red line on Gr, gr, and Blue line on Gb, gb) are arranged, corresponding to each pixel cell, pixel 1 (Gr11), pixel 1 (Gr′11), pixel 1 (gr′11), and pixel 1 (gr11). Similarly, Red (hereinafter abbreviated as “R” and “r”) corresponds to each of the pixel cells, and is indicated by pixel 1 (R11), pixel 1 (R′11), pixel 1 (r′11), and pixel 1 (r11). . Similarly, Blue (hereinafter abbreviated as “B” and “b”) corresponds to each of the pixel cells, and is indicated by pixel 1 (B11), pixel 1 (B′11), pixel 1 (b′11), and pixel 1 (b11). . Green on the remaining Blue line is also indicated as pixel 1 (Gb11), pixel 1 (Gb′11), pixel 1 (gb′11), and pixel 1 (gb11). The pixel arrangement in FIG. 6 is arranged in the vertical and horizontal directions in units of 16 pixels in which 2 × 2 pixels and one color block are Bayer arranged in 2 × 2. This color filter is called Quad Bayer.

Although the motion detection method has been described in FIGS. 1 to 5 without considering the color filter arrangement, the color filter arrangement in FIG. 6 can also be processed by performing processing in the same color pixel block. The motion detection results 1 to 5 can be similarly applied to FIG.

The pixel 1 (Gr11), the pixel 1 (Gr′11), the pixel 1 (R11), and the pixel 1 (R′11) that are paired in the horizontal direction have different readouts for the readout gates 2 and 2 ′. Wirings (READ1, READ1 ′) are applied to change the photoelectric conversion time within the same color pixel (for example, Gr11, Gr′11). Similarly, the pixel 1 (gr′11), the pixel 1 (gr11), the pixel 1 (r′11), and the pixel 1 (r11), which are paired in the horizontal direction of the next line, have different readout wirings. (READ2, READ2 ′) is applied, and the photoelectric conversion time is changed within the same color pixel (for example, gr′11, gr11). As in FIG. 1, the first photoelectric conversion times of the first pixel 1 (Gr11) and the pixel 1 (gr11) adjacent in the oblique direction are made the same. The second photoelectric conversion time of the second pixel 1 (gr′11) and the pixel 1 (Gr′11) adjacent in the remaining diagonal direction is also made the same, and the second photoelectric conversion time is made longer than the first photoelectric conversion time. (For example, the photoelectric conversion time is doubled). This situation is the same as in FIG. Similarly, the photoelectric conversion time of 2 × 2 pixels in the other color blocks is also changed. The circuit configuration at the lower ends of the vertical signal lines 6 and 6 'is the same as that shown in FIG. The driving method of the motion detection color imaging apparatus according to the present invention shown in FIG. 4 is the same as that shown in FIG. 1, and a color digital signal is output from the ADC circuit 8 and a motion detection signal is output from the horizontal signal line 12.

FIGS. 7A and 7B are diagrams illustrating how the motion detection signal changes when the object position of the subject changes in the motion detection color imaging apparatus according to the present invention.

FIG. 7A is a diagram showing a repetitive arrangement of color blocks based on a specific component drawing of the motion detection color imaging apparatus according to the present invention shown in FIG. The pixel 1, the gate of the readout transistor 2, the floating junction 3, the gate of the reset transistor 4, the gate of the amplification transistor 5, and the vertical signal lines 6 and 6 ', which are the basic configuration of FIG. 6, are shown with reference to an actual layout diagram. . One pixel cell in FIG. 1 (pixel 1 (a11), pixel 1 (a′11), pixel 1 (b′11), and pixel 1 (b11)) corresponds to a color block. In FIG. Corresponding color blocks (Gr11, Gr′11, gr′11, gr11). For the sake of convenience, FIG. 6A shows coordinate positions of A, B, C, and D in the horizontal direction and 1, 2, 3, 4, 5, 6, 7, and 8 in the vertical direction as pixel coordinates.

FIG. 7A also includes a diagram in which the relative position between the object end of the subject and the pixel arrangement is changed. In FIG. 7A, as Case 1, the black object end portion extends in the horizontal direction, and the vertical end portion position is at the center of coordinate 3 on the upper side and at the center of coordinate 6 on the lower side. Is in the initial state. A case in which the object moves downward by one pixel during the first photoelectric conversion time t1 shown in FIGS. 2B and 3B is indicated by Timing1. Timing 1 is the timing immediately after the movement for one pixel is completed and reset. A case in which the object moves further downward by one pixel during the next first photoelectric conversion time t1 is indicated by Timing2. As shown in FIG. 4A, the movement for two pixels is completed in the movement period of 2 × t1, and the timing immediately before the reading from the pixel is Timing2. It is assumed that the object movement of the subject is performed at a constant speed.

FIG. 7 (b) shows the output of each pixel at each coordinate at the three timings (Initial, Timing1, and Timing2) shown in FIG. The table shows how the signal looks.
In calculating this output, the same preconditions as in FIG. Details are obtained by the same calculation method as in FIG. 4B, and only the result is described as a table corresponding to three timings (Initial, Timing1, and Timing2). A motion detection signal is generated at the contour portion of the moving object. The motion detection signal output pattern table is halved in both the horizontal and vertical directions, and is a 2 × 4 table in FIG. 7B.

FIG. 8A is a diagram showing the repetitive arrangement of unit cells in the same motion detection imaging apparatus according to the present invention as in FIG. 7A. FIG. 8A also shows the coordinate positions of A, B, C, and D in the horizontal direction and 1, 2, 3, 4, 5, 6, 7, and 8 in the vertical direction as pixel coordinates.
Using this, the output pattern of the motion detection signal when the object of the subject does not move is determined, and the object position of the subject is set one pixel up and down around the initial position in FIG. 7A. Estimated for 3 cases moved.

FIG. 8A also includes a diagram in which the relative position between the object end of the subject and the pixel arrangement is changed. In FIG. 8A, as Case 4, the end of the black object extends in the horizontal direction, and the end position in the vertical direction comes to the center of coordinate 2 on the upper side and to the center of coordinate 5 on the lower side. If you are. Cases 5 and 6 were obtained when the black object was sequentially shifted downward by one pixel from Case 4. The black objects in both Cases 4, 5, and 6 remain stationary.

FIG. 8B is a table showing what kind of output is obtained in each pixel at each coordinate in the case 4, 5, 6, and what the motion detection signal obtained by calculating them is. Yes. The preconditions for calculating this output are the same as those in FIG.
Details are obtained by the same calculation method as in FIG. 5B, and only the result is described. In three cases (Case 4, 5, 6), the output pattern table of the motion detection signal is half in both the horizontal and vertical directions. FIG. 8B shows a 2 × 4 table. The calculation results are all 0 for every color block.

In the above description, it is assumed that the object of the subject is a black object whose end extends in the left-right direction, and when the black object is moved in the up-down direction and the relative position in the up-down direction with respect to the pixel is changed. The calculation result was as follows.
The same calculation is possible even when the object end of the subject extends in the vertical direction and moves in the horizontal direction with respect to the pixel array. In the present invention, the multiplication operation is performed with 2 × 2 pixel cells in the pixel block, and the result is equivalent.

FIG. 9 is a diagram showing an overall configuration of a motion detection color imaging apparatus according to another embodiment of the present invention. The arrangement of the solid-state imaging device pixels for motion detection of the present invention is a configuration in which the floating junction 3, the amplifying transistor 5, and the reset transistor 4 are shared by the four pixels while maintaining the characteristic that one cell is formed by 2 × 2 4 pixels. It is.
The color filter arrangement for each color block is the same as in FIG. For example, the upper left block includes pixel 1 (Gr11), pixel 1 (Gr′11), pixel 1 (gr′11), and pixel 1 (gr11) corresponding to each pixel cell. The pixel arrangement in FIG. 9 is arranged in vertical and horizontal directions in units of 4 × 4 = 16 pixels in which four 2 × 2 pixel color blocks are Bayer arranged.

The pixel 1 (Gr11), the pixel 1 (Gr′11), the pixel 1 (R11), and the pixel 1 (R′11) that are paired in the horizontal direction have different readouts for the readout gates 2 and 2 ′. Wirings (READ1, READ1 ′) are applied, and the photoelectric conversion time is changed within the same color pixel (for example, Gr11, Gr′11) as in FIG. Similarly, the pixel 1 (gr′11), the pixel 1 (gr11), the pixel 1 (r′11), and the pixel 1 (r11), which are paired in the horizontal direction of the next line, have different readout wirings. (READ2, READ2 ′) is applied, and the photoelectric conversion time is changed within the same color pixel (for example, gr′11, gr11). Similar to FIG. 6, the first photoelectric conversion times of the first pixel 1 (Gr11) and the pixel 1 (gr11) adjacent in the oblique direction are made the same. The second photoelectric conversion time of the second pixel 1 (gr′11) and the pixel 1 (Gr′11) adjacent in the remaining diagonal direction is also made the same, and the second photoelectric conversion time is made longer than the first photoelectric conversion time. ing. This situation is the same as in FIG.

The 2 × 2 pixel photoelectric conversion time in other color blocks is similarly changed. There is only one vertical signal line 6 for each color block. The circuit configuration at the lower end of FIG. 9 is slightly different from that of FIG. However, this is a difference in configuration in the motion detection circuit 7 and the ADC circuit 8, and the horizontal scanning circuit 13, the vertical scanning circuit 14, the horizontal readout gate 11, and the horizontal readout line 12 are the same as those in FIG. A color digital signal is output from the ADC circuit 8 and a motion detection signal is output from the horizontal signal line 12.

FIG. 10 is a diagram showing the basic unit configuration of the color block based on the specific component drawing of the motion detection color imaging apparatus of the present invention of FIG. The pixel 1 of the Green color block shown as an example is composed of four pixels Gr11, Gr′11, gr′11, and gr11. The read gates 2 and 2 ', the floating junction 3, the gate of the reset transistor 4 and the gate of the amplification transistor 5 are arranged as shown in FIG. In FIG. 10, the output circuit portion can be shared by four pixels, and the degree of integration can be improved.

FIGS. 11A, 11B, and 11C are diagrams for explaining the 2 × 2 basic pixel block and the circuit ahead of the vertical signal line in FIG. 9 and driving. As a general form, the figure is described with a configuration without a color filter in contrast to FIG.

FIG. 11A is a diagram for explaining the unit cell layout, motion detection circuit configuration, and driving method according to another embodiment of the present invention shown in FIG. The arrangement of the pixels is 2 × 2 four pixels (pixel 1 (a11), pixel 1 (a′11), pixel 1 (b′11), pixel 1 (b11)) to form one cell. Different readout wirings (READ1, READ1 ′) are applied to the readout gates 2, 2 ′ of the pixel 1 (a11) and the pixel 1 (a′11) which are paired in the horizontal direction, respectively, and the remaining horizontal direction Different readout wirings (READ2 ′, READ2) are applied to the readout gates 2 ′, 2 of the pixel 1 (b′11) and the pixel 1 (b11) which are paired with each other. The charges of the four pixels flow into one floating junction 3 and are read out by the vertical signal line 6.

In FIG. 11A, a branch is made to several circuits from the junction of the end of the vertical signal line 6 to the LOAD Tr (load transistor). LOAD
The potential of Tr is connected to the positive side of the capacitor and the differential amplifier circuit 15 through the switch circuit (SW1). The potential of LOAD Tr is also connected to the negative side of the differential amplifier circuit 15 through the amplifier 16 via the switch circuit (SW2). The output of the differential amplifier 15 is referred to as a differential output 17. This differential output constitutes a motion detection signal.

After the potential of the LOAD Tr branches, it passes through the switch circuits (SW3, SW4), and only one of them merges again through the amplifier 22 and is input to the ADC circuit. The amplifier 22 performs a function of aligning pixel outputs having different photoelectric conversion times. Specifically, since pixel outputs having different photoelectric conversion times alternately appear, in synchronization with this, SW3 and SW4 are changed, and pixels having a long photoelectric conversion time (a′11 and b′11 in the figure). Is half the output through amplifier 22.

FIG. 11B shows the photoelectric conversion time change of the output level of the pair of pixel 1 (a11) and pixel 1 (a′11) and another pair of pixel 1 (b′11) and pixel 1 (b11). FIG.

FIG. 11C is a diagram for explaining the timing of RESET, READ1, 1 ′, READ2, 2 ′, output change appearing on the vertical signal line, and differential output. The pixel 1 (a11) and the pixel 1 (a′11) adjacent in the horizontal direction are sequentially applied to READ1, 1 ′, and a read pulse is applied (timing T1, T2), and transferred to the vertical signal line 6. The output of the vertical signal line 6 and the output of the vertical signal line 6 ′ at the point of joining after passing through the amplifier 22 are shown in the figure. Since the difference in photoelectric conversion time is corrected, normal imaging output is obtained, and serial input to the ADC circuit. The ADC circuit sequentially outputs a11 and a′11.

The waveform of the differential output 17 in FIG. 11A is as shown in FIG. 11C, but by taking the difference between the two (a circuit is not shown), a motion signal is output. Although the noise component is added to the differential output 17 ', the noise component can be easily removed through a normal CDS circuit (not shown). As described above, the present invention is effective even in the 4-pixel 1-cell configuration in which the output circuits shown in FIGS. 9 and 10 are shared, and both the normal image signal and the motion detection signal can be output.

The pixel sizes described above are all the same, and the sensitivity is assumed to be the same for the same color. By changing the photoelectric conversion time twice, the output difference changes twice. For this reason, the signal is reduced to ½ with an amplifier, and the output is made uniform to obtain a difference signal to obtain a motion detection signal. Providing an amplifier for each column not only complicates the configuration of the motion detection circuit, but also increases power consumption.

FIGS. 12A, 12B, and 12C are diagrams illustrating the layout of a unit cell, the configuration of a motion detection circuit, and a driving method according to another embodiment of the present invention. The arrangement of the pixels is 2 × 2 four pixels (pixel 1 (a11), pixel 1 (a′11), pixel 1 (b′11), pixel 1 (b11)) to form one cell. Different readout wirings (READ1, READ1 ′) are applied to the gates of the readout transistors 2 and 2 ′ of the pixel 1 (a11) and the pixel 1 (a′11) which are paired in the horizontal direction, respectively. Different readout wirings (READ2 ′ and READ2) are applied to the gates of the readout transistors 2 and 2 ′ of the pixel 1 (b′11) and the pixel 1 (b11) which are paired in the horizontal direction. . The charges of the four pixels flow into one floating junction 3 and are read out by the vertical signal line 6. These are not different from FIG.

In FIG. 12A, the first pixel set (pixel 1 (a11), pixel 1 (b11)) of the first photoelectric conversion time and the second pixel set (pixel 1 (b) of the second photoelectric conversion time. '11), the pixel size of the pixel 1 (a'11)) is changed, and the sensitivity is changed. Here, an example in which the sensitivity of the second pixel group is set to ½ times the sensitivity of the first pixel group while the second photoelectric conversion time is set to twice the first photoelectric conversion time will be described. .

FIG. 12B is a diagram illustrating the photoelectric conversion time change of the output level of the pixel 1 (a11), the pixel 1 (a′11), the pixel 1 (b′11), and the pixel 1 (b11). The sensitivity of the first pixel group (pixel 1 (a11), pixel 1 (b11)) is high, and reset (RS) is performed halfway. The first pixel group has a double sensitivity, but has a feature that the photoelectric conversion time is halved. For this reason, the output of the first pixel group and the second pixel group is the same at the readout timing.

FIG. 12C shows the driving method of the embodiment shown in FIGS. 12A and 12B, and is a diagram for explaining the timing of RESET, READ1, 1 ′, the change in output appearing on the vertical signal line, and the differential output. It is. The pixel 1 (a11) and the pixel 1 (a'11) adjacent in the horizontal direction are sequentially applied to READ1, 1 ', and a read pulse is applied (timing T6, T8), and transferred to the vertical signal line 6. Since the difference between the output of the vertical signal line 6 and the photoelectric conversion time is originally corrected by the sensitivity of the pixel, it becomes a normal imaging output and enters the ADC circuit serially. The ADC circuit sequentially outputs a11 and a′11.

In FIG. 12A, the motion detection signal processing circuit at the end of the vertical signal line 6 is composed of a circuit similar to the CDS circuit. This is because the difference in photoelectric conversion time is not the difference in output, so that an amplifier is unnecessary and the circuit is simplified. The waveform of the differential output 17 is as shown in FIG. This is because the amplifier is no longer necessary, and the reset noise components are the same and can be eliminated by the difference. Similarly, the same processing is performed at b′11 and b11, and the motion detection circuit signal of the present invention can be obtained by combining with the calculation results of a11 and a′11.
As described above, even in the 4-pixel 1-cell configuration in which the output circuit shown in FIG. 12 is shared, the present invention is effective and can output both a normal image signal and a motion detection signal.

FIG. 13 is a diagram showing a repetitive arrangement of color blocks based on a specific component drawing of the motion detection color imaging apparatus having a 4-pixel 1-cell configuration sharing the output circuit shown in FIG. The 4-pixel 1-cell configuration sharing the output circuit is based on FIG. 10, and the sensitivity difference between the pixels of FIG. 13 is expressed by changing the pixel size. The size of the normal pixel size is controlled by the size of the microlens on the top surface of the pixel. The arrangement of the color filters is the same as in FIG.

FIG. 14 is a repetitive configuration in which an output circuit is efficiently arranged based on a specific component drawing of the motion detection color imaging device having a 4-pixel 1-cell configuration in which the output circuit shown in FIG. 13 is shared. FIG. The driving method and the like are the same as in FIG.

FIG. 15A is a diagram showing the repetitive arrangement of unit cells in the detection imaging apparatus according to the present invention shown in FIG. The fact that the direction of movement can be easily determined according to the present invention will be described below. In FIG. 15A, the coordinates of the pixel are A, B, C, D, E, F, G, H, I, J, K, L in the horizontal direction, 1, 2, 3, 4 in the vertical direction. The coordinate position is shown.

15B, similarly to FIG. 12B, photoelectric conversion of the output levels of the pixel 1 (a11), the pixel 1 (a′11), the pixel 1 (b′11), and the pixel 1 (b11) is performed. It is a figure which shows a time change. The sensitivity of the first pixel group (pixel 1 (a11), pixel 1 (b11)) is doubled, but the photoelectric conversion time is halved. Therefore, at the readout timing (Timing 2), the first pixel The output of the set and the second pixel set is the same. Although reset (RS) is performed on the way, this timing is set to Timing 1 and the start is set to Initial.

FIG. 15A also includes a diagram in which the positions of the black object end of the subject and the pixel arrangement are changed. In the figure, the end of the black object extends in the vertical direction, the horizontal end position is at the boundary between the B coordinate and the C coordinate on the left side, and at the boundary between the H coordinate and the I coordinate on the right side. Set to the Initial state. A case in which the object moves to the right side by two pixels during the first photoelectric conversion time t1 is indicated by Timing1. Timing 1 is the timing when the movement for two pixels is completed. During the next first photoelectric conversion time t1, a case where the object further moves to the right side by two pixels is indicated by Timing2. As shown in FIG. 15B, the timing when the movement of 4 pixels is completed in the 2 × t1 movement period is Timing2. It is assumed that the movement of the object is performed at a constant speed.

FIG. 15 (c) shows the output of each pixel at each coordinate at the three timings (Initial, Timing1, Timing2) shown in FIG. The table shows how the signal looks.
In calculating this output, the second photoelectric conversion time is set to be twice as long as the first photoelectric conversion time, the pixel output of the black object is zero, and the pixel corresponding to the second photoelectric conversion time (other than the black object) ( The output of the pixel marked with 'is set to 1. Other preconditions are the same as those in FIG.

In FIG. 15C, the pixels corresponding to the first photoelectric conversion time (a11, a12, a13,..., B11, b12, b13...) Are immediately read out and reset in Timing1. 0. On the other hand, in the pixels corresponding to the second photoelectric conversion time (a′11, a′12, a′13, b′11, b′12, b′13. For pixels that correspond to the black pattern and are not exposed, it is zero. Although the detailed description is omitted below, Timing 1 and Timing 2 are as shown in the table of FIG. When the motion detection signal is calculated from this Timing 2, the detailed description is omitted, but the table of the motion detection signal is obtained. Here, the direction of motion is represented by a sign of ±. That is, in the case of a black object, the sign of the head part of the movement is minus, and the sign of the tail part of the movement is plus. In the case of a white object, the sign plus and minus are reversed. That is, in the present application, at the same time as the motion detection information appears in the peripheral portion of the moving object, the direction of motion can be detected by the code.

In order to verify the effect of the present invention, a conventional example (the method of Patent Document 1 and the method of the present invention were compared with each other on the same subject. Even when there was no relatively fine pattern, a background with no movement was observed. When the number of noises was compared, it was confirmed that the noise of the present invention was reduced by about two digits with respect to the conventionally known example, and the noise suppression effect of the present application at the edge of the object was confirmed. The difference becomes larger as the subject becomes finer, and the moving object extraction accuracy can be increased.

In order to verify other effects of the present invention, the code for the direction of movement of the subject was confirmed in the actual imaging state, and as expected, the difference in the calculated code was confirmed at the edge of the moving subject.

As described above, in the present application, it is possible to extract the motion detection portion and to obtain information on the direction of motion inside the sensor without performing complicated signal processing in the subsequent stage, which greatly simplifies the system. I can do it.

FIG. 16 is a diagram showing the overall configuration of a motion detection system to which the present invention is applied.
The motion detection system 30a of the present invention includes an imaging device 32a including an imaging lens 31, a motion determination circuit 33a, a motion determination threshold setting circuit 34 for setting a determination level for determining motion, and an imaging signal of the imaging device 32a. A color signal processing circuit 35 for processing the video signal to be reproduced on the monitor, a video signal output circuit 37 for outputting the video signal 1, and a motion determination signal output circuit for outputting the determination result of the motion determination 36 or the like.

The motion determination threshold value setting circuit 34 sets the level threshold value 1 for counting so as not to count the difference signal of the noise level, and sets the count number to be the threshold value 2. When the count number exceeds this threshold value 2, it is determined that there is a moving object.

The motion determination circuit 33a counts the number of times that the level of the motion detection signal output from the imaging device 32a is greater than the threshold value 1. If the count number exceeds the threshold 2, it is determined that there is a moving object, and the determination signal ON / OFF is set to the HI level. When there is no movement, the LO level is set.

  When it is determined that there is no motion using the motion determination signal (ON / OFF) generated by the motion determination circuit 33a, the circuit operation and signal processing operation of the color signal processing circuit 35 and the video signal output circuit 37 are stopped. By doing so, the power consumption of the motion detection system 30a can be reduced. Similarly, the power consumption of the imaging device 32a can be reduced by controlling the timing circuit of the imaging device 32a.

As a method 1 for stopping a processing operation of a circuit or apparatus using a motion determination signal, an input signal for color signal processing is blocked. For example, the 10-bit digital value of the input signal is switched to all zeros. Since the signal after the color signal processing circuit becomes zero and the switching operation (0⇔1 switching) of the digital circuit stops, the power consumption of the circuit operation can be reduced.
As a stop method 2, each circuit or device is provided with a reset switch for returning to the original state when a malfunction occurs. By keeping the reset switch in the ON state, the circuit processing of each circuit or device is kept stopped, so that power consumption can be reduced.

FIG. 17 is an overall configuration diagram showing a modified example of the motion detection system to which the present invention is applied.
The motion detection system 30b has an advantage that a conventional transmission cable can be used by reducing the motion determination signal output circuit 36 of FIG. The motion determination circuit 33b outputs a motion determination signal (ON / OFF) and also outputs a determination code. The motion determination signal (ON / OFF) stops the circuit operation and signal processing operation of the color signal processing circuit 35 and the video signal output circuit 37 as in FIG.

The determination code obtained by coding the ON / OFF signal of the determination signal is embedded in the video signal using the motion determination signal addition circuit 38. In general, shooting information different from a video signal is embedded in a blanking period in which a valid video signal is not included. Similarly, the motion determination signal is encoded and embedded in the blanking period to output the video signal 2.

FIG. 18 is a configuration diagram in which a determination circuit 33a is provided on the horizontal signal line 12 from which a motion detection signal is output in the overall configuration diagram of the motion detection imaging apparatus according to the present invention shown in FIG. The determination output signal ON / OFF of the determination circuit 33a is input to the timing generation circuit 24. By controlling the timing generation circuit 24, it is possible to add a function for changing the driving method of the vertical scanning circuit 14, the horizontal scanning circuit 13, the motion detection circuit 7, the ADC circuit 8, and the like. The determination output signal ON / OFF can also be output to the outside. Specific driving methods that can be performed by adding the determination circuit 33a shown in FIG. 18 will be sequentially described below.

As an application example 1 of adding the determination circuit 33a shown in FIG. 18, there is a system that saves power in a monitoring device that monitors the presence or absence of a moving object. In the conventional motion monitoring system, since the imaging camera is always driven, the power consumption of the solid-state imaging device and the power consumption for driving the signal processing system in the subsequent stage are large. This always requires almost constant power consumption regardless of the presence or absence of moving objects. However, in the motion detection imaging apparatus according to the present invention, the number of readout scanning lines of the vertical scanning circuit 14 is changed according to the level of the motion detection signal in the determination circuit 33a, or the readout vertical signal line is similarly changed by the horizontal scanning circuit 13. You can change the number. It is possible to change the ratio of the number of horizontal scanning lines to be thinned out and the number of vertical signal lines to be thinned out according to the pattern of the motion detection signal. In this case, the vertical scanning circuit 14 and the horizontal scanning circuit 13 are controlled by controlling the timing generation circuit based on information from the determination circuit 33a.

As a method of thinning out the number of scanning lines, first, the entire screen is imaged to detect the presence or absence of a moving object. When there is no moving object, the number of scanning lines is thinned out, and the thinned state is maintained until the moving object appears. Since the thinning interval changes depending on the situation, several thinning patterns may be prepared and which one to select may be changed depending on the pattern of the motion detection signal. When a moving object appears, it is sufficient to return to the all-pixel imaging mode again.
If the area where the moving object is present is biased, the entire image is divided into blocks, the number of scanning lines is increased only in the area where the moving object exists, and a drive mode is prepared in which the other areas are reduced. It is also possible to select a pattern in the existing area.

As another application example 2 of adding the determination circuit 33a shown in FIG. 18, when the motion detection signal corresponding to the moving object is small, the photoelectric conversion accumulation period is changed, and the gain of the amplifier is changed accordingly. As a result, the motion detection signal can be increased. In this case, information enters the motion detection circuit 7 and the accumulation time control circuit 25 from the determination circuit 33a. It is only necessary to find a condition that maximizes the motion detection signal by repeating several times and set a photoelectric conversion accumulation period suitable for the moving object.

As another application example 3 of adding the determination circuit 33a shown in FIG. 18, when there is no moving object, the power source of the signal processing system in the subsequent stage is set to the standby state, and the motion detection signal appears significantly. In addition, it may be used as an ON / OFF signal for a subsequent power supply that is turned on again. In this way, when there is no movement, a significant energy saving can be achieved by turning off the power of the signal processing circuit in the subsequent stage, in addition to setting the energy saving mode by thinning-out operation of the imaging device. Especially in a monitoring system in a place where there is little movement, the latter stage power is turned on only when there is movement compared to the conventional system that always requires the latter stage processing, and usually only the thinning drive of the imaging apparatus of the present invention is required. , Significant (for example, 1/1000 to 1/100000) energy saving can be realized. This feature is a great advantage when the present invention is applied to a monitoring system in a place where the power supply of the mountainous area does not reach, and reaches a level that can be sufficiently driven by a power supply of a compact solar battery and capacitor set.

As another application example 4 of adding the determination circuit 33a shown in FIG. 18, the dynamic range may be expanded by changing the conversion level of the ADC based on the information of the motion detection signal. In the case where the motion detection signal corresponding to the moving object is small, as described in the application example 3, the method of increasing the motion detection signal by changing the photoelectric conversion accumulation period and changing the gain of the amplifier accordingly is described. The dynamic range of the normal image can be expanded in the region where the motion detection signal is output. Specifically, by adding two pixels having different photoelectric conversion accumulation times, the dynamic range can be expanded, and the ADC can be set appropriately. The same applies to the method of achieving a high dynamic range by adding pixels having different pixel sizes shown in FIG.

In the motion detection imaging apparatus according to the present invention, as described above, it is possible to obtain normal pixel imaging information simultaneously with the motion detection signal, and to determine that normal pixel imaging information is not output to the outside based on the motion information. Functions can be provided. The function of outputting only the imaging information of the portion with the movement to the outside and not outputting the image information without the movement to the outside is very useful for the surveillance camera application. In other words, in a system for monitoring with a plurality of surveillance cameras, monitoring by a conventional surveillance camera has been a heavy load on the surveillance staff. However, only a camera with motion or only a portion with motion is displayed. This greatly reduces the burden on the observer.

Although the preferred embodiments of the solid-state imaging device for motion detection of the present invention have been described in detail, the present invention is not limited to the specific embodiments, and is within the scope of the gist of the present invention described in the claims. Various modifications and changes can be made within.

For example, the ratio of the photoelectric conversion time is not limited to 1: 2, and the ratio can be changed according to the movement of the subject. At this time, the gain of the amplifier may be changed (amplified or reduced) correspondingly. Even when the sensitivity of the pixel is changed, it is not limited to 1: 2.
The layouts of the pixel cells in FIGS. 13 and 14 may be arranged rotated by 90 degrees or 45 degrees.

In the present invention, 2 × 2 pixels adjacent to each other as a pixel block are set as a cell unit. However, in a color image pickup device having a normal Bayer arrangement, the same calculation process is performed on a 2 × 2 pixel block of the same color every other pixel. Also good. Naturally, in this case, background noise without movement increases, but a conventional Bayer color filter can be used as it is. Further, by performing the multiplication operation processing of the present application, the background noise can be reduced as compared with the conventional calculation only in the horizontal direction.

1: Pixel 2: Read transistor, READ: Read wire 3: Floating junction 4: Reset transistor RESET (RS); Reset wire, reset pulse 5: Amplifying transistor 6, 6 ′: Vertical signal line 7: Motion detection circuit 8: ADC (Analog
to Digital Converter) circuit 10: pixel group 11: horizontal readout gate (motion detection signal)
12: Horizontal signal line (motion detection signal)
13: horizontal scanning circuit 14: vertical scanning circuit 15, 15 ': differential amplifier 16, 16': amplifier 17, 17 ': differential output 18: addition circuit 19: addition output 20: noise removal circuit 21: noise removal addition Output 22: Amplifier 24: Timing generation circuit 25: Accumulation (photoelectric conversion) time control circuit 30a, 30b: Motion detection system 31: Optical lens 32: Motion detection image sensor 33a, 33b: Motion determination circuit 34: Motion detection determination threshold 35 : Color signal processing 36: Motion determination signal output circuit 37: Video signal output circuit 38: Motion determination signal addition circuit LOAD Tr: Load transistors SW1, 2, 3, 4: Switch circuits Gr, Gr ′, gr, gr ′, R , R ′, r, r ′, B, B ′, b, b ′: Color filters

Claims (11)

A pixel unit including a pixel group in which pixels for converting light collected by the imaging lens into an electrical signal are two-dimensionally arranged, and a driving unit that drives the pixel group;
In the motion detection solid-state imaging device including motion detection means for detecting motion from the captured image signal in the pixel unit,
In the pixel group, adjacent 2 × 2 4 pixels are used as a unit pixel block,
The motion detection solid-state imaging device according to claim 1, wherein the motion detection means includes calculation means for calculating two sets of signal differences in which two pixels in the unit pixel block are paired.
A pixel unit including a pixel group in which pixels for converting light collected by the imaging lens into an electrical signal are two-dimensionally arranged, and a driving unit that drives the pixel group;
Motion detection means for detecting motion from the captured image signal in the pixel unit;
Motion determining means for determining motion based on the output signal of the motion detecting means;
In a motion detection system comprising signal processing means for processing the imaging signal,
The pixel group includes four adjacent 2 × 2 pixels as a unit pixel block,
The motion detection system includes a calculation unit that calculates two sets of signal differences in which two pixels in the unit pixel block are paired.
A color filter that separates the wavelength of light is formed on-chip above the pixel, and the color filter has the same color of the adjacent 2 × 2 unit pixel blocks of 4 pixels, and four sets of the unit pixel blocks are formed. The motion detection solid-state imaging device according to claim 1 or the motion detection system according to claim 2, wherein the motion detection solid-state imaging device according to claim 1 is arranged in a two-dimensional manner in a Bayer array .
Three types of color filters for separating the wavelength of light are arranged on the upper part of the pixel in one unit pixel block of 2 × 2 adjacent pixels, and 3 colors are arranged in 2 × 2 unit blocks of adjacent unit pixel blocks. The motion detection solid-state imaging device according to claim 1 or the motion detection system according to claim 2, wherein the color filters are arranged two-dimensionally according to a Bayer array .
3. The motion detection solid-state imaging device according to claim 1, wherein a floating junction that converts a signal charge photoelectrically converted by the pixel into a readout voltage is shared by adjacent 2 × 2 pixels. The motion detection system described in 1.
The pixel unit includes a readout transistor for reading out a photoelectrically converted signal from the pixel and a readout signal line for controlling the readout transistor,
3. The motion detection solid-state imaging device according to claim 1, wherein the pixel group includes two readout signal lines connected to the readout transistor for each pixel in order to make photoelectric conversion times different. Motion detection system.
In the 2 × 2 unit pixel block, the two readout signal lines are arranged so that two pixels arranged obliquely in a hatched positional relationship have the same photoelectric conversion time. The motion detection solid-state imaging device or the motion detection system according to claim 6, wherein the motion detection solid-state imaging device or the motion detection system is alternately connected to a readout transistor.
The motion detection means further includes signal amplification means for adjusting the signal amount by the reciprocal of the photoelectric conversion time ratio so that the signal amounts having different photoelectric conversion times read from the pixel unit become the same signal amount. The motion detection solid-state imaging device or motion detection system according to claim 6.
The motion detection solid-state imaging device according to claim 1 or the motion detection system according to claim 2, wherein the motion detection means is provided at an upper end or a lower end of the pixel unit .
Based on the determination signal of the movement determination means,
The motion detection system according to claim 2, wherein the number of execution pixels read from the pixel group is reduced by controlling timing generation means for controlling the driving means.
Based on the determination signal of the movement determination means,
The motion detection system according to claim 2, further comprising a motion direction determination unit that determines a direction of movement of the subject based on a sign of a motion detection signal of the motion detection unit.