JP5509937B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

  Conventionally, in an imaging device using a CCD (Charge Coupled Device), when intense light is incident, image quality is deteriorated due to generation of smear. Various imaging devices that perform such smear correction have been proposed (see Patent Document 1 as an example).

JP 2007-110375 A

  By the way, in the smear correction of the imaging apparatus, when the smear position detection is performed in a frame different from the correction target frame, there is a possibility that the smear correction is performed on a portion that does not include the smear component of the image depending on the shooting situation. In particular, in a scene where the subject brightness is very high, if the smear correction is performed at an incorrect position, the appearance of the image is significantly impaired.

  In view of the above circumstances, there is provided means for preventing image degradation due to erroneous smear correction in a scene with extremely high subject brightness.

An imaging apparatus according to one aspect includes an imaging device that captures an image of a subject, a smear determination unit, a luminance determination unit, a smear correction unit, and a motion detection unit . The smear determining unit determines whether a smear component is included in the output of the image sensor. The luminance determination unit determines whether the luminance of the subject is equal to or higher than a threshold value. The smear correction unit executes a smear correction process for reducing a smear component included in the output of the image sensor when the smear determination unit determines that the smear component is included. The motion detection unit detects the motion of the device. Also, smear correction unit does not perform the smear correction processing when the brightness of the subject is equal to or larger than the threshold and the motion detector detects the movement of the device, or the brightness of the subject is a threshold and detects the motion of the motion detection section apparatus The smear correction process is executed when the brightness of the subject is less than the threshold value.

  The imaging device according to the one aspect described above may further include a light amount adjusting unit and a photographing control unit. The light amount adjustment unit adjusts the amount of light incident on the image sensor. When the luminance of the subject is equal to or higher than the threshold value, the photographing control unit executes control for reducing the exposure amount of the image sensor by the light amount adjusting unit and performs control for extending the charge accumulation time of the image sensor.

  In the one aspect described above, the luminance determination unit corresponds to either a subject luminance value at which proper exposure cannot be performed or a case where one photometric value when a divided photometric measurement is performed exceeds a predetermined value. Sometimes, it may be determined that the luminance of the subject is equal to or higher than a threshold value.

  In the imaging apparatus of one aspect, the smear component correction process is not executed when the luminance of the subject is equal to or higher than the threshold value, thereby preventing image degradation due to erroneous smear correction in a scene with very high subject luminance.

1 is a block diagram illustrating a configuration example of an imaging apparatus (electronic camera) according to an embodiment. A diagram schematically showing part of the circuit configuration of an image sensor (A): a diagram showing an example of smear in a past frame, (b): a diagram showing an example of smear in a frame to be corrected, (c): a diagram showing an example of erroneous correction of smear correction processing The figure which shows an example of the possibility of execution of a smear correction process

<Description of One Embodiment>
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus (electronic camera) according to an embodiment. The electronic camera includes a photographing lens 11, a first lens driving unit 12, a blur correction lens 13, a second lens driving unit 14, an ND (Neutral Density) filter 15, a filter driving unit 16, and a blur detection unit 17. An image sensor 18, an image pickup circuit 19, a camera microcomputer 20, an image compression circuit 21, a first memory 22, a display image creation circuit 23, a monitor 24, a recording I / F 25, a release button 26, and And a second memory 27. Here, the 1st lens drive part 12, the 2nd lens drive part 14, the filter drive part 16, the blur detection part 17, the image pick-up element 18, the image pick-up circuit 19, the image compression circuit 21, the 1st memory 22, and the display image creation circuit 23 The recording I / F 25, the release button 26, and the second memory 27 are connected to the camera microcomputer 20, respectively.

  The photographing lens 11 is composed of a plurality of lens groups including a zoom lens and a focus lens. The lens position of the photographic lens 11 is adjusted in the optical axis direction by the first lens driving unit 12. For the sake of simplicity, FIG. 1 shows the photographing lens 11 as a single lens.

  The shake correction lens 13 is a lens for correcting camera shake occurring in the electronic camera, and can swing in a direction perpendicular to the optical axis of the photographing lens 11. Further, the second lens driving unit 14 has a vertical swing mechanism that swings the shake correction lens 13 in the vertical direction and a horizontal swing mechanism that swings the shake correction lens 13 in the horizontal direction. The second lens driving unit 14 swings the blur correction lens 13 under the control of the camera microcomputer 20. Thereby, the imaging position of the subject image is shifted, and image blurring on the light receiving surface of the image sensor 18 is suppressed.

  The ND filter 15 is a light-reducing member that attenuates transmitted light by a predetermined amount (for example, equivalent to three stages in apex value). The ND filter 15 is driven back and forth in a direction orthogonal to the optical axis of the photographic lens 11 by the filter driving unit 16. When the ND filter 15 is inserted on the optical path of the subject light, the amount of light incident on the image sensor 18 is limited by the ND filter 15. On the other hand, when the ND filter 15 is retracted out of the optical path of the subject light, the amount of light incident on the image sensor 18 is not limited. The filter driving unit 16 drives the ND filter 15 to advance and retreat according to instructions from the camera microcomputer 20.

  The shake detection unit 17 includes a vertical angular velocity sensor that detects the pitch of the electronic camera and a horizontal angular velocity sensor that detects the roll of the electronic camera. For each of the above angular velocity sensors, for example, a piezoelectric vibration angular velocity sensor that detects Coriolis force due to rotation is used.

  The image sensor 18 captures (captures) an image of a subject formed by the photographing lens 11 and outputs an image signal to the image capturing circuit 19. The image sensor 18 of one embodiment is configured with a CCD image sensor. Note that the charge accumulation time of the image sensor 18 and the signal readout timing are controlled by the camera microcomputer 20.

  Further, in the shooting mode of the electronic camera, the image sensor 18 shoots a still image or a moving image for recording accompanied by recording on a nonvolatile recording medium in response to an imaging start instruction by the release button 26. In addition, the imaging device 18 continuously captures images for observation (through images) at predetermined intervals even during standby for recording still images for recording. The through-image data (or the moving image data) acquired in time series is used for moving image display on the monitor 24 and various arithmetic processes by the camera microcomputer 20. Note that, during moving image shooting in the shooting mode, the electronic camera may record the through image on a non-volatile storage medium.

  FIG. 2 is a diagram schematically showing a part of the circuit configuration of the image sensor 18. On the light receiving surface of the image sensor 18, light receiving elements 31 that perform photoelectric conversion are two-dimensionally arranged. Further, the image sensor 18 has a plurality of vertical readout lines 32 for reading out pixel signals of the respective light receiving elements 31 and one horizontal readout line 33. The vertical readout line 32 is arranged for each column of the light receiving elements 31, and the output end of the vertical readout line 32 is connected to the horizontal readout line 33. The output of the horizontal readout line 33 is connected to the imaging circuit 19.

  The light receiving element 31 of the image sensor 18 is divided into effective pixels and light shielding pixels. The effective pixel is a pixel that accumulates signal charges according to the brightness of the image of the subject. Then, an image signal of the captured image is generated based on the output of the effective pixel. On the other hand, the light shielding pixel is a pixel whose surface is covered with a light shielding film. The light shielding pixel detects the charge (dark current component) accumulated in the light receiving element 31 due to a temperature change or the like. Then, a black level corresponding to the black color of the photographed image is determined based on the output of the light shielding pixels. As an example, on the light receiving surface of the image sensor 18, a light shielding pixel is formed in a ring pixel portion around the effective pixel region. In the example of FIG. 2, the shaded pixel region is indicated by a broken line.

  Returning to FIG. 1, the imaging circuit 19 is a circuit that sequentially performs analog signal processing, A / D conversion processing, and digital signal processing on an input image signal. The output of the imaging circuit 19 is connected to the camera microcomputer 20.

  The camera microcomputer 20 is a processor that comprehensively controls the operation of the electronic camera. The camera microcomputer 20 functions as an image processing unit 20a, a photographing control unit 20b, a smear determination unit 20c, a luminance determination unit 20d, and a smear correction unit 20e by executing a program stored in the second memory 27.

  The image processing unit 20 a performs image processing on the digital image signal input from the imaging circuit 19. The image processing includes, for example, contour enhancement, white balance adjustment processing, and format conversion processing for an image signal.

  The imaging control unit 20b performs auto focus (AF) control and automatic exposure (AE) control. For example, the imaging control unit 20 b performs a known contrast detection AF calculation using the through image, and adjusts the lens position of the focus lens via the first lens driving unit 12. In addition, the photographing control unit 20b performs divided photometry in the photographing screen using the through image, and performs AE control (driving of the ND filter 15, adjustment of the charge accumulation time of the image sensor 18, etc.) based on the divided photometry result. Run. When performing camera shake correction, the imaging control unit 20 b uses the output of the shake detection unit 17 to determine the drive position of the shake correction lens 13. The presence / absence of motion detection of the electronic camera is also used as one of determination conditions for determining whether or not to perform smear correction described later.

  The smear determination unit 20c determines whether a smear component is included in the output of the image sensor 18 using the output of the light-shielded pixels of the image sensor 18. In general, when a very high-brightness object is photographed with a CCD image sensor, charges are generated from other than the light receiving element 31 in the vicinity of a pixel having a large amount of incident light. Then, the charge transferred by the charge transfer line (vertical readout line 32) of the image sensor 18 and generated by signals other than the light receiving element 31 with respect to signal charges accumulated in a pixel different from a pixel having a large amount of incident light. When charges are integrated, smears (white streaks) appear in the image.

  As an example, the smear determining unit 20c adds and averages the outputs of the light shielding pixels in each column along the vertical direction of the light shielding pixel region. As a result, it is possible to obtain an average value of light-shielding pixels for one horizontal line. The random noise of the pixels becomes almost zero by averaging in the above-mentioned average value. On the other hand, smear components appearing along the vertical readout line 32 are not averaged in the addition average value. Therefore, the smear determining unit 20c obtains the difference between the addition average values in the adjacent columns, and determines that the column in which the above average addition value is larger than the predetermined threshold compared to the adjacent columns includes a smear component. Good.

  The luminance determination unit 20d determines whether or not there is an extremely high-brightness subject (subject whose subject luminance is equal to or higher than a threshold) in the shooting screen. Note that the determination result by the luminance determination unit 20d is one of the determination conditions for determining whether to perform smear correction described later.

  As an example, when the subject luminance value (BV value) obtained by the AE calculation is a subject luminance value at which proper exposure cannot be performed, the luminance determination unit 20d determines that the luminance of the subject is equal to or higher than the threshold value. Here, the subject luminance value at which proper exposure cannot be performed is determined according to the shortest shutter time of the electronic camera. For example, consider an example of an electronic camera in which the shortest shutter time is 1/6000 seconds. In this electronic camera, if the subject brightness value of the photographic scene increases so that proper exposure cannot be achieved in 1/6000 seconds, the brightness determination unit 20d determines that the brightness of the subject is equal to or greater than the threshold value.

  As another example, when one photometric value when the divided photometry is performed on the shooting screen exceeds a predetermined value, the luminance determination unit 20d determines that the luminance of the subject is equal to or higher than the threshold value. For example, when the gradation range is 8 bits (value of 0 to 255) and any of the divided photometric values in the shooting screen exceeds “235”, the luminance determination unit 20d determines that the luminance of the subject is equal to or greater than the threshold value. Judge that there is.

  The smear correction unit 20e performs a smear correction process for reducing the smear component from the captured image when the smear determination unit 20c determines that the smear component includes the smear component. The smear correction process is performed by a known method. That is, the smear correction unit 20e subtracts the value of the smear component from the pixel value of the photographed image in the smeared column of the photographed image.

  As will be described later, even when the smear determining unit 20c determines that the smear component is included, in a case where the appearance of the photographed image is greatly deteriorated when incorrect smear correction is performed, the smear correcting unit 20e The correction process is not executed (see FIG. 4).

  The image compression circuit 21 performs image compression processing at a predetermined compression ratio in the JPEG format on the image signal processed by the image processing unit 20a.

  The first memory 22 is a buffer memory that temporarily stores data in the pre-process and post-process of image processing. As an example, the first memory 22 is configured by an SDRAM which is a volatile storage medium.

  The display image creation circuit 23 creates display data for displaying the captured image on the monitor 24. In addition, the monitor 24 is composed of a liquid crystal display panel disposed on the back of the housing of the electronic camera. When operating in the shooting mode, the camera microcomputer 20 displays the above-described through image or moving image on the monitor 24 as a moving image so that the user can visually recognize the subject being shot.

  The recording I / F 25 has a connector for connecting a nonvolatile storage medium 28. The recording I / F 25 executes data writing / reading with respect to the storage medium 28 connected to the connector. The storage medium 28 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the storage medium 28.

  The release button 26 receives an input of a recording still image or moving image shooting timing. The second memory 27 is composed of a nonvolatile storage medium such as a flash memory. The second memory 27 stores a program executed by the camera microcomputer 20. An example of the operation in the shooting mode by executing this program will be described later.

  Next, an operation example in the shooting mode in one embodiment will be described. For example, in a mode for capturing a still image for recording, the electronic camera captures a still image for recording at a timing instructed by the release button 26. On the other hand, in a mode for capturing a moving image for recording, the electronic camera captures a moving image for recording during a moving image capturing period determined by pressing the release button 26. In any shooting mode, the electronic camera displays a moving image of the state of the subject within the shooting range using a through image or the like.

  Further, at the time of shooting in the above shooting mode, the smear determining unit 20c determines whether a smear component is included in the output of the image sensor 18 using the output of the light-shielding pixel. When a high-luminance subject exists in the shooting screen, the smear determination unit 20c determines that a smear component is included. And when satisfy | filling the below-mentioned conditions, the smear correction | amendment part 20e correct | amends the smear of a picked-up image. In the through image and moving image smear correction processing, the smear correction unit 20e detects the smear position and the correction amount in the correction target frame from a frame earlier than the correction target frame.

  Here, the reason why the smear correction process may not be performed in the electronic camera according to the embodiment although the smear has occurred will be described.

  In capturing a through image or a moving image, unnecessary charges cannot be discharged from the image sensor 18 with the light receiving surface of the image sensor 18 shielded by a mechanical shutter (not shown). In a bright scene, the charge accumulation time of the image sensor 18 is shortened by exposure control, so that the smear component included in the image signal relatively increases. Therefore, generally, when removing smear from a moving image for recording or a display image on the monitor 24, smear correction processing by the smear correction unit 20e is useful.

  However, for example, when the electronic camera moves greatly due to panning or camera shake at the time of shooting, the position where the smear occurred in the past frame (see FIG. 3A) and the smear of the correction target frame The position (see FIG. 3B) is different. If the smear correction unit 20e performs the smear correction process in the above case, the smear detection position in the past frame is erroneously corrected although it is normal in the correction target frame (see FIG. 3C). ).

  In addition, the above-described erroneous correction is not so conspicuous in, for example, a case where a slight smear is generated in a fluorescent lamp portion in an indoor scene. On the other hand, in a case where a noticeable smear is generated in a captured image due to a very bright subject, the above-described erroneous correction significantly deteriorates the appearance of the captured image.

  Therefore, the smear correction unit 20e of one embodiment determines whether or not the smear correction process can be performed under the following conditions (# 1) to (# 4). FIG. 4 shows an example of whether or not the smear correction process can be executed in a table format.

  (# 1) When the smear determination unit 20c determines that the smear component is not included in the output of the image sensor 18, the smear correction unit 20e does not execute the smear correction process. This is because smear does not occur in the image sensor 18 in the case of (# 1) above.

  (# 2) When the smear determination unit 20c determines that a smear component is included in the output of the image sensor 18, and the luminance determination unit 20d determines that the luminance of the subject is less than the threshold, the smear correction unit 20e Execute correction processing. In the case of (# 2) above, smear has occurred in the image sensor 18 and the brightness of the smear is relatively small. Therefore, even if erroneous correction occurs in the smear correction process, there is relatively little adverse effect, and the smear correction unit 20e executes smear correction.

  (# 3) The smear determining unit 20c determines that a smear component is included in the output of the image sensor 18, and the luminance determining unit 20d determines that the luminance of the subject is equal to or higher than the threshold, and the electronic camera moves. When it is determined that there is, the smear correction unit 20e does not execute the smear correction process.

  In the case of (# 3) above, smear has occurred in the image sensor 18 and the brightness of the smear is very strong. However, in the case of the above (# 3), since the electronic camera is moving and there is a high possibility that the appearance of the photographed image is significantly deteriorated due to erroneous correction of the smear correction process, the smear correction unit 20e does not execute the smear correction process. .

  (# 4) The smear determination unit 20c determines that a smear component is included in the output of the image sensor 18, and the luminance determination unit 20d determines that the luminance of the subject is equal to or higher than the threshold, and the electronic camera moves. If it is determined that there is no smear correction unit 20e, the smear correction unit 20e executes a smear correction process.

  In the case of (# 4), unlike the case of (# 3), the electronic camera does not move. Therefore, since the possibility of erroneous correction of the smear correction process is low, the smear correction unit 20e executes the smear correction process.

  As described above, the electronic camera according to the embodiment does not perform the smear correction process when a very bright subject exists in the shooting screen even if smear occurs. For this reason, it is possible to suppress adverse effects caused by erroneous correction of the smear correction process.

  In the case of (# 3), the imaging control unit 20b may execute the following control in order to suppress the occurrence of smear. First, the imaging control unit 20b decreases the exposure amount per unit time of the image sensor 18 by inserting the ND filter 15. Further, the imaging control unit 20b sets the charge accumulation time of the image sensor 18 to be long. Thus, if the exposure time is reduced and the charge accumulation time is lengthened, the amount of smear component charge is relatively small with respect to the amount of signal charge accumulated in the light receiving element 31. Therefore, even in a situation where the smear correction process cannot be performed, it is possible to improve the image quality of the captured image to some extent. Note that an ND filter may be manually attached to the electronic camera. As described above, in the case where the ND filter is manually mounted, the camera microcomputer 20 may cause the monitor 24 to display a display prompting the user to mount the ND filter.

<Supplementary items of the embodiment>
(Modification 1) In the above embodiment, an example in which the motion detection unit 17 detects the movement of the device on the electronic camera side has been described. However, the present invention may detect motion on the electronic camera side using a plurality of photographed images photographed in time series. As an example, the camera microcomputer 20 executes subject matching between through images, for example. Then, the camera microcomputer 20 may detect the movement on the electronic camera side from the shift amount of the position of each subject that matches between the through images. In the first modification, the camera microcomputer 20 can also detect the presence / absence of movement of a high-luminance subject.

  (Modification 2) The determination of whether or not the smear correction process shown in FIG. 4 can be executed is merely an example. For example, in the case of (# 2) illustrated in FIG. 4, the smear correction unit 20e may not perform the smear correction process when the electronic camera or the high-luminance subject moves.

  (Modification 3) In the said embodiment, the example which adjusts the incident light quantity to the image pick-up element 18 with the ND filter 15 was shown. However, in the present invention, for example, a mechanical shutter having a plurality of aperture blades may be used.

  From the above detailed description, features and advantages of the embodiments will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

DESCRIPTION OF SYMBOLS 15 ... ND filter; 16 ... Filter drive part; 17 ... Blur detection part; 18 ... Image pick-up element; 20 ... Camera microcomputer; 20a ... Image processing part; 20b ... Shooting control part; 20c ... Smear judgment part; 20e ... smear correction part

Claims (3)

An image sensor for capturing an image of a subject;
A smear determining unit that determines whether a smear component is included in the output of the image sensor;
A luminance determination unit that determines whether the luminance of the subject is equal to or higher than a threshold;
A smear correction unit that performs a smear correction process for reducing a smear component included in the output of the image sensor when the smear determination unit determines that the smear component is included;
A motion detector for detecting the motion of the device ,
The smear correction unit is configured and the motion detector luminance is more than the threshold value of the subject is not performing the smear correction processing upon detecting movement of the device, the luminance of the subject is greater than or equal to the threshold value and the movement detection when the part does not detect the movement of the device, or imaging device brightness of the subject is to perform the smear correction process when it is below the threshold value. The imaging device according to claim 1,
A light amount adjusting unit that adjusts the amount of light incident on the imaging element;
An imaging control unit that executes control to reduce the exposure amount of the image sensor with the light amount adjustment unit and to increase the charge accumulation time of the image sensor when the luminance of the subject is equal to or greater than the threshold; ,
An imaging apparatus further comprising: In the imaging device according to claim 1 or 2,
The brightness determination unit determines whether the subject has a brightness value at which proper exposure cannot be performed, or when one of the photometric values when the photometric screen is divided and metered exceeds a predetermined value. An imaging apparatus that determines that the luminance is equal to or higher than the threshold value.

Images