JP5772096B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

  Conventionally, various techniques related to an imaging apparatus including a light emitting unit that illuminates a subject in an auxiliary manner have been considered. For example, in the invention of Patent Document 1, when the amount of illumination light emitted by the light emitting unit is adjusted to a specific subject at a certain distance, the exposure amount is over for another subject existing at a different distance. It is intended to solve the problem of being under. In order to solve such a problem, the invention of Patent Document 1 discloses a technique that includes a plurality of light emitting devices and performs so-called multiple-flash photography.

Patent No. 3976905

  In the invention of Patent Document 1, it is necessary to provide a plurality of light emitting devices, and to control these light emitting devices in conjunction with each other. Therefore, there is a problem that not only the whole system is enlarged, but also the control in the system is complicated.

  An object of the imaging apparatus of the present invention is to obtain an image with favorable brightness in the entire image without including a plurality of light emitting devices (light emitting units).

An imaging apparatus according to the present invention includes: an imaging unit that repeats imaging of a subject a plurality of times to generate a plurality of images; a light emitting unit that illuminates the subject with a different light emission amount in the plurality of times of imaging; and the plurality of images A low-pass processing unit that performs low-pass processing on at least one image to generate a low-pass image , a first gradation curve having an input / output characteristic that enhances contrast at a gradation intermediate part, and gradation A setting unit for setting a gradation curve used for gradation conversion processing of the low-pass image generated by the low-pass processing unit, among the second gradation curve having input / output characteristics that weaken the contrast in the intermediate portion of A gradation conversion unit that performs gradation conversion processing on the low-pass image generated by the low-pass processing unit using the gradation curve set by the setting unit; Based on the low-pass image gradation conversion processing has been performed by the conversion unit, and a synthesizing unit for synthesizing the plurality of images generated by the imaging unit.

  The image processing apparatus further includes a white balance adjustment unit that performs white balance adjustment on the plurality of images generated by the imaging unit, and the white balance adjustment unit performs automatic adjustment individually for each of the plurality of images. The white balance adjustment based on the above may be performed.

  Further, the light emitting unit may end the illumination of the subject substantially simultaneously with the end of exposure at the time of imaging by the imaging unit.

  In addition, the low-pass processing unit may generate the low-pass image based on an image with the smallest amount of light emission during imaging by the imaging unit among the plurality of images.

  In addition, the light emitting unit measures the distance of a plurality of areas in the subject and obtains an appropriate light emission amount for each of the plurality of areas based on the distance measurement result. The light emission amount may be determined.

  Further, the low-pass processing unit is based on an image generated by the imaging unit when illumination is performed by the light-emitting unit with the appropriate light emission amount in the shortest distance region among the plurality of regions. The low-pass image may be generated.

  According to the present invention, it is possible to obtain an image with favorable brightness in the entire image without providing a plurality of light emitting devices (light emitting units).

It is a figure which shows the structure of the electronic camera 1 of 1st Embodiment. It is a functional block diagram of electronic camera 1 of a 1st embodiment. It is a flowchart which shows operation | movement at the time of imaging | photography of the electronic camera 1 of 1st Embodiment. It is a figure explaining a low pass filter. It is a figure explaining the gradation curve Gm1. It is a figure explaining the gradation curve Gm1.

<First Embodiment>
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. In the first embodiment, a single-lens reflex electronic camera will be described as an example of the imaging apparatus of the present invention.

  FIG. 1 is a diagram illustrating a configuration of an electronic camera 1 according to the first embodiment. As shown in FIG. 1, an electronic camera 1 includes a photographing lens 2, an aperture 3, a quick return mirror 4, a sub mirror 5, a diffusion screen 6, a condenser lens 7, a pentaprism 8, a beam splitter 9, an eyepiece lens 10, and an imaging lens. 11, a photometric sensor 12, a shutter 13, an image sensor 14, and a focus detection unit 15.

  The photometric sensor 12 is, for example, a photometric sensor capable of split photometry. The imaging element 14 is a semiconductor device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). For example, the focus detection unit 15 detects a focus state of the photographic lens 2 by performing phase difference type focus detection.

  The electronic camera 1 includes a monitor 16 such as a liquid crystal monitor that displays an image generated by imaging, a light emitting unit 17 that illuminates a subject, a light emission control unit 18 that controls the light emitting unit 17, and a control unit 19 that controls each unit. Is further provided. ON / OFF of the light emission by the light emitting unit 17 may be a manual setting based on a user instruction or an automatic setting by the control unit 19. Moreover, it is preferable to be able to switch between manual setting and auto setting based on a user instruction. Further, the amount of light emitted by the light emitting unit 17 can be controlled by the control unit 19 and the light emission control unit 18. The control unit 19 includes a memory (not shown) therein, and records a program for controlling each unit in advance.

  When not photographing, that is, when photographing is not performed, the quick return mirror 4 is disposed at an angle of 45 ° as shown in FIG. The light beam that has passed through the photographing lens 2 and the diaphragm 3 is reflected by the quick return mirror 4 and guided to the eyepiece lens 10 through the diffusion screen 6, the condenser lens 7, the pentaprism 8, and the beam splitter 9. The user confirms the composition by viewing the subject image through the eyepiece 10. On the other hand, the light beam split upward by the beam splitter 9 is re-imaged on the imaging surface of the photometric sensor 12 via the imaging lens 11. Further, the light beam transmitted through the quick return mirror 4 is guided to the focus detection unit 15 via the sub mirror 5.

  On the other hand, at the time of photographing, the quick return mirror 4 is retracted to the position indicated by the broken line, the shutter 13 is opened, and the light flux from the photographing lens 2 is guided to the image sensor 14.

  FIG. 2 is a functional block diagram of the electronic camera 1 according to the first embodiment. 2, the electronic camera 1 includes a timing generator 20, a signal processing unit 21, an A / D conversion unit 22, a buffer memory 23, a bus 24, a card interface 25, and a compression / decompression unit in addition to the configuration of FIG. 26, each part of the image display unit 27 is provided. The timing generator 20 supplies output pulses to the image sensor 14. The image data of the image generated by the image sensor 14 is temporarily stored in the buffer memory 23 via the signal processing unit 21 (including a gain adjustment unit corresponding to the imaging sensitivity) and the A / D conversion unit 22. . The buffer memory 23 is connected to the bus 24. The card interface 25, the control unit 19, the compression / decompression unit 26, and the image display unit 27 described in FIG. The card interface 25 is connected to a removable memory card 28 and records image data of images on the memory card 28. The control unit 19 is connected to a switch group 29 (including a release button (not shown)) of the electronic camera 1, a timing generator 20, and a photometric sensor 12. Further, the image display unit 27 displays an image or the like on the monitor 16 provided on the back surface of the electronic camera 1.

  The operation at the time of photographing of the electronic camera 1 having the above-described configuration will be described using the flowchart shown in FIG. The process described below is a process suitable for photographing in a dark place indoors or outdoors. The following processing may be executed based on a user operation via the switch group 29 or the like, or may be executed based on automatic determination by the control unit 19.

  In step S <b> 1, the control unit 19 performs photometric calculation according to the photometric result obtained by the photometric sensor 12. The photometric calculation is performed in the same manner as in the known technique.

  In step S <b> 2, the control unit 19 determines whether or not the user has instructed to start shooting via the switch group 29. Then, the control unit 19 repeats the photometry calculation described in step S1 until it is determined that the start of shooting is instructed, and when it is determined that the start of shooting is instructed, the process proceeds to step S3.

  In step S3, the control unit 19 determines the light emission amount. The control unit 19 determines a light emission amount for the first imaging (first light emission amount) and a light emission amount for the second imaging (second light emission amount) described later. The first light emission amount is, for example, a small light amount appropriate for a subject with a distance of about 2 meters, and the second light emission amount is, for example, a large light amount appropriate for a subject with a distance of about 4 meters.

  In step S <b> 4, the control unit 19 controls each unit, and based on the result of the photometric calculation performed in step S <b> 1, the image sensor 14 captures a subject image and generates an image. Note that the control unit 19 controls the light emission control unit 18 at the time of imaging, and causes the light emitting unit 17 to emit light in synchronization with the imaging. This light emission is performed based on the first light emission amount determined in step S3. However, the light emission control unit 18 controls the exposure end time by the image sensor 14 and the light emission end time by the light emitting unit 17 substantially simultaneously according to the instruction of the control unit 19. This is to reduce the influence of stationary light as much as possible during imaging in step S4.

  The RGB values of the image generated by the imaging in step S4 are hereinafter referred to as R1 [x, y], G1 [x, y], and B1 [x, y], respectively.

  The image data of the image generated by the image sensor 14 is temporarily stored in the buffer memory 23 via the signal processing unit 21 and the A / D conversion unit 22.

  In step S5, the control unit 19 reads the image data from the buffer memory 23 and performs normal image processing. Normal image processing includes white balance adjustment, interpolation processing, color tone correction processing, gradation conversion processing, and the like. Since the specific method of each process is the same as that of a well-known technique, description is abbreviate | omitted. However, regarding the white balance adjustment, the control unit 19 performs so-called auto white balance adjustment by automatic adjustment.

  In step S <b> 6, the control unit 19 controls each unit again, and based on the result of the photometric calculation performed in step S <b> 1, the image sensor 14 captures a subject image and generates an image. Note that the control unit 19 controls the light emission control unit 18 at the time of imaging, and causes the light emitting unit 17 to emit light in synchronization with the imaging. This light emission is performed based on the second light emission amount determined in step S3. However, as in step S4, the light emission control unit 18 controls the exposure end time by the image sensor 14 and the light emission end time by the light emitting unit 17 substantially simultaneously according to the instruction of the control unit 19.

  The RGB values of the image generated by the imaging in step S6 are hereinafter referred to as R2 [x, y], G2 [x, y], and B2 [x, y], respectively.

  The image data of the image generated by the image sensor 14 is temporarily stored in the buffer memory 23 via the signal processing unit 21 and the A / D conversion unit 22.

  In step S7, the control unit 19 reads image data from the buffer memory 23 and performs normal image processing. Normal image processing includes white balance adjustment, interpolation processing, color tone correction processing, gradation conversion processing, and the like, as in step S5. Since the specific method of each process is the same as that of a well-known technique, description is abbreviate | omitted. However, regarding the white balance adjustment, the control unit 19 performs so-called auto white balance adjustment by automatic adjustment, as in step S5.

  In step S8, the control unit 19 performs low pass processing on the image data of the image generated in step S4, and generates a low pass image LY1 [x, y]. First, based on R1 [x, y], G1 [x, y], B1 [x, y] generated in step S4, the control unit 19 uses the following formula 1 to calculate the luminance value Y1 [x, y ]

Y1 [x, y] = kr · R1 [x, y] + kg · G1 [x, y] + kb · B1 [x, y] (Formula 1)
In Equation 1, kr, kg, and kb are predetermined coefficients for calculating the luminance value Y1 [x, y].

  Further, the low-pass image LY1 [x, y] is obtained by using the filter Lpw having a relatively wide width r illustrated in FIG.

  The filter Lpw shown in FIG. 4 is, for example, a filter having a width r of about 10 times or more compared to a filter used for general contour enhancement processing.

  In step S9, the control unit 19 performs gradation conversion processing on the low-pass image LY1 [x, y] generated in step S8, and obtains a low-pass image LY1t [x, y] after the gradation conversion processing. The gradation conversion process is performed using the following Expression 3.

LY1t [x, y] = Gm1 (LY1 [x, y]) (Formula 3)
Gm1 in Equation 3 is the gradation curve Gm1 shown in FIG. As shown in FIG. 5, the gradation curve Gm1 has an input / output characteristic having a characteristic that takes into account the characteristic of changing the contrast in the middle part of the gradation. The gradation curve Gm1 (a) shown in FIG. 5 is an example of a gradation curve that takes into account the characteristic of enhancing contrast in the middle part of the gradation. In step S9, by performing the gradation conversion process using the gradation curve Gm1 (a), the images to be combined can be more clearly separated and combined in the image combining process described later. Also, the gradation curve Gm1 (b) shown in FIG. 5 is an example of a gradation curve that takes into account the characteristic of weakening the contrast in the middle part of the gradation. In step S9, by performing the gradation conversion process using the gradation curve Gm1 (b), it is possible to suppress the conspicuous boundary portion in the combined image during the image combining process described later.

  Further, by adjusting the gradation curve Gm1, it is possible to adjust the brightness boundary between the bright part and the dark part of the image to be synthesized in the image synthesis process described later. For example, the gradation curve Gm1 (c) shown in FIG. 6 uses the image generated in step S4 at the time of synthesis (an image captured in a state where there is a small amount of light emission) limited to a dark portion, and is used in step S6. 6 is an example of a gradation curve for widely using the image generated in (1) (image captured in a state where there is a large amount of light emission) up to the dark side. In addition, the gradation curve Gm1 (d) shown in FIG. 6 is generated in step S6 when the image generated in step S4 (image captured in a state where there is a small amount of light emission) is used in a wide luminance range. It is an example of the gradation curve for limiting and using the image (image imaged in the state with light emission of large light emission amount) limited to a bright part.

  The gradation curve Gm1 described above may be set based on a user operation via the switch group 29 or the like, or may be set based on automatic determination by the control unit 19. For the automatic determination by the control unit 19, for example, a gradation curve Gm1 capable of extracting an optimum portion of each synthesized image may be set based on a known scene analysis or the like. Alternatively, the control unit 19 may automatically generate the optimum gradation curve Gm1.

  Note that the low-pass image LY1t [x, y] that has been subjected to gradation conversion processing using such a gradation curve Gm1 is used as a mask in the image composition processing described later.

  In step S10, the control unit 19 performs an image synthesis process on the image generated in step S4 and the image generated in step S6 based on the low-pass image LY1t [x, y] subjected to the gradation conversion process in step S9. Do. The image composition processing is performed for each of the RGB values using the following equations 4 to 6.

Rmix [x, y] = (R1 [x, y] · LY1t [x, y] + R2 [x, y] · (255−LY1t [x, y])) ÷ 255 (Formula 4)
Gmix [x, y] = (G1 [x, y] · LY1t [x, y] + G2 [x, y] · (255−LY1t [x, y])) ÷ 255 (Formula 5)
Bmix [x, y] = (B1 [x, y] · LY1t [x, y] + B2 [x, y] · (255−LY1t [x, y])) ÷ 255 (Formula 6)
In Expressions 4 to 6, 255 is derived from the fact that the RGB value is 256 gradations. The image generated in step S4 is an image captured in a state where there is a small amount of light emitted by the light emitting unit 17, and is an image in which the foreground portion is suitably expressed. On the other hand, the image generated in step S6 is an image captured in a state where there is a large amount of light emitted by the light emitting unit 17, and in the image generated in step S4, a portion that is not suitably expressed (by the light emitting unit 17). The portion where the illumination does not reach is a suitably expressed image. By combining the images with the low-pass image LY1t [x, y] subjected to the gradation conversion process in step S9 by the image combining process described in Expressions 4 to 6, the light emitting unit 17 reduces the amount of light. A composite image having a preferable brightness can be obtained in the entire image including the portion illuminated with the light and the portion illuminated with a large amount of light.

  In step S11, the control unit 19 stores the image data of the composite image composed of Rmix [x, y], Gmix [x, y], and Bmix [x, y] generated in step S10 via the card interface 25. The data is recorded on the card 28, and the series of processing is completed. The RGB values of the composite image may be converted to YCbCr and recorded, or may be recorded via the compression / decompression unit 26 before the image data of the composite image is recorded on the memory card 28. Depending on the image compression processing (JPEG compression processing, etc.) may be performed.

  As described above, according to the first embodiment, the imaging unit that repeats imaging of the subject a plurality of times to generate a plurality of images, and the light emitting unit that illuminates the subject with different light emission amounts in the plurality of imagings, A low-pass processing unit that performs low-pass processing on at least one of the plurality of images and generates a low-pass image, and a combining unit that combines the plurality of images generated by the imaging unit based on the low-pass image With. Therefore, it is possible to obtain an image with favorable brightness in the entire image without providing a plurality of light emitting devices (light emitting units).

  In particular, according to the first embodiment, an image captured in a state of being illuminated with a small amount of light by the light emitting unit 17 and an image captured in a state of being illuminated with a large amount of light by the light emitting unit 17 are combined. Since the weighting is obtained based on the low-pass image LY1 [x, y], the dark part and the bright part are roughly distinguished, and there is no decrease in contrast or saturation in an area where the dark part or the bright part is 100%. Natural synthesis is possible. On the other hand, in the conventional method of creating a plurality of images of a subject and simply synthesizing them, the bright portions of the images may be faded by combining the bright portions of each image. Many.

  Moreover, by this synthesis, among the images captured in a state where the light emitting unit 17 is illuminated with a small amount of light, the subject area on the short distance side appropriately illuminated with the small amount of light and the light emitting unit 17 are illuminated with a large amount of light. Among the images picked up in this state, a subject area on the long distance side that is appropriately illuminated with a large amount of light is combined. Therefore, it is possible to obtain a composite image in which a subject existing at a short distance and a subject existing at a long distance are appropriately illuminated.

Second Embodiment
The second embodiment of the present invention will be described below with reference to the drawings. The second embodiment is a modification of the first embodiment. In the second embodiment, only parts different from the first embodiment will be described, and parts similar to those in the first embodiment will be described using the same reference numerals as those in the first embodiment.

  The electronic camera of the second embodiment has the same configuration (see FIGS. 1 and 2) as the electronic camera 1 described in the first embodiment. However, the light emission part 17 and the light emission control part 18 have a symptom control function.

  The operation at the time of photographing of the electronic camera 1 of the second embodiment is basically the same as that of the first embodiment. Hereinafter, different points will be described.

  In the second embodiment, instead of step S3, step S4, and step S6, the control unit 19 uses the under eye (for the first imaging, so that the first light emission amount is appropriate for a subject at a short distance. For example, imaging is performed by setting a threshold value so that the light emission amount is stopped at 1 Ev under), and for the second image pickup, the second light emission amount is appropriate for a long-distance subject. For example, imaging is performed by setting a threshold value so that the light emission amount is stopped at 1 Ev over).

  As described above, according to the second embodiment, based on the image generated by the imaging in which the amount of light emitted by the light emitting unit is adjusted with a small amount of light, and the image generated by the imaging in which the amount of light is adjusted with a large amount of light. Generate a composite image. Therefore, the same effect as that of the first embodiment can be obtained.

  Note that in each of the above-described embodiments, three or more images may be generated by performing imaging in a state where the light emission amounts by the light emitting units are different three or more times. For example, imaging may be performed in three stages of small light amount, medium light amount, and large light amount with different light emission amounts to generate three images. In this case, the first two images (small light amount image and medium light amount image) may be combined in the same manner as in the above-described embodiments, and a large light amount image may be combined with the combined image. That is, it is preferable to perform each combining process on the basis of an image with a small amount of light emission, that is, an image in which an object at a short distance is appropriately illuminated.

  Further, in each of the above-described embodiments, the present invention can be similarly applied to a case where the electronic camera 1 includes a known multipoint distance measuring device. In this case, when a subject is detected at a plurality of distances, an appropriate amount of light emission is obtained for each subject, imaging is performed a plurality of times, and the generated plurality of images may be combined.

  Further, in each of the above-described embodiments, an example has been shown in which the low-pass image used for weighting when performing the image composition processing is generated based on an image generated by imaging in a state where the light emission amount by the light emitting unit is the smallest. The present invention is not limited to this example. For example, a low-pass image may be generated based on an image other than the above-described image according to the state of the scene, composition, or the like.

  Further, in each of the above-described embodiments, the example in which the technique of the present invention is realized in the electronic camera 1 has been described. However, the present invention is not limited to this. For example, the present invention can be similarly applied to a compact type electronic camera, a movie camera for taking a moving image, and the like. Moreover, in the electronic camera 1, the example which comprised the light emission part 17 as integral with the electronic camera 1 was shown, However, This invention is applied similarly to the electronic camera which can attach or detach a light emission member with respect to a camera main body. Can do.

  Further, the image processing described in the above-described embodiments may be realized by software by a computer and an image processing program. In this case, what is necessary is just to make it the structure which implement | achieves a part or all of the process of the flowchart demonstrated in each embodiment with a computer. When realized by a computer, for example, the image data generated in step S4 in FIG. 3 and the image data of the image generated in step S6 are supplied to the computer, and the processing in steps S5, S7 to S11 in FIG. May be executed. Further, the image data of the image processed in step S5 in FIG. 3 and the image data of the image processed in step S7 are supplied to the computer, and the processing in steps S8 to S11 in FIG. good. By adopting such a configuration, it is possible to perform the same processing as in the above-described embodiments.

  The present invention can be implemented in various other forms without departing from the spirit or main features thereof. For this reason, the above-described embodiments are merely examples in all respects and should not be interpreted in a limited manner. The scope of the present invention is indicated by the scope of claims, and is not restricted by the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Electronic camera, 2 ... Shooting lens, 14 ... Image sensor, 17 ... Light emission part, 19 ... Control part

Claims (6)

An imaging unit that repeats imaging of a subject multiple times and generates a plurality of images;
A light emitting unit that illuminates the subject with a different light emission amount in the plurality of times of imaging;
A low-pass processing unit that performs low-pass processing on at least one of the plurality of images to generate a low-pass image;
Of the first gradation curve having an input / output characteristic that enhances contrast in the middle part of gradation and the second gradation curve having an input / output characteristic that weakens contrast in the middle part of gradation, the low-pass processing A setting unit for setting a gradation curve used for gradation conversion processing of the low-pass image generated by the unit;
A gradation conversion unit that performs gradation conversion processing on the low-pass image generated by the low-pass processing unit using the gradation curve set by the setting unit;
An image pickup apparatus comprising: a combining unit that combines the plurality of images generated by the image pickup unit based on a low-pass image subjected to the tone conversion process by the tone conversion unit. The imaging device according to claim 1,
A white balance adjustment unit that performs white balance adjustment on the plurality of images generated by the imaging unit;
The white balance adjustment unit performs the white balance adjustment based on automatic adjustment individually for each of the plurality of images. The imaging device according to claim 1,
The light emitting unit ends the illumination of the subject substantially simultaneously with the end of exposure during imaging by the imaging unit. The imaging device according to claim 1,
The imaging apparatus, wherein the low-pass processing unit generates the low-pass image based on an image having the smallest light emission amount at the time of imaging by the imaging unit among the plurality of images. The imaging device according to claim 1,
The light emitting unit measures the distances of a plurality of regions in the subject, and obtains an appropriate light emission amount for each of the plurality of regions based on a distance measurement result, whereby the light emission in each of the plurality of imaging operations. An imaging apparatus characterized by determining an amount. The imaging apparatus according to claim 5,
The low-pass processing unit is configured to perform the low-pass processing based on an image generated by the imaging unit when illumination is performed by the light-emitting unit with an appropriate light emission amount in the shortest distance region among the plurality of regions. An imaging apparatus characterized by generating an image.

Images