JP5125478B2 - Imaging apparatus and image effect providing program - Google Patents

Description

  The present invention relates to an imaging apparatus that applies an image effect to a captured image, and an image effect applying program.

  Conventionally, for example, a cross filter is attached to a lens of a digital camera to capture an image, or an image effect is applied to an image obtained by imaging, and an image with an image effect is generally obtained. Yes.

For example, when an image effect is applied to an image obtained by imaging (hereinafter referred to as an original image), if the image effect is directly applied to the original image, it takes time to apply the image effect. An image to be used as an image effect (hereinafter referred to as a mask image) is created from the reduced image obtained by reducing the image, and the created mask image is enlarged so as to have the same angle of view as the original image, and then combined with the original image. (See Patent Document 1).
JP 2006-213121 A

  However, it is effective to apply an image effect to an original image captured in an environment with a dark background, such as at night or in a dark place, but it can be applied to an original image captured in an environment with a bright background, such as indoors. When the image effect is applied, the color of the pixel corresponding to the mask image pattern may be saturated, and the applied image effect tends to be unnatural.

  The present invention has been invented in order to solve the above-described problems, and an imaging apparatus capable of imparting an appropriate image effect to an image captured in a bright environment, and an image The purpose is to provide an effect grant program.

  An image pickup apparatus according to a first aspect of the invention uses an image reduction unit that reduces an original image obtained at the time of imaging, and a first image that generates a first mask image using the reduced image generated by the image reduction unit. Based on the comparison result by the generating means, the calculating means for calculating the brightness in the reduced image, the comparing means for comparing the brightness calculated by the calculating means with a preset threshold value, A second image generating means for generating a second mask image obtained by subtracting the first mask image and the reduced image, an image enlarging means for enlarging the second mask image, and an enlarged second mask image. Image synthesizing means for synthesizing the original image.

  According to a second aspect of the present invention, in the first aspect of the invention, the image processing apparatus further includes a filter processing unit that performs a filter process on the first mask image, and the second image generation unit includes the first mask image after the filter process. And generating the second mask image using the reduced image.

The third aspect of the invention, in the first and second aspects of the present invention, an area detection means for detecting a region of interest in the reduced image, the first stroke image generator, the corresponding to the detected the target regions the One mask image is created.

A fourth invention, Te third invention smell, before Symbol area detection means calculates the luminance of each pixel of the reduced image, among the luminance of each pixel calculated, the target luminance is high pixel It is detected as a region.

According to a fifth invention, in the third or fourth invention, further comprising color information acquisition means for acquiring color information in a peripheral portion of the region of interest detected by the area detection means, wherein the first image generation means Is characterized in that the first mask image is generated based on the size of the region of interest and the color information at the periphery of the region of interest.

  In a sixth aspect based on any one of the first to fifth aspects, the first mask image has the same image size as the reduced image, and a preset pattern is formed at a position corresponding to the attention area. It consists of the image which was made.

  According to a seventh invention, in the sixth invention, the pattern has a shape in which a plurality of lines are radially extended from the region of interest as a base point.

  An image effect imparting program according to an eighth aspect of the present invention is a method of acquiring a reduced image by reducing an original image obtained at the time of imaging, and a step of generating a first mask image based on the reduced image; Calculating the luminance in the reduced image, comparing the luminance in the reduced image with a preset threshold value, and calculating a difference between the first mask image and the reduced image based on the comparison. Generating a second mask image, enlarging the second mask image, and synthesizing the enlarged second mask image with the original image. .

  ADVANTAGE OF THE INVENTION According to this invention, generation | occurrence | production of the whiteout which arises when an image effect is provided to the image imaged in the bright environment can be suppressed. By suppressing the occurrence of this whiteout, it is not necessary to give an uncomfortable feeling to the applied image effect.

  FIG. 1 is a schematic diagram illustrating an example of the configuration of the digital camera 10. The digital camera 10 is centrally controlled by the CPU 15. Since the operation signals of the operation unit 16 and the release button 17 are input to the CPU 15, the CPU 15 controls each unit of the digital camera 10 based on the input operation signals.

  The imaging optical system 20 includes a zoom lens 22 and a focus lens 23 in addition to the imaging lens 21. The zoom lens 22 is moved along the optical axis direction by the lens driving mechanism 24 when changing the imaging magnification at the time of imaging. The focus lens 23 is finely moved in the optical axis direction by the lens driving mechanism 24 when performing focus adjustment during imaging.

  The CCD 25 receives subject light incident via the imaging optical system 20, and photoelectrically converts the subject light by accumulating signal charges corresponding to the received light amount. The accumulation of signal charges in the CCD 25 and the output of the accumulated signal charges are controlled by the driver 26.

  The signal charge accumulated by the CCD 25 is output to the AFE circuit 27 as an analog image signal. The AFE circuit 27 includes an AGC circuit and a CDS circuit (not shown). The AFE circuit 27 performs analog processing such as gain control and noise removal on the input analog image signal.

  The DFE circuit 28 converts the analog image signal subjected to the analog processing by the AFE circuit 27 into a digital image signal. The converted digital image signals are collected frame by frame and stored in the buffer memory 30 as digital image data. The driver 26, the AFE circuit 27, and the DFE circuit 28 are controlled based on the operation timing in the timing generator (TG) 31, respectively. Reference numeral 32 denotes a bus, and each part of the digital camera 10 is electrically connected via the bus 32.

  The image processing unit 35 performs image processing such as contour compensation, gamma correction, and white balance adjustment on the digital image data stored in the buffer memory 30. The digital image data subjected to this image processing is subjected to format processing (image post-processing) for compression by a storage method such as the JPEG method, and then stored in the buffer memory 30 again. The digital image data stored in the buffer memory 30 is compressed using a compression rate corresponding to a predetermined compression method.

  For example, when a through image is displayed on the LCD monitor 36 provided in the digital camera 10, compression processing is executed on the digital image data after the image processing using a compression rate that matches the resolution of the LCD monitor 36. And output to the display control unit 37.

  On the other hand, when storing digital image data, a compression process using a compression rate suitable for image storage is performed and stored in the built-in memory 38. The digital image data stored in the built-in memory 38 is written into a storage medium 40 such as a memory card or an optical disk via the media controller 39. As the compression rate for image storage, in addition to the compression rate set for the image storage resolution set in advance in the digital camera 10, the compression rate is set according to the resolution set by the user operating the operation unit 16 in advance. Compression rate.

  The strobe device 41 emits strobe light toward the subject when, for example, the luminance of the subject in the photometric sensor 43 is a predetermined value or less. The strobe light emission timing and amount of light emitted from the strobe device 41 are controlled by a strobe control unit 42.

  In the following, the image data before reduction processing is the original image data, the reduced image data is reduction image data, and the image decoration image data created based on the reduction image data is primary mask image data or secondary This will be described as mask image data. The display of the original image data, reduced image data, primary mask image data, and secondary mask image data will be referred to as an original image, a reduced image, a primary mask image, and a secondary mask image, respectively.

  In addition to reducing the original image data stored in the built-in memory 38 at a predetermined reduction rate, the image enlargement / reduction unit 50 enlarges mask image data described later at a predetermined enlargement rate. In order to increase the processing speed when creating mask image data, which will be described later, the reduction ratio described above may be increased. However, the enlargement ratio when the mask image is enlarged also increases. The image quality of the enlarged mask image is degraded. Therefore, the reduction rate of the original image data when creating the mask image data is set in consideration of the processing speed and the image quality of the mask image. On the other hand, the enlargement ratio is an enlargement ratio at which the size of the mask image is the same as the size of the original image. As a method for reducing and enlarging the image data, a known bilinear method and a bicubic method can be cited.

  The image generation unit 51 generates mask image data to be combined with the original image data. As the mask image data, primary mask image data created from the size (number of pixels) of the attention area detected by the area detection unit 52 described later and the color information acquired by the color information acquisition unit 53 described later. And secondary mask image data obtained by performing difference processing using primary mask image data and reduced image data.

  As shown in FIG. 2, when the primary mask image data is displayed on the LCD monitor 36, for example, a primary mask image MI1 having the same angle of view as the reduced image is displayed. The primary mask image MI1 is an image in which patterns such as cross patterns 66 and 67 having high luminance are arranged on a black background (hatched area 65 indicated by a dot in FIG. 2), for example. The cross patterns 66 and 67 described above have a shape in which a plurality of lines starting from a pixel serving as the center of a region of interest described later are radially extended. The length and thickness of the lines of the cross patterns 66 and 67 are determined based on the size (number of pixels) of the region of interest. Note that table data in which the thickness and length of the cross pattern correspond to the size of the attention area is stored in advance, and the size and thickness of the cross pattern may be determined from the extracted size of the attention area.

  The colors of the cross patterns 66 and 67 are determined based on the color information acquired by the color information acquisition unit 53 described later. That is, the gradation value of each pixel is determined so that the luminance decreases from the center side of the cross pattern toward the tip of each line. Note that the number of lines in the cross pattern and the angle of the lines described above may be arbitrarily selected by the user in advance or may be set in advance. Further, the cross pattern may be arranged for all the attention areas, or the attention area where the cross pattern is arranged may be selected depending on the size of the attention area.

  On the other hand, the secondary mask image is an image created by a difference process between the primary mask image and the reduced image. For example, as shown in FIG. 3A, the gradation value in the pixels constituting the cross pattern of the primary mask image changes so as to approach the background color toward the tip. On the other hand, as shown in FIG. 3B, in the reduced image, the gradation value of the pixel corresponding to the cross pattern changes irregularly. By performing the difference processing between the primary mask image and the reduced image, the gradation value of the pixel in the secondary mask image gradually decreases (see FIG. 3C). That is, by performing this difference processing, when the secondary mask image and the reduced image are combined, the gradation value of the pixel corresponding to the cross pattern becomes the gradation value of the cross pattern of the primary mask image. For example, since the background of the primary mask image is black, when this difference processing is performed, the gradation value of the pixel corresponding to the background in the primary mask image becomes a negative value, Since there is no negative value, when the difference process is executed on the black pixel, the gradation value (R, G, B) of the pixel of the secondary mask image is (0, 0, 0), That is, it becomes black.

  The image generation unit 51 includes an area detection unit 52 and a color information acquisition unit 53. The region detection unit 52 detects a region of interest in the reduced image. As the detection of the attention area, pixels with high luminance are extracted from the reduced image, and among the extracted pixels, an area adjacent so that a plurality of pixels are substantially circular or substantially elliptical is defined as the attention area. To do. When the attention area is detected, a pixel that is the center of the attention area is also obtained.

  FIG. 4 is a diagram illustrating an example of an image obtained by capturing an indoor exhibit. In the case of this image, since the spot light is irradiated to the exhibit, the light emitting part (symbol 71 in the figure) of the light stand that irradiates the spot light and the portion of the exhibit that is irradiated with the spot light (Reference numeral 72 in the figure) is a high luminance area, that is, an attention area.

  The color information acquisition unit 53 acquires, as color information, an average value of gradation values of pixels having substantially the same luminance among the peripheral pixels of the region of interest detected by the region detection unit 52.

  The filter processing unit 54 performs low-pass filter (LPF) processing on the generated mask image data. In this LPF process, among the 9 pixels 71 to 79 in total of 3 × 3, the weight for the pixel 75 at the center is “2”, and the weight for the pixels 71 to 74 and the pixels 76 to 79 is “1”. A process for performing the convolution calculation. As shown in FIG. 5, for example, a total of nine pixels 71 to 79 of 3 × 3 are described. The pixel 77 is red, the pixels 74 and 78 are pink, the pixels 71 to 73, the pixels 75 and 76, and the pixel 79 are In the case of white, the pixel 75 changes from white to light pink by performing this LPF process. Since this processing is performed using all the pixels that make up the mask image, an image with a clear boundary between the pattern and the background in the mask image is processed into an image with a blurred boundary between the pattern and the background. Can do.

  The image composition unit 55 synthesizes the mask image data generated by the image generation unit 51 and the reduced image data, or synthesizes the mask image data enlarged by the image enlargement / reduction unit 50 and the original image data. The image data synthesized by the image synthesis unit 55 is stored in the built-in memory 38.

  The luminance calculation unit 56 calculates the luminance of the reduced image using the reduced image data. Specifically, the luminance at each pixel of the reduced image is calculated, and the average value of the calculated luminance of each pixel is calculated.

  The luminance determination unit 57 compares the average luminance value calculated by the luminance calculation unit 56 with a preset threshold value. In this comparison, when the average value of luminance is lower than the threshold value, it is determined that the original image data is image data acquired in a dark environment. Therefore, the primary mask image data is used as mask image data. used. On the other hand, when the average luminance value is higher than the threshold value, it is determined that the original image data is data acquired in a bright environment. In such a case, the image generation unit 51 The difference between the primary mask image data and the reduced image data is used to create secondary mask image data. The created secondary mask image data is used as mask image data.

  Next, a processing procedure for applying an image effect to image data captured by the digital camera 10 will be described based on a flowchart of FIG. It should be noted that it may be possible for the user to select in advance whether or not to apply an image effect to the captured image data, or an image effect may be automatically applied to all image data. When the imaging process is executed by the digital camera 10, the image data acquired by imaging is used as original image data, and the process proceeds to step S <b> 101.

  Step S101 is processing for creating reduced image data from original image data. Note that reduced image data is created separately from the original image data and stored in the built-in memory 38 by the processing in step S101.

  Step S102 is processing for detecting a region of interest using the reduced image data created in step S101. In step S102, pixels with high luminance are extracted from the pixels constituting the reduced image, and attention is paid to adjacent areas so that a plurality of extracted pixels are substantially circular or elliptical. Detect as a region. At the time of detecting this attention area, the pixel serving as the center is specified. That is, when the illuminated exhibit shown in FIG. 4 is imaged, the light stand 71 of the light stand and the area 72 illuminated by the light stand in the exhibit are detected as the attention area.

  Step S103 is processing for detecting color information of pixels at the periphery of the attention area extracted in step S102. In step S103, pixels having substantially the same luminance are detected from the peripheral area of the attention area, and a value obtained by averaging the gradation values of the detected pixels is acquired as color information.

  Step S104 is processing for creating primary mask image data. Since the size (number of pixels) of the attention area extracted in step S102 is known, the size of the pattern arranged in the primary mask image, that is, each line of the cross pattern is determined from the size of the attention area. The number of pixels indicating the length and the number of pixels indicating the thickness are respectively determined. Also, using the acquired color information, the color (RGB value or YCrCb value) of each pixel of the cross pattern is determined so that the luminance decreases from the base point of the cross pattern toward the tip of each line. With these determinations, primary mask image data is generated with a pixel at the center of the region of interest as a base point and a black pattern on the background and a cross pattern with the pixel corresponding to the center of the region of interest as the base point.

  Step S105 is an LPF process for the primary mask image pattern created in step S104. By performing this LPF process, the mask image becomes an image in which the boundary between the cross pattern and the background is blurred.

  Step 106 is a process of calculating luminance from the reduced image data. Specifically, it is a process of calculating the luminance of each pixel of the reduced image and obtaining the average value of the calculated luminance.

  Step S107 is a process of comparing the calculated average value of luminance and a threshold value. As a result of this comparison, if the average luminance value exceeds the threshold value, the process proceeds to step S108. On the other hand, if the average luminance value is equal to or less than the threshold value, the process proceeds to step S110.

  Step S108 is processing for generating secondary mask image data by difference processing using primary mask image data and reduced image data. If it is determined in step S107 that the average luminance value of the reduced image data exceeds the threshold value, the original image data is determined to be image data acquired in a bright environment. When the primary mask image data enlarged to 1 is combined or the primary mask image data is combined with the reduced image data, the gradation values of the pixels constituting the cross pattern are saturated. Accordingly, by performing the processing of step S108, the gradation value of the pixels constituting the cross pattern in the synthesized image becomes the gradation value of the pixels constituting the cross pattern of the primary mask image data, and the image is synthesized. Can be prevented from being saturated. When the secondary mask image data is generated in step S108, the generated secondary mask image data becomes mask image data used for the subsequent processing.

  Step S109 is processing for combining the mask image data and the reduced image data. That is, since the secondary mask image data is created in step S108, the secondary mask image data and the reduced image data are combined in the process of step S109. As shown in FIG. 7, for example, when a mask image is combined with an image of an indoor exhibit, an image in which a cross pattern is given to the light emitting unit of the light stand and the exhibit illuminated by the light stand. CI. For example, when the secondary mask image data is used as the mask image data, the secondary mask image data is combined with the reduced image data, so that the gradation value of each pixel constituting the cross pattern is not saturated and combined. An image that does not give a sense of incongruity to the cross pattern in the processed image is acquired.

  On the other hand, when the average value of the brightness calculated by the process of step S107 is equal to or less than the threshold value, the process proceeds to step S110. In step S110, the primary mask image data generated by the image generation unit 51 is used as mask image data, thereby synthesizing the mask image data and the reduced image data.

  When the process of step S109 or step S110 ends, the process proceeds to step S111. In this step S111, the synthesized image is displayed on the LCD monitor 38.

  Step S112 is processing for selecting whether or not to change the synthesized image. If the synthesized image is not changed in step S112, the process proceeds to step S113. If the synthesized image is changed, the process returns to step S102. Note that changing the combined image means, for example, changing the pattern of the mask image to be combined, changing the number of lines of the cross pattern when the selected pattern is a cross pattern, or changing the pattern of the cross pattern. For example, changing the inclination angle.

  Step S113 is a process for enlarging the mask image data. By performing the processing in step 105, the mask image data is an image in which the boundary between the background and the cross pattern is blurred. Therefore, even if the mask image data is enlarged in step S109, the image quality of the mask image data is deteriorated. You do n’t have to.

  Step S114 is a process of combining the original image data and the enlarged mask image data. Since the image quality of the mask image data enlarged in step S114 has not deteriorated, there is no sense of incongruity with the cross pattern synthesized with the original image. In addition, for example, even in the case of image data acquired in an environment where the original image is bright, when the mask image data and the original image data are combined, the gradation values of the pixels constituting the cross pattern are saturated. There is no. Thereby, for example, an image equivalent to an image obtained by imaging using a cross-screen filter can be created in a pseudo and high speed manner. Note that step S115 is processing for storing the created composite image data in the built-in memory.

  In this embodiment, the reduced image and the mask image are combined and then displayed on the LCD monitor 36. However, the present invention is not limited to this, and the combined image of the original image and the enlarged mask image is reduced. It may be displayed on the LCD monitor 36.

  In the present embodiment, the image enlargement / reduction unit 50, the image generation unit 51, the filter processing unit 54, the image synthesis unit 55, the luminance calculation unit 56, and the luminance determination unit 57 are configured separately from the image processing unit 35 and the CPU 15. However, the present invention is not limited to this, and the image processing unit 35 or the CPU 15 may include the functions of these units.

  In the present embodiment, a cross pattern is cited as a mask image pattern. However, the present invention is not limited to this. For example, a musical note or a heart may be used.

  In the present embodiment, an area with high luminance is detected as an attention area, and a cross pattern is synthesized with the detected attention area. For example, the background color information of the original image or the reduced image and the attention area It is also possible to acquire the color information and to add a cross pattern to the region of interest consisting of the same color information as the background color information. In addition, it is also possible to acquire background color information and adjust the color of the pattern of the mask image data to be synthesized in accordance with the acquired background color information.

  In the present embodiment, a digital camera having a function of giving an image effect to a captured image has been described. However, the present invention is not limited to this. For example, image processing that can execute the procedure shown in the flowchart of FIG. It may be a device. Moreover, the program which can make a computer perform the procedure shown in the flowchart of FIG. 6 and the storage medium which memorize | stores this program and can be read with a computer may be sufficient.

It is a functional block which shows an example of the outline of a digital camera. It is a figure which shows an example of a mask image. It is a figure which shows an example of the gradation value of the pixel used as the cross pattern of a primary mask image, the gradation value of the pixel of a corresponding reduced image, and the gradation value of the pixel in the secondary mask image produced by a difference process. It is a figure which shows an example of the image which imaged the illuminated exhibit. It is a figure which shows an example of a low-pass filter process. It is a flowchart which shows the process sequence which provides an image effect. It is a figure which shows an example of the image to which the image effect was provided.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Digital camera, 38 ... Built-in memory, 50 ... Image expansion / contraction part, 51 ... Image generation part, 52 ... Area | region detection part, 53 ... Color information acquisition part, 54 ... Filter processing part, 55 ... Image composition part, 56 ... Luminance Calculation unit, 57 ... luminance determination unit, 66, 67 ... cross pattern

Claims (8)

Image reduction means for reducing the original image obtained at the time of imaging;
First image generation means for generating a first mask image using the reduced image generated by the image reduction means;
Brightness detection means for detecting brightness in the reduced image;
Second image generation means for generating a second mask image obtained by subtracting the first mask image and the reduced image based on the brightness detected by the brightness detection means;
Image enlarging means for enlarging the second mask image;
Image synthesizing means for synthesizing the enlarged second mask image with the original image;
An imaging apparatus comprising: The imaging device according to claim 1,
Filter processing means for performing filter processing on the first mask image;
The image pickup apparatus, wherein the second image generation unit creates the second mask image using the first mask image after the filtering process and the reduced image. The imaging device according to claim 1 or 2,
Comprising an area detecting means for detecting an attention area in the reduced image;
The first stroke image generator is an imaging apparatus characterized by creating said first mask image corresponding to the detected the target area. The imaging device according to claim 3.
It said region detecting means calculates the luminance of each pixel of the previous SL reduced image of the luminance of each pixel calculated, an imaging device and detects the high brightness pixels as the region of interest. In the imaging device according to claim 3 or 4,
Further comprising color information acquisition means for acquiring color information at a peripheral edge of the region of interest detected by the area detection means;
The imaging apparatus according to claim 1, wherein the first image generation unit generates the first mask image based on a size of the attention area and color information at a peripheral portion of the attention area. In the imaging device according to any one of claims 1 to 5,
The imaging apparatus according to claim 1, wherein the first mask image is an image having the same image size as the reduced image and having a preset pattern formed at a position corresponding to the region of interest. The imaging device according to claim 6,
The image pickup apparatus according to claim 1, wherein the pattern has a shape in which a plurality of lines are radially extended with a center of the attention area as a base point. Obtaining a reduced image by reducing the original image obtained at the time of imaging;
Generating a first mask image based on the reduced image;
Detecting the luminance in the reduced image;
Generating a second mask image obtained by subtracting the first mask image and the reduced image based on the detected luminance;
Enlarging the second mask image;
Combining the enlarged second mask image with the original image;
An image effect providing program that causes a computer to execute the above.

Images