JP5711005B2 - Flicker correction method and image processing apparatus - Google Patents

Description

  The present invention relates to a flicker correction method and an image processing apparatus.

  As a general method of flicker correction of moving images, a plurality of past frame images are averaged to generate a reference image (an image without flicker), and the ratio between the generated reference image and a frame image to be subjected to flicker correction is corrected. A technique is known in which a frame image to be calculated as a coefficient and subjected to flicker correction is multiplied by a correction coefficient. Also, a technique is known that performs flicker correction using an average value of three frame images as a reference image when a 60 fps moving image is captured under fluorescent lighting in a region where the frequency of a commercial power supply is 50 Hz (for example, Patent Document 1).

  Hereinafter, the generation principle of the reference image in the conventionally known flicker correction will be described. FIG. 9 is a graph showing a blinking frequency of a light source and a frame image capturing timing which are conventionally known. The horizontal axis of the graph indicates time, and the vertical axis indicates the brightness of the light source. Note that the brightness when the light source is the brightest is 1, and the brightness when the light source is the darkest is -1. The brightness between the light source brightness and the darkest brightness is set to 0. A curve 900 indicates the blinking frequency of the light source (which is a sine curve as an ideal state). Points 901 to 903 indicate the imaging timing of the frame image.

  Flicker is caused by a difference between the blinking frequency of the light source and the imaging frequency (frame rate). In the illustrated example, the blinking frequency of the light source is 100 Hz, and the frame rate of the moving image to be captured is 60 fps. The first frame is imaged at time 0 ms (point 901 in the figure). Since the brightness of the light source at this time is just the median value, the brightness of the captured frame image is also the median value. The second frame is imaged at time 16.67 ms (point 902 in the figure). Since the brightness of the light source at this time is darker than the median value, the brightness of the captured frame image is darker than the median value. Further, the imaging of the third frame is performed at time 33.33 ms (point 903 in the figure). Since the brightness of the light source at this time is brighter than the median value, the brightness of the captured frame image is brighter than the median value. The imaging of the fourth frame is performed at a time of 50 ms (point 904 in the figure), and the brightness of the light source at this time is the median value as in the imaging of the first frame. Thereafter, the above pattern is repeated.

  In the three frame images captured in the first to third frames, the average value of the pixel values is calculated for each pixel having the same coordinates in each frame image, so that the brightness of the light source is exactly the center. An image (reference image) having the same brightness as that of the frame image captured when the value is obtained is obtained. That is, the reference image can be obtained by averaging the three frame images captured in the first to third frames. Specifically, the pixel value at the coordinate (1, 1) in the frame image captured in the first frame, the pixel value at the coordinate (1, 1) in the frame image captured in the second frame, and the image in the third frame The average value with the pixel value of the coordinate (1, 1) in the frame image is calculated, and the calculated value is set as the pixel value of the coordinate (1, 1) of the reference image. Similarly, a reference image is generated by calculating an average value for pixel values of other coordinates. In this manner, the reference image can be generated by averaging the three frame images captured in the first to third frames.

  For example, when flicker correction is performed on the frame image captured in the third frame, the ratio of the pixel values of the reference image and the frame image captured in the third frame is calculated as a correction coefficient, and 3 frames By multiplying the pixel value of the frame image captured by the eye by the correction coefficient, the flicker of the frame image captured by the third frame can be corrected. Note that a frame image to be subjected to flicker correction is a correction target image.

  In addition, although the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be picked up is 60 fps, the reference image is generated using the three frame images. Even when the frame rate of the moving image is different from the above-described example, the reference image can be generated using a plurality of frame images. Note that the number of frame images necessary to generate the reference image can be calculated from the frequency b (Hz) of the light source and the frame rate a (fps) of the moving image to be captured. Specifically, the least common multiple of the reciprocal 1 / a of the frame rate a (fps) of the moving image to be captured and the reciprocal 1 / b of the blinking frequency b (Hz) of the light source is calculated. Then, the numerator value when the calculated denominator value of the least common multiple is the same value as the frame rate is the number of images necessary for generating the reference image.

  In the above description and the following description, the exposure time (the time during which the shutter of the image sensor is open) is set to 0 for ease of explanation. It does not matter if it is In this case, the brightness of the frame image is the average brightness within the exposure time if the image sensor is a global shutter, and the brightness changes for each line of the frame image if it is a rolling shutter. Absent.

JP 2001-16508 A

  However, in recent years, image sensors that can capture images at a high frame rate have been developed, and cameras that take advantage of these features have been developed. In addition, the number of frame images necessary for generating a reference image used for flicker correction varies depending on the combination of the blinking frequency of the light source and the frame rate of the moving image to be captured. In general, the higher the frame rate, the greater the number of frame images required to generate a reference image used for flicker correction. Therefore, in an imaging device that can capture a moving image with a frame rate of 60 fps or more, three or more frame images may be required.

  As an example in which the number of frame images necessary for generating a reference image increases, for example, the blinking frequency of the light source is 50 Hz and the frame rate of the moving image to be captured is 240 fps. In this case, in order to generate the reference image, the above method theoretically requires 12 frame images. Therefore, when performing flicker correction under such conditions, a memory for storing 12 frame images is required, and it is necessary to always generate the reference image by averaging the latest 12 frame images. For this reason, every time flicker correction processing is performed on each frame image, a time or arithmetic circuit for averaging the 12 frame images is required. Therefore, the conventionally known flicker correction methods have a problem that the memory, the circuit scale, and the processing time are increased.

  The present invention has been made in view of the above circumstances, and even when flicker correction is performed when capturing a high frame rate moving image, the flicker can reduce the memory, circuit scale, and processing time. An object is to provide a correction method and an image processing apparatus.

In the present invention, the calculation unit calculates the least common multiple of 1 / a and 1 / b from the frame rate a [fps] of the moving image to be captured and the blinking frequency b [Hz] of the light source, and the calculated least common multiple A calculation step of calculating a minimum divisor n excluding 1 included in the numerator value when the denominator value is converted to a, and a reference image generation unit that sets a cycle of the least common multiple multiplied by the blinking frequency b to n A reference image generating step for generating a reference image by averaging n frame images captured at equally divided timings, and a correcting step for performing flicker correction of the captured moving image based on the reference image; , Including a flicker correction method.

  Further, in the flicker correction method of the present invention, in the reference image generation step, when the frame image having the same brightness as the frame image captured most recently is obtained among the n frame images, the reference image is determined. It is characterized by updating.

  In the flicker correction method of the present invention, in the reference image generation step, the reference image is updated when a predetermined time or more has elapsed since the previous update.

Further, the present invention provides an imaging unit that captures a moving image having a frame rate a [fps] under a light source having a blinking frequency b [Hz], the frame rate a [fps], and the blinking frequency b [Hz]. Calculating a least common multiple of / a and 1 / b, and calculating a minimum divisor n excluding 1 included in a numerator value when the calculated denominator of the least common multiple is converted to a; a reference image generation unit that generates a reference image n frame images of the captured cycle the least common multiple and the blink frequency b times at a timing n equal parts by averaging, the reference to the reference image generating unit has generated An image processing apparatus comprising: a correction unit that performs flicker correction on the moving image captured by the imaging unit based on an image.

According to the present invention, the calculating unit calculates the least common multiple of 1 / a and 1 / b from the frame rate a [fps] of the moving image to be captured and the blinking frequency b [Hz] of the light source, and calculates the minimum The smallest divisor n except 1 included in the numerator value when the denominator value of the common multiple is converted to a is calculated. In addition, the reference image generation unit generates a reference image by averaging n frame images captured at a timing obtained by dividing the period obtained by multiplying the least common multiple by the blinking frequency b into n equal parts. In addition, the correction unit performs flicker correction on the captured moving image based on the reference image. As a result, the number of frame images used when generating the reference image can be reduced. Therefore, the capacity of the storage unit that stores the frame image can be reduced, and the processing time for creating the reference image or the circuit resources in the hardware processing can be further reduced. Therefore, even when flicker correction is performed when capturing a high frame rate moving image, the memory, circuit scale, and processing time can be further reduced.

It is the graph which showed the blinking frequency of the light source in 1st Embodiment, and the imaging time timing of the frame image. It is the graph which showed the blinking frequency of the light source in 1st Embodiment, and the imaging time timing of the frame image. FIG. 6 is a diagram summarizing the relationship between the number of frame images necessary for generating a reference image and the number of frame images that need to be stored for generating the reference image in the first embodiment. 1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment. 3 is a flowchart illustrating an operation procedure of the image processing apparatus according to the first embodiment. In the second embodiment, the number n of frame images necessary for generating the reference image, the number m of frame images that need to be stored to generate the reference image, and the storage period of the frame image for generating the reference image It is the figure which summarized the relationship with t. 10 is a flowchart illustrating an operation procedure of the image processing apparatus according to the second embodiment. In the second embodiment, the number n of frame images necessary for generating the reference image, the number m of frame images that need to be stored to generate the reference image, and the storage period of the frame image for generating the reference image It is the figure which summarized the relationship between t and the update interval of a reference image. It is the graph which showed the blink frequency of the light source known conventionally, and the imaging timing of the frame image.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. First, the principle of generating a reference image used for flicker correction in this embodiment will be described. FIG. 1 is a graph showing the blinking frequency of the light source and the frame image capturing time timing in this embodiment. The horizontal axis of the graph indicates time, and the vertical axis indicates the brightness of the light source. Note that the brightness when the light source is the brightest is 1, and the brightness when the light source is the darkest is -1. The brightness between the light source brightness and the darkest brightness is set to 0. A curve 1001 indicates the blinking frequency of the light source (which is a sine curve as an ideal state). Points 101 to 112 indicate times when frame images from the first frame to the twelfth frame are captured.

  In the illustrated example, the blinking frequency of the light source is 100 Hz, and the frame rate of the moving image to be captured is 240 fps. The first frame is imaged at time 0 ms (point 101 in the figure). Since the brightness of the light source at this time is just the median value, the brightness of the captured frame image is also the median value. In addition, the imaging of the second frame is performed at a time of 4.16 ms (point 102 in the figure). Since the brightness of the light source at this time is brighter than the median value, the brightness of the captured frame image is brighter than the median value. Further, the imaging of the third frame is performed at a time of 8.33 ms (point 103 in the figure). Since the brightness of the light source at this time is darker than the median value, the brightness of the captured frame image is darker than the median value. The imaging time after the fourth frame and the brightness of the light source are as shown in the figure.

  When the flicker frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 240 fps, a conventionally known flicker correction method uses 12 to generate a reference image that is an image having a central brightness. One frame image is required. The method for calculating the required number of frame images is as described above.

  However, referring to the graph shown in FIG. 1, it can be seen that an image having a central brightness can be generated without using 12 frame images. For example, if the frame image captured at the first frame (point 101 in the figure) and the frame image captured at the seventh frame (point 107 in the figure) are averaged, it can be seen that an image having a central brightness can be generated. . It can also be seen that if the frame image captured in the second frame (point 102 in the figure) and the frame image captured in the eighth frame (point 108 in the figure) are averaged, an image with a central brightness can be generated. . Similarly, by averaging two frame images such as averaging a frame image captured at the third frame (point 103 in the figure) and a frame image captured at the ninth frame (point 109 in the figure). It can be seen that an image having a central brightness can be generated.

  This can be explained by the principle that “the sum of the sine values when the sine curve is equally divided is always equal to 0 regardless of the number”. In the present embodiment, the blinking frequency of the light source is a sine curve. The timing for capturing the frame image is an interval that equally divides a cycle obtained by multiplying the blink cycle of the light source (cycle based on the blink frequency of the light source) by an integer. For this reason, the brightness of an image obtained by averaging frame images taken at intervals equal to a cycle obtained by multiplying the blinking cycle of the light source by an integer is exactly the center value. For example, the brightness of an image obtained by averaging 12 frame images taken at the timing of dividing the sine curve into 12 parts is equal to the brightness of the image obtained by averaging two frame images taken at the timing of dividing the sine curve into 2 parts. In addition, both are 0 values in terms of sine values, that is, the brightness at the center of the light source.

Based on the above-described characteristics, a method for calculating the number of frame images (necessary frame images) necessary for generating the reference image and the number of necessary frame images (saved images) to be stored is generalized as follows. .
(1) The number n of frame images necessary for generating a reference image is the minimum divisor of the number of frame images required for generating a reference image using a conventional method (however, 1 is excluded).
(2) The number m of frame images that need to be stored in order to generate a reference image is determined based on the number of frame images required when a reference image is generated using a conventional method. Subtract the value obtained by dividing the number of frame images required for image generation by the number n of frame images required for generating the reference image calculated in (1), and add the number of correction target images (1). Value.

For example, as shown in FIG. 1, when the blinking frequency of the light source is 100 Hz and the moving image frame rate is 240 fps, the number n of frame images necessary for generating the reference image is as follows.
The number of frame images required for generating a reference image using the conventional method is “12”.
-The minimum divisor (excluding 1) of the number of frame images required for generating a reference image using the conventional method is "2"
Therefore, the number n of frame images necessary for generating the reference image is “2”.

As shown in FIG. 1, when the blinking frequency of the light source is 100 Hz and the moving image frame rate is 240 fps, the number m of frame images that need to be stored to generate the reference image is as follows: It becomes.
The number of frame images required for generating a reference image using the conventional method is “12”.
The number n of frame images necessary for generating the reference image is “2”.
The value obtained by dividing the number of frame images “12” necessary for generating the reference image using the conventional method by the number of frame images “2” required for generating the reference image is “6”.
The number of frame images necessary for generating the reference image using the conventional method is changed from the number of frame images “12” required for generating the reference image using the conventional method to generate the reference image. The value obtained by subtracting the value “6” divided by the number of necessary frame images is “6”.
-The number of frame images necessary for generating the reference image is calculated from the number of frame images required for generating the reference image using the conventional method to the number of frame images required for generating the reference image using the conventional method. The value obtained by subtracting the value divided by the number of images “6” and adding the number of correction target images “1” is “7”.
Therefore, the number m of frame images that need to be stored in order to generate the reference image is “7”.

  That is, when the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 240 fps, in order to generate a reference image for performing flicker correction of the frame image of the 12th frame using the conventional method It is necessary to save all 12 frame images of the first to twelfth frames and to average the 12 frame images. However, when generating a reference image for performing flicker correction of the frame image of the 12th frame using the method of the present embodiment, only the 7 frame images of the 6th to 12th frames are stored, Only two of the frame images need be averaged.

  Note that the two frame images necessary for generating the reference image are two frame images captured at a timing obtained by equally dividing the cycle obtained by multiplying the blinking cycle of the light source by 5 into two equal parts. In this example, there are two saved images (the sixth frame image) and the correction target image (the 12th frame image), which are captured most recently. This is based on the above-mentioned principle that “the sum of sine values when the sine curve is equally divided is always equal to 0 even if it is equally divided”.

  Next, as an example different from the example shown in FIG. 1, a case where the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 90 fps will be described. FIG. 2 is a graph showing the blinking frequency of the light source and the frame image capturing time timing in this embodiment. The horizontal axis of the graph indicates time, and the vertical axis indicates the brightness of the light source. As in FIG. 1, the brightness when the light source is the brightest is 1, and the brightness when the light source is the darkest is -1. The brightness between the light source brightness and the darkest brightness is set to 0. A curve 2001 indicates the blinking frequency of the light source (which is a sine curve as an ideal state). Points 201 to 209 indicate times when the frame images of the first to ninth frames are captured.

  In the illustrated example, the blinking frequency of the light source is 100 Hz, and the frame rate of the moving image to be captured is 90 fps. The first frame is imaged at time 0 ms (point 201 in the figure). Since the brightness of the light source at this time is just the median value, the brightness of the captured frame image is also the median value. In addition, the imaging of the second frame is performed at time 11.11 ms (point 202 in the figure). Since the brightness of the light source at this time is brighter than the median value, the brightness of the captured frame image is brighter than the median value. Further, the imaging of the third frame is performed at time 22.22 ms (point 203 in the figure). Since the brightness of the light source at this time is brighter than that at the time of imaging of the second frame, the brightness of the captured frame image is brighter than that of the frame image of the second frame. The imaging time after the fourth frame and the brightness of the light source are as shown in the figure.

  Even in this case, the number n of frame images necessary for generating the reference image and the number m of frame images that need to be stored to generate the reference image are calculated using the same method as described above. be able to.

Specifically, as shown in FIG. 2, when the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 90 fps, the number n of frame images necessary for generating the reference image is as follows. .
The number of frame images required when generating a reference image using the conventional method is “9”
The minimum divisor (excluding 1) of the number of frame images required when generating a reference image using the conventional method is “3”.
Accordingly, the number n of frame images necessary for generating the reference image is “3”.

Also, as shown in FIG. 2, when the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 90 fps, the number m of frame images that need to be stored to generate the reference image is as follows: It becomes as follows.
The number of frame images required when generating a reference image using the conventional method is “9”
The number n of frame images necessary for generating the reference image is “3”.
The value obtained by dividing the number of frame images “9” necessary for generating the reference image using the conventional method by the number of frame images “3” required for generating the reference image is “3”.
From the number of frame images “9” required when generating a reference image using the conventional method, the number of frame images required when generating the reference image using the conventional method is used to generate the reference image. The value obtained by subtracting the value “3” divided by the number of necessary frame images is “6”.
-The number of frame images necessary for generating the reference image is calculated from the number of frame images required for generating the reference image using the conventional method to the number of frame images required for generating the reference image using the conventional method. The value obtained by subtracting the value divided by the number of images “6” and adding the number of correction target images “1” is “7”.
Therefore, the number m of frame images that need to be stored in order to generate the reference image is “7”.

  That is, when the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 90 fps, in order to generate a reference image for flicker correction of the ninth frame image, It is necessary to save all nine frame images of the first to ninth frames and average the nine frame images. However, in the case of generating a reference image for performing flicker correction of the frame image of the ninth frame by the method of the present embodiment, only the seven frame images of the third to ninth frames are stored, of which It is only necessary to average the three frame images.

  Note that the three frame images necessary for generating the reference image are three frame images captured at a timing obtained by dividing the cycle obtained by multiplying the blinking cycle of the light source by 10 into three equal parts. In this example, among the stored frame images, the most recently stored image (the third frame image), the correction target image (the ninth frame image), and the most previously stored image Three images are stored images (sixth frame image) captured at an intermediate timing with the correction target image. This is based on the above-mentioned principle that “the sum of sine values when the sine curve is equally divided is always equal to 0 even if it is equally divided”.

  FIG. 3 shows the number of frame images necessary for generating a reference image and the reference image when shooting moving images having various frame rates when the light / dark frequency of the light source is 100 Hz. It is the figure which put together the relationship with the number of the frame images which need to be preserve | saved for this. In the illustrated example, they are summarized in a table format. For each column, the frame rate of the moving image to be captured, the number of frame images required when generating a reference image using a conventional method, The number of frame images necessary for generating a reference image using the method, the number of frame images that need to be stored when generating the reference image using a conventional method, and the method of this embodiment are used. The number of frame images that need to be saved when generating a reference image is shown in association with each other.

  For example, when shooting a moving image with a frame rate of 15 fps, the number of frame images required for generating a reference image using the conventional method is “3”, and the reference image using the method of this embodiment is used. The number of frame images required when generating the image is “3”, and the number of frame images that need to be stored when generating the reference image using the conventional method is “3”. This indicates that the number of frame images that need to be saved when a reference image is generated using the method of the embodiment is “three”.

  In addition, when shooting a moving image with a frame rate of 240 fps, the number of frame images necessary for generating a reference image using the conventional method is “12”, and the reference image is generated using the method of this embodiment. The number of frame images required when generating the image is “2”, and the number of frame images that need to be stored when generating the reference image using the conventional method is “12”. This indicates that the number of frame images that need to be saved when the reference image is generated using the method of the embodiment is “7”. The case of shooting a moving image of another frame rate is as illustrated.

  As shown in the figure, when shooting a moving image with a high frame rate (for example, a frame rate of 90 fps or more), the method of the present embodiment is more than the number of frame images required when generating a reference image using a conventional method. The number of frame images required for generating a reference image using is smaller. In addition, when shooting a high frame rate moving image, the reference image is generated using the method of this embodiment rather than the number of frame images that need to be stored when the reference image is generated using the conventional method. The number of frame images that need to be saved is smaller.

  Next, the configuration of the image processing apparatus in the present embodiment will be described. FIG. 4 is a block diagram showing the configuration of the image processing apparatus according to this embodiment. In the illustrated example, the image processing apparatus 1 includes an input unit 11, a required number calculation unit 12 (calculation unit), an imaging unit 13, a temporary storage unit 14, a reference image generation unit 15, and a gain calculation unit 16. , A correction unit 17, a storage unit 18, and a control unit 19.

  The input unit 11 inputs the frame rate of the moving image to be captured to the required number calculation unit 12 and the imaging unit 13. Further, the input unit 11 inputs the blinking frequency of the light source to the required number calculation unit 12. For example, the input unit 11 includes a switch, a numeric keypad, and the like, and receives input from the user of the frame rate of the moving image to be captured and the blinking frequency of the light source. Then, the input unit 12 inputs the frame rate of the moving image to be captured, which is input from the user, to the required number calculation unit 12 and the imaging unit 13, and inputs the blinking frequency of the light source to the required number calculation unit 12. . Further, for example, the frame rate of the moving image to be captured and the blinking frequency of the light source are determined in advance, and the input unit 11 sets the predetermined frame rate of the moving image to be captured to the necessary number calculation unit 12 and the imaging unit 13 And a predetermined blinking frequency of the light source is input to the required number calculation unit 12.

  The necessary number calculation unit 12 generates the reference image and the number of frame images necessary for generating the reference image based on the frame rate of the moving image to be captured and the flicker frequency of the light source input from the input unit 11. To calculate the number of frame images that need to be saved. The imaging unit 13 captures a moving image based on the frame rate of the moving image to be captured, which is input from the input unit 11. The temporary storage unit 14 temporarily stores a frame image of a moving image captured by the imaging unit 13. The reference image generation unit 15 generates a reference image using the frame image temporarily stored in the temporary storage unit 14. The gain calculation unit 16 calculates a gain value for correcting the correction target image using the correction target image captured by the imaging unit 13 and the reference image generated by the reference image generation unit 15. The correction unit 17 corrects the flicker of the correction target image using the gain value calculated by the gain calculation unit 16. The storage unit 18 stores the frame image corrected by the correction unit 17 as a moving image. The control unit 19 controls each unit included in the image processing apparatus 1.

Next, the operation procedure of the image processing apparatus 1 will be described. FIG. 5 is a flowchart showing an operation procedure of the image processing apparatus 1 in the present embodiment.
(Step S101) The user inputs the frame rate of the moving image to be captured and the blinking frequency of the light source to the input unit 11 of the image processing apparatus 1 before shooting the moving image. The input unit 11 inputs the frame rate of the image to be captured, which is input by the user, to the necessary number calculation unit 12 and the imaging unit 13, and inputs the blinking frequency of the light source to the necessary number calculation unit 12. Thereafter, the process proceeds to step S102.

(Step S102) The required number calculation unit 12 calculates the number n of frame images necessary for generating a reference image based on the frame rate of the moving image to be captured and the blinking frequency of the light source input in the process of step S101. And the number m of frame images that need to be stored in order to generate the reference image. Further, the necessary number calculation unit 12 generates the reference image generation using the calculated number n of frame images necessary for generating the reference image and the number m of frame images that need to be stored for generating the reference image. Input to the unit 15 and the control unit 19. Thereafter, the process proceeds to step S103.
(Step S <b> 103) When starting the shooting of a moving image, the user presses a shooting start button provided in the input unit 11. When the shooting start button included in the input unit 11 is pressed, the control unit 19 causes the imaging unit 13 to start shooting a moving image. Under the control of the control unit 19, the imaging unit 13 starts shooting a moving image at a frame rate specified by the frame rate of the moving image to be captured, which is input in step S <b> 101. Thereafter, the process proceeds to step S104.

  (Step S <b> 104) When the user finishes shooting a moving image, the user presses a shooting end button provided in the input unit 11. If the shooting end button included in the input unit 11 has not been pressed, the control unit 19 determines to continue moving image shooting, and proceeds to the process of step S105. In addition, when the shooting end button included in the input unit 11 is pressed, the control unit 19 determines that the moving image shooting is not continued, and causes the imaging unit 13 to end the shooting of the moving image. The imaging unit 13 ends the moving image shooting based on the control of the control unit 19. Thereafter, the process ends.

(Step S105) The imaging unit 13 captures a frame image (correction target image) for one frame. Thereafter, the process proceeds to step S106.
(Step S106) The control unit 13 causes the temporary storage unit 14 to store the frame image for one frame captured by the imaging unit 13 in the process of step S105. Thereafter, the process proceeds to step S107.

(Step S107) The reference image generation unit 15 stores frames necessary for generating a reference image out of m frame images stored in the temporary storage unit 14 that need to be stored in order to generate a reference image. The n images are averaged to generate a reference image. Thereafter, the process proceeds to step S108. When the number of frame images stored in the temporary storage unit 14 is less than m, the process returns to step S104.
(Step S108) The gain calculation unit 16 uses the correction target image captured by the imaging unit 13 in the process of step S105 and the reference image generated by the reference image generation unit 15 in the process of step S107. A gain value for correcting flicker is calculated. Thereafter, the process proceeds to step S109.

(Step S109) The correction unit 17 corrects the flicker of the correction target image captured by the imaging unit 13 in the process of step S105, using the gain value calculated by the gain calculation unit 16 in the process of step S108. Thereafter, the process proceeds to step S110.
(Step S110) The control unit 19 stores the correction target image corrected by the correction unit 17 in the process of step S109 in the storage unit 18 as a moving image. Thereafter, the process proceeds to step S111.
(Step S111) The control unit 19 deletes unnecessary frame images from the frame images stored in the temporary storage unit. Thereafter, the process returns to step S104.

  Next, a specific example of the operation procedure of the image processing apparatus 1 described with reference to FIG. 5 will be described by taking as an example a case where the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 240 fps. First, in the process of step S <b> 101, the input unit 11 inputs the frame rate “240 fps” to the required number calculation unit 12 and the imaging unit 13, and inputs the blinking frequency “100 Hz” of the light source to the required number calculation unit 12. Subsequently, in the process of step S102, the required number calculation unit 12 sets the number m “7” of frame images that need to be stored in order to generate the reference image, and the number of frame images required to generate the reference image. The number n “2” is calculated.

  Subsequently, in the process of step S <b> 103, the imaging unit 13 starts shooting a moving image with a frame rate of “240 fps”. Subsequently, in the process of step S104, if the control unit 19 determines to continue moving image shooting, the imaging unit 13 captures a frame image (correction target image) in step S105, and step S106. In this process, the data is stored in the temporary storage unit 14.

  Subsequently, in the process of step S107, the average of the two “2” frame images necessary for generating the reference image out of the “7” frame images that need to be stored for generating the reference image. An image, that is, a reference image is generated. The two frame images necessary for generating the reference image are the saved image captured in the past and the correction target image. The selection of the two frame images is based on the principle of equally dividing the sine curve described above. As a frame image necessary for generating the reference image, a cycle obtained by multiplying the blinking cycle of the light source by 5 times is exactly the same. Two frame images picked up at the timing divided into two are selected.

  Subsequently, in step S108, the reference image is divided by the correction target image to calculate a gain value for correcting flicker of the correction target image. Subsequently, in the process of step S109, the correction target image is multiplied by the gain value calculated in the process of step S109 to perform flicker correction. Subsequently, in step S110, the correction target image subjected to the flicker correction is stored in the storage unit 18 as a moving image. Subsequently, in the process of step S <b> 111, the stored image captured in the past among the temporary storage unit 14 is deleted from the temporary storage unit 14. Thereafter, the processing from step S105 to step S111 is repeatedly executed until the photographing end button is pressed.

  Next, a specific example of the operation procedure of the image processing apparatus 1 described with reference to FIG. 5 will be described by taking as an example the case where the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 90 fps. First, in the process of step S <b> 101, the input unit 11 inputs the frame rate “90 fps” to the required number calculation unit 12 and the imaging unit 13, and inputs the blinking frequency “100 Hz” of the light source to the required number calculation unit 12. Subsequently, in the process of step S102, the required number calculation unit 12 sets the number m “7” of frame images that need to be stored in order to generate the reference image, and the number of frame images required to generate the reference image. The number n “3” is calculated.

  Subsequently, in the process of step S <b> 103, the imaging unit 13 starts shooting a moving image with a frame rate of “90 fps”. Subsequently, in the process of step S104, if the control unit 19 determines to continue moving image shooting, the imaging unit 13 captures a frame image (correction target image) in step S105, and step S106. In this process, the data is stored in the temporary storage unit 14.

  Subsequently, in the process of step S107, the average of the three frame images “3” necessary to generate the reference image from among the “7” frame images that need to be stored to generate the reference image. An image, that is, a reference image is generated. The frame image necessary to generate the reference image at this time was captured at a timing just between the stored image captured most recently, the correction target image, and the stored image captured most recently and the correction target image. Three images with saved images. The selection of the three frame images is based on the principle of equally dividing the sine curve described above. As a frame image necessary for generating the reference image, a cycle obtained by multiplying the blinking cycle of the light source by 10 times is used. Three frame images captured at the timing divided into three equal parts are selected.

  The processing in steps S108 to S111 is similar to the processing in the case where the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 240 fps. Further, the processing from step S105 to step S111 is repeatedly executed until the photographing end button is pressed.

  By performing the flicker correction of the image by the above-described method, it is possible to reduce the number of frame images used for creating the reference image. Therefore, the capacity of the storage unit that stores the frame image can be reduced, and the processing time for creating the reference image or the circuit resources in the hardware processing can be further reduced. That is, the image processing apparatus and the image processing method in the present embodiment can further reduce the memory, circuit scale, and processing time even when performing flicker correction when capturing a high frame rate moving image.

  At any frame rate, in the process of step S108, a gain value for each pixel of the frame image may be obtained, or a gain value for each line of the frame image may be obtained, or the entire frame image may be obtained. A single gain value may be obtained. In the process of step S109, the flicker of the correction target image may be corrected for each unit for which the gain value is obtained in step S108.

  For example, if the imaging device captures images using the global shutter method, the light source brightness is constant when the entire frame image is captured, so flicker is corrected by obtaining one gain value for the entire frame image. It is desirable to do. In this case, the average value of the entire reference image and the average value of the corrected image obtained in step S107 may be obtained, and one correction gain value may be calculated based on the average value. Further, if the image pickup device picks up an image by the rolling shutter system, it is desirable to correct the flicker by obtaining a gain value for each line of the frame image because the brightness of the light source is different for each line. In this case, an average value for each line of the reference image and an average value for each line of the corrected image may be obtained, and a correction gain value for each line may be calculated.

  Further, in the present embodiment, the description has been made on the assumption that the same subject is photographed in all of the plurality of images used in step S107. However, in an actual photographing scene, the movement of the camera such as camera shake or the entire subject moves. There is a possibility that the position of the entire image is shifted due to or a part of the image is mismatched due to the movement of a part of the subject. Even in such a case, in order to calculate an accurate correction gain value, for example, a reference image is generated after correcting the positional deviation of the entire image by using a camera shake correction function generally used in a camera or the like. Alternatively, the movement of the portion of the subject is detected using the moving object detection function, and the portion having movement may not be used for calculating the average value for each line or the entire screen in the above description.

  In this embodiment, the flicker correction operation flow at the time of moving image shooting has been described. However, since the flicker correction is a process for correcting the brightness, the corrected image is the same as the image before the correction except for the brightness. There is no change. Therefore, it is possible to easily combine with image processing not disclosed in the present embodiment other than flicker correction.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment, a reference image is generated for each correction target image. However, in the present embodiment, one reference image is generated for a plurality of correction target images. That is, the reference image is updated every plural frames. As a result, the capacity of the storage unit for storing the frame image can be further reduced, and the processing time for creating the reference image or the circuit resources in the hardware processing can be further reduced. Note that the image processing apparatus 1 in the present embodiment has the same configuration as the image processing apparatus 1 in the first embodiment.

  Next, the timing for generating the reference image in the present embodiment will be described with reference to FIG. As described above, FIG. 1 is a graph showing the blinking frequency of the light source and the imaging time timing of the frame image when the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 240 fps. In the illustrated example, when the correction target image is the frame image of the seventh frame, the reference image is generated by averaging the frame image of the first frame and the frame image of the seventh frame. This is the same as in the first embodiment.

  Next, when the correction target image is the frame image of the eighth frame, in the first embodiment, the frame image of the second frame and the frame image of the eighth frame are averaged to generate the reference image. However, in the present embodiment, the reference image generated for performing the flicker correction of the frame image of the seventh frame is used for the flicker correction of the frame image of the eighth frame. In the present embodiment, the reference image generated for flicker correction of the seventh frame image is used for correcting the seventh to twelfth frame images. When a frame image having the same brightness as that of the seventh frame image (13th frame image) is acquired, a new reference image is generated (the reference image is updated).

When a new reference image is generated when a frame image having the same brightness as the frame image acquired when the reference image was generated last time is acquired, the frame image (necessary frame required for generating the reference image) The method of calculating the number of images) and the number of frame images (stored images) that need to be stored is generalized as follows.
(1) The number n of frame images necessary for creating a reference image is the minimum divisor (excluding 1) of the number of frame images necessary for generating a reference image using a conventional method. .
(2) The number m of frame images that need to be stored in order to generate a reference image is the same as the number n of frame images required for creating the reference image in (1). The frame image storage cycle t for generating the reference image (the cycle in which the frame images that need to be stored for generating the reference image among the frame images appear) is generated when the reference image is generated using a conventional method. Is the value obtained by dividing the number of frame images necessary for the above by the number n of frame images necessary for creating the reference image.

For example, as shown in FIG. 1, when the blinking frequency of the light source is 100 Hz and the moving image frame rate is 240 fps, the number n of frame images necessary for generating the reference image is as follows.
The number of frame images required for generating a reference image using the conventional method is “12”.
-The minimum divisor (excluding 1) of the number of frame images required for generating a reference image using the conventional method is "2"
Therefore, the number n of frame images necessary for generating the reference image is “2”.

As shown in FIG. 1, when the blinking frequency of the light source is 100 Hz and the moving image frame rate is 240 fps, the number m of frame images that need to be stored to generate the reference image is as follows: It becomes.
The number n of frame images necessary for generating the reference image is “2”.
Accordingly, the number m of frame images that need to be stored in order to generate the reference image is “2”.

Further, as shown in FIG. 1, when the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 240 fps, the storage period t of the frame image for creating the reference image is as follows.
The number of frame images required for generating a reference image using the conventional method is “12”.
-The number n of frame images necessary for creating the reference image is “2”.
The value obtained by dividing the number of frame images “12” necessary for generating the reference image using the conventional method by the number of frame images “2” necessary for generating the reference image is “6”.
Therefore, the frame image storage cycle t is “6”. That is, every time six frame images are acquired, one frame image is stored.

  FIG. 6 shows a case where the number n of frame images necessary for generating a reference image and a reference image are generated in the case where moving images having various frame rates are taken when the light / dark frequency of the light source is 100 Hz. FIG. 6 is a diagram summarizing the relationship between the number m of frame images that need to be stored in order to be stored and the storage period t of frame images for creating a reference image. In the example shown in the figure, they are summarized in a table format. For each column, the frame rate of a moving image to be shot, the number n of frame images necessary for generating a reference image using a conventional method, and the present embodiment The number n of frame images required when generating a reference image using the method of FIG. 5, the number m of frame images that need to be stored when generating a reference image using a conventional method, The number m of frame images that need to be stored when generating a reference image using the method, the storage period t of the frame image for generating the reference image when generating the reference image using the conventional method, A frame image storage cycle t for creating a reference image in the case of generating a reference image using the method of the present embodiment is shown in association with each other.

  For example, when shooting a moving image with a frame rate of 120 fps, the number n of frame images required for generating a reference image using the conventional method is “6”, and the reference using the method of this embodiment is used. The number n of frame images necessary for generating an image is “2”, and the number m of frame images that need to be stored when generating a reference image using the conventional method is “6”. Yes, the number m of frame images that need to be stored when the reference image is generated using the method of the present embodiment is “2”, and the reference in the case of generating the reference image using the conventional method The storage period t of the frame image for image creation is “1”, and the storage cycle t of the frame image for creation of the reference image when the reference image is generated using the method of the present embodiment is “3”. It is shown that.

  In addition, when shooting a moving image with a frame rate of 240 fps, the number n of frame images necessary for generating a reference image using the conventional method is “12”. The number n of frame images necessary for generating an image is “2”, and the number m of frame images that need to be stored when generating a reference image using a conventional method is “12”. Yes, the number m of frame images that need to be stored when the reference image is generated using the method of the present embodiment is “2”, and the reference in the case of generating the reference image using the conventional method The storage cycle t of the frame image for image creation is “1”, and the storage cycle t of the frame image for creation of the reference image when the reference image is generated using the method of the present embodiment is “6”. It is shown that. The case of shooting a moving image of another frame rate is as illustrated.

  As shown in the drawing, when shooting a moving image with a high frame rate (for example, a frame rate of 90 fps or more), the number of frame images required in the case of generating a reference image using a conventional method is larger than that of the present embodiment. The number n of frame images necessary for generating a reference image using the method is smaller. Also, when shooting a high frame rate moving image, the reference image is generated using the method of this embodiment rather than the number m of frame images that need to be stored when the reference image is generated using the conventional method. In this case, the number m of frame images that need to be saved is smaller. In addition, when shooting a high frame rate moving image, the reference image is generated using the method of the present embodiment rather than the frame image storage cycle t for generating the reference image when the reference image is generated using the conventional method. The storage period t of the frame image for creating the reference image in the case of generating is larger.

Next, the operation procedure of the image processing apparatus 1 will be described. FIG. 7 is a flowchart showing an operation procedure of the image processing apparatus 1 in the present embodiment.
(Step S201) The user inputs the frame rate of the moving image to be captured and the blinking frequency of the light source to the input unit 11 of the image processing apparatus 1 before shooting the moving image. The input unit 11 inputs the frame rate of the image to be captured, which is input by the user, to the necessary number calculation unit 12 and the imaging unit 13, and inputs the blinking frequency of the light source to the necessary number calculation unit 12. In addition, the control unit 19 initializes the value of the number of imaging T used to determine the timing for storing the frame image for creating the reference image to 0. Thereafter, the process proceeds to step S202.

(Step S202) The required number calculating unit 12 needs to save a frame image that is stored in order to generate a reference image based on the frame rate of the moving image to be captured and the blinking frequency of the light source, which are input in the process of Step S201. The number m of frames, the number n of frame images necessary for generating the reference image, and the storage period t of the frame image for creating the reference image are calculated. The required number calculation unit 12 also calculates the calculated number m of frame images that need to be stored to generate a reference image, the number n of frame images required to generate the reference image, and the reference image creation. The frame image storage period t is input to the reference image generation unit 15 and the control unit 19. Thereafter, the process proceeds to step S203.
(Step S203) When the user starts shooting a moving image, the user presses a shooting start button provided in the input unit 11. When the shooting start button included in the input unit 11 is pressed, the control unit 19 causes the imaging unit 13 to start shooting a moving image. Under the control of the control unit 19, the imaging unit 13 starts shooting a moving image at a frame rate specified by the frame rate of the moving image to be captured, which is input in step S <b> 201. Thereafter, the process proceeds to step S204.

  (Step S <b> 204) When the user finishes shooting a moving image, the user presses a shooting end button provided in the input unit 11. When the shooting end button included in the input unit 11 is not pressed, the control unit 19 determines to continue moving image shooting and proceeds to the process of step S205. In addition, when the shooting end button included in the input unit 11 is pressed, the control unit 19 determines that the moving image shooting is not continued, and causes the imaging unit 13 to end the shooting of the moving image. The imaging unit 13 ends the moving image shooting based on the control of the control unit 19. Thereafter, the process ends.

(Step S205) The imaging unit 13 captures a frame image (correction target image) for one frame. Thereafter, the process proceeds to step S206.
(Step S206) The control unit 13 adds 1 to the value of the number of times T of imaging. Thereafter, the process proceeds to step S207.
(Step S207) The control unit 13 determines whether or not the value of the number of times of imaging T is the same as the value of the storage cycle t of the frame image for creating the reference image. If the control unit 13 determines that the value of the number of times of imaging T and the value of the storage period t of the frame image for creating the reference image are the same value, the process proceeds to step S208. The process proceeds to S212.

(Step S208) The control unit 13 causes the temporary storage unit 14 to store the frame image for one frame captured by the imaging unit 13 in the process of step S205. Thereafter, the process proceeds to step S209.
(Step S209) The reference image generation unit 15 is necessary for generating a reference image out of the number m of frame images stored in the temporary storage unit 14 that need to be stored in order to generate the reference image. The number n of frame images is averaged to generate a reference image. The reference image generation unit 15 stores the generated reference image in the temporary storage unit 14. When the temporary storage unit 14 has already stored the reference image, the reference image generation unit 15 updates the reference image to a newly generated reference image. Thereafter, the process proceeds to step S210. If the number of frame images stored in the temporary storage unit 14 is less than m, the process proceeds to step S211.

(Step S210) The control unit 19 deletes unnecessary frame images from the frame images stored in the temporary storage unit 14. Thereafter, the process proceeds to step S211.
(Step S211) The control unit 13 initializes the value of the number of imaging times T to zero. Thereafter, the process proceeds to step S212.

(Step S212) The gain calculation unit 16 corrects flicker of the correction target image using the correction target image captured by the imaging unit 13 in the process of step S205 and the reference image stored in the temporary storage unit 14. A gain value is calculated. Thereafter, the process proceeds to step S213.
(Step S213) The correction unit 17 uses the gain value calculated by the gain calculation unit 16 in the process of step S212 to correct the flicker of the correction target image captured by the imaging unit 13 in the process of step S205. Thereafter, the process proceeds to step S214.
(Step S214) The control unit 19 stores the correction target image corrected by the correction unit 17 in the process of step S213 in the storage unit 18 as a moving image. Thereafter, the process returns to step S204.

  Next, a specific example of the operation procedure of the image processing apparatus 1 described with reference to FIG. 7 will be described by taking as an example a case where the blinking frequency of the light source is 100 Hz and the frame rate of the moving image to be captured is 240 fps. First, in the process of step S <b> 201, the input unit 11 inputs the frame rate “240 fps” to the required number calculation unit 12 and the imaging unit 13, and inputs the blinking frequency “100 Hz” of the light source to the required number calculation unit 12. In addition, the value of the number of imaging T is initialized to zero. Subsequently, in the process of step S202, the required number calculation unit 12 sets the number m “2” of frame images that need to be stored in order to generate the reference image, and the frame image required to generate the reference image. The number n “2” and the storage period t “6” of the frame image for creating the reference image are calculated.

  Subsequently, in step S <b> 203, the imaging unit 13 starts shooting a moving image with a frame rate of “240 fps”. Subsequently, in the process of step S204, if the control unit 19 determines to continue moving image shooting, the imaging unit 13 captures a frame image (correction target image) in step S205, and step S206. In this process, the control unit 19 adds 1 to the value of the number T of imaging. Subsequently, in the process of step S207, the control unit 19 determines whether or not the value of the number of imaging times T and the value of the storage period t of the frame image for creating the reference image are the same value. If the control unit 13 determines that the value of the number of times of imaging T and the value of the storage period t of the frame image for creating the reference image are the same value, the process proceeds to step S208. The process proceeds to S212.

  In the process of step S208, the control unit 19 causes the temporary storage unit 14 to store the frame image captured by the imaging unit 13 in the process of step S205. Subsequently, in the process of step S <b> 209, an average image of the “two” frame images necessary for generating the reference image, that is, a reference image is generated and stored in the temporary storage unit 14. The frame image “2 sheets” necessary for generating the reference image is the same as the frame image “2 sheets” that needs to be stored to generate the reference image. Subsequently, in the process of step S <b> 210, the stored image captured in the past in the temporary storage unit 14 is deleted from the temporary storage unit 14. Subsequently, the value of the number T of times of imaging is initialized to 0 in the process of step S211.

  In the process of step S212, the reference image stored in the temporary storage unit 14 when the previous process of step S209 is executed is divided by the correction target image, and a gain value for correcting flicker of the correction target image is calculated. Subsequently, in the process of step S213, the correction target image is multiplied by the gain value calculated in the process of step S212 to perform flicker correction. Subsequently, in step S214, the correction target image subjected to the flicker correction is stored in the storage unit 18 as a moving image. Thereafter, the processing from step S204 to step S214 is repeatedly executed until the photographing end button is pressed.

  As described above, in the flicker correction method for the correction target image in the present embodiment, it is not necessary to generate the reference image for each frame, so that the number of frame images necessary for generating the reference image can be reduced. Therefore, the capacity of the storage unit that stores the frame image can be reduced, and the processing time for creating the reference image or the circuit resources in the hardware processing can be further reduced. Further, in the apparatus that performs flicker correction using the flicker correction method of the present embodiment, it is possible to reduce the number of second generations for creating a reference image, so that power consumption can be further reduced.

  In the flicker correction method described above, a reference image that is temporally separated from the correction target image is used for flicker correction of the correction target image. For example, when the subject is moving, There is a concern that a difference may occur between the corrected image and an appropriate flicker correction gain cannot be obtained. However, in the imaging system at a high frame rate assumed in the present embodiment, even if a reference image is generated for each of a plurality of frames, the reference image update cycle is when a moving image with a frame rate of 60 fps is captured. The interval is almost the same as the imaging interval of one frame. For this reason, it is considered that generating a reference image for each of a plurality of frames has no significant adverse effect.

  FIG. 8 shows the case where the number n of frame images necessary for generating the reference image and the reference image are generated when moving images having various frame rates are taken when the light / dark frequency of the light source is 100 Hz. FIG. 6 is a diagram summarizing the relationship among the number m of frame images that need to be stored in order to save, the storage period t of frame images for creating reference images, and the update interval of reference images.

  In the example shown in the figure, they are summarized in a table format. For each column, the frame rate of a moving image to be shot, the number n of frame images necessary for generating a reference image using a conventional method, and the present embodiment The number n of frame images required when generating a reference image using the method of FIG. 5, the number m of frame images that need to be stored when generating a reference image using a conventional method, The number m of frame images that need to be stored when generating a reference image using the method, the storage period t of the frame image for generating the reference image when generating the reference image using the conventional method, The reference image creation frame image storage cycle t and the reference image update interval in the case of generating a reference image using the method of this embodiment are shown in association with each other.

  For example, when shooting a moving image with a frame rate of 120 fps, the number n of frame images required for generating a reference image using the conventional method is “6”, and the reference using the method of this embodiment is used. The number n of frame images necessary for generating an image is “2”, and the number m of frame images that need to be stored when generating a reference image using the conventional method is “6”. Yes, the number m of frame images that need to be stored when the reference image is generated using the method of the present embodiment is “2”, and the reference in the case of generating the reference image using the conventional method The storage period t of the frame image for image creation is “1”, and the storage cycle t of the frame image for reference image creation when the reference image is generated using the method of the present embodiment is “3”. The reference image update interval is 2 It shows that it is a ms. The case of shooting a moving image of another frame rate is as illustrated.

  As shown in the figure, when a moving image with a high frame rate (for example, a frame rate of 90 fps to 480 fps) is shot, the frame image storage cycle t (reference image creation cycle) for creating a reference image is increased. The update interval of the reference image is 25 ms to 33 ms. The update interval of the reference image when shooting a moving image with a frame rate of 15 fps to 60 fps is 17 ms to 67 ms. From this, it can be seen that, when shooting a high frame rate moving image, even if flicker correction is performed by a method of generating a reference image for each of a plurality of frames, the update interval of the reference image does not change significantly.

  In this embodiment, the frame image storage cycle t (reference image update cycle) for creating the reference image is automatically obtained based on the frame rate of the moving image to be shot and the blinking frequency of the light source. However, the storage period t of the frame image for creating the reference image can be set to a longer period. For example, when the movement of the subject is slow, the storage period t of the frame image for creating the reference image can be further increased. In order to further increase the storage period t of the frame image for creating the reference image, the storage period t of the frame image for creating the reference image may be set to a value that is an integral multiple of the value obtained by the above-described calculation. As a result, the processing time and power consumption can be further reduced.

  Although the first embodiment and the second embodiment of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment and does not depart from the gist of the present invention. Range design etc. are also included.

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 11 ... Input part, 12 ... Required number calculation part, 13 ... Imaging part, 14 ... Temporary storage part, 15 ... Reference | standard image generation part, 16. ..Gain calculation unit, 17 ... correction unit, 18 ... storage unit, 19 ... control unit

Claims (4)

The calculation unit calculates the least common multiple of 1 / a and 1 / b from the frame rate a [fps] of the moving image to be captured and the blinking frequency b [Hz] of the light source, and calculates the denominator value of the calculated least common multiple. a calculation step of calculating a minimum divisor n excluding 1 included in the numerator value when converted to a;
Reference image generating unit, a reference image generating step of generating a reference image n frame images of the captured cycle the least common multiple and the blink frequency b times at a timing n equal parts by averaging,
A correction unit that performs flicker correction of the captured moving image based on the reference image;
A flicker correction method comprising: 2. The reference image generation step, wherein the reference image is updated when a frame image having the same brightness as the most recently captured frame image is obtained from the n frame images. The flicker correction method described in 1. The flicker correction method according to claim 2, wherein in the reference image generation step, the reference image is updated when a predetermined time or more has elapsed since the previous update. An imaging unit that captures a moving image with a frame rate a [fps] under a light source having a blinking frequency b [Hz];
The least common multiple of 1 / a and 1 / b is calculated from the frame rate a [fps] and the blinking frequency b [Hz], and the denominator value of the calculated least common multiple is converted to a. A calculation unit for calculating a minimum divisor n excluding 1 included in the value;
A reference image generation unit that generates a reference image n frame images of the captured cycle the least common multiple and the blink frequency b times at a timing n equal parts by averaging,
A correction unit that performs flicker correction of the moving image captured by the imaging unit based on the reference image generated by the reference image generation unit;
An image processing apparatus comprising:

Images