JP7730693B2 - Imaging device, flicker detection method and program - Google Patents

Description

The present invention relates to an imaging device, a flicker detection method, and a program, and in particular to technology for calculating characteristics related to periodic changes in the amount of light (called flicker) from a subject.

In recent years, the sensitivity of image sensors installed in imaging devices such as digital cameras and mobile phones has been increasing. As a result, it is now possible to capture bright images with reduced subject blur by setting a high shutter speed (shortening the exposure time) to capture a subject in a relatively dark environment, such as indoors, compared to outdoors during the day.

Furthermore, fluorescent lights, which are commonly used as indoor light sources, are known to cause flicker, a phenomenon in which the amount of light in a subject image changes periodically, due to the influence of commercial power supply frequencies. If a subject is captured using a high shutter speed under a light source that causes such flicker, uneven exposure and color may occur within a single image (screen), or there may be variations in exposure and color temperature between multiple images captured in succession.

Patent Document 1 proposes a technology for detecting flicker based on multiple images continuously captured at a rate that is the least common multiple of the flicker frequencies (100 Hz and 120 Hz) caused by two commercial power supply frequencies, 50 Hz and 60 Hz.

JP 2014-220763 A

In recent years, the use of light-emitting diodes (LEDs) as light sources has been increasing. LEDs use a different method of supplying current than fluorescent lamps, controlling the drive current with a rectifier circuit. As a result, the amount of light changes at a different cycle and waveform than the commercial power frequency. Therefore, flicker occurs under an LED light source, just like under a fluorescent lamp, but the frequency of the flicker change in light amount is different from that under light sources such as fluorescent lamps.

Patent Document 1 discloses a method for detecting flicker at frequencies (100 Hz and 120 Hz) caused by two commercial power supply frequencies, 50 Hz and 60 Hz, such as under light sources such as fluorescent lamps, but does not mention how to deal with flicker that occurs under LED light sources.

Therefore, if it were impossible to detect flicker, which occurs when the amount of light changes at a frequency different from the commercial power frequency, then capturing images with reduced flicker effects would require the user to perform cumbersome operations such as manually adjusting the shutter speed (exposure time).

The object of the present invention is to reduce the complicated operations required of users to capture images with reduced flicker effects by effectively detecting flicker with simple operations regardless of the light source.

In order to achieve the above object, the imaging device of the present invention comprises an imaging element, a drive control means for controlling the drive of the imaging element, and a flicker detection means for detecting flicker based on a signal output from the imaging element , wherein the drive control means controls the imaging element to be driven at n different frame rates, where n is a natural number greater than or equal to 3, for the purpose of detecting flicker, the least common multiple of the n frame rates used for detecting flicker does not include any of the n frame rates, and the flicker detection means detects flicker based on a flicker detection signal obtained for each of the n frame rates.

The present invention allows for effective detection of flicker with simple operations, regardless of the light source, thereby reducing the number of complicated operations required by the user to capture images with reduced effects of flicker.

1 is a block diagram illustrating the configuration of a camera body 100, a lens unit 200, and a light emitting device 300, which are an embodiment of an imaging device embodying the present invention. FIG. 10 is a diagram illustrating an example of a shutter speed setting (index) table according to the present invention. 4 is a flowchart showing a flicker reduction process according to the first embodiment of the present invention. 4 is a flowchart illustrating a flicker detection process according to the first embodiment of the present invention. 5A to 5C are diagrams illustrating an example of a method for selecting a plurality of imaging periods when detecting flicker according to the first embodiment of the present invention. 10A and 10B are diagrams illustrating an example of a modification of the method for selecting a plurality of imaging periods when detecting flicker according to the first embodiment of the present invention. 10 is a diagram (graph) illustrating an example of the relationship between a method for determining each imaging period for flicker detection and the number of imaging periods according to the present embodiment; 10A and 10B are diagrams illustrating an example of luminance changes based on images continuously obtained by a global shutter method. 10A and 10B are diagrams illustrating an example of luminance changes based on images continuously obtained by a rolling shutter method. 10A and 10B are diagrams illustrating setting values of exposure time (shutter speed) in a first pattern of a plurality of imaging periods for flicker detection according to the first embodiment of the present invention. FIG. 10 is a diagram illustrating setting values of exposure time (shutter speed) in a second pattern of a plurality of imaging periods for flicker detection according to the first embodiment of the present invention. 4 is a flowchart illustrating a process for determining a flicker reduction exposure time according to the first embodiment of the present invention. 10A and 10B are diagrams illustrating an example of a method for setting an ideal flicker-reducing exposure time when flicker that changes at a predetermined light intensity change frequency occurs according to the present invention. 4 is a flowchart illustrating a shutter speed selection process according to the first embodiment of the present invention. 10A and 10B are diagrams illustrating an example of the relative relationship between a shutter speed selected by the shutter speed selection process according to the first embodiment of the present invention and an ideal shutter speed for reducing the effect of flicker. 10A to 10C are diagrams illustrating exemplary notification images displayed on a display unit 102 by the display processing according to the first embodiment of the present invention. 10A and 10B are diagrams illustrating exemplary notification images when no flicker is detected by the display process according to the first embodiment of the present invention. 10A and 10B are diagrams illustrating exemplary notification images displayed on the display unit 102 by the display processing according to the second embodiment of the present invention. 13A and 13B are diagrams illustrating an example of a transition screen for flicker reduction processing during live view display according to the third embodiment of the present invention. 10A and 10B are diagrams illustrating an example of a method for sampling captured images based on images continuously obtained by a rolling shutter method. 10A and 10B are diagrams illustrating an example of a change in flicker level when a plurality of sampling periods are mixed. 6 is a diagram illustrating an example of a case where the imaging period of the captured image used for detecting flicker described with reference to FIG. 5 is shifted to a higher speed. FIG. 7A and 7B are diagrams illustrating an example of a case where the imaging period of the captured image used for detecting flicker described with reference to FIG. 6 is shifted to a higher speed.

(First embodiment)
(Basic configuration of the imaging device)
Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating the configuration of a camera body 100, a lens unit 200, and a light emitting device 300, which are embodiments of an imaging device embodying the present invention. Note that one or more of the functional blocks shown in FIG. 1 may be implemented by hardware such as an ASIC or a programmable logic array (PLA). Alternatively, they may be implemented by a programmable processor (microprocessor, microcomputer) such as a CPU or MPU executing software. Alternatively, they may be implemented by a combination of software and hardware.

Therefore, even if different functional blocks are described as the main operating entities in the following description, they may be realized as the same hardware.

First, we will explain the various components that make up the camera body 100. The camera body 100 is equipped with a frame memory (not shown), which functions as a storage unit that temporarily stores signals (video signals) and can read them out when needed. Frame memory is generally also called RAM, and in recent years, DDR3-SDRAM (DUAL DATA RATE 3 - SYNCHRONOUS DYNAMIC RAM) and the like are often used. Using this frame memory enables a variety of processing functions.

The image sensor 101 is an imaging unit that uses a charge-storage solid-state image sensor, such as a CMOS or CCD, to receive the light beam from the subject guided into the camera body 100 via the lens unit 200 and convert it into an electrical image signal. The image (signal) obtained using the image sensor 101 under drive control by the CPU 103 (described below) is handled as various image signals, such as for live view display, flicker detection, and captured images for recording. Since the electrical signal obtained by the image sensor 101 is an analog value, it also has the function of converting it into a digital value. An evaluation value (photometric value) related to the brightness of the subject can be detected based on the image signal output from the image sensor 101. The exposure time of the image sensor 101 can also be controlled according to a shutter speed that can be set as an exposure control value for the image sensor 101.

The mechanical shutter 104 is a light-oblique device that can travel in a direction parallel to the signal scanning direction of the image sensor 101. The exposure time of the image sensor 101 can be controlled by adjusting the exposure aperture formed by the multiple shutter blades of the mechanical shutter according to the shutter speed described above. The exposure time adjustment of the present invention can be achieved by utilizing or combining the mechanical shutter 104 with a so-called electronic shutter that adjusts the signal reset and readout timing of the image sensor 101.

The display unit 102 is a display device that can be viewed by the user, and allows the user to check the operating status of the camera body 100. For example, the display unit 102 displays images that have been image-processed based on the image signal of the subject, settings menus, etc. However, an LCD (Liquid Crystal Display) or organic EL (Organic Electroluminescence) may also be used as the display unit 102. By displaying the image acquired by the image sensor 101 and setting conditions such as exposure control values in real time on the display unit 102 while the subject is being captured, so-called live view display is possible. Note that the display unit 102 in this embodiment is equipped with a resistive or capacitive thin-film element known as a touch panel, and doubles as an operation unit that the user can touch.

The CPU 103 is a control means capable of comprehensively controlling each part of the camera body 100 and the accessories attached to the camera body 100. A ROM (read only memory) and a RAM (random access memory) are connected to the CPU 103. The ROM (not shown) is a non-volatile storage element that stores programs for operating the CPU 103, various adjustment parameters, and the like. Programs read from the ROM are expanded into volatile RAM (not shown) and executed. Generally, RAM is a slower, lower-capacity element than frame memory (not shown).

Next, the lens unit 200 will be described in detail. The lens unit 200 is an accessory that can be attached to and detached from the camera body 100, and is a so-called interchangeable lens that includes a lens group 201 such as a focus lens, zoom lens, and shift lens. For example, the focusing lens included in the lens group 201 can adjust the focus on a subject by adjusting the lens position in the direction of the lens's optical axis.

The diaphragm 202 is a light amount adjustment component for adjusting the amount of light associated with the subject's light flux guided into the camera body 100 via the lens unit 200. In this embodiment, light amount adjustment is possible by adjusting the aperture diameter of the diaphragm 202, which is achieved by changing the aperture value as an exposure control value associated with the diaphragm's aperture diameter.

The LPU 203 is a control unit that controls each part of the lens unit 200, and can control the driving of, for example, the lens group 201 and the aperture 202. The LPU 203 is connected to the CPU 103 of the camera body 100 via a group of terminals (not shown), and can drive each part of the lens unit 200 in response to control instructions from the CPU 103.

Next, the light-emitting device 300 will be described in detail. The light-emitting device 300 is an external light-emitting device that can be attached and detached via a connector (not shown) provided on the camera body 100. The SPU 301 is a control unit that controls each part of the light-emitting device 300, and is primarily responsible for controlling light emission and communication with the camera body 100. The SPU 301 is connected to the CPU 103 of the camera body 100 via a group of contacts (not shown), and can drive each part of the light-emitting device 300 in response to control instructions from the CPU 103.

The above describes each part of the imaging device according to the first embodiment of the present invention, but the present invention is not limited to the above-mentioned configuration. For example, the camera body 100 may be configured to incorporate devices equivalent to the lens unit 200 and light-emitting device 300.

(How to set the shutter speed)
Next, a method for setting the shutter speed, which is an exposure control value for controlling the exposure time of the image sensor 101 according to this embodiment, will be described in detail with reference to Fig. 2. Fig. 2 is a diagram for explaining an example of a shutter speed setting (index) table according to the present invention.

It is generally known that the shutter speed can be changed in 1/2 or 1/3 stops of light intensity, but in this embodiment, the shutter speed can be adjusted in finer steps to address flicker that occurs under LED light sources that periodically blink at various frequencies. Specifically, in this embodiment, the shutter speed can be adjusted in 1/4 stop increments for shutter speeds between 1/8192.0 and 1/4871.0, and in 1/8 stop increments for shutter speeds between 1/4096.0 and 1/2233.4. Furthermore, the shutter speed can be set in 1/16 stop increments for shutter speeds between 1/2048.0 and 1/1069.3, and in 1/32 stop increments for shutter speeds between 1/1024.0 and 1/523.2. Furthermore, the shutter speed can be adjusted in 1/64 increments from 1/512.0 to 1/258.8, in 1/128 increments from 1/256.0 to 1/128.7, and in 1/256 increments from 1/128.0 to 1/50.0.

Note that in the table shown in Figure 2, some shutter speeds have been omitted to improve visibility. The index values in the table shown in Figure 2 are used in the shutter speed selection process for reducing flicker, which will be described later.

Furthermore, the camera body 100 according to this embodiment prioritizes the use of the electronic shutter, allowing for the shutter speed to be freely set from a high shutter speed of less than 1/8000 seconds as described above to a slower shutter speed of even longer than 1/50 seconds (not shown). The shutter method (use of the electronic shutter and mechanical shutter 104 alone or in combination) can be changed at any time by the user, for example, manually via a menu screen displayed on the display unit 102.

(Flicker reduction processing)
Next, the flicker reduction process according to this embodiment will be described with reference to the flowchart shown in Fig. 3. Fig. 3 is a flowchart showing the flicker reduction process according to the first embodiment of the present invention.

First, flicker reduction processing is initiated in response to a predetermined operation, such as a manual operation by the user based on a menu displayed on the display unit 102 or the like. Note that the flicker reduction processing according to this embodiment is a process that controls the shutter speed (i.e., exposure time) to reduce the effects of detected flicker, thereby preventing unevenness caused by flicker from occurring in moving images such as live view displays. The flicker reduction processing according to the present invention is not limited to this; for example, a method of reducing flicker may involve applying a gain to the image to reduce unevenness, in addition to adjusting the shutter speed.

When flicker reduction processing begins, first in step S301, the CPU 103 repeats the processing of S301 until flicker detection processing (detection processing) begins. If it is determined in step S301 that detection has begun, the CPU 103 executes flicker detection processing in step S302. Details of the flicker detection processing will be described later.

Next, in step S303, the CPU 103 determines whether flicker has occurred based on the processing result of step S302. If it is determined in step S303 that flicker has been detected, the process proceeds to step S304; if it is determined that flicker has not been detected, the process proceeds to step S306. Note that, as a result of flicker detection, flicker is considered to have been detected if flicker of a predetermined level or above has occurred. The method for calculating the flicker level will be described later.

In step S304, the CPU 103 determines an exposure time (shutter speed) that will reduce the effects of the previously detected flicker (flicker reduction exposure time determination process). Details of the flicker reduction exposure time determination process will be described later.

Next, in step S305, the CPU 103 executes shutter speed selection processing to select an arbitrary shutter speed that can reduce the effects of flicker, based on the information regarding the exposure time suitable for reducing flicker determined in step S304. Details of this shutter speed selection processing will be described later.

In step S306, the CPU 103 executes display processing to display the results of the processing in steps S304 and S305, such as the flicker detection result (detection or non-detection) and selectable values for shutter speeds that can reduce the effects of flicker. Details of this display processing will be described later. The flicker reduction processing described above makes it possible to obtain an image in which the effects of flicker are reduced, regardless of the flicker frequency, and to display or record the image based on this image.

(Flicker detection processing)
Next, the flicker detection process (flicker detection process) according to this embodiment will be described with reference to Fig. 4. As described above, unlike light sources such as fluorescent lamps, LED light sources and the like use a rectifier circuit to control the drive current, and therefore, changes in the light intensity (blinking), i.e., flicker, occur at a cycle different from the power supply frequency used to drive the light source. Therefore, when detecting flicker caused by light sources such as LEDs, it is not possible to narrow down the frequency to be detected to a specific value, as is the case with the drive power supply frequency, and therefore it is necessary to analyze whether or not flicker occurs over a wide range of frequencies.

On the other hand, if the light intensity change frequency of the flicker (the blinking period of the light source) and the imaging period when continuously capturing images of a subject match or are an integer multiple (hereinafter, this state will be referred to as synchronization), changes in light intensity (blinking) between consecutively captured images will be suppressed. In this case, for example, in a live view display in which images are displayed consecutively, image quality degradation such as unevenness caused by flicker will not occur. However, still images captured by capturing images at an arbitrary shutter speed may experience exposure unevenness caused by flicker. Note that even if the imaging frame rate for live view display images matches the light intensity change frequency of the flicker, if a video for recording is captured at a different frame rate, the video may experience exposure unevenness and brightness fluctuations caused by flicker.

A known method for identifying the frequency of changes in light intensity due to flicker is to detect and compare the difference in light intensity (brightness/darkness) between images obtained by continuous imaging. Therefore, when using this method to identify the frequency of changes in light intensity due to flicker, it is necessary to adjust the frequency of changes in light intensity due to flicker and the imaging cycle (frame rate) so that they are not synchronized.

In this embodiment, therefore, the occurrence of flicker is detected by analyzing the frequency of change in light intensity of the flicker at multiple imaging cycles. This method, by analyzing the frequency of change in light intensity of the flicker at multiple frequencies, makes it possible to avoid synchronization between the frequency of change in light intensity of the flicker and the imaging cycle, enabling effective detection processing for flicker over a wide range of frequencies.

Figure 4 is a flowchart of flicker detection processing according to the first embodiment of the present invention. As shown in Figure 4, in step S401, the CPU 103 performs a photometric calculation of the subject (subject photometry) to determine the exposure when capturing an image of the subject for flicker detection processing. Any method may be used for the photometric calculation. For example, in this embodiment, an evaluation value is obtained based on the average value of the image signal obtained by accumulating charge for photometric calculation using the image sensor 101. The CPU 103 then calculates the representative luminance (photometric value) of the subject as the photometric result based on the obtained evaluation value. In this case, the photometric value is calculated by dividing the angle of view corresponding to the image signal into multiple blocks, calculating the average value of the signals output from the corresponding pixels for each block, and averaging the average values calculated for each block to calculate the photometric value (representative luminance). The unit of photometric value is the so-called APEX (Additive System of Photographic Exposure) system, where 1 BV is one step of brightness value, but other units may also be used.

Next, in step S402, the CPU 103 adjusts the imaging cycle to an imaging cycle (non-frame rate) for flicker detection. Details of how to adjust the imaging cycle for flicker detection will be described later.

Next, in step S403, the CPU 103 determines an exposure control value (changes the exposure) based on the photometric value determined earlier. The exposure control values in this embodiment are the shutter speed (i.e., accumulation time), aperture value, and shooting sensitivity (ISO sensitivity), and are parameters that can adjust the brightness of the image obtained by capturing an image of the subject. The determined exposure control value is stored in the RAM mentioned above, and the exposure of the camera body 100 is changed, and acquisition of an image for flicker detection begins.

Next, in step S404, the CPU 103 determines whether there is a change in luminance in the acquired image (i.e., whether flicker is occurring). As mentioned above, if the blinking cycle of the light source and the imaging cycle of the subject are synchronized, flicker cannot be detected correctly, so the presence or absence of a change in luminance is determined based on the acquired image. If it is determined that there is no change in luminance in the acquired image, it is determined that the imaging cycle and the light intensity change frequency of the flicker related to the subject are synchronized, or that flicker is not occurring, and the detection operation at the current frame rate (imaging cycle) is skipped.

If it is determined that a change in brightness has been detected in the acquired image (YES in step S404), in step S405 the CPU 103 analyzes (detects) the presence or absence of flicker at multiple different frequencies. The method of detecting flicker at multiple frequencies in step S405 will be described in detail below.

Next, in step S406, the CPU 103 determines whether detection has been completed for a predetermined number (n) of imaging cycles. If it is determined that detection has not been completed for the predetermined number of imaging cycles (NO in step S406), the process returns to step S402, the imaging cycle (frame rate) is changed, and the process from step S403 onwards is repeated.

If it is determined that detection has been completed for a predetermined number of imaging cycles (YES in step S406), in step S407 the CPU 103 identifies the frequency of the subject's flicker based on the detection results up to step S405. In the processing of step S407, the presence or absence of flicker of multiple different frequencies has been detected for multiple imaging cycles (frame rates).

Therefore, the level of flicker detected for each frequency is compared, and the flicker with the highest level is the final detection result as the currently occurring flicker of the subject. In this embodiment, the level of flicker is compared using the magnitude of the change in light quantity (the magnitude of the amplitude in the curve showing the regular change in light quantity), but this is not limited to this. For example, a configuration may be adopted in which the stability of the change in light quantity, in addition to the level of flicker, is compared.

Here, we will explain in detail the imaging cycle (frame rate) for flicker detection described above. As previously described, the camera body 100 according to this embodiment performs flicker detection processing at multiple imaging cycles. For example, consider the case where the imaging cycle is switched between 100 fps and 120 fps to detect the flicker light intensity change frequency. In this case, for flicker whose light intensity changes at a cycle of k (k is a natural number) × 100 Hz, such as 100 Hz, 200 Hz, or 300 Hz, which are integer multiples of the 100 fps imaging cycle, the imaging cycle and the flicker light intensity change frequency are synchronized, making it impossible to detect the flicker. On the other hand, consider the case where the flicker light intensity change frequency is detected at a 120 fps imaging cycle. In this case, for flicker whose light intensity changes at a cycle of m (m is a natural number) × 120 Hz, such as 120 Hz, 240 Hz, or 360 Hz, which are integer multiples of the 120 fps imaging cycle, the imaging cycle and the flicker light intensity change frequency are synchronized, making it impossible to detect the flicker. Note that 600 Hz and 1200 Hz, which satisfy the conditions of both k x 100 Hz (k is a natural number) and m x 120 Hz (m is a natural number), are the least common multiples of 100 Hz and 120 Hz. When flicker occurs, where the light intensity changes at such frequencies, the frequency of the flicker's light intensity change will be synchronized with both the 100 fps and 120 fps imaging cycles, and the flicker cannot be detected correctly using images obtained at either imaging cycle.

For example, in light sources with rectifier circuits, such as LED light sources, the adjusted power supply frequency generally falls within the range of 50 Hz to 1000 Hz. Therefore, flicker with the aforementioned 600 Hz light intensity change frequency can occur with LED light sources, but depending on the imaging cycle, the flicker may not be detected correctly. In other words, even if flicker detection is performed using images acquired at two imaging cycles, there are frequencies at which the flicker cannot be detected correctly among the wide range of flicker frequencies that are expected to occur with LED light sources, etc.

In the above example, we have described flicker that changes at a frequency that is exactly the same as an integer multiple of the imaging cycle (frame rate), but even if the frequency does not match the integer multiple of the imaging cycle, the accuracy of flicker detection may decrease. For example, for flicker that changes at a frequency close to an integer multiple of the imaging cycle when acquiring images for flicker detection, the impact of exposure unevenness that occurs in the image is small, and flicker detection may take a long time or may not be possible.

In this embodiment, therefore, in order to effectively detect flicker over a wide range of frequencies that can occur under an LED light source, the number n of multiple imaging cycles (frame rates) used during flicker detection is adjusted to satisfy the condition "n ≥ 3 (n) is a natural number." In other words, flicker detection is performed over n or more imaging cycles, where n is a natural number greater than or equal to 3.

Note that the higher the frequency of changes in light intensity of the flicker being detected, the more accurately the frequency of changes in light intensity of the flicker can be detected by increasing the number n of imaging cycles used for detection. However, increasing the number of imaging cycles used for flicker detection may extend the period required for flicker detection, and it is also necessary to consider the possibility of a release time lag or a decrease in the display frame rate of the live view image. Therefore, in this embodiment, the number of imaging cycles used for flicker detection is set to n = 3, as this is the sampling number that can effectively detect flicker that may occur in light sources that are generally expected to be used in many cases, such as LED light sources.

Next, we will explain how to select specific values for each of the n imaging periods.

In this embodiment, a reference imaging cycle is first set. For example, assume that the reference imaging cycle is 100 fps. The light intensity change frequency of flicker synchronized with an imaging cycle of 100 fps is an integer multiple of 100 Hz, and if flicker occurs at this light intensity change frequency, the flicker cannot be detected correctly.

Note that even when sampling at an imaging cycle of 200 fps, which is twice the standard imaging cycle of 100 fps, the same problem occurs as when sampling at a standard imaging cycle of 100 fps. That is, if an integer multiple of the imaging cycle for obtaining images for flicker detection matches an integer multiple of the frequency of change in light intensity of the flicker, the imaging cycle and the frequency of change in light intensity will be synchronized, making it impossible to correctly detect flicker based on the images obtained for sampling.

In this embodiment, therefore, the remaining n-1 (2 in this embodiment) imaging periods are set as n imaging periods (n = 3 in this embodiment) between the reference imaging period and the next imaging period that is an integer multiple of the reference imaging period. For example, if there are three imaging periods and the reference imaging period is 100 fps, the remaining imaging periods between 100 fps and 200 fps are set as multiple imaging periods for detecting flicker, in addition to 100 fps. Note that in this embodiment, each imaging period (frequency) is set so that the least common multiple of the n imaging periods is equal to or greater than a predetermined frequency. For example, since the blinking frequency of an LED light source is generally 10,000 Hz or less, the frequency of each imaging period is determined so that the least common multiple of the n imaging periods (frame rates) is equal to or greater than 10,000. Furthermore, to reduce the effects of flicker in the camera body 100, each imaging period (frequency) is set so that the least common multiple of the n imaging periods is greater than the reciprocal of the upper limit of the highest shutter speed that can be set by the camera body 100. This configuration makes it possible to effectively detect flicker that occurs in light sources whose light intensity changes at high frequencies (e.g., 200 Hz or higher), such as LED light sources, while also reducing the effects of the detected flicker by adjusting the shutter speed.

Figure 5 is a diagram illustrating an example of a method for selecting multiple imaging periods when detecting flicker according to the first embodiment of the present invention. In order to accurately detect the light intensity change frequency of flicker, the imaging periods are spaced as far apart as possible to ensure a sufficient difference between one of the multiple imaging periods and the light intensity change frequency (blinking period of the light source) of the flicker to be detected, allowing for good flicker detection. Therefore, in this embodiment, as shown in Figure 5(a), the range of imaging periods to be detected (100 fps to 200 fps) is divided at predetermined intervals, and flicker is detected by setting imaging periods spaced apart by 1/3 of the imaging period 2.

Specifically, in this embodiment, as shown in Figure 5(a), the three imaging periods are: a reference imaging period of 100 fps, 100 fps x 2^(1/3) = 125.99 fps ≒ 126 fps, and 100 fps x 2^(2/3) = 158.74 fps ≒ 159 fps. These three imaging periods differ by a factor of 2^(1/3) = 1.2599 ≒ 1.26, resulting in a difference of approximately 26% between the imaging periods. This configuration ensures that even when flicker detection is performed by dividing the wide frequency range from 50 to over 1000 Hz into multiple ranges, there is no significant deviation between the frequencies targeted for detection within each range. Furthermore, it is possible to ensure that one of the imaging periods is sufficiently different from the light intensity change frequency of the flicker being detected. In other words, if n imaging periods are set and flicker detection is performed at each imaging period, by setting each imaging period to a multiple of 2 (1/n), it is possible to suppress a decrease in detection accuracy for each frequency to be detected.

Figure 5(b) is a diagram illustrating an example of the correspondence between n imaging cycles and the light intensity change frequency of the flicker to be detected. In this embodiment, flicker is detected based on the image obtained by the imaging cycle that is the furthest away from the light intensity change frequency of the flicker to be detected among the n imaging cycles. Specifically, in this embodiment, as shown in Figure 5(b), flicker detection is performed based on a data table that divides the light intensity change frequencies of flicker from 50 Hz to 1008 Hz into ranges (A) to (P) for the three imaging cycles shown in Figure 5(a).

In this embodiment, the effects of flicker are reduced by capturing an image of a subject at a shutter speed that is the reciprocal of the flicker's light intensity change frequency, thereby setting an imaging period synchronized with the flicker's light intensity change frequency. Therefore, when there is a discrepancy between the ideal shutter speed synchronized with the flicker's light intensity change frequency and the actual shutter speed, a slower shutter speed will have a greater effect on the image (such as uneven exposure) than a faster shutter speed. For example, consider the case where shutter speeds of 1/101 second and 1/1001 second are set for flicker with a light intensity change frequency of 100 Hz and 1000 Hz, respectively, which are 1 Hz off the ideal shutter speed for reducing the effects of flicker. In both cases, there is a 1 Hz discrepancy between the shutter speed capable of reducing the effects of flicker and the actual shutter speed. However, while the discrepancy is 1% at a shutter speed of 1/100 second, it is only 0.1% at a shutter speed of 1/1000 second. In other words, the faster shutter speed results in less flicker in the image for a 1 Hz change in shutter speed. However, as the shutter speed increases, the period over which changes in light intensity due to flicker are captured becomes longer, increasing the likelihood of obtaining an image in which changes in light intensity are smoothed. Therefore, when detecting flicker with a light intensity change frequency that is reduced at shutter speeds above a predetermined value (for example, 1/25 seconds or longer), the detection range in the low-frequency region of flicker can be adjusted and widened as appropriate.

In this embodiment, as shown in Figure 5(b), the range of light intensity change frequencies of the flicker to be detected is divided into multiple ranges, and the detection ranges are set so that the frequencies in each successive range differ by 2^(1/3) = 1.26 times. For example, while range (N) shown in Figure 5(b) is 159 to 200 Hz, the next range (C) detects flicker from 200 to 252 Hz, which is approximately 1.26 times the range (N).

As shown in Figure 5(b), the range of flicker light intensity change frequencies that can be detected using the same imaging cycle differs by approximately two times between successive ranges. For example, the detection frequencies for ranges (A), (B), and (c) shown in Figure 5(b), which correspond to an imaging cycle of 159 fps, range from 50 Hz, 100 Hz, and 200 Hz to 63 Hz, 126 Hz, and 252 Hz, respectively. This is because the light intensity change due to flicker is the same at integer multiples of each frequency. With the above-described configuration, the imaging device according to this embodiment can detect flicker over a wide range of frequencies with stable accuracy.

In this embodiment, the imaging cycles used to detect flicker differ by a factor of m (1/n) (m and n are natural numbers). While the above description assumes m = 2, this is not limiting. For example, the imaging cycle may be set to m = 3. In this case, the difference between each imaging cycle becomes greater, and therefore the detection accuracy for the light intensity change frequency of the flicker to be detected may be lower than when m = 2. However, when the same frequency range is to be detected, m = 3 requires less time for detection than m = 2, making it more suitable for detecting a wider range of light intensity change frequencies of flicker.

Here, a method (variant) of selecting n imaging periods that is different from the method described above will be described with reference to FIG. 6. FIG. 6 is a diagram illustrating an example of a variant of the method of selecting multiple imaging periods when detecting flicker according to the first embodiment of the present invention. The difference between this variant and the example described above with reference to FIG. 5 is the method of setting n imaging periods for the range of imaging periods to be detected.

In this modified example, as shown in Figure 6(a), multiple imaging periods are set by equally dividing the range of imaging periods to be detected. That is, if the range of imaging periods for flicker detection (100 fps to 200 fps) is set to 100% range, n imaging periods are set to imaging periods that differ by 33% and 66%, respectively, from the reference imaging period of 100 fps. Specifically, the three imaging periods are set to the reference imaging period of 100 fps, 100 fps x 1.333 = 133.333... fps ≒ 133 fps, and 100 fps x 1.666 = 166.666 fps ≒ 167 fps.

The differences between the three imaging periods mentioned above are 133.333/100 = 1.33333, 166.666/133.33 = 1.25, and 200/166.666 = 1.2, which means that the imaging periods differ by more than 20%.

Figure 6(b) is a diagram illustrating an example of the correspondence between the n imaging cycles shown in Figure 6(a) and the light intensity change frequency of the flicker to be detected. As shown in Figure 6(b), in this modified example, as in Figure 5(b) described above, flicker is detected based on the image obtained by the imaging cycle that is the furthest away from the light intensity change frequency of the flicker to be detected among the n imaging cycles.

Here, we will explain the differences between the multiple imaging periods used for flicker detection. As mentioned above, as the number of multiple imaging periods used for flicker detection increases, the differences between each imaging period become smaller, but the time required for sampling increases. Therefore, in order to accurately detect flicker in a short period of time, it is preferable that the differences between each imaging period be as large as possible, and that the number of imaging periods used for sampling be as small as possible, within the range that allows detection of a wide range of flicker light intensity change frequencies.

5, the period from the reference imaging period to a period twice the reference imaging period is set to 100%, and the period is divided into increments of 1/n of 2. In this case, the multiple imaging periods for flicker detection differ at intervals shown in the following formula (1).
{2^(1/n)-1}×100[%] (Formula 1)

Also, as explained with reference to Figure 6, consider the case where the interval between the reference imaging cycle and a cycle twice the reference imaging cycle is set to 100%, and the interval therebetween is changed by 100/n [%].

As calculated when n = 3, the difference is smallest between an imaging cycle that is 100% x (n-1)/n away from the reference imaging cycle and an imaging cycle that is twice the reference imaging cycle, and the difference is [200/{100 + {(100 x (n-1)/n}-1] x 100 [%] = {200n/(200n-100)-1} [%] = {2n/(2n-1)-1} x 100 [%] = {1/(2n-1)} x 100 [%]] This is calculated using Equation 2. In other words, when each imaging period differs by 100/n%, the multiple imaging periods (frame rates) used to detect flicker differ from each other by at least a ratio of [{2n/(2n-1)-1}] x 100% or more. Note that in the camera body 100 according to the first embodiment of the present invention, the multiple imaging periods (frame rates) used to detect flicker differ from each other by at least a ratio of [{2n/(2n-1)}-1] x 100% or more. This also includes the case where the period from the reference imaging period to a period twice the reference imaging period is set to 100%, and the interval therebetween is divided by 1/n power of 2.

Based on the above-described Equations 1 and 2, the relationship between the method for determining each imaging period and the difference in the number of imaging periods is shown in Figure 7. Figure 7 is a diagram (graph) that illustrates an example of the relationship between the method for determining each imaging period for flicker detection and the number of imaging periods according to this embodiment. As shown in Figure 7, Equation 2, shown by the solid line, produces a smaller difference between imaging periods depending on the number n of imaging periods than Equation 1, shown by the dashed line. Note that this condition also applies when considering a larger number n of imaging periods not shown in Figure 7. In other words, while the above examples described two different methods for determining imaging periods, it can be seen that for both methods, the difference between imaging periods is greater than or equal to the value calculated by Equation 2. While two examples of imaging periods for flicker detection have been described above, the imaging periods for flicker detection are not limited to these. As long as the imaging device according to the present invention can accurately detect flicker, it is sufficient that the imaging periods (frame rates) n, which are three or more different natural numbers, are configured such that the least common multiple of the n imaging periods does not include any of the n imaging periods. For example, even if the imaging periods for flicker detection are set to 50 Hz, 150 Hz, 300 Hz, etc., the luminance changes in the images obtained at each imaging period do not change over the same period, making it impossible to accurately detect flicker. For the imaging device of this embodiment, it is preferable that each imaging period for flicker detection is a high rate of 100 fps or more, and that the least common multiple of each imaging period is adjusted so that it does not fall below 10,000 Hz, which is a frequency that may be used as the flicker frequency of a light source such as an LED.

Next, we will explain the details of the process in step S405 mentioned above, which analyzes (detects) the occurrence of flicker at multiple different frequencies. The imaging device according to this embodiment extracts changes in luminance over time based on the luminance of continuously acquired images, and analyzes the periodicity of the luminance changes to detect the frequency of changes in light intensity of flicker. Note that the luminance changes that occur in an image differ depending on the image acquisition method used for detection. For example, the luminance changes in an image differ when an object is captured using a so-called global shutter method such as a CCD and when an object is captured using a so-called rolling shutter method such as a CMOS. Below, we will explain how luminance changes when an image is captured using each of the above methods.

First, referring to Figure 8, we will explain the change in luminance of an image obtained using the global shutter method. Figure 8 is a diagram that illustrates an example of the change in luminance based on images obtained continuously using the global shutter method. When capturing an image of a subject affected by the flickering of a light source due to flicker, a captured image is obtained that is affected by the intensity of the flickering of the light source. When the luminance of the entire captured image is measured, a photometric value that is affected by the intensity of the flickering of the light source is obtained.

Note that the luminance in this description may refer to a luminance signal calculated by multiplying the R, G1, G2, and B signals in a Bayer array RAW image by a certain coefficient, or it may refer to the color signals of the R, G1, G2, and B signals themselves. It may also refer to color and luminance signals obtained from a sensor array other than the Bayer array.

Then, for the captured images obtained using the above-mentioned method, the difference or ratio in brightness (photometric value) between multiple consecutive images is calculated. Alternatively, an average image of multiple images is used as a reference image, and the difference/ratio in brightness between each image and the reference image is calculated. By plotting the brightness change for each image obtained in this way, it is possible to detect the progression of brightness change in the images, as shown in Figure 8.

Next, referring to Figure 9, we will explain the change in brightness of an image obtained using the rolling shutter method. Figure 9 is a diagram that illustrates an example of the change in brightness based on images obtained continuously using the rolling shutter method. When driving a sensor using the rolling shutter method, the exposure and readout timing differs for each row (line) of the sensor, so the impact of flickering of the light source due to flicker differs for each row (line), and the change in brightness that occurs in the vertical direction of the image differs.

Therefore, when driving the sensor (image sensor 101 in this embodiment) using the rolling shutter method, luminance changes due to light source flicker can be extracted by obtaining an integral value for each row (line) of the captured image. Specifically, as shown in FIG. 9, the luminance change for the same line in consecutive frames N-1 and N of images obtained by continuously capturing images of a subject is extracted. In this case, an integral value is calculated for each row of the captured image corresponding to the Nth and N-1th frames. As described above with respect to the global shutter method, this integral value may be a luminance signal obtained by multiplying the color signal by a certain ratio, or it may be the integral of the color signal itself. By comparing the integral values for the Nth and N-1th frames for each row and calculating the difference/ratio, luminance changes in the vertical direction (i.e., the scanning direction of the sensor) of the captured image can be detected, as shown in FIG. 9.

The frames to be compared do not have to be two consecutive frames. For example, the signal values of multiple captured images can be averaged to obtain an average image, and the average image can be used as a reference image. The integral value for each line can be compared with the integral value for each line of the Nth frame to calculate the change in brightness in the vertical direction of the image.

By analyzing images captured using the rolling shutter method using the method described above, it is possible to detect the vertical luminance change in the captured image, as described above, and this luminance change represents the flickering of the light source (i.e., changes in the light intensity due to flicker).

Next, we will explain a method for analyzing the frequency of luminance changes from the progression of luminance changes in an image. A common method for converting a signal that changes over time into frequency components is the Fourier transform. In this case, a signal that changes over time, f(t), is converted into a frequency function, F(ω).

If we focus on the exponential function in Equation 3, it is generally known that the exponential function can be expanded into trigonometric functions with real and imaginary parts due to the relationship between the Maclaurin expansion and the nth derivative of a trigonometric function (shown in Equation 4 below).

Furthermore, since the integral can be calculated by taking the transition of the change in the image signal as f(t) and the sampling interval of the transition of the change as dt, Equation 4 can be expressed as Equation 5 below.
F(ω)=A(ω)+j×B(ω) (Formula 5)

Since this is a complex function of frequency ω, its magnitude is calculated as |F(ω)|. If the transition of luminance change in the image includes a luminance change component due to frequency ω, |F(ω)| will be a large value, and if the transition of luminance change in the image does not include a luminance change component due to frequency ω, |F(ω)| will be a small value. In other words, |F(ω)| can be considered the flicker level for each frequency. Therefore, by calculating each frequency component using Equation 5 above for a wide range of frequencies to be detected, it is possible to detect the presence or absence of luminance changes due to flickering of the light source over a wide frequency range (i.e., the frequency of change in light intensity of flicker).

Furthermore, if the progression of brightness changes does not include at least one cycle of the light source blinking (one cycle of the flicker light intensity change), the target frequency cannot be detected properly and may be mistaken for another frequency. Therefore, it is preferable to continue capturing images of the subject for at least one cycle of the frequency to be detected, and then detect each of the above-mentioned frequencies (i.e., the flicker light intensity change frequency) based on the images obtained by this capture.

Next, we will specifically explain the exposure operation during flicker detection in step S403 mentioned above. As mentioned above, if the imaging cycle when detecting flicker and the light source blinking frequency (flicker light intensity change frequency) are synchronized, it becomes difficult to effectively detect flicker based on sampled images. Furthermore, in addition to the imaging cycle, if the exposure time (i.e., shutter speed) when capturing an image of a subject is synchronized with the light source blinking frequency, no effective brightness change occurs in the image obtained in this state, making it difficult to detect flicker.

In this embodiment, therefore, during each imaging cycle when flicker detection is performed, an exposure time (shutter speed) is set that is synchronized with each imaging cycle so as not to synchronize with frequencies other than that of the imaging cycle. In other words, when detecting flicker, it is preferable to capture an image of the subject with an exposure time (shutter speed) that is 1/N (N is an integer) of the imaging cycle (frame rate) for detection.

Figure 10 is a diagram illustrating exemplary exposure time (shutter speed) setting values for a first pattern of multiple imaging cycles for flicker detection according to the first embodiment of the present invention. For example, as described above, if the multiple imaging cycles for flicker detection are 100 fps, 126 fps, and 159 fps, the subject is captured by setting the exposure time as shown in Figure 10.

FIG. 11 is a diagram illustrating exemplary exposure time (shutter speed) setting values for a second pattern of multiple imaging cycles for flicker detection according to the first embodiment of the present invention. For example, as described above, if the multiple imaging cycles for flicker detection are 100 fps, 133 fps, and 167 fps, the subject is captured by setting the exposure time as shown in FIG. 11.

As shown in Figures 10 and 11, by acquiring images for flicker detection with an exposure time that is 1/N (N is an integer) of the imaging period (frame rate) for flicker detection, it is possible to prevent synchronization between the exposure time and the frequency of changes in light intensity of the flicker.

Furthermore, if the exposure conditions differ at each frequency of the flicker light intensity change, differences will occur in the detected flicker level, reducing the accuracy of detection. Therefore, in this embodiment, by performing exposure operations for the multiple imaging cycles described above based on the photometry results of step S401, differences in exposure amount between imaging cycles are suppressed, enabling stable flicker level detection.

By adopting the configuration described above, the imaging device according to this embodiment can stably and effectively detect flicker across a wide range of possible frequencies of change in light intensity due to flicker.

Next, with reference to FIG. 12, the details of the flicker reduction exposure time determination process executed in step S304 described above will be described. FIG. 12 is a flowchart of the flicker reduction exposure time determination process according to the first embodiment of the present invention. First, in step S1201, the CPU 103 reads from memory the light intensity change frequency of the flicker detected by the flicker detection process executed in step S302 described above.

Next, in step S1202, the CPU 103 calculates the ideal exposure time (IdealFlkExpTime) for reducing the effects of the detected flicker based on the reciprocal of the light intensity change frequency of the flicker read in step S1201. For example, if the light intensity change frequency of the detected flicker is 540.0 Hz, then IdealFlkExpTime = 1/540.0.

Next, in step S1203, the CPU 103 acquires the currently set shutter speed (CurTv). The current shutter speed CurTv may be, for example, the shutter speed set manually by the user. In this embodiment, it is assumed that the shooting mode of the camera body 100 has been set to manual mode in advance, and that all of the multiple exposure control values (parameters) have been manually set by the user.

Next, in step S1204, the CPU 103 executes initialization processing to integer-multiply the ideal flicker reduction exposure time IdealFlkExpTime. Specifically, in step S1204, the integer N is set to 1, and information about the ideal flicker reduction exposure time IdealFlkExpTime before integer multiplication is stored as PreIdealFlkExpTime.

Next, in step S1205, the CPU 103 compares the currently set shutter speed CurTv obtained in step S1203 with the ideal flicker reduction exposure time IdealFlkExpTime. If the value of CurTv is equal to or less than IdealFlkExpTime (i.e., the exposure time is short), the process proceeds to step S1207 (YES determination in step S1205). On the other hand, if CurTv is greater than IdealFlkExpTime (the exposure time is long), the process proceeds to step S1206 (NO determination in step S1205).

In step S1206, the CPU 103 retains the current ideal flicker reduction exposure time as the previous PreIdealFlkExpTime, increments the integer N by one, and then multiplies the ideal flicker reduction exposure time by an integer N. Specifically, in step S1206, IdealFlkExpTime is assigned to PreIdealFlkExpTime, N is incremented to N=N+1, and IdealFlkExpTime is multiplied by an integer N. The processing of step S1206 is repeated until the shutter speed currently set in step S1205 becomes equal to or less than the ideal flicker reduction exposure time (CurTv≦IdealFlkExpTime). In other words, the processing of step S1206 is performed to bring the ideal flicker reduction exposure time IdealFlkExpTime as close as possible to the currently set shutter speed. With this processing, CurTv is positioned between IdealFlkExpTime and PreIdealFlkExpTime, so for example, the flicker reduction exposure time can be narrowed down to an exposure time close to the shutter speed set by the user.

Next, in step S1207, the CPU 103 compares the absolute value of the difference between the IdealFlkExpTime and PreIdealFlkExpTime values and CurTv. If the determination in step S1207 is NO, the current flicker reduction exposure time determination process ends. This is because it can be determined that the currently set ideal flicker reduction exposure time, IdealFlkExpTime, is closer to the current shutter speed than PreIdealFlkExpTime.

On the other hand, if step S1207 returns YES, it can be determined that the previously set ideal flicker reduction exposure time, PreIdealFlkExpTime, is closer to the current shutter speed than the currently set ideal flicker reduction exposure time. Therefore, in this case, processing proceeds to step S1208, where the CPU 103 replaces the previously set PreIdealFlkExpTime with the ideal flicker reduction exposure time, IdealFlkExpTime, and ends the current flicker reduction exposure time determination process.

The flicker reduction exposure time determination process according to this embodiment described above makes it possible to determine an exposure time (shutter speed) for reducing flicker at a value close to the shutter speed set by the user. This configuration makes it possible to obtain an image in which the effects of flicker are reduced, while suppressing any discrepancies in the intended shooting effect caused by the user adjusting the shutter speed.

Figure 13 is a diagram illustrating an example of a method for setting the ideal flicker reduction exposure time when flicker that changes at a predetermined light intensity change frequency occurs, according to the present invention. Figure 13(a) shows, for example, a case where the shutter speed is set to 1/5792.6 by the user (CurTv = 1/5792.6). Figure 13(b) shows, for example, a case where the shutter speed is set to 1/250.5 by the user (CurTv = 1/250.5).

For example, if the light intensity change frequency of the detected flicker is 540.0 Hz, then in the example shown in FIG. 13(a), the ideal flicker reduction exposure time IdealFlkExpTime would be 1/540.0. Furthermore, for the same flicker light intensity change frequency, in the example shown in FIG. 13(b), the ideal flicker reduction exposure time IdealFlkExpTime would be 1/270.0. The change in light intensity of flicker is the same for integer multiples of the frequency. Therefore, even if a subject is captured at a shutter speed slower than the reciprocal of the flicker light intensity change frequency, which is the reciprocal of an integer multiple of the flicker frequency, the effects of flicker can be reduced. Therefore, if the shutter speed set by the user is equal to or lower than the reciprocal of the light intensity change frequency of the detected flicker, then the ideal flicker reduction exposure time can be set to the reciprocal of the integer multiple of the flicker frequency that is the smallest difference from the shutter speed set by the user.

Next, with reference to FIG. 14, the shutter speed selection process executed in step S305 described above will be described in detail. FIG. 14 is a flowchart related to the shutter speed selection process according to the first embodiment of the present invention. First, in step S1401, the CPU 103 executes initialization processing to select an arbitrary shutter speed from the shutter speed setting (index) table described above with reference to FIG. 2. Specifically, in step S1401, the CPU 103 sets a settable flicker reduction shutter speed (SetPosFlkTv) from the shutter speed setting table, with index i=1 in the shutter speed setting table. Note that in this embodiment, as shown in FIG. 2, with index i=1, SetPosFlkTv=1/8192.0.

Next, in step S1402, the CPU 103 increments the index i of the shutter speed setting table by 1. Next, in step S1403, the CPU 103 compares the absolute value of the difference between SetPosFlkTv and the ideal flicker reduction exposure time IdealFlkExpTime described above for each shutter speed [i] in the shutter speed setting table. If the difference between SetPosFlkTv and IdealFlkExpTime is less than or equal to the difference between shutter speed [i] and IdealFlkExpTime (NO in step S1403), the process proceeds to step S1405.

On the other hand, if it is determined that the difference between SetPosFlkTv and IdealFlkExpTime is greater than the difference between shutter speed [i] and IdealFlkExpTime (YES in step S1403), the process proceeds to step S1404. Then, in step S1404, the CPU 103 selects a settable flicker reduction shutter speed based on the determination result of step S1403. Specifically, in step S1404, the CPU 103 sets the settable flicker reduction shutter speed SetPosFlkTv to the shutter speed [i] corresponding to index i in the current shutter speed setting table, and then proceeds to step S1405.

Next, in step S1405, the CPU 103 determines whether index i in the shutter speed setting table is equal to or greater than the maximum index. If the current index [i] is smaller than the maximum index (NO in step S1405), the process returns to step S1402 and repeats steps S1402 to S1405. Note that the maximum index in this embodiment is 600, as shown in FIG. 2. If it is determined in step 1405 that the current index [i] has reached the maximum index (YES in step S1405), the current SetPosFlkTv is selected as the settable flicker reduction shutter speed, and the shutter speed selection process ends.

In the example described above, the shutter speed selection process was performed for all indexes that can be referenced in the shutter speed setting table, but this is not limited to this. For example, if the currently set shutter speed CurTv is acquired during the flicker reduction exposure time determination process, the settable flicker reduction shutter speed may be determined by narrowing it down to the vicinity of CurTv. Specifically, if a specific value is recorded as the currently set shutter speed CurTv, the index corresponding to the shutter speed closest to CurTv is identified. Then, the difference between the ideal flicker reduction exposure time and the shutter speed corresponding to that index and other indexes adjacent to that index is calculated, and the shutter speed with the smallest difference is determined as the settable flicker reduction shutter speed. This configuration is particularly effective when a specific shutter speed has been set by the user. By adopting this configuration, the deviation from the shutter speed intended by the user is minimized, and the number of indexes to be compared is significantly reduced, thereby reducing the processing time and processing load associated with the shutter speed selection process.

By performing the shutter speed selection process described above, the camera body 100 can select, from among the shutter speeds that can be set, a shutter speed that can effectively reduce the effects of pre-detected flicker. In other words, the camera body 100 of this embodiment can select (set) the shutter speed that is closest to the ideal shutter speed IdealFlkExpTime, which is the ideal shutter speed for reducing the effects of detected flicker, from among the shutter speeds that can be set.

Figure 15 is a diagram illustrating an example of the relative relationship between the shutter speed selected by the shutter speed selection process according to the first embodiment of the present invention and the ideal shutter speed for reducing the effects of flicker. In Figure 15, it is assumed that the flicker light intensity change frequency is 540.0 Hz and the ideal flicker reduction exposure time IdealFlkExpTime is 1/540.0. Figure 15(a) shows the case where the shutter speed (CurTv) currently set by the user is 1/5792.6, and Figure 15(b) shows the case where the shutter speed (CurTv) currently set by the user is 1/250.5.

In Figure 15(a), the difference between Tv = 1/546.4 indicated by index 58 in the shutter speed setting table and IdealFlkExpTime Tv = 1/540.0 is shown as Δ58. Also, in Figure 15(a), the difference between Tv = 1/534.7 indicated by index 59 in the shutter speed setting table and IdealFlkExpTime Tv = 1/540.0 is shown as Δ59. In the case shown in Figure 15(a), Δ59 < Δ58, so Tv = 1/534.7 is selected as SetPosFlkTv by the shutter speed selection process described above.

In addition, in Figure 15(b), the difference between Tv = 1/273.2 indicated by index 119 in the shutter speed setting table and IdealFlkExpTime Tv = 1/270.0 is shown as Δ119. In addition, in Figure 15(b), the difference between Tv = 1/270.2 indicated by index 120 in the shutter speed setting table and IdealFlkExpTime Tv = 1/270.0 is shown as Δ120. In the case shown in Figure 15(b), Δ120 < Δ119, so Tv = 1/270.2 is selected as SetPosFlkTv by the shutter speed selection process described above.

As explained above, the camera body 100 of this embodiment can effectively detect the light intensity change frequency of flicker occurring in the current shooting environment, and the ideal shutter speed (exposure time) that reduces the effects of the detected flicker, in the shortest possible time.

Furthermore, the camera body 100 of this embodiment can set a shutter speed that takes into account the shutter speed currently set by the user, etc., as the ideal shutter speed for reducing the effects of flicker. Therefore, the camera body 100 of this embodiment can detect a shutter speed that can reduce the effects of flicker while minimizing any discrepancies in the exposure conditions and shooting effects intended by the user.

Furthermore, the camera body 100 of this embodiment can automatically select (set) the shutter speed that is closest to the ideal shutter speed that can reduce the effects of flicker among the shutter speeds that the camera body 100 can set. Therefore, the camera body 100 of this embodiment can automatically select (set) a shutter speed that can reduce the effects of flicker without the user needing to manually adjust the shutter speed.

Next, details of the display processing of step S306 described above according to the first embodiment of the present invention will be described with reference to Figures 16 and 17. Figure 16 is a diagram illustrating an example of a notification image displayed on the display unit 102 according to the display processing according to the first embodiment of the present invention.

Of these, Figure 16(a) shows a case where a 540.0 Hz flicker is detected, CurTv is 1/5792.6, and SetPosFlkTv is 1/534.7. Figure 16(b) shows a case where a 540.0 Hz flicker is detected, CurTv is 1/250.5, and SetPosFlkTv is 1/270.2. Figure 17 is a diagram illustrating an example of a notification image when no flicker is detected using the display processing according to the first embodiment of the invention.

The detected flicker area 1601 displays information indicating the frequency of change in light intensity of the flicker detected based on the method described above (540.0 Hz in the illustrated example).

The selectable shutter speed area 1602 displays the settable flicker reduction shutter speed SetPosFlkTv calculated using the method described above (Figure 16(a) shows 1/534.7, and Figure 16(b) shows 1/250.5).

The current shutter speed area 1603 displays the shutter speed currently set on the camera body 100, for example, as manually set by the user (1/5792.6 in Figure 16(a) and 1/270.2 in Figure 16(b)).

The first user selection icon 1604 displays an option for when the user does not agree to the change to the configurable flicker reduction shutter speed SetPosFlkTv displayed on the notification screen. The second user selection icon 1605 displays an option for when the user agrees to the change to the configurable flicker reduction shutter speed SetPosFlkTv displayed on the notification screen.

Furthermore, if the flicker detection process does not detect flicker at a predetermined level or above, as shown in FIG. 17, a message 1701 indicating that flicker was not detected and an icon 1702 allowing the user to input confirmation or non-confirmation are displayed on the display unit 102.

As described above, if flicker at a predetermined light intensity change frequency is detected by the flicker detection process, various icons and text such as those shown in Figures 16(a) and (b) are displayed on the display unit 102 to prompt the user to change the shutter speed. This configuration makes it possible to easily set a shutter speed that can reduce the effects of flicker, while reducing the need for the user to manually adjust the shutter speed to reduce the effects of flicker. Therefore, the camera body 100 of the present invention can capture images that reduce the effects of flicker at a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

Note that the method of informing the user of the frequency of change in light intensity of flicker, the shutter speed that can reduce the effects of flicker, and the method of changing the shutter speed are not limited to those described above. For example, in the above example, a notification image is displayed on the display unit 102, but the notification image may also be displayed on another display device or an external device connected to the camera body 100. Furthermore, notification does not have to be limited to image display. Various notification methods, such as audio guidance or changing the lighting state or color of a lamp (not shown) provided on the camera body 100, can also be used.

Furthermore, the camera body 100 according to this embodiment employs a method of prompting the user to confirm whether or not to change to a settable flicker reduction shutter speed, but this is not limiting. For example, the camera may be configured to automatically change to a settable flicker reduction shutter speed without the user's consent, or the camera may be configured to determine whether or not to prompt the user about whether or not to change to a settable flicker reduction shutter speed depending on the shooting mode.

In this case, if the shooting mode is an auto mode in which the camera body 100 automatically determines each parameter related to exposure control, it is preferable for the camera body 100 to automatically set the settable flicker reduction shutter speed. On the other hand, if the shooting mode is a manual mode in which the user manually sets each parameter related to exposure control (exposure control value), it is preferable to employ a method of asking the user whether or not to change the shutter speed, as in the example described above.

Furthermore, as described above, the camera body 100 according to this embodiment uses the electronic shutter with priority, but this is not limited to this. For example, the camera body 100 may be configured to use the mechanical shutter 104 to adjust the exposure time of the image sensor 101 in accordance with an arbitrary shutter speed.

When capturing an image of a subject using the mechanical shutter 104 with a high shutter speed, the timing of the mechanical shutter 104's movement relative to the ideal exposure time may deviate depending on changes in the physical characteristics of the mechanical shutter 104 and differences in the environment. In other words, if the shutter speed set as the settable flicker reduction shutter speed SetPosFlkTv is high, it may not be possible to capture an image of the subject with an exposure time that properly reduces the effects of flicker.

Therefore, when adjusting the exposure time using the mechanical shutter 104, the settable flicker reduction shutter speed SetPosFlkTv may be limited so that the shutter speed is equal to or faster than a predetermined speed. The predetermined speed (shutter speed) may be any value that ensures that the deviation (i.e., error) between the ideal exposure time and the timing of exposure and shading of the image sensor 101 caused by driving the mechanical shutter 104 falls within a predetermined range. In this embodiment, the predetermined shutter speed is illustratively set to 1/4000 seconds. In this case, the aforementioned shutter speed setting table may be used within a range excluding indexes corresponding to shutter speeds of 1/4000 seconds or less, or new table data may be used to determine the settable flicker reduction shutter speed.

Note that the camera body 100 of this embodiment may be configured to dynamically adjust whether to use the electronic shutter or the mechanical shutter 104 depending on the value of the settable flicker reduction shutter speed SetPosFlkTv. For example, when the shutter speed is faster than 1/4000 seconds, only the electronic shutter may be used, and at other shutter speeds, both the electronic shutter and the mechanical shutter 104 may be used.

Second Embodiment
In the first embodiment described above, a configuration was described in which only one arbitrarily settable flicker reduction shutter speed was notified to the user. In contrast, in this embodiment, a configuration in which a plurality of options are notified to the user as settable flicker reduction shutter speeds will be described with reference to FIG. 18 . Note that the configurations and basic driving methods of the camera body 100, lens unit 200, and light-emitting device 300, which are the imaging apparatus according to this embodiment, are substantially the same as those in the first embodiment described above, and therefore the same reference numerals are used for the respective components, and their description will be omitted. This embodiment differs from the first embodiment described above in the display processing in step S306.

Figure 18 is a diagram illustrating an example of a notification image displayed on the display unit 102 by display processing according to the second embodiment of the present invention. Of these, Figure 18(a) shows a case where a 540.0 Hz flicker is detected, CurTv is 1/5792.6, and SetPosFlkTv is 1/534.7. Also, Figure 18(b) shows a case where a 540.0 Hz flicker is detected, CurTv is 1/250.5, and SetPosFlkTv is 1/270.2.

The detected flicker area 1801 displays information indicating the frequency of change in light intensity of the detected flicker. The current shutter speed area 1802 displays the shutter speed currently set on the camera body 100, for example, as manually set by the user (1/5792.6 is shown in Figure 18(a), and 1/270.2 is shown in Figure 18(b)).

The selectable shutter speed first candidate area 1803 displays the settable flicker reduction shutter speed SetPosFlkTv calculated based on the method described in the first embodiment as the first candidate shutter speed selectable by the user. Note that the selectable shutter speed first candidate area 1803 shows 1/534.7 in Figure 18(a) and 1/270.2 in Figure 18(b).

The second selectable shutter speed candidate area 1804 displays the shutter speed corresponding to the index whose difference from IdealFlkExpTime is next smallest after SetPosFlkTv as the second candidate shutter speed that the user can select. Note that the second selectable shutter speed candidate area 1804 shows 1/546.4 in Figure 18(a) and 1/273.2 in Figure 18(b).

In the selectable shutter speed candidate area 1805, if there is a shutter speed that is more effective at reducing the effects of flicker, regardless of the difference from CurTv, the corresponding shutter speed is displayed as another candidate shutter speed that the user can select. For example, in Figure 18(a), the selectable shutter speed candidate area 1805 displays 1/270.2, which is close to Tv=1/270.0, which is twice the IdealFlkExpTime Tv=1/540.0. If 540 Hz flicker is detected, Tv=1/270.2 has a larger difference from CurTv, but is more effective at reducing the effects of flicker than SetPosFlkTv (1/534.7).

The shutter speed selection icon 1806 displays an icon that allows the user to select from available shutter speed candidates. Among these icons, a white arrow indicates that no available shutter speeds exist, while a black arrow indicates that available shutter speeds exist. In Figure 18(a), there are no other SetPosFlkTv candidates in the first selectable shutter speed candidate area 1803, so a white arrow icon is displayed next to the first selectable shutter speed candidate area 1803n. This is also true for the example shown in Figure 18(b). Furthermore, in Figure 18(a), there is another shutter speed (1/180.0) that is highly effective at reducing the effects of flicker in the selectable shutter speed candidate area 1805, so a black arrow icon is displayed next to the selectable shutter speed candidate area 1805. In Figure 18 (b), there is another shutter speed (1/135.0) that is highly effective in reducing the effects of flicker in the selectable shutter speed candidate area 1805, so a black arrow icon is displayed next to the selectable shutter speed candidate area 1805.

As explained above, the camera body 100 of this embodiment can notify the user of multiple candidates, in addition to the arbitrary SetPosFlkTv, as shutter speeds that can reduce the effects of flicker. This configuration allows the user to easily set the shutter speed they desire from multiple candidates that can reduce the effects of flicker, while reducing the need for manual operation by the user to adjust the shutter speed to reduce the effects of flicker. Therefore, the camera body 100 of the present invention can capture images that reduce the effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

(Third embodiment)
In the first embodiment described above, an example was described in which a specific notification screen was displayed on the display unit 102. In contrast to this, in this embodiment, a configuration for performing flicker detection processing during live view display in which captured images are displayed sequentially will be described with reference to Fig. 19. Note that the configurations and basic driving methods of the camera body 100, lens unit 200, and light-emitting device 300, which are the image capture device according to this embodiment, are substantially the same as those in the first embodiment described above, and therefore the same reference numerals are used for the respective components, and description thereof will be omitted.

Figure 19 is a diagram illustrating an example of a transition screen to flicker reduction processing during live view display according to a third embodiment of the present invention. Note that, although this embodiment describes a configuration in which live view display is performed on the display unit 102, a configuration in which live view display is performed on an electronic viewfinder (not shown) may also be used. Note that during live view display, sampling (charge accumulation) for flicker detection is performed in the image sensor 101 at a timing different from the timing of charge accumulation for obtaining the captured image used for live view display.

As shown in FIG. 19, flicker detection icon 1901 is an icon display for indicating that flicker has been detected when flicker is detected by the flicker detection process described above in the first embodiment. Note that if a flicker detection process different from the flicker detection process described above can be performed, icon 1901 may be configured to display the same, or an icon different from icon 1901 may be used. Here, other flicker detection processes may include processing to detect specific flicker (100 Hz, 120 Hz) that occurs due to periodic changes in the commercial power supply.

Furthermore, icon 1901 may be configured to be displayed only when flicker is detected, or may be configured to be constantly displayed as an icon and have its display content changed (updated) depending on whether flicker is detected. Furthermore, the CPU 103 may be configured to perform control so that flicker detection processing is executed when the user presses flicker detection icon 1901.

The flicker reduction menu and beyond icon 1902 is an icon for transitioning the display content of the display unit 102 to the notification screen described in the first and second embodiments when the icon is pressed (including by touch) by the user. In other words, the camera body 100 according to this embodiment allows the user to transition directly to the notification screen during live view display without going through another user interface such as a menu screen.

As described above, the camera body 100 of this embodiment can detect flicker that changes over a wide range of frequencies even when the subject is being photographed, such as during live view display, and transition to imaging that reduces the effects of flicker, with simple user operations. This configuration reduces the number of manual operations required by the user to detect flicker, while allowing the user to easily set the shutter speed they desire from among multiple options that can reduce the effects of flicker. Therefore, the camera body 100 of the present invention can perform imaging that reduces the effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

(Fourth embodiment)
In the first embodiment described above, the flicker reduction exposure time determination process was described when the current shutter speed CurTv was set in advance. In contrast, in this embodiment, the flicker reduction exposure time determination process will be described when a specific shutter speed (CurTv) is not set, for example, by manual operation by the user. Note that the configurations and basic driving methods of the camera body 100, lens unit 200, and light-emitting device 300, which are the imaging device according to this embodiment, are substantially the same as those in the first embodiment described above, and therefore the same reference numerals are used for the respective components, and their description will be omitted.

In addition to the auto mode and manual mode mentioned above, the shooting modes that can be set on the camera body 100 include a priority mode in which the user manually sets an exposure control value and other exposure control values are automatically set. As a priority mode, the camera body 100 according to this embodiment can set, for example, a shutter speed priority mode in which the user can manually set the shutter speed.

For example, in an automatic exposure control state where the camera body 100's shooting mode is set to auto mode or the like, the user cannot set an arbitrary shutter speed. Therefore, in the flicker reduction exposure time determination process in the first embodiment described above, there is little need to determine the ideal flicker reduction exposure time IdealFlkExpTime taking into account the current shutter speed CurTv.

In this embodiment, the ideal flicker reduction exposure time IdealFlkExpTime is determined based on the result of a determination as to whether the current shutter speed CurTv is the shutter speed CurUserTv manually set by the user. Specifically, in the camera body 100 of this embodiment, the CPU 103 determines whether CurTv ≠ CurUserTv. If this determination determines that CurTv ≠ CurUserTv, the CPU 103 sets the shutter speed in the shutter speed setting table that has the smallest difference from the ideal flicker reduction exposure time as the settable flicker reduction shutter speed.

If the above-described configuration is applied to the flicker reduction exposure time determination process described above, steps S1203 and S1205 and subsequent steps are unnecessary. In this case, the ideal flicker reduction exposure time IdealFlkExpTime is set to an exposure time that is the reciprocal of the light intensity change frequency of the detected flicker, but this is not limited to this. For example, as described in the second embodiment, to maximize the effect of reducing the effects of flicker, a settable flicker reduction shutter speed may be set so that the difference between the ideal flicker reduction exposure time multiplied by an integer N is minimized. In this case, the settable shutter speeds according to the shutter speed setting table are repeatedly compared with integer multiples of the ideal flicker reduction exposure time IdealFlkExpTime. The shutter speed with the smallest difference is then selected as the settable flicker reduction shutter speed SetPosFlkTv.

For example, in the first and second embodiments described above, it is assumed that CurTv has been set, and the value of the settable flicker reduction shutter speed SetPosFlkTv is determined taking into account the difference from CurTv, but this is not limited to this. For example, the camera body 100 may compare the difference between the shutter speed corresponding to each index and the frequency of change in light intensity of flicker and the reciprocal of its integer multiple, and set the value with the smallest difference as the settable flicker reduction shutter speed SetPosFlkTv. In this case, a range of the frequency of change in light intensity of flicker that can be reduced by shutter speeds that can be set on the camera body 100 may be determined, and only the reciprocals of frequencies that fall within this range may be compared.

In this embodiment, the determination as to whether CurTv≠CurUserTv may be made based on the shooting mode currently set on the camera body 100.

As explained above, the camera body 100 of this embodiment can calculate the optimal shutter speed that can effectively reduce the effects of flicker, which changes over a wide range of frequencies, even if the user has not set an arbitrary shutter speed. This configuration makes it possible to easily set the shutter speed that can most effectively reduce the effects of flicker, regardless of the shooting conditions of the camera body 100, and without requiring complex operations by the user.

Therefore, the camera body 100 according to the present invention can capture images that reduce the effects of flicker across a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

Fifth Embodiment
In the first embodiment described above, flicker reduction processing related to capturing a subject when acquiring a still image was explained. In contrast, in this embodiment, flicker reduction processing related to capturing a subject when acquiring a moving image will be explained. Note that the configurations and basic driving methods of the camera body 100, lens unit 200, and light-emitting device 300, which are the imaging device according to this embodiment, are substantially the same as those of the first embodiment described above, so the same reference numerals are used for each component and their explanations will be omitted.

When capturing moving images, the shutter speed that can be set is limited by the update period of each frame that forms the moving image. In other words, there are shutter speeds that cannot be set depending on the recording frame rate of the moving image.

Furthermore, even among the available shutter speeds, there are some that are not desirable as shutter speeds when capturing moving images. For example, if the shutter speed is short, the exposure time for one frame will be short, and the time difference between each frame that forms the moving image will be large, making the movement of the subject in the moving image appear less smooth.

In this embodiment, the ideal flicker reduction exposure time is set as the longest exposure time that can be set at the set frame rate of the video when capturing a video. Note that the flicker reduction exposure time and the settable flicker reduction shutter speed may not match. Therefore, if the settable flicker reduction shutter speed selected based on the newly determined ideal flicker reduction exposure time is a value that cannot be set at the current frame rate of the video, the settable flicker reduction shutter speed is adjusted. Specifically, the shutter speed that is closest to the newly determined ideal flicker reduction exposure time among the shutter speeds that are not limited by the frame rate of the video is set as the settable flicker reduction shutter speed.

In this embodiment, the process of comparing with CurTv in the flicker reduction exposure time determination process described above can be omitted. However, the longest exposure time among the ideal flicker reduction exposure times (integer multiples thereof) whose difference from the current shutter speed CurTv falls within a predetermined range may be set as the final ideal flicker reduction exposure time.

As described above, the camera body 100 of this embodiment is capable of detecting flicker that changes over a wide range of frequencies and capturing images with reduced effects of flicker, while preventing degradation of the quality of the moving image, even when capturing an image of a subject to acquire a moving image. With this configuration, the camera body 100 of this embodiment can easily set a shutter speed that reduces the effects of flicker when capturing both still images and moving images, without requiring any additional operation by the user. Therefore, the camera body 100 of the present invention can capture images with reduced effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

Sixth Embodiment
In the first embodiment described above, a configuration was described in which an ideal flicker reduction exposure time was set so as to minimize the difference with respect to the current shutter speed CurTv. In contrast, in this embodiment, a method for setting an ideal flicker reduction exposure time that can reduce the effects of camera shake and subject blur will be described. Note that the configurations and basic driving methods of the camera body 100, lens unit 200, and light-emitting device 300, which are the image capture device according to this embodiment, are substantially the same as those in the first embodiment described above, and therefore the same reference numerals are used for the respective components, and their description will be omitted.

Generally, the longer the shutter speed (exposure time), the greater the chance of capturing an image with blurred subject parts due to camera shake or subject movement (so-called subject blur) during capture. In other words, to reduce blur in images, it is desirable to keep the shutter speed as short as possible.

The camera body 100 according to this embodiment determines the ideal flicker reduction exposure time to be shorter than the predetermined exposure time in the flicker reduction exposure time determination process according to the first embodiment described above. The predetermined exposure time may be any value that can reduce the effects of subject blur in the image, but in this embodiment, the predetermined exposure time is illustratively set to 1/125 seconds.

In this embodiment, the process of comparing with CurTv in the flicker reduction exposure time determination process described above can be omitted. However, the ideal flicker reduction exposure time may be determined to be an exposure time (an integer multiple of) the ideal flicker reduction exposure time, the difference from the current shutter speed CurTv being within a predetermined range, and shorter than the predetermined exposure time.

Furthermore, when conditions for reducing blur (for example, a specific shooting scene (such as a sports scene)) are set as shooting conditions for the camera body 100, the camera may be configured to set an ideal flicker reduction exposure time that reduces the effects of blurring of the subject.

As described above, the camera body 100 of this embodiment is capable of detecting flicker that changes over a wide range of frequencies and capturing images with reduced effects of flicker, while suppressing the effects of subject blur in images. With this configuration, the camera body 100 of this embodiment can easily set a shutter speed that reduces the effects of flicker, without requiring any additional operations by the user, even when specific shooting conditions are set that aim to reduce blur. Therefore, the camera body 100 of the present invention can capture images with reduced effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in images caused by flicker.

Seventh Embodiment
In this embodiment, a description will be given of flicker reduction processing that occurs during light-emitting imaging using a light-emitting device 300. Note that the configurations and basic driving methods of the camera body 100, which is the imaging device according to this embodiment, the lens unit 200, and the light-emitting device 300 are substantially the same as those of the first embodiment described above, and therefore the same reference numerals are used for the various components, and descriptions thereof will be omitted.

When capturing images with flash using the light-emitting device 300, the settable flicker reduction shutter speed is limited by the synchronization speed, which is determined based on the timing at which the image sensor 101 is exposed and the timing at which the light is emitted by the light-emitting device 300. That is, the camera body 100 according to this embodiment sets the settable flicker reduction shutter speed from candidate shutter speeds that are slower than the synchronization speed of the light-emitting device 300. Specifically, the CPU 103 determines whether or not to perform flash capture using the light-emitting device 300. If it is determined that flash capture will be performed, the CPU 103 limits the selectable shutter speeds in the shutter speed setting table to a range slower than the synchronization speed of the light-emitting device 300.

In this embodiment, the process of comparing with CurTv in the flicker reduction exposure time determination process described above can be omitted. However, it is also possible to configure the final ideal flicker reduction exposure time to be the tuning speed of the light-emitting device 300 that has the smallest difference from the current shutter speed CurTv among the ideal flicker reduction exposure times (integer multiples of the ideal flicker reduction exposure time).

As described above, the camera body 100 of this embodiment is capable of detecting flicker that changes over a wide range of frequencies and capturing images with reduced effects of flicker, while maintaining appropriate illumination of the subject, even when capturing images with light using a light-emitting device. With this configuration, the camera body 100 of this embodiment can easily set a shutter speed that reduces the effects of flicker when capturing images with light, without requiring any additional operations by the user. Therefore, the camera body 100 of the present invention can capture images with reduced effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

Eighth Embodiment
In this embodiment, a detection method for suppressing erroneous detection of flicker will be described in addition to the flicker detection method described in the first embodiment. First, sampling of captured images obtained by the rolling shutter method will be described with reference to Fig. 20. Fig. 20 is a diagram illustrating an example of a method for sampling captured images based on images continuously obtained by the rolling shutter method.

As shown in Figure 20, by dividing an image captured using the rolling shutter method vertically and analyzing the signal for each divided region, it is possible to sample image signals at a frequency even shorter (faster) than the capture period of the captured image. Note that Figure 20 shows an example in which an image captured at a capture period of 100 fps is divided vertically into N regions. If the readout time of the rolling shutter is R [ms], N samples are taken during a period of R [ms], making it possible to sample image signals for flicker detection at a frequency of R/N [ms]. This configuration makes it possible to detect even flicker that changes at high frequencies. Meanwhile, because the capture period of a single captured image before division is 100 fps, macroscopically speaking, this also means that image signals for flicker detection are sampled at a capture period of 10 ms. In other words, as described above, when using a method in which one captured image is divided into multiple parts to obtain more detailed image signals for flicker detection, the first sampling period for obtaining the captured image itself and the second sampling period (R/N [ms]) obtained when each captured image is divided are mixed together.

As a result, when flicker is detected based on images of a subject flickering at a certain frequency K [Hz] captured at a 100 fps imaging cycle, the analysis may result in a peak indicating K±100 [Hz] in addition to the peak indicating the flicker level related to the frequency K [Hz].

Figure 21 is a diagram illustrating an example of changes in flicker level when multiple sampling periods are mixed. As shown in Figure 21, the mixture of two sampling periods causes a so-called wave beat phenomenon, and it can be determined that multiple flicker level rises occur in locations other than those corresponding to the actual flicker frequency k [Hz]. The wave beat phenomenon is generally a phenomenon in which, in a situation where two waves are generated, a different frequency is observed due to the superposition of the two signals. When it comes to detecting the flicker level, this wave swell reduction can sometimes make it impossible to accurately detect the flicker frequency. The following equation (6) explains this in detail.

If the flicker frequency of the subject is _f2 and the observed frequency is _f1 , the wave beat phenomenon caused by the mixing of two sampling periods will affect the imaging frequency fB, and _the observed frequency _f1 will be _f2 ± _fB . As a result, the flicker frequency cannot be detected correctly, and there is a possibility of erroneous detection.

Below, specific methods for addressing the above-mentioned problem will be described. As a first method, there is a method in which the combination of the imaging cycle (frame rate) for acquiring captured images and the frequency of the flicker to be analyzed is different from that in the above-mentioned embodiment. For example, when detecting the flicker frequency _f2 of the subject at an imaging frequency _fB , captured images obtained at an imaging cycle other than _fB can be used in analyzing the flicker frequency of _f2 ± _fB . In this case, the detection result of _the imaging frequency _fB is not used under conditions in which frequencies f2 ± _fB may be observed, and therefore there is no influence from the wave beat phenomenon.

The second method is a method for reducing the influence of the two sampling periods. The reading of the next image is adjusted so that it starts immediately after the reading of the previous image. In other words, the shorter the so-called vertical blanking period, the smaller the influence of the first sampling period. If the vertical blanking period is zero, the influence of the first sampling period is theoretically reduced. Therefore, the second method described above involves adjusting the readout time so that the vertical blanking period is shorter. Specifically, the imaging period used for detection is changed to a faster period (frame rate). FIG. 22 is a diagram illustrating an example of shifting the imaging period of the captured image used for flicker detection described in FIG. 5 to a faster period. FIG. 23 is a diagram illustrating an example of shifting the imaging period of the captured image used for flicker detection described in FIG. 6 to a faster period. By shifting the imaging period to a faster period as shown in FIGS. 22 and 23 , the influence of the first sampling period is reduced, and the peak flicker level caused by the frequency f ₂ ±f _B can be suppressed. As shown in FIGS. 22 and 23, instead of just shifting n imaging periods by one step to the faster speed side, the configuration may be such that all n imaging periods are shifted to the faster speed side.

Next, a third method is a method of determining the flicker frequency of the subject using the peak of the flicker level at _f2 [Hz] and the peak of the flicker level at _f2 ± _fB [Hz]. In this case, the frequency at which the flicker level is maximum is not simply determined to be the flicker frequency of the subject (the frequency of change in the light intensity of the flicker). In the third method, when a significant rise similar to three peaks, namely the frequency _f2 [Hz] and _f2 ± _fB [Hz], is detected as a change in the flicker level, the frequency _f2 , which is the center value of the three, is determined to be the flicker frequency of the subject (the frequency of change in the light intensity of the flicker). When this configuration is adopted, flicker detection is required based on images captured at the imaging frequency _fB for the frequencies _f2 [Hz] and _f2 ± _fB [Hz], resulting in a greater computational load than in the first embodiment described above. However, as shown in Figures 5(b) and 6(b), if the combination of any imaging cycle and the frequency to be detected is separated, there are cases where the frequency corresponding to _f2 ± _fB [Hz] is not subject to detection at the imaging frequency _fB . Detecting the frequency range of the flicker to be detected for all n imaging cycles requires a very large amount of calculation, but if there is sufficient leeway in the calculation cost or if a dedicated processor for calculation can be provided, flicker can be detected with high accuracy by adopting the third method. That is, it is sufficient to perform an analysis of the flicker level for all frequencies to be detected for each of the n imaging cycles and detect the frequency of change in light intensity of the flicker.

While preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and various modifications and variations are possible within the spirit and scope of the invention. For example, in the above-described embodiments, a digital camera was described as an example of an imaging device for implementing the present invention, but the present invention is not limited to this. For example, the present invention may be configured to employ imaging devices other than digital cameras, such as portable devices such as digital video cameras and smartphones, wearable devices, in-vehicle cameras, and security cameras.

In addition, while the above-described embodiment describes a configuration capable of detecting and reducing flicker that changes over a wide range of frequencies without specifying a light source, the present invention is not limited to this. For example, a configuration may be adopted in which a specific light source is designated in advance and flicker is detected according to the frequency range in which it is likely to occur. In this case, for example, similar to the shutter speed setting table shown in FIG. 2, table data may be prepared for each light source (or group of similar light sources), and the shutter speed that is likely to be set for each table data may be limited based on the frequency at which the light intensity of the light source changes. With this configuration, it is possible to efficiently set a shutter speed that can reduce the effects of flicker according to the flicker that is likely to occur with each light source, thereby reducing the amount of table data as much as possible while effectively reducing the effects of flicker.

The imaging device according to the present invention may be configured so that the user can select whether or not to execute the flicker detection method and the method for reducing the effects of flicker described in the above-described embodiment as one of the modes or menu settings that can be set by the camera body 100. For example, the camera body 100 may be configured to have at least a first mode that can detect flicker that changes in luminance at a high frequency of 200 Hz or more, and so that the user can freely switch between switching on and off the first mode.

Furthermore, the camera body 100 according to the present invention may be configured to have a second mode for detecting flicker (100 Hz and 120 Hz) attributable to commercial power supply frequencies as described in the aforementioned Patent Document 1 and for reducing the effects of the flicker. In this case, the camera body 100 only needs to be configured to be able to execute the high-frequency flicker detection and reduction methods described in the above-mentioned embodiments only when the first mode is set, out of the above-mentioned first and second modes.

In addition, in the above-described embodiment, the various components constituting the imaging system, centered around the CPU 103, work in cooperation with one another to control the operation of the device as a whole, but this is not limited to this. For example, a (computer) program following the flow illustrated in each of the above-described figures may be stored in advance in the ROM of the camera body 100. The program may then be executed by a microprocessor such as the CPU 103 to control the operation of the entire imaging system. Furthermore, as long as the program has the functionality of a program, it may take any form, such as object code, a program executed by an interpreter, or script data supplied to an OS. Furthermore, the recording medium for supplying the program may be, for example, a magnetic recording medium such as a hard disk or magnetic tape, or an optical/magneto-optical recording medium.

Furthermore, in the above-described embodiment, a digital camera was described as an example of an imaging device for implementing the present invention, but the present invention is not limited to this. For example, various imaging devices may be used, such as portable devices such as digital video cameras and smartphones, wearable devices, and security cameras.

(Other embodiments)
The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more of the functions.

100 Camera body 101 Image sensor 102 Display unit 103 CPU
200 photographing lens 300 light emitting device

Claims (16)

An imaging element;
a drive control means for controlling the drive of the imaging element;
a flicker detection means for detecting flicker based on a signal output from the imaging element;
and
the drive control means controls the imaging device to be driven at n different frame rates, n being a natural number of 3 or more, for flicker detection;
the least common multiple of the n frame rates used for detecting flicker does not include any of the n frame rates;
The imaging device is characterized in that the flicker detection means detects flicker based on a flicker detection signal obtained for each of the n frame rates. The imaging device described in claim 1, characterized in that the n frame rates used for flicker detection have cycles that differ from each other by 1/nth power of 2 as a predetermined interval. The imaging device described in claim 1, characterized in that the n frame rates used for flicker detection have cycles that differ from each other by 100/n [%] at predetermined intervals. The imaging device described in any one of claims 1 to 3, wherein each of the n frame rates used for flicker detection is 100 fps or higher. 5. The imaging device according to claim 1, wherein the least common multiple of the n frame rates used for detecting flicker is 10,000 or greater. 5. The imaging device according to claim 1, wherein the least common multiple of the n frame rates used for detecting flicker is greater than the reciprocal of the upper limit of the high-speed shutter speed that can be set by the imaging device. The imaging device described in claim 2 or 3, characterized in that the flicker detection means detects flicker within a predetermined range of light intensity change frequencies, and detects flicker based on the flicker detection signal obtained by driving the imaging element at one of the n frame rates for each light intensity change frequency of the flicker to be detected. The imaging device described in claim 7, characterized in that the flicker light intensity change frequencies detected by the n frame rates are shifted by the specified interval for each frame rate. The imaging device described in claim 7 or 8, characterized in that the drive control means controls the drive of the imaging element so that imaging is performed for at least one cycle of the light intensity change frequency of each flicker to be detected, at each of the n frame rates used for flicker detection. 10. The imaging device according to claim 1 , wherein the drive control means changes the number n of frame rates used for detecting flicker in accordance with a range of light intensity change frequencies of flicker to be detected. 11. The imaging device according to claim 1, wherein the drive control means changes the n frame rates used for detecting flicker in accordance with a range of light intensity change frequencies of the flicker to be detected. 12. The imaging device according to claim 1, wherein the flicker detection means controls to change the n frame rates used for detecting flicker when no flicker is detected based on the flicker detection signal. 13. The imaging device according to claim 1 , further comprising: a notifying unit that notifies a light intensity change frequency of the flicker detected by the flicker detecting unit. the imaging device has a plurality of modes for detecting flicker,
The imaging device according to any one of claims 1 to 13 , characterized in that when a second mode corresponding to detecting flicker that changes at a light intensity change frequency of 200 Hz or more is set , which is different from a first mode that detects flicker at a frequency resulting from a commercial power supply frequency, the flicker detection means detects flicker based on the flicker detection signal acquired for each of the n frame rates. a drive control step of controlling the drive of the imaging element;
a flicker detection step of detecting flicker based on a signal output from the imaging element;
and
the drive control step controls the imaging device to be driven at n different frame rates, n being a natural number of 3 or more, for flicker detection;
the least common multiple of the n frame rates used for detecting flicker does not include any of the n frame rates;
The flicker detection method is characterized in that the flicker detection step detects a frequency of change in light intensity of a flicker based on a flicker detection signal acquired for each of the n frame rates. A computer-readable program for causing a computer to execute the flicker detection method according to claim 15 .