JP7780910B2 - Imaging device, image processing method, and image processing device - Google Patents

Description

The present invention relates to an imaging device, an image processing method, and an image processing device.

There are many different types of cameras in the world. Familiar cameras include compact digital cameras and smartphone cameras that are used for snapshots and portraits. On the other hand, cameras for special purposes include broadcast cameras used by television stations and surveillance cameras installed at building entrances or outdoors. There are also camera systems installed in automobiles, ships, and aircraft, as well as special surveillance cameras installed on airport runways to monitor for obstacles. Of course, there are also multipurpose cameras that can be adapted for multiple specialized uses.

For example, surveillance cameras installed outdoors take images over long periods of time without an operator present. In this case, the brightness level of the images output by the surveillance camera changes from moment to moment due to changes in the environment, such as weather and lighting conditions. Considering the use of surveillance cameras for detecting monitored objects, it is necessary to maintain a certain brightness level of the images. One way to increase the brightness level is to increase the exposure time (hereinafter referred to as the slow shutter function). However, cameras used for specific purposes do not have a slow shutter function, and brightness levels cannot be adjusted using a slow shutter.

One method for increasing the brightness level of an image without using a slow shutter is to add multiple image frames to generate a high-brightness image frame, as described in Patent Document 1 below. Furthermore, in the method of Patent Document 1, when the number of added image frames is changed, gain adjustment is performed so that the brightness level of the output high-brightness image frame remains constant before and after the change. For example, in the method of Patent Document 1, when the number of added image frames is halved, the gain for the changed image frame is doubled. Furthermore, Patent Document 2 below describes a method for adjusting the gain according to the output of the frame memory to prevent blown-out highlights caused by frame addition.

Japanese Patent Application Laid-Open No. 2008-263316 Japanese Patent Application Publication No. 11-261880

By applying the above-mentioned video frame addition process, it is possible to increase the brightness level of the video even without a slow shutter function. However, with the method described in the above document, a high-brightness video frame is suddenly output after the frame addition operation begins, resulting in a brightness level gap before and after this. Note that the gain adjustment in Patent Document 1 is a function that adjusts the gain to maintain a constant brightness level in response to an increase or decrease in the number of frames added during the frame addition operation, and is unable to eliminate the brightness level gap described above. The gain adjustment in Patent Document 2 is also a function that adjusts the gain to prevent whiteout during the frame addition operation, and is unable to eliminate the brightness level gap described above.

Abrupt changes in brightness levels such as those described above degrade image quality. In addition to this, reducing factors that degrade image quality due to frame addition, even if only slightly, is important when performing high-sensitivity shooting with an imaging device that does not have a slow shutter function. Therefore, one aspect of the present invention aims to enable high-sensitivity shooting with an imaging device that does not have a slow shutter function, and to suppress the degradation of image quality that accompanies high-sensitivity shooting.

According to a first aspect of the present invention, an imaging device is provided that includes a signal processing unit that generates video frames from an output signal of an image sensor; a frame storage unit that generates a high-brightness video frame using multiple video frames output from the signal processing unit when a high-sensitivity shooting function is turned on and switches the output video frame to the high-brightness video frame; and a gain adjustment unit that adjusts the brightness level of video frames output over multiple frame periods before switching to the high-brightness video frame so that the brightness level gradually approaches the brightness level of the high-brightness video frame during multiple frame periods before switching to the high-brightness video frame.

Furthermore, according to a second aspect of the present invention, there is provided an imaging device comprising: a signal processing unit that applies filtering to video frames generated from an output signal of an image sensor; a first black level correction unit that applies black level correction to the video frames after the filtering process has been applied; and a frame storage unit that, when the high-sensitivity shooting function is turned on, outputs a high-brightness video frame generated using multiple video frames after the black level correction has been applied.

Furthermore, according to a third aspect of the present invention, there is provided an image processing method in which, when the high-sensitivity shooting function is turned on, a computer executes processing including: generating a high-brightness image frame using multiple image frames generated from the output signal of the image sensor, switching the output image frame to the high-brightness image frame; and adjusting the brightness levels of the image frames output during multiple frame periods before switching to the high-brightness image frame so that the brightness levels gradually approach the brightness level of the high-brightness image frame during multiple frame periods before switching to the high-brightness image frame.

Furthermore, according to a fourth aspect of the present invention, there is provided an image processing method in which a computer executes processing including applying filtering to image frames generated from an output signal of an image sensor, applying black level correction to the image frames after the filtering has been applied, and, when a high-sensitivity shooting function is turned on, outputting a high-brightness image frame generated using multiple image frames after the black level correction has been applied.

Furthermore, according to a fifth aspect of the present invention, a computer program for implementing the video processing method according to either the third or fourth aspect described above on a computer can be provided. Furthermore, according to a sixth aspect of the present invention, a computer-readable storage medium storing the computer program according to the fifth aspect can be provided. Furthermore, according to a seventh aspect of the present invention, a video processing device for executing the video processing method according to either the third or fourth aspect described above can be provided.

One aspect of the present invention makes it possible to perform high-sensitivity shooting with an imaging device that does not have a slow shutter function, and also to suppress the degradation of image quality that accompanies high-sensitivity shooting.

1 is a block diagram schematically showing the configuration of an imaging device according to a first embodiment. 10A and 10B are diagrams for explaining pixel addition processing by a signal processing unit. 10 is a table showing an example of first correction data; FIG. 2 is a diagram illustrating a configuration of a frame storage unit. 10 is a timing chart for explaining an example of operation of the frame storage unit (in the case of storing two frames); 10 is a timing chart for explaining an example of operation of the frame storage unit (in the case of storing three frames); 10 is a table showing an example of second correction data. 10 is a timing chart for explaining an example of operation of a gain adjustment unit (in the case of accumulating two frames); 10 is a timing chart for explaining an example of the operation of the gain adjustment unit (in the case of accumulating three frames); 10 is a flowchart illustrating a flow of video frame processing according to the first embodiment. FIG. 10 is a block diagram schematically illustrating the configuration of an imaging device according to a second embodiment. FIG. 10 is a block diagram schematically illustrating the configuration of an imaging device according to a third embodiment.

Embodiments of the present invention will be described below with reference to the accompanying drawings. Note that elements having substantially the same functions in this specification and drawings may be designated by the same reference numerals, and redundant explanations may be omitted.

1. First embodiment
A first embodiment of the present invention will be described below.

[1-1. Configuration of imaging device]
The configuration of an imaging device according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram that schematically shows the configuration of the imaging device according to the first embodiment. Note that the imaging device 10 shown in Fig. 1 is an example of the imaging device according to the first embodiment. Furthermore, the imaging device 10 does not have the slow shutter function described above.

As shown in FIG. 1, the imaging device 10 has a camera head unit 11, a camera control unit 12, and a display device 13. The camera head unit 11 may be simply referred to as a camera head or as an optical unit. The camera control unit 12 may be referred to as a CCU (Camera Control Unit).

In the example of Figure 1, the camera head unit 11 and the camera control unit 12 are shown separately, but they may also be an integrated imaging device in which both are housed in the same housing. The following explanation will refer to the example configuration shown in Figure 1, but the physical shape of the imaging device 10 and the combination of functions provided in each of the camera head unit 11 and the camera control unit 12 can be modified as desired depending on the implementation, and such modifications naturally fall within the technical scope of the first embodiment.

The camera head unit 11 is connected to the camera control unit 12 via a communication line such as a fiber optic cable, coaxial cable, or LAN (Local Area Network) cable. Control signals output from the camera control unit 12 and video signals output from the camera head unit 11 are transmitted via the communication line. For example, when using a fiber optic cable, a communication line of up to approximately 10 km in length can be stretched, making it easy to use not only as a broadcast camera, but also as a surveillance camera operating in a large area such as an airport.

The display device 13 is connected to the camera control unit 12 and displays the video output from the camera control unit 12. The display device 13 is, for example, a display device such as an LCD (Liquid Crystal Display) or ELD (Electro-Luminescence Display). In the example of FIG. 1, the display device 13 is directly connected to the camera control unit 12, but the display device 13 may be external to the imaging device 10 or connected to the imaging device 10 via a video output interface. The installation and connection methods of the display device 13 can be modified as desired depending on the implementation, and such modifications naturally fall within the technical scope of the first embodiment.

(Camera head configuration)
As shown in FIG. 1, the camera head unit 11 includes an optical system 111 , an image sensor 112 , and a signal processing unit 113 .

The optical system 111 includes, for example, a lens unit 111a, an optical filter 111b, and a spectroscopic element 111c. The lens unit 111a is composed of one or more lens groups, each of which includes one or more lenses. The lens unit 111a may include an iris. The optical filter 111b may be an ND (Neutral Density) filter for dimming, an Ir filter that attenuates infrared light, or a color filter that transmits any of R (red), G (green), or B (blue), and may be arranged on the path of light that passes through the lens unit 111a and reaches the image sensor 112.

The spectroscopic element 111c is an optical element such as a prism that separates light transmitted through the lens unit 111a, and is arranged between the lens unit 111a and the image sensor 112. In the example of Figure 1, the imaging device 10 is equipped with three image sensors 112. For example, the three image sensors 112 correspond to R, G, and B, respectively. In this case, the spectroscopic element 111c separates the light transmitted through the lens unit 111a into the three components of R, G, and B, and guides these three components to the three image sensors 112, respectively. Note that if the imaging device 10 is equipped with a single image sensor 112, the spectroscopic element 111c is omitted.

The image sensor 112 is a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal-Oxide Semiconductor) sensor. As described above, the image sensor 112 may be composed of multiple sensors. The readout method of the image sensor 112 may be either a rolling shutter method or a global shutter method. However, from the perspective of suppressing rolling shutter distortion, it is more preferable to use a global shutter CMOS sensor. The video signal output from the image sensor 112 is input to the signal processing unit 113.

The signal processing unit 113 has an ADC (Analog to Digital Converter) function 113a and a pixel addition function 113b. The ADC function 113a converts the analog video signal output from the image sensor 112 into a digital signal (hereinafter referred to as a video frame). The pixel addition function 113b applies filtering processing to each video frame to reduce noise and/or increase the brightness level (gain increase). For example, the pixel addition function 113b processes each video frame using the method shown in Figure 2. Figure 2 is a diagram for explaining the pixel addition processing by the signal processing unit.

In the example of Figure 2, n is a parameter indicating the horizontal position of a pixel, and m is a parameter indicating the vertical position of a pixel. Furthermore, p(n,m) represents the pixel value of the pixel located at coordinates (n,m). For example, if the pixel value after filtering is P(n,m), the pixel addition function 113b calculates P(n,m) according to the following formula (1).

In equation (1), a0, a1, a2, and a3 are filter coefficients. Setting a0, a1, a2, and a3 to all 1 increases the brightness level of the video frame. In addition, because the pixel values of surrounding pixels are added, the above filtering process also has the effect of a low-pass filter that reduces high-frequency components. By changing the combination of filter coefficients, various filtering processes are possible. For example, by making the total value of the coefficients greater than 1, it can function as a filter with increased gain, and by making the total value less than 1, it can function as a noise reduction filter that reduces noise without increasing gain.

For the sake of convenience, the example in Figure 2 and the above formula (1) target four pixels (two horizontal pixels and two vertical pixels) for filtering processing, but the combination of target pixels is not limited to this. For example, the seven filtering patterns shown in Table 1 below can be set as combinations of pixels to be filtered. Note that the +0 dB pattern corresponds to a case where filtering processing by the pixel addition function 113b is not applied.

Furthermore, the above equation (1) can be expanded as in the following equation (2). In equation (2), a _mn is a filter coefficient applied to the pixel value p(m, n) at the coordinate position (m, n). By setting the filter coefficients of pixels other than the target pixel to 0, each of the above patterns can be expressed, and by setting the filter coefficient of the target pixel, the frequency characteristics of the video frame can be manipulated. The signal processing unit 113 calculates P(m, n) for all m and n using the pixel addition function 113b, and outputs a video frame having the calculated P(m, n) as the pixel value to the camera control unit 12.

(Configuration of camera control unit)
1 again, the camera control unit 12 includes a control unit 120, a first black level correction unit 121, a frame accumulation unit 122, a second black level correction unit 123, a gain adjustment unit 124, and a video processing unit 125.

The control unit 120 controls the operation of the camera head unit 11 and camera control unit 12 according to the first embodiment. The control unit 120 is a processor such as a CPU (Central Processing Unit), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or GPU (Graphic Processing Unit). The processor is connected to memory such as ROM (Read Only Memory) or RAM (Random Access Memory), and/or a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). The memory and/or storage device can store a computer program for implementing the functions of the control unit 120.

(Regarding the first black level correction)
The first black level correction unit 121 corrects the black level of the video frame output from the camera head unit 11 (signal processing unit 113) using a preset correction value M1. The correction value M1 is the average brightness level of the video frame output from the camera head unit 11 in a light-shielded state. The black level can be corrected by subtracting the correction value from the brightness level of the video frame captured in a light-unshielded state. However, when the gain of the sensor output is increased, dark current noise is amplified, and therefore the appropriate black level correction value also changes depending on the magnitude of the gain.

In the case of the imaging device 10, the pixel addition function 113b of the signal processing unit 113 applies the above-described filtering process to the video frame. If the content of this filtering process changes, the magnitude of the black level to be corrected will fluctuate. Therefore, the first black level correction unit 121 obtains a correction value M1 corresponding to the content of the filtering process applied by the signal processing unit 113, and corrects the black level by subtracting the correction value M1 from the luminance level of the video frame.

In the example of Figure 1, data on correction value M1 is held by first black level correction unit 121 as first correction data 121a. However, first correction data 121a may also be held by control unit 120 and provided to first black level correction unit 121. First correction data 121a may have a structure such as that shown in Figure 3, for example. Figure 3 is a diagram showing an example of first correction data. Note that, for ease of explanation, Figure 3 shows the relationship between the contents of the filtering process (the seven filtering patterns shown in Table 1 above) and correction value M1 in the form of a diagram, but the data structure is not limited to this example.

For example, if the pixel addition function 113b of the signal processing unit 113 applies a filtering pattern that simply adds two horizontal pixels and two vertical pixels (see Figure 2), the first black level correction unit 121 uses the correction value M1 (+12 dB) corresponding to that pattern to correct the black level of the video frame output from the signal processing unit 113. Note that the correction value M1 corresponding to each filtering pattern can be determined in advance using a filtered video frame captured in a light-blocking state. Furthermore, the filtering pattern used by the signal processing unit 113 may be determined by the control unit 120, and the correction value M1 applied by the first black level correction unit 121 may be specified by the control unit 120.

In the above explanation, we introduced a method in which a correction value M1 corresponding to each filtering pattern is prepared in advance as first correction data 121a. However, as a variant, the difference value between the correction value M1 (0+dB) when no filtering processing is applied and the correction value when another filtering pattern is applied (e.g., M1 (+12dB)) may be stored as first correction data 121a. In this case, the first black level correction unit 121 uses the correction value M1 (0+dB) and the difference value to determine the correction value (e.g., M1 (+12dB)) corresponding to the applied filtering pattern, and performs black level correction using the determined correction value.

(About frame accumulation)
The video frame after black level correction by the first black level correction unit 121 is output to the frame accumulation unit 122. When the high-sensitivity shooting mode is on, the frame accumulation unit 122 accumulates multiple video frames in a frame memory and adds the multiple video frames together to generate a high-brightness video frame. When the high-sensitivity shooting mode is off, the frame accumulation unit 122 directly outputs the input video frames without adding them together.

As shown in FIG. 1, the frame storage unit 122 has frame memories 122a and 122b, and a register 122c. In the following explanation, for ease of explanation, frame memory 122a may be referred to as side A, and frame memory 122b may be referred to as side B. However, please note that the notations A and B are merely used to distinguish between the two, and do not imply any relationship such as priority or higher/lower ranking.

The frame accumulation unit 122 has a configuration, for example, as shown in Figure 4. Figure 4 is a diagram schematically illustrating the configuration of the frame accumulation unit. Note that the configuration example in Figure 4 is merely an example for explanatory purposes, and that equivalent functionality can be achieved during implementation by appropriately combining hardware elements such as circuits and/or software elements, and that the scope of application of the technology according to the first embodiment is not limited to the example in Figure 4.

In Figure 4, the elements of the frame storage unit 122 include frame memories 122a and 122b, as well as switches SWin and SWout and adders 122d and 122e.

Switch SWin is a switch for switching the frame memory that stores video frames. When switch SWin connects input In to frame storage unit 122 to terminal Ain on the frame memory 122a side, the video frames are stored in frame memory 122a (side A). When switch SWin connects input In to frame storage unit 122 to terminal Bin on the frame memory 122b side, the video frames are stored in frame memory 122b (side B).

Switch SWout is a switch used to switch the frame memory from which a video frame is read. When switch SWout connects terminal Aout on the frame memory 122a side to output Out from frame storage unit 122, a video frame is read from frame memory 122a (side A). When switch SWout connects terminal Bout on the frame memory 122b side to output Out from frame storage unit 122, a video frame is read from frame memory 122b (side B).

The adder 122d adds the video frame read from the frame memory 122a with the video frame input to the terminal Ain. The video frame after addition output from the adder 122d is written to the frame memory 122a. Similarly, the adder 122e adds the video frame read from the frame memory 122b with the video frame input to the terminal Bin. The video frame after addition output from the adder 122e is written to the frame memory 122b. The operation of the switches SWin and SWout, and the timing of reading and writing the video frames, are controlled by the control unit 120.

(Explanation of operation)
The operation of the frame storage unit 122 will now be described in detail with reference to Figures 5 and 6. Figure 5 is a timing chart illustrating an example of operation of the frame storage unit (when storing two frames). Figure 6 is a timing chart illustrating an example of operation of the frame storage unit (when storing three frames).

The example in Figure 5 shows a case where, after starting the accumulation of video frames in the frame memory, two video frames are added and output. Figure 5 shows the output timing (frame period) from image sensor 112 [sensor output], the operation of frame memory 122a [frame memory (side A)], the operation of frame memory 122b [frame memory (side A)], the value of register 122c [register (number of accumulated frames)], and which side the video frame is output from [output video]. The horizontal axis of the timing chart shown in Figure 5 is the time axis, with time progressing to the right.

The number written in the block of the sensor output represents the frame period. For example, the block written as k represents the kth frame period. In the example of Figure 5, the accumulation operation of video frames begins from the kth frame period. The number of accumulated video frames, N, is 2. During the frame period before the accumulation operation begins, no video frames are added, and one video frame is output directly. In this case, the adders 122d and 122e do not operate, and data is written alternately to sides A and B, and the one video frame written is output as is.

For example, during the (k-2)th frame period, the video frame output from the camera head unit 11 is written to side B, and at the same time, the video frame on side A is read out and output from the frame storage unit 122. In this case, switch SWin connects input In to terminal Bin on the side B, and switch SWout connects terminal Aout on the side A to output Out.

During the (k-1)th frame period, the video frame output from the camera head unit 11 is written to side A, and at the same time, the video frame on side B is read out and output from the frame storage unit 122. In this case, switch SWin connects input In to terminal Ain on the A side, and switch SWout connects terminal Bout on the B side to output Out. Note that before the storage period begins, the value of register 122c, which represents the number of stored frames, remains 0.

When the accumulation operation begins, adders 122d and 122e begin functioning and add video frames. During the kth frame period, which occurs immediately after the accumulation operation begins, a new video frame input to frame accumulation unit 122 is written to side B, and at the same time, the video frame on side A is read out and output from frame accumulation unit 122. At this time, switch SWin connects input In to terminal Bin on the side B, and switch SWout connects terminal Aout on the side A to output Out.

During the following (k+1)th frame period, the video frame on side B is read out and input to the adder 122e. A new video frame input to the frame storage unit 122 is also input to the adder 122e and added to the video frame read out from side B. The video frame added by the adder 122e is then written to side B. At this time, the value of register 122c (number of stored frames) is rewritten to 1. During this period, switch SWin connects input In to terminal Bin on the side B.

In this example, the number of video frames N to be added is 2, so when the value of register 122c becomes 1, the video frame stored in the frame memory is output. Therefore, the added video frame on side B is read out and output from the frame storage unit 122. At this time, switch SWout connects terminal Bout on the side B to output Out. When frame storage is complete and the added video frame is output, the value of register 122c is reset to 0. Note that no processing is performed on side A during this frame period.

During the following (k+2)th frame period, the video frame on side B is read out and output from the frame storage unit 122. A new video frame input to the frame storage unit 122 is written to side A. At this time, switch SWin connects input In to terminal Ain on the A side, and switch SWout connects terminal Bout on the B side to output Out.

During the following (k+3)th frame period, the video frame on plane A is read out and input to adder 122d. A new video frame input to frame storage unit 122 is also input to adder 122d and added to the video frame read out from plane A. The video frame added by adder 122d is then written to plane A. At this time, the value of register 122c (number of stored frames) is rewritten to 1. During this period, switch SWin connects input In to terminal Ain on the A side.

In this example, the number of video frames N to be added is 2, so when the value of register 122c becomes 1, the video frame stored in the frame memory is output. Therefore, the added video frame on side A is read out and output from the frame storage unit 122. At this time, switch SWout connects terminal Aout on the side A to output Out. When frame storage is complete and the added video frame is output, the value of register 122c is reset to 0. Note that no processing is performed on side B during this frame period.

In the subsequent (k+4)th frame period and thereafter, the frame accumulation unit 122 accumulates video frames while switching between the A and B sides, and adds the video frames using the addition units 122d and 122e, thereby generating high-brightness video frames.

When the number N of video frames to be added is 3, the operation of the frame accumulation unit 122 is as shown in Figure 6. The operation of the frame accumulation unit 122 during the frame period before the start of the accumulation operation is the same as in the example of Figure 5. However, the operation during the (k+1)th and subsequent frame periods differs from the example of Figure 5. For example, during the (k+1)th frame period, the video frame on plane A is output from the frame accumulation unit 122. Also, the process of reading the video frame on plane B, inputting it to the adder 122e, and adding it to the video frame newly input to the frame accumulation unit 122 is the same as in the example of Figure 5, but the video frame after the addition is not output from the frame accumulation unit 122 after being written to plane B. Also, the value of register 122c remains 1.

In the following (k+2)th frame, the video frame on side B is read out and input to the adder 122e. A new video frame input to the frame storage unit 122 is also input to the adder 122e and added to the video frame read out from side B. The video frame added by the adder 122e is then written to side B. At this time, the value of register 122c (number of stored frames) is rewritten to 2. During this period, the switch SWin connects the input In to the terminal Bin on the side B.

In the example of Figure 6, the number of video frames N to be added is 3, so when the value of register 122c reaches 2, the video frame stored in the frame memory is output. Therefore, the added video frame on side B is read out and output from the frame storage unit 122. At this time, switch SWout connects terminal Bout on the side B to output Out. When frame storage is complete and the added video frame is output, the value of register 122c is reset to 0. Note that no processing is performed on side A during this frame period.

In the following (k+3)th frame period, the video frame on side B is read out and output from the frame accumulation unit 122. A new video frame input to the frame accumulation unit 122 is written to side A. At this time, switch SWin connects input In to terminal Ain on the A side, and switch SWout connects terminal Bout on the B side to output Out. In the following (k+4)th frame period and beyond, video frames are similarly accumulated while switching between sides A and B, and the frame accumulation unit 122 generates a high-brightness video frame by adding the video frames using adders 122d and 122e.

As described above, the frame storage unit 122 stores video frames using frame memories 122a and 122b, and adds multiple video frames using adders 122d and 122e to generate high-brightness video frames. Furthermore, even during a frame period when a video frame is stored in one frame memory, a video frame is output from the other frame memory. Therefore, while a typical slow shutter function results in periods when no video is output, such periods do not occur in the imaging device 10 described above.

Here, the function of the adders 122d and 122e will be further explained. One method of adding multiple video frames is to simply add the pixel values of corresponding pixels in the multiple video frames. As an applied example, there is also a method of weighting and adding the pixel values of corresponding pixels. If the pixel value of a pixel located at position (n, m) in the j-th video frame is denoted as q(n, m, j) and the weighting coefficient for the pixel value of the j-th video frame is denoted as _rj , the pixel value Q(n, m, t) of the video frame after addition is given by the following equation (3):

When _r0 = _r1 = 1 and _rj (j ≥ 2) = 0, a video frame obtained is a simple addition of two video frames. Furthermore, when the weighting coefficient _rj is greater than 1, a video frame with increased gain is obtained. Additionally, depending on the setting of the weighting coefficient _rj , a filter effect such as a low-pass filter can also be produced. One method of incorporating the elements of the weighting coefficient _rj into the adders 122d and 122e is to associate the weighting coefficient _rj with the value of the register 122c in advance, and when adding video frames, the adders 122d and 122e apply the weighting coefficient _rj corresponding to the value of the register 122c to each pixel value.

As described above, when the high-sensitivity shooting mode is on, the frame accumulation unit 122 generates a high-brightness video frame using multiple video frames. The video frame output from the frame accumulation unit 122 is then input to the second black level correction unit 123.

(Regarding invalid frame elimination)
As described above, the frame accumulation unit 122 performs the addition process of video frames to obtain a high-brightness video frame. However, there is a possibility that an invalid video frame may be output from the camera head unit 11 for some reason, and adding such an invalid video frame would degrade the video quality. Note that recent image sensors are capable of outputting video at various timings, and may intentionally output invalid video frames that produce unwanted video (black). If such invalid video frames are used as is, they could become noise. Therefore, during the above-described accumulation operation, the frame accumulation unit 122 avoids adding invalid video frames among the video frames output from the camera head unit 11 (i.e., video frames generated from the output signal of the image sensor 112) and prevents them from being used to generate a high-brightness video frame. By applying this mechanism, it is possible to avoid degradation of video quality due to invalid video frames. Note that various factors may cause invalid video frames to be output, including, for example, noise contained in the output signal of the image sensor 112, as well as some quality degradation factors that occur during video processing by the signal processing unit 113 and/or the first black level correction unit 121. The reason for invalidation is not particularly limited in this embodiment.

(Regarding the second black level correction)
As already explained, the black level of each video frame is corrected by the first black level correction unit 121 before the frame accumulation unit 122. However, because the frame accumulation unit 122 adds multiple video frames, there is a possibility that the black level may fluctuate due to amplification of remaining noise depending on the number of additions (number of accumulated frames). Therefore, in the first embodiment, the black level is further corrected by the second black level correction unit 123.

The second black level correction unit 123 corrects the black level of the video frames output from the frame accumulation unit 122 using a correction value M2 that corresponds to the number of accumulated frames. The correction value M2 is set to the average brightness level of the video frames output from the frame accumulation unit 122 when shooting is performed in a light-blocking state. By calculating the average brightness level while changing the number of accumulated frames, data on the correction value M2 that corresponds to the number of accumulated frames (second correction data 123a) is obtained. Figure 7 is a chart showing an example of the second correction data.

When the number of accumulated frames is 1 (for example, the frame period before the kth frame in Figure 5), the video frame is output directly, so there is no black level fluctuation due to the addition process, and the black level has already been corrected by the first black level correction unit 121. Therefore, when the number of accumulated frames is 1, the second black level correction unit 123 does not perform black level correction, and outputs the video frame output from the frame accumulation unit 122 directly to the gain adjustment unit 124.

When the number of accumulated frames is two or more (for example, the (k+1)th frame period or later in Figure 5), the second black level correction unit 123 obtains a correction value M2 corresponding to the number of accumulated frames and corrects the black level by subtracting the correction value M2 from the luminance level of the video frame. For example, when the number of accumulated frames is two, the second black level correction unit 123 corrects the black level of the video frame using the correction value M2(2) of the second correction data 123a. The video frame after black level correction is output to the gain adjustment unit 124.

(Regarding gain adjustment)
As described above, when the imaging device 10 is in high-sensitivity shooting mode, a high-brightness video frame is output by adding multiple video frames using the function of the frame accumulation unit 122. The brightness level of the output video frame is multiplied by the number of frames added (doubled when adding two frames) compared to the brightness level of a single video frame input to the frame accumulation unit 122. Therefore, a gap in brightness level occurs when switching from an unadded video frame to a high-brightness video frame.

To eliminate the above gap, the gain adjustment unit 124 gradually increases the gain of the unadded video frames over a period of several frames before the high-brightness video frame is output, adjusting the brightness level to approach that of the high-brightness video frame. The gain adjustment unit 124 also similarly adjusts the brightness level of the video frames output during the period of several frames before the high-brightness video frame is output after the accumulation operation begins. This type of gain adjustment results in a smooth change in brightness level when switching to high-sensitivity shooting mode.

The gain adjustment unit 124 may be configured as a variable gain circuit in which the gain adjustment range can be set arbitrarily, or as a step gain circuit in which the gain adjustment range is set to a predetermined value (e.g., +6 dB). When a variable gain circuit is used, the gain adjustment range can be determined based on the luminance level (or number of accumulated frames) of a high-luminance video frame and the gain adjustment period (the number of frame periods in which gain adjustment is performed). When a step gain circuit is used, the gain adjustment period (the number of frame periods in which gain adjustment is performed) can be determined based on the luminance level (or number of accumulated frames) of a high-luminance video frame.

Now, the operation of the gain adjustment unit 124 will be described in more detail with reference to Figures 8 and 9. Figure 8 is a timing chart illustrating an example of operation of the gain adjustment unit (in the case of accumulating two frames). Figure 9 is a timing chart illustrating an example of operation of the gain adjustment unit (in the case of accumulating three frames).

The example in Figure 8 shows a case where, after starting the accumulation of video frames in the frame memory, two video frames are added together and output. In this case, the brightness level of the high-brightness video frame that is output is twice that of the single video frame that is input. Figure 8 shows the output timing (frame period) from image sensor 112 [sensor output], the operation of frame memory 122a [frame memory (side A)], the operation of frame memory 122b [frame memory (side A)], the value of register 122c [register (number of accumulated frames)], and which side the video frame is output from [output video].

In the example of Figure 8, similar to the example of Figure 5 already described, the accumulation operation of video frames begins from the kth frame period. In the frame period before the accumulation operation begins, no video frames are added, and unadded video frames are output. On the other hand, after the accumulation operation begins, in the frame period after the first addition process is completed, high-brightness video frames are output. In the example of Figure 8, high-brightness video frames are output in the (k+1)th frame period and onwards, and unadded video frames are output in the frame periods before the kth frame.

When the high-sensitivity shooting mode is turned on, the imaging device 10 does not immediately start the accumulation operation, but instead sets a frame period (hereinafter referred to as the gain adjustment period) for gain adjustment by the gain adjustment unit 124. In the example of Figure 8, the gain adjustment period is the (k-6)th to kth frame period. In other words, the gain adjustment period is set to the period consisting of a predetermined number of frame periods (6 in the example of Figure 8) before the accumulation operation starts and the frame period until a high-brightness video frame is output after the accumulation operation starts.

As shown in Figure 8, the amount of amplification applied in each frame period within the gain adjustment period is set to a constant value, and this constant value may be preset so that the same brightness level as a high-brightness video frame is reached in the frame period immediately prior to the start of the accumulation operation (the (k-1)th frame period in the example of Figure 8). Furthermore, this constant value may be preset so that the same brightness level as a high-brightness video frame is reached in the frame period at the end of the gain adjustment period (the kth frame period in the example of Figure 8).

When the number N of video frames to be added is 3, the operation of the frame accumulation unit 122 and gain adjustment unit 124 is as shown in Figure 9. The operation of the frame accumulation unit 122 and gain adjustment unit 124 before the start of the accumulation operation is the same as in the example of Figure 8. After the accumulation operation starts, unadded video frames are output for two frame periods (the kth and (k+1)th frame periods in the example of Figure 9) until a high-brightness video frame is output. Therefore, in the example of Figure 9, the frame periods before the (k+1)th frame become the gain adjustment period. Note that the amount of amplification applied in each frame period within the frame adjustment period is the same as in the example of Figure 8.

Video frames output from the gain adjustment unit 124 are input to the video processing unit 125. Video frames input before the gain adjustment period are through-output video frames with no gain adjustment, video frames input during the gain adjustment period are gain-adjusted video frames with no addition, and video frames input after the gain adjustment period are high-brightness video frames. Because the brightness levels of these video frames change smoothly, there are no gaps in the brightness levels of the output video, resulting in high-quality video with little discomfort.

The above explanation describes the operation when high-sensitivity shooting mode is turned on, but even when high-sensitivity shooting mode is turned off, gaps in the brightness level can occur if gain adjustment is not performed. Therefore, contrary to the method described above, during a predetermined number of frame periods after the accumulation operation ends (second gain adjustment period), the gain adjustment unit 124 gradually decreases the amplification amount so that the amplification amount becomes 0 immediately after the second gain adjustment period, while increasing the gain of unadded video frames during each frame period within the second gain adjustment period. As a result, the brightness of the image changes smoothly even when high-sensitivity shooting mode is turned off, preventing gaps in the brightness level.

(Regarding processing and display by the video processing unit)
The image processing unit 125 applies predetermined image processing, such as AWB (Auto White Balance), gamma correction, and knee correction, to the input image frames. As described above, unlike a slow shutter, the imaging device 10 adds image frames together to increase the brightness level, and also outputs image frames during the accumulation period, eliminating any periods during which images are not output. Furthermore, gain adjustment by the gain adjustment unit 124 prevents sudden changes in brightness levels. This makes it possible to apply image processing, such as AWB, and achieve higher image quality. Furthermore, black level correction by the first black level correction unit 121 and the second black level correction unit 123 further improves image quality.

The video processing unit 125 applies predetermined video processing to the video frames and then causes the display device 13 to display the series of video frames. Note that the video processing unit 125 may output the series of video frames to a video output interface in addition to or instead of the display device 13. As a variation, the video processing unit 125 may analyze the series of input video frames, detect objects (e.g., obstacles on a runway) contained in the video frames, and display information about the detected objects on the display device 13. As another variation, the video processing unit 125 may output the series of video frames to a detection device that analyzes video and detects objects.

The configuration of the imaging device according to the first embodiment has been described above.

[1-2. Video frame processing flow]
Next, the flow of video frame processing (processing by the camera control unit 12) according to the first embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart for describing the flow of video frame processing according to the first embodiment.

(S101) A video frame output from the camera head unit 11 is input to the first black level correction unit 121. The first black level correction unit 121 references the first correction data 121a and selects a correction value M1 that corresponds to the filtering pattern applied by the signal processing unit 113 of the camera head unit 11. The first black level correction unit 121 then corrects the black level of the video frame using the selected correction value M1. The video frame after black level correction is output to the frame accumulation unit 122.

(S102) The control unit 120 determines whether the camera is in high-sensitivity shooting mode. If the camera is in high-sensitivity shooting mode, the process proceeds to S103. On the other hand, if the camera is not in high-sensitivity shooting mode, the process proceeds to S107.

(S103) The control unit 120 determines whether it is a gain adjustment period. For example, when high-sensitivity shooting mode is turned on, a period of a predetermined number of frames set before the start of accumulation operation and a frame period in which unadded video frames are output after the start of accumulation operation are set as the gain adjustment period. If it is a gain adjustment period, processing proceeds to S104. On the other hand, if it is not a gain adjustment period, processing proceeds to S105.

(S104) The frame accumulation unit 122 outputs the video frames without adding them. The gain adjustment unit 124 increases the gain of the unadded video frames output from the frame accumulation unit 122. At this time, the gain adjustment unit 124 gradually increases the amount of amplification, and increases the gain of the unadded video frames so that, by the last frame period of the gain adjustment period, they have the same brightness level as the high-brightness video frame that is output subsequently. The gain-adjusted video frames are subjected to predetermined video processing by the video processing unit 125 and then output externally. When the processing of S104 is completed, the series of processes shown in Figure 10 ends.

(S105) The frame accumulation unit 122 reads out and outputs high-brightness video frames from the frame memories. Furthermore, while the frame accumulation unit 122 is storing video frames in one frame memory, it outputs video frames read out from the other frame memory.

(S106) The video frame output from the frame accumulation unit 122 in S105 is input to the second black level correction unit 123. The second black level correction unit 123 references the second correction data 123a and selects a correction value M2 corresponding to the number of accumulated frames. The second black level correction unit 123 then corrects the black level of the video frame using the selected correction value M2. The video frame after black level correction is output to the video processing unit 125 without undergoing gain adjustment by the gain adjustment unit 124, and is output externally after predetermined video processing is applied by the video processing unit 125. When the processing of S106 is completed, the series of processes shown in FIG. 10 ends.

(S107) The video frame after black level correction by the first black level correction unit 121 skips the processing by the frame accumulation unit 122, second black level correction unit 123, and gain adjustment unit 124, and is subjected to predetermined video processing by the video processing unit 125 before being output externally. When the processing of S107 is completed, the series of processes shown in Figure 10 ends.

The example in Figure 10 explains the operation when the high-sensitivity shooting mode is turned on, but the gain adjustment method during the second gain adjustment period described above changes when the high-sensitivity shooting mode is turned off. As already explained, during the second gain adjustment period, the gain of unadded video frames is increased while the amplification amount is gradually reduced, so that the amplification amount becomes 0 in the last frame period of the second gain adjustment period. This is different from the example in Figure 10, but those skilled in the art will easily understand the operation when the high-sensitivity shooting mode is turned off from the explanation in Figure 10.

The above describes the first embodiment of the present invention.

2. Second embodiment
Next, the configuration of an imaging device according to the second embodiment will be described with reference to Fig. 11. Fig. 11 is a block diagram schematically showing the configuration of the imaging device according to the second embodiment. Note that the imaging device 10a shown in Fig. 11 is an example of the imaging device according to the second embodiment. Furthermore, the imaging device 10a does not have the slow shutter function described above.

As shown in FIG. 11, the imaging device 10a has a camera head unit 11, a camera control unit 12a, and a display device 13. The configuration of the camera head unit 11 is the same as that of the camera head unit 11 in the first embodiment. The configuration of the display device 13 is also the same as that of the display device 13 in the first embodiment. Therefore, detailed descriptions of the configurations of the camera head unit 11 and the display device 13 will be omitted. Furthermore, of the elements that make up the camera control unit 12a, elements that have substantially the same functions as the elements that make up the camera control unit 12 in the first embodiment are assigned the same reference numerals.

The camera control unit 12a has a control unit 120, a first black level correction unit 121, a frame accumulation unit 122, a gain adjustment unit 124, and an image processing unit 125. The camera control unit 12a differs from the camera control unit 12 of the first embodiment in the presence or absence of a second black level correction unit 123. The camera control unit 12a does not include the second black level correction unit 123, and high-brightness image frames output from the frame accumulation unit 122 are also input to the image processing unit 125 without undergoing black level correction.

In the second embodiment, the second black level correction unit 123 is omitted, but the first black level correction unit 121 applies black level correction to each video frame before the frame accumulation unit 122, so sufficient video quality can be obtained depending on the application. Furthermore, omitting the second black level correction unit 123 makes the circuit configuration slightly simpler during implementation, which may contribute to reducing design and manufacturing costs.

The configuration of the imaging device according to the second embodiment has been described above.

3. Third Embodiment
Next, the configuration of an imaging device according to a third embodiment will be described with reference to Fig. 12. Fig. 12 is a block diagram schematically illustrating the configuration of an imaging device according to the third embodiment. Note that the imaging device 10b shown in Fig. 12 is an example of the imaging device according to the third embodiment. Furthermore, the imaging device 10b does not have the slow shutter function described above.

As shown in FIG. 12, the imaging device 10b has a camera head unit 11, a camera control unit 12b, and a display device 13. The configuration of the camera head unit 11 is the same as that of the camera head unit 11 in the first embodiment. The configuration of the display device 13 is also the same as that of the display device 13 in the first embodiment. Therefore, detailed descriptions of the configurations of the camera head unit 11 and the display device 13 will be omitted. Furthermore, of the elements that make up the camera control unit 12b, elements that have substantially the same functions as the elements that make up the camera control unit 12 in the first embodiment are assigned the same reference numerals.

The camera control unit 12b has a control unit 120, a first black level correction unit 121, a frame accumulation unit 122, a second black level correction unit 123, and a video processing unit 125. The difference between the camera control unit 12b and the camera control unit 12 of the first embodiment is the presence or absence of a gain adjustment unit 124. The camera control unit 12b does not include the gain adjustment unit 124, and unadded video frames output from the frame accumulation unit 122 during the gain adjustment period are also input to the video processing unit 125 without undergoing gain adjustment.

In the third embodiment, the gain adjustment unit 124 is omitted, so a gap in brightness level occurs when a high-brightness video frame is first output. However, the first black level correction unit 121 and the second black level correction unit 123 correct the black level before and after the frame accumulation unit 122, significantly improving video quality compared to conventional methods. By omitting the gain adjustment unit 124, the circuit configuration becomes slightly simpler during implementation, which may contribute to reducing design and manufacturing costs. Therefore, if the application does not notice a gap in brightness level, there is an advantage to adopting the configuration of the third embodiment.

The configuration of the imaging device according to the third embodiment has been described above.

<4. Notes>
The above-described first to third embodiments are intended to explain the technical ideas described in the following supplementary notes, and in other words, the technical ideas described in the following supplementary notes can be derived from the above-described explanations of the first to third embodiments. Of course, it should be noted that the descriptions in the supplementary notes shown here are not intended to limit the technical scope of the present invention, but are intended to aid in understanding the technical ideas disclosed in this specification.

[Appendix 1]
a signal processing unit that generates a video frame from an output signal of the image sensor;
a frame storage unit that generates a high-luminance image frame using the plurality of image frames output from the signal processing unit when the high-sensitivity image capturing function is turned on, and switches the image frame to be output to the high-luminance image frame;
a gain adjustment unit that adjusts the luminance level of the video frame output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance level gradually approaches the luminance level of the high-luminance video frame;
An imaging device comprising:
[Appendix 2]
The frame storage unit
a first frame memory and a second frame memory;
storing a plurality of video frames output from the signal processing unit in one of the first frame memory and the second frame memory to generate the high-luminance video frame, outputting the video frame in the other frame memory during the storage operation, and after outputting the high-luminance video frame, switching the frame memory used for storage with the frame memory used for output during the storage operation.
2. The imaging device according to claim 1.
[Appendix 3]
the number of video frames whose luminance levels are adjusted by the gain adjustment unit is determined based on the number of video frames used to generate the high-luminance video frame and a preset adjustment amount of luminance levels to be applied to one video frame.
3. The imaging device according to claim 1 or 2.
[Appendix 4]
The storage operation by the frame storage unit includes a process of adding together the video frame to be stored output from the signal processing unit and the video frame read from the frame memory used for storage, with a weighting factor applied thereto, and a process of writing the video frame after the addition into the frame memory used for storage.
3. The imaging device according to claim 2.
[Appendix 5]
the frame storage unit does not use invalid video frames among the video frames generated from the output signal of the image sensor for generating the high-luminance video frames;
An imaging device according to any one of appendices 1 to 4.
[Appendix 6]
a first black level correction unit that corrects a black level of a video frame output from the signal processing unit and outputs the corrected video frame to the frame storage unit;
the frame storage unit generates a high-luminance video frame using the plurality of video frames output from the first black level correction unit when the high-sensitivity shooting function is turned on;
An imaging device according to any one of appendices 1 to 5.
[Appendix 7]
a second black level correction unit that corrects a black level of the high-luminance video frame by using a second correction value corresponding to the number of video frames used to generate the high-luminance video frame;
7. The imaging device according to claim 6.
[Appendix 8]
the first black level correction unit corrects the black level of the video frame output from the signal processing unit by using a first correction value corresponding to the content of the filtering process applied to the video frame by the signal processing unit;
8. The imaging device according to claim 6 or 7.

[Appendix 9]
a signal processing unit that applies filtering to video frames generated from the output signals of the image sensor;
a first black level correction unit that applies black level correction to the video frame after the filtering process;
a frame storage unit that outputs a high-brightness image frame generated using the plurality of image frames after the black level correction is applied when the high-sensitivity image capturing function is turned on;
An imaging device comprising:
[Supplementary Note 10]
a second black level correction unit that corrects a black level of the high-luminance video frame by using a second correction value corresponding to the number of video frames used to generate the high-luminance video frame;
10. The imaging device according to claim 9.
[Appendix 11]
the first black level correction unit corrects a black level of the video frame output from the signal processing unit by using a first correction value corresponding to the content of the filtering process applied to the video frame by the signal processing unit;
11. The imaging device according to claim 9 or 10.
[Appendix 12]
the signal processing unit applies the filtering process to a pixel group in a rectangular region including a target pixel in the video frame, a pixel group arranged in a horizontal direction including the target pixel, or a pixel group arranged in a vertical direction including the target pixel, by multiplying the pixel group by a filter coefficient and performing an addition process;
An imaging device according to any one of appendices 9 to 11.

[Appendix 13]
When the high-sensitivity photographing function is turned on, a high-luminance image frame is generated using a plurality of image frames generated from the output signal of the image sensor, and the image frame to be output is switched to the high-luminance image frame;
adjusting the luminance levels of the video frames output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance levels gradually approach the luminance level of the high-luminance video frame;
A video processing method in which a computer executes a process including the steps of:

[Appendix 14]
applying a filtering process to a video frame generated from the output signal of the image sensor;
applying black level correction to the filtered video frames;
When the high-sensitivity shooting function is turned on, outputting a high-brightness video frame generated using the plurality of video frames after the black level correction is applied;
A video processing method in which a computer executes a process including the steps of:

[Appendix 15]
a frame storage unit that generates a high-luminance image frame using a plurality of image frames generated from an output signal of the image sensor when the high-sensitivity image capturing function is turned on, and switches the image frame to be output to the high-luminance image frame;
a gain adjustment unit that adjusts the luminance level of the video frame output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance level gradually approaches the luminance level of the high-luminance video frame;
A video processing device comprising:

[Appendix 16]
a first black level correction unit that applies black level correction to the video frame that has been generated from the output signal of the image sensor and that has been subjected to the filtering process;
a frame storage unit that outputs a high-brightness image frame generated using the plurality of image frames after the black level correction is applied when the high-sensitivity image capturing function is turned on;
A video processing device comprising:

The above describes preferred embodiments of the present invention with reference to the accompanying drawings, but the present invention is not limited to these examples. It is clear that a person skilled in the art could conceive of various modifications or alterations within the scope of the claims, and these naturally fall within the technical scope of the present invention.

REFERENCE SIGNS LIST 10, 10a, 10b Imaging device 11 Camera head unit 12, 12a, 12b Camera control unit 13 Display device 111 Optical system 112 Image sensor 113 Signal processing unit 120 Control unit 121 First black level correction unit 122 Frame accumulation unit 123 Second black level correction unit 124 Gain adjustment unit 125 Video processing unit

Claims (13)

a signal processing unit that generates a video frame from an output signal of the image sensor;
a frame storage unit that generates a high-luminance image frame using the plurality of image frames output from the signal processing unit when the high-sensitivity image capturing function is turned on, and switches the image frame to be output to the high-luminance image frame;
a gain adjustment unit that adjusts the luminance level of the video frame output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance level gradually approaches the luminance level of the high-luminance video frame;
the frame storage unit has a first frame memory and a second frame memory, stores a plurality of video frames output from the signal processing unit in one of the first frame memory and the second frame memory to generate the high-luminance video frame, outputs the video frame in the other frame memory during a storage operation, and, after the high-luminance video frame is output, swaps the frame memory used for storage with the frame memory used for output during the storage operation . a signal processing unit that generates a video frame from an output signal of the image sensor;
a frame storage unit that generates a high-luminance image frame using the plurality of image frames output from the signal processing unit when the high-sensitivity image capturing function is turned on, and switches the image frame to be output to the high-luminance image frame;
a gain adjustment unit that adjusts the luminance level of the video frame output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance level gradually approaches the luminance level of the high-luminance video frame;
wherein the number of video frames whose brightness levels are adjusted by the gain adjustment unit is determined based on the number of video frames used to generate the high-brightness video frame and a preset adjustment amount of brightness level to be applied to one video frame . The storage operation by the frame storage unit includes a process of adding together the video frame to be stored output from the signal processing unit and the video frame read from the frame memory used for storage, with a weighting factor applied thereto, and a process of writing the video frame after the addition into the frame memory used for storage.
The imaging device according to claim 1 . the frame storage unit does not use invalid video frames among the video frames generated from the output signal of the image sensor for generating the high-luminance video frames;
The imaging device according to any one of claims 1 to 3 . a first black level correction unit that corrects a black level of a video frame output from the signal processing unit and outputs the corrected video frame to the frame storage unit;
the frame storage unit generates a high-luminance video frame using the plurality of video frames output from the first black level correction unit when the high-sensitivity shooting function is turned on;
The imaging device according to any one of claims 1 to 4 . a signal processing unit that generates a video frame from an output signal of the image sensor;
a frame storage unit that generates a high-luminance image frame using the plurality of image frames output from the signal processing unit when the high-sensitivity image capturing function is turned on, and switches the image frame to be output to the high-luminance image frame;
a gain adjustment unit that adjusts the luminance level of the video frame output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance level gradually approaches the luminance level of the high-luminance video frame;
a first black level correction unit that corrects a black level of a video frame output from the signal processing unit and outputs the corrected video frame to the frame storage unit;
a second black level correction unit that corrects a black level of the high-luminance video frame by using a second correction value corresponding to the number of video frames used to generate the high-luminance video frame;
wherein when the high-sensitivity shooting function is turned on, the frame accumulation unit generates a high-luminance video frame using the plurality of video frames output from the first black level correction unit. the first black level correction unit corrects the black level of the video frame output from the signal processing unit by using a first correction value corresponding to the content of the filtering process applied to the video frame by the signal processing unit;
7. The imaging device according to claim 5 . When the high-sensitivity photographing function is turned on, a high-luminance image frame is generated using a plurality of image frames generated from the output signal of the image sensor, and the image frame to be output is switched to the high-luminance image frame;
adjusting the luminance levels of the video frames output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance levels gradually approach the luminance level of the high-luminance video frame;
A video processing method in which a computer executes a process including:
the computer has a first frame memory and a second frame memory;
the processing further includes storing the generated plurality of video frames in one of the first frame memory and the second frame memory to generate the high-luminance video frame, outputting the video frame in the other frame memory during the storage operation, and, after the high-luminance video frame is output, switching the frame memory used for storage with the frame memory used for output during the storage operation.
Image processing method. When the high-sensitivity photographing function is turned on, a high-luminance image frame is generated using a plurality of image frames generated from the output signal of the image sensor, and the image frame to be output is switched to the high-luminance image frame;
adjusting the luminance levels of video frames output during a plurality of frame periods before switching to the high-luminance video frame so as to gradually approach the luminance level of the high-luminance video frame, wherein the number of video frames whose luminance levels are adjusted is determined based on the number of video frames used to generate the high-luminance video frame and a preset adjustment amount of luminance level applied to one video frame;
A video processing method in which a computer executes a process including the steps of: when the high-sensitivity shooting function is turned on, correcting the black level of a video frame generated from an output signal of the image sensor, generating a high-luminance video frame using the corrected video frames, correcting the black level of the high-luminance video frame using a second correction value corresponding to the number of video frames used to generate the high-luminance video frame, and switching the video frame to be output to the high-luminance video frame;
adjusting the luminance levels of the video frames output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance levels gradually approach the luminance level of the high-luminance video frame;
A video processing method in which a computer executes a process including the steps of: a frame storage unit that generates a high-luminance image frame using a plurality of image frames generated from an output signal of the image sensor when the high-sensitivity image capturing function is turned on, and switches the image frame to be output to the high-luminance image frame;
a gain adjustment unit that adjusts the luminance level of the video frame output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance level gradually approaches the luminance level of the high-luminance video frame;
wherein the frame accumulation unit has a first frame memory and a second frame memory, accumulates the generated plurality of video frames in one of the first frame memory and the second frame memory to generate the high-luminance video frame, outputs the video frame in the other frame memory during an accumulation operation, and, after outputting the high-luminance video frame, swaps the frame memory used for accumulation with the frame memory used for output during the accumulation operation . a frame storage unit that generates a high-luminance image frame using a plurality of image frames generated from an output signal of the image sensor when the high-sensitivity image capturing function is turned on, and switches the image frame to be output to the high-luminance image frame;
a gain adjustment unit that adjusts the luminance level of the video frame output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance level gradually approaches the luminance level of the high-luminance video frame;
wherein the number of video frames whose luminance levels are adjusted by the gain adjustment unit is determined based on the number of video frames used to generate the high-luminance video frame and a preset amount of luminance level adjustment to be applied to one video frame. a first black level correction unit that corrects the black level of a video frame generated from an output signal of the image sensor;
a frame storage unit that generates a high-luminance image frame using the plurality of image frames output from the first black level correction unit when a high-sensitivity image capturing function is turned on, and switches the image frame to be output to the high-luminance image frame;
a second black level correction unit that corrects a black level of the high-luminance video frame by using a second correction value corresponding to the number of video frames used to generate the high-luminance video frame;
a gain adjustment unit that adjusts the luminance level of the video frame output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance level gradually approaches the luminance level of the high-luminance video frame;
A video processing device comprising: