JP7204441B2 - IMAGING DEVICE, CONTROL METHOD AND PROGRAM THEREOF - Google Patents

Description

The present invention relates to an imaging apparatus, its control method, and program.

2. Description of the Related Art Imaging apparatuses typified by digital cameras generate image data for viewing by performing development processing on raw image information (RAW data) acquired by an image sensor. Here, the development process refers to a series of processes in which debayer processing (demosaicing) is performed to convert the signals into signals composed of luminance and color difference, and then noise removal and optical distortion correction are performed on each signal.

When continuously shooting still images, an area having a size capable of storing a plurality of RAW data at one time is secured in the buffer memory, and the RAW data stored in the buffer memory are sequentially developed.

In addition, newly acquired RAW data is stored in the empty space of the buffer memory created by developing the RAW data.

In photographing a still image, development processing takes much time as compared to signal processing when acquiring RAW data from an image sensor. Therefore, if continuous shooting is continued, the amount of RAW data waiting for development processing will gradually increase, and the buffer memory will run out of space.

Due to the recent trend toward higher pixel counts such as 6K and 8K, the data size of RAW data per sheet has increased, and the time required for development processing has also increased. As a result, it becomes difficult for the buffer memory to become free, and continuous shooting is interrupted prematurely.

In recent years, there has also been developed a technique for reducing the data size of RAW data by compressing the RAW data itself. For example, the technique described in Patent Document 1 classifies RAW data into R, G1, B, and G2 planes. Then, according to Patent Document 1, each color plane is transformed into frequency components using a discrete wavelet transform to generate a plurality of subbands as shown in FIG. After that, encoding is performed for each subband, thereby efficiently compressing the RAW data.

A single wavelet transform produces four subbands LL, HL, LH, and HH. Here, L represents a low frequency range and H represents a high frequency range. The numbers before L and H represent decomposition levels. For example, 1HL represents a subband image of decomposition level=1, in which the horizontal direction is the high frequency component and the vertical direction is the low frequency component. Note that there is no limit to the number of wavelet transforms. Further, the second and subsequent wavelet transforms are recursively performed on the subband LL obtained by the previous wavelet transform. Therefore, there is only one sub-band LL when the wavelet transform is performed the planned number of times. Since FIG. 1 is an example in which wavelet transform is performed twice, seven subbands are generated.

By compressing the RAW data using the above technology to reduce the data size, more RAW data can be stored in the buffer memory during continuous shooting. It is possible to increase the number of photographs.

JP 2017-85319 A

However, in the case of compression using subband coding as described in Patent Document 1, it is necessary to pay attention to allocation of buffer memory to each subband.

When allocating buffer memory to each subband, it is desirable to refer to the code amount of each subband when RAW data is encoded. However, it is difficult to determine buffer memory allocation after encoding due to the control of the imaging apparatus, which starts shooting after specifying a memory address where encoded data is to be stored. Therefore, it is conceivable to use a value preset in the imaging device, for example, a ratio of coded data of each subband statistically obtained in advance from a large amount of image data, to allocate the buffer memory of each subband.

Storing encoded data in the buffer memory during continuous shooting will now be described with reference to FIGS.

FIG. 2A shows five RAW data obtained during continuous shooting, and FIG. 2B and FIG. showing.

FIGS. 2B and 2C show how buffer memories secured on the memory so as to store encoded data for five frames are assigned to each subband. FIGS. 2B and 2C store encoded data obtained by performing wavelet transform twice on one of the four planes R, G1, B, and G2 shown in FIG. ing.

When the ratio of the encoded data of each subband in the image data obtained by imaging is close to the preset value, as shown in FIG. can be used as intended. However, if the ratio of coded data in each subband of image data obtained by imaging is biased and deviates from the preset value, the buffer memory cannot be used efficiently. FIG. 2(C) shows a case where the size of encoded data of 1HL subband is larger than the preset value. FIG. 2(C) shows that the encoded data of the 1HL subband exceeds the allocated buffer memory when encoding the fourth image data. As shown in the figure, at the time of encoding the fourth image, the encoded data of the 1HL subband exceeds the buffer memory although the encoded data of the other subbands could be stored in the buffer memory. . Therefore, the encoded data for the fourth sheet cannot be recorded as reproducible encoded data. In addition, since the continuous shooting stops at the time when the image data for the fourth image is obtained, the fifth image cannot be shot.

SUMMARY OF THE INVENTION An object of the present invention is to provide a technique that enables more efficient use of the buffer memory during continuous shooting than ever before.

In order to solve this problem, for example, the imaging device of the present invention has the following configuration. i.e.
An imaging device for recording image data obtained from imaging means on a recording medium,
an encoding means for generating encoded data for each subband by frequency-converting image data obtained by imaging to obtain a plurality of subbands, and quantizing and entropy-encoding each subband;
a memory functioning as a buffer for storing the encoded data obtained by the encoding means in the recording medium;
a control means for recording the encoded data obtained by the encoding means on the recording medium through the memory;
In the continuous shooting mode, the control means comprises: a first buffer memory for temporarily storing a predetermined number of coded data of the first image group from the beginning; securing in the memory a second buffer memory for temporarily storing encoded data of a second image group;
In the first buffer memory, an area having a size corresponding to a preset ratio is assigned to each subband, and encoded data of each subband obtained from the first image group is temporarily stored in each of the areas. store in
In the second buffer memory, an area having a size corresponding to the code amount ratio of each subband of the first image group is allocated to each subband, and each of the areas is obtained from the second image group. It is characterized by temporarily storing encoded data of each subband.

According to the present invention, it is possible to efficiently use the buffer memory during continuous shooting.

The figure which shows the example of subband generation by wavelet transform. FIG. 4 is a diagram showing an example of buffer memory and storage of encoded data; FIG. 4 is a diagram showing an example of separating RAW data into plain images; 1 is a block configuration diagram of an imaging device according to a first embodiment; FIG. FIG. 5 is a diagram showing an example of setting a quantization step size; 4A and 4B are diagrams showing a usage example of a buffer memory of the imaging apparatus according to the first embodiment; FIG. 4 is a flow chart showing a processing procedure of the imaging device according to the first embodiment; FIG. 10 is a diagram showing a usage example of a buffer memory of an imaging device according to the second embodiment; FIG. 2 is a block configuration diagram of an imaging device according to a second embodiment; 9 is a flow chart showing the processing procedure of the imaging device according to the second embodiment.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Note that the configurations in the embodiments shown below are merely examples, and the present invention is not limited to the illustrated configurations.

[First embodiment]
FIG. 4 is a block configuration diagram of an imaging device 400 according to the first embodiment. The imaging apparatus 400 includes a CPU 401, an optical unit 402, an imaging unit 403, a plane transform unit 404, a discrete wavelet transform unit 405, a quantization unit 406, an entropy coding unit 407, an address setting unit 408, . It has a memory I/F 409 , a memory 410 , a recording processing unit 411 and a recording medium 412 .

The CPU 401 has a memory (not shown) that stores control programs executed by the CPU, and controls each component of the imaging apparatus 400 .

The optical unit 402 is composed of a lens optical system capable of optical zoom including an optical lens, an aperture, a focus control unit, and a lens driving unit. do. The imaging unit 403 has an imaging sensor having filters for each of the R, G, and B color components arranged on its surface, and generates an electric signal corresponding to the amount of light incident through the filter for each color component. The imaging unit 403 also has an A/D converter that converts an electrical signal from the imaging sensor into digital data, and supplies the obtained digital data to the plane conversion unit 404 . Note that the imaging sensor is an imaging element such as a CCD image sensor or a CMOS sensor. It is also assumed that the filters for each color component arranged on the surface of the imaging sensor have a Bayer array. Therefore, the digital data output from the imaging unit 403 is also Bayer array image data, and the pixel data of each color component is arrayed before development processing. Therefore, hereinafter, image data represented by pixel data in the Bayer array is referred to as RAW data.

A plane conversion unit 404 converts the RAW data from the imaging unit 403 into a plurality of planes, each of which is composed of data of a single component. Specifically, the plane conversion unit 404 converts the RAW data into an R plane consisting of only R component pixels, a B plane consisting of only B component pixels, and a G1 plane consisting of only G1 component pixels. , and G2 planes composed of only G2 component pixels. Then, the plane transformation unit 404 further transforms the R plane, B plane, G1 plane, and G2 plane obtained by transformation into a plurality of planes, and supplies them to the discrete wavelet unit 405 . For example, the plane conversion unit 404 generates a Y plane representing luminance and Co, Cg, and Dg planes representing three color differences from the R plane, B plane, G1 plane, and G2 plane according to the following equations.
Y={R+B+G1+G2}/4
Co = RB
Cg={G0+G1}/2-{R+B}/2
Dg=G0-G1
Note that the plane conversion unit 404 may supply the R plane, B plane, G1 plane, and G2 plane obtained from the RAW data to the discrete wavelet unit 405 . Also, the plane conversion unit 404 may be equipped with a function or circuit for executing a plurality of calculation formulas, and the CPU 401 may select the calculation formula to be used.

The discrete wavelet transform unit 405 receives each of the planes supplied from the plane transform unit 404 as multi-valued image data of a single color component (monochrome), performs discrete wavelet transform, which is one of frequency transforms, and converts the planes into a plurality of sub-pixels. generate a band. Then, the discrete wavelet transform unit 405 supplies the generated multiple subbands to the quantization unit 406 in a preset order. Note that the CPU 401 sets the number of decompositions (number of discrete wavelet transform executions) by the discrete wavelet transform unit 405 .

The quantization unit 406 quantizes the transform coefficients forming the subbands of each plane sent from the discrete wavelet transform unit 405 based on a predetermined quantization step size. Here, the predetermined quantization step size may be set for each subband by the CPU 401 as shown in FIG. 5, for example.

FIG. 5 is a diagram assuming that there are Y, Co, Cg, and Dg planes, and shows an example in which wavelet transform is performed once for each plane.

FIG. 5 shows an example in which the CPU 401 sets a base quantization step size of “4” to the quantization section 406 . The quantization unit 406 has a preset weighting factor for each subband of each plane of Y, Co, Cg, and Dg, and multiplies the quantization step size by the weighting factor to obtain each Calculate the actually used quantization step size for each subband of the plane. Note that the CPU 401 may individually set the quantization step size used in the quantization section 406 .

The entropy coding unit 407 performs entropy coding using a Huffman code, an arithmetic code, or the like on the transform coefficients after quantization of each subband by the quantization unit 406, and generates coded data of each subband. .

The entropy coding unit 407 also notifies the address setting unit 408, which will be described later, of the size (code amount) of the coded data of each subband.

Also, the entropy coding unit 407 stores the coded data of each subband in the memory 410 based on the address specified by the address setting unit 408 which will be described later. A part of this memory 410 is used as a buffer for temporarily storing encoded data to be stored in the recording medium 412 .

The address setting unit 408 sets an address that the entropy coding unit 407 refers to when writing the coded data of each subband to the memory 410 .

In the present embodiment, the address setting unit 408 uses preset information, such as the ratio of encoded data of each subband statistically obtained from a large amount of image data in advance, in the setting of the first buffer memory, which will be described later. , to set the address based on.

Also, in this embodiment, the address setting unit 408 sets an address based on the size of the encoded data of each subband sent from the entropy encoding unit 407 in setting the second buffer memory, which will be described later. Also, the address setting unit 408 may be incorporated as one of the functions of the control program of the CPU 401 . Details of address setting will be described later.

The memory I/F 409 arbitrates access requests to the memory 410 from each processing unit including the entropy coding unit 407 , that is, performs read/write control for the memory 410 .

The memory 410 is a storage area composed of a volatile memory for storing various data including the coded data of each subband sent from the entropy coding section 407 .

The recording processing unit 411 records the encoded data stored in the memory 410 onto the recording medium 412 according to the recording command from the CPU 401 . A recording medium 412 is a recording medium configured with a non-volatile memory, and is typically a removable memory card typified by an SD card.

Next, allocation of buffer memory on the memory 410 by the address setting unit 408 will be described with reference to FIG.

FIG. 6 shows allocation of buffer memory in memory 410 and allocation to each subband.

In this embodiment, a first buffer memory for storing the RAW data of the first image during continuous shooting, and RAW data for the second and subsequent images (in the example of FIG. A second buffer memory for storing the RAW data of the first to fifth sheets) is secured on the memory 410, respectively. It is assumed that the maximum number of images for continuous shooting is set from a preset menu screen.

Here, it is assumed that the encoded data size per sheet is uniformly 4000 KB.

Also, to simplify the explanation, allocation to subbands for one of the four planes separated from the RAW data will be explained. In this embodiment, the above 4000 KB is evenly allocated to four planes, and the encoded data size per sheet is 4000 KB/4=1000 KB.

In this embodiment, the first buffer memory allocates the buffer area based on the area ratio of each subband.

Subband division by the discrete wavelet transform halves the horizontal and vertical coefficients each time the decomposition level increases by one level, so the area is reduced to 1/4. Therefore, each size of 2LL/2HL/2LH/2HH has a relationship of 1/4 with respect to each size of 1HL/1LH/1HH.

In the present embodiment, allocation to each subband is not performed based on the size of the encoded data that is actually generated. It is desirable to have a margin against Encoded data can be stored reliably by having a margin.

In this embodiment, the size of the first buffer memory has a margin of 20% with respect to 1000 KB, and 1200 KB is secured.

Based on the above, allocation of the first buffer memory to each subband is performed as follows.

<Proportion of each subband in allocation of the first buffer memory>
2LL: 2HL: 2LH: 2HH: 1HL: 1LH: 1HH
= 1:1:1:1:4:4:4
<Allocation of first buffer memory>
2LL subband: 1200 KB x 1 ÷ (1 + 1 + 1 + 1 + 4 + 4 + 4) =
1200KB x 1/16 = 75KB
2HL subband: 1200KB x 1/16 = 75KB
2 LH sub-band: 1200 KB x 1/16 = 75 KB
2HH subband: 1200KB x 1/16 = 75KB
1HL subband: 1200KB x 4/16 = 300KB
1 LH subband: 1200 KB x 4 ÷ 16 = 300 KB
1HH subband: 1200KB x 4/16 = 300KB

Further, in carrying out the present embodiment, when allocating the first buffer memory, the coded data of each subband obtained statistically in advance from a large amount of image data is used instead of the area ratio of each subband. It may be based on the ratio of Further, in the embodiment, an example has been given in which the first buffer is evenly allocated to each of the four planes generated from RAW data. I don't mind.

The second buffer memory is assigned to each subband based on the encoded data generated from the first piece of RAW data during continuous shooting stored in the first buffer memory. This is the result of focusing on the fact that the tendency of the amount of code obtained from the RAW data of the second and subsequent sheets is the same as that of the first sheet of RAW image data.

In the example of FIG. 6, the coded data size of each subband stored in the first buffer memory is as follows.

<Code amount of leading RAW data>
2LL sub-band: 65KB
2HL subband: 70KB
2LH subband: 65KB
2HH subband: 50KB
1HL subband: 280KB
1LH subband: 240KB
1HH subband: 230KB

When the size of the encoded data of each subband is converted into a ratio, it is as follows. Here, the coded data size of each subband is divided by 50 KB, which is the minimum value of the coded data size of each subband.

<Ratio of Code Amount of Each Subband in Leading RAW Data>
2LL: 2HL: 2LH: 2HH: 1HL: 1LH: 1HH
=1.3:1.4:1.3:1.0:5.6:4.8:4.6
In the explanation here, the second buffer memory secures a size that can store RAW data for four shots of the second to fifth shots in continuous shooting of five shots, so the overall size of the second buffer memory is is 1000 KB×4 sheets=4000 KB.

In addition, in order to reliably store RAW data for four sheets, a margin is provided as in the first buffer memory. Here, assuming that the second buffer memory has a 20% margin like the first buffer memory, the overall size of the second buffer memory is 4000 KB×1.2=4800 KB. The allocation to each subband, together with the above ratios, is as follows.

<Second buffer memory allocation based on first sheet>
2LL subband: 4800 KB x 1.3 ÷ (1.3 + 1.4 + 1.3 + 1.0 + 5.6 + 4.8 + 4.6) = 4800 KB x 1.3 ÷ 20 = 312 KB
2HL subband: 4800 KB x 1.4 ÷ 20 = 336 KB
2 LH sub-band: 4800 KB x 1.3 ÷ 20 = 312 KB
2HH subband: 4800 KB x 1.0 ÷ 20 = 240 KB
1HL subband: 4800 KB x 5.6 ÷ 20 = 1344 KB
1 LH subband: 4800 KB x 4.8 ÷ 20 = 1152 KB
1HH subband: 4800 KB x 4.6 ÷ 20 = 1104 KB

In continuous shooting of still images, temporally adjacent images have a high correlation, so there is a high possibility that the size of coded data of each subband is close between temporally adjacent RAW data. Therefore, as shown in FIG. 6, when allocating a buffer memory for storing a plurality of pieces of RAW data, referring to the encoding result of the first piece makes it possible to use the buffer memory without waste. It becomes possible.

Next, details of the operation of the imaging device 400 according to the first embodiment will be described with reference to the flowchart shown in FIG.

As an operation here, storing the encoded RAW data in the buffer memory of the memory 410 and recording the encoded RAW data already stored in the buffer memory on the recording medium 412 are executed simultaneously. shall be The processing described below is the main portion of the former storage processing.

Also, when storing encoded data, if there is a possibility that the already stored encoded data will be overwritten, the encoded data corresponding to that will not be output.

First, the CPU 401 secures a first buffer memory and a second buffer memory on the memory 410 (S701).

Subsequently, in S702, the CPU 401 controls the address setting unit 408, assigns each subband to the first buffer memory based on the area ratio of the subband, and sets it as the output destination of the entropy coding unit 407. do.

Next, in S703, the CPU 401 acquires the first RAW data from the imaging unit 403, controls the plane transform unit 404, the discrete wavelet transform unit 405, the quantization unit 406, and the entropy coding unit 407, Encoded data for each of a plurality of subbands of each plane is generated and written to the first buffer memory. Note that the top address when the entropy encoding unit 407 stores in the first buffer memory is the address set by the address setting unit 408 .

Subsequently, in S704, the CPU 401 controls the address setting unit 408, and based on the code amount (encoded data size) of each subband sent from the entropy encoding unit 407, the RAW data of the second and subsequent sheets. A second buffer memory in which the coded data of is stored is allocated to each subband. The address setting unit 408 sets the assigned address of each subband to the entropy encoding unit 407 as an output destination.

Next, in step S705, the CPU 401 causes the imaging unit 403 to acquire RAW data (raw data of the second and subsequent frames) again, and converts the plane transform unit 404, the discrete wavelet transform unit 405, the quantization unit 406, and the entropy code. It causes the encoding unit 407 to start generating and storing encoded data for each of a plurality of subbands.

At the same time, the CPU 401 starts writing the encoded data in the first or second buffer to the recording medium 412 in S706.

Subsequently, in S707, the CPU 401 causes the encoded data generated in S705 to overwrite the encoded data recorded in the second buffer memory that has already been encoded and has not been written to the recording medium 412. determine whether or not to

If overwriting is determined, in S708 the CPU 401 suspends the output of the encoded data to the buffer, and confirms that the encoded data recorded in the second buffer memory has been written to the recording medium. After that, the writing of the RAW encoded data is restarted.

If it is determined in S707 that the encoded data will not be overwritten, or if there is a free space for storage, the CPU 401 advances the process to S709. In S709, the CPU 401 causes the encoded data from the entropy encoding unit 407 to be stored in the second buffer memory.

In S710, the CPU 401 determines whether continuous shooting is continuing. Continuing means that the user is pressing the release button and the maximum number of continuous shots (five in the embodiment) has not been reached. When the number of continuous shots reaches the upper limit, the continuous shooting process is terminated even if the release button is pressed. If it is determined that continuous shooting is being continued, the CPU 401 subsequently returns the process to S705 in order to start encoding the acquired RAW data.

On the other hand, if it is determined that continuous shooting has ended, the CPU 401 advances the process to S711, writes all the encoded data in the buffer memory that has not been output to the recording medium 412, and ends the process.

According to the first embodiment described above, during continuous shooting, when the encoded data is recorded on the recording medium via the buffer memory for temporarily storing the encoded data, even if the generated code amount of the sub-band is uneven, also allows for efficient buffer memory utilization.

In the above embodiment, the upper limit of the number of images for continuous shooting is set in advance, but the upper limit of the number of images may not necessarily be set. As in the above-described embodiment, when data is written to the recording medium 412 while storing encoded data in the buffer memory during continuous shooting, an empty area is generated in the buffer memory due to the writing. Therefore, when acquiring the next RAW data, as long as there is an empty area for storing the code amounts of all subbands of the RAW data for one frame, it is possible to continue continuous shooting by pressing the release button. In this case, the size of each subband secured in the second buffer memory may be determined according to the code amount ratio of each subband of the first RAW data, as described above. Then, in S710, it is determined whether or not the release button is pressed and the estimated "code amount+margin" of all subbands of the next RAW data exists. Then, if both conditions are satisfied, the CPU 401 may return the processing to S705.

In the imaging apparatus 400 according to the first embodiment, when allocating the second buffer memory, the size of the encoded data of the HL/LH/HH subbands of the lowest resolution level is changed to that of the high resolution level. It may be used to allocate buffer memory for the HL/LH/HH sub-bands.

In this case, from the code amount of 1HL/1LH/1HH, a value corresponding to the code amount of 2HL/2LH/2HH is calculated using the area ratio as described below, and is used for buffer setting.

<Code amount of sub-band with decomposition level = 1>
1HL subband: 280KB
1LH subband: 240KB
1HH subband: 230KB
<Value corresponding to code amount of sub-band with decomposition level = 2>
2 HL sub-bands: size of 1 HL sub-band ÷ 4 = 280 KB ÷ 4 = 70 KB
2 LH sub-bands: Size of 1 LH sub-band ÷ 4 = 240 KB ÷ 4 = 60 KB
2HH subband: Size of 1HH subband/4=230KB/4=57.5KB

In addition, in the overall size of the first and second buffer memories of the present embodiment, the encoded data size per sheet is equally divided by four planes to secure it. Alternatively, it may be based on the coded data size ratio for each plane, which is determined in advance by statistically obtaining from a large amount of image data.

In this case, it is assumed that the plane conversion unit 404 converts the R plane, B plane, G1 plane, and G2 plane into one luminance plane and three chrominance planes as in Patent Document 1, for example. ing.

In the following, since the luminance plane has a greater influence on the image than the chrominance plane, buffers are allocated at a larger ratio than the chrominance plane.

<Ratio of plane data size>
Plane 0: Plane 1: Plane 2: Plane 3
= 2:1:1:1
<Buffer memory allocation for each plane>
Plane 0: 4800 KB x 2 ÷ (2 + 1 + 1 + 1) = 1920 KB
Plane 1: 4800 KB x 1 ÷ (2 + 1 + 1 + 1) = 960 KB
Plane 2: 4800 KB x 1 ÷ (2 + 1 + 1 + 1) = 960 KB
Plane 3: 4800 KB x 1 ÷ (2 + 1 + 1 + 1) = 960 KB
In this embodiment, only the first image is recorded in the first buffer memory. may be used to allocate memory areas between sub-bands of the second buffer memory.

As described above, according to this embodiment, the size of the area of each subband allocated in the second buffer memory secured in the memory 410 is set to the size of each subband actually stored in the first buffer memory. , the second buffer memory can be used without waste, and continuous shooting can be prevented from being interrupted more than before.

Note that in the above embodiment, the first buffer memory is reserved in the memory 410 for the first image in the continuous shooting mode, and the second buffer memory is reserved for the second and subsequent images that are subsequently captured. An example is shown in which the memory 410 is reserved for an image for . However, the first buffer memory is reserved for the first group of images of a predetermined number from the beginning in the continuous shooting mode, and the second buffer memory is reserved for the second group of images to be captured subsequent to the first group of images. may be reserved for the group of images. By securing the first buffer memory for a plurality of images, the ratio of the actual code amount of each band has the same meaning as the ratio of the average value corresponding to the number of images, and the second buffer memory is used. It is possible to estimate the size of the area to be allocated in the memory 410 with higher accuracy, and the waste area of the memory 410 can be made smaller.

[Second embodiment]
A second embodiment will be described. The imaging apparatus according to the second embodiment assumes that the imaging parameters change and the pattern changes. Here, the imaging parameter indicates an exposure value, an F value, an ISO value, and the like.

If the F-number changes at any time during continuous shooting, the degree of resolution of the subject changes, and as a result, it is conceivable that the patterns of temporally adjacent images will change. A case in which the F value changes during continuous shooting is assumed to be continuous shooting in the shutter priority mode.

Also, if the exposure value changes as needed during continuous shooting, the brightness of temporally adjacent images may change, and as a result, the patterns of temporally adjacent images may change. A case in which the exposure value changes during continuous shooting is assumed to be continuous shooting in the exposure bracket shooting mode.

Also, if the ISO value changes at any time during continuous shooting, the amount of noise in temporally adjacent images may change, and as a result, the pattern may change in temporally adjacent images. A case in which the ISO value changes during continuous shooting is assumed to be continuous shooting in the ISO bracket shooting mode.

As described above, if the imaging parameters such as the exposure value, F value, and ISO value change during continuous shooting, there is a high probability that the pattern between temporally adjacent images will change, and the correlation between the images will be lost. Therefore, the size ratio of code data for each subband between adjacent images also changes.

Therefore, by utilizing the fact that temporally adjacent images have a high degree of pattern correlation as described in the first embodiment, allocation to each subband can be efficiently performed based on the encoding result of the first image. It becomes difficult to establish the premise of doing so. In other words, in the case where the imaging parameters such as the exposure value, the F value, and the ISO value change during continuous shooting, if the buffer memory is allocated as described in the imaging apparatus of the first embodiment, then in FIG. Conversely, as shown, the number of continuous shots may decrease.

In the example of FIG. 8A, among the encoded data of each subband of the first piece of RAW data, large size encoded data is generated in the 1HL subband, whereas the second and subsequent pieces of RAW data RAW data represents an example in which large-sized encoded data is generated in one LH subband.

Here, allocation of the second buffer memory in the example of FIG. 8A is the same as that used in the description of the imaging apparatus of the first embodiment of the present invention.

<Second buffer memory allocation based on first sheet>
2LL subband: 312KB
2HL subband: 336KB
2LH subband: 312KB
2HH subband: 240KB
1HL subband: 1344KB
1LH subband: 1152KB
1HH subband: 1104KB

In the example of FIG. 8(A), according to the encoding result of the first sheet, more than 1 HL sub-band is allocated, but since the encoded data of 1 LH sub-band becomes large in the second and subsequent sheets, 4 At the time of the first sheet, the buffer memory to which 1 LH subband is allocated is exceeded.

Here, as shown in FIG. 8B, by assigning the second buffer memory in the same manner as the first buffer memory, it is possible to suppress the decrease in the number of continuous shots.

In the example of FIG. 8B, by allocating the second buffer memory at the same ratio as that of the first buffer memory, if the allocation is made according to the encoding result of the first one, the recording of the fourth sheet is performed. This indicates that recording on the fourth sheet is possible, whereas recording on the fourth sheet is not possible.

The setting for allocating the second buffer memory at the same ratio as the first buffer memory is as follows.

Numerical values used in the following description are the same as those used in the description of the imaging apparatus of the first embodiment.
<Proportion of each subband in second buffer memory allocation>
2LL: 2HL: 2LH: 2HH: 1HL: 1LH: 1HH
= 1:1:1:1:4:4:4
<Allocation of second buffer memory>
2LL subband: 4800 KB x 1 ÷ (1 + 1 + 1 + 1 + 4 + 4 + 4) =
4800KB x 1/16 = 300KB
2HL subband: 4800KB x 1/16 = 300KB
2LH sub-band: 4800KB x 1/16 = 300KB
2HH subband: 4800KB x 1/16 = 300KB
1HL subband: 4800KB x 4/16 = 1200KB
1 LH subband: 4800 KB x 4/16 = 1200 KB
1HH subband: 4800KB x 4/16 = 1200KB

The configuration of the imaging apparatus according to the second embodiment will be described with reference to the block diagram of FIG.

In addition, in the configuration of the imaging device 900 of the second embodiment, the same reference numerals are given to the same components as those of the imaging device 400 of the first embodiment, and the description thereof will be omitted.

A user setting unit 901 allows the user to set imaging parameters to be monitored when acquiring temporally adjacent images, which will be described later. In the second embodiment, the shooting parameters to be monitored are set through the setting of the continuous shooting mode. Modes for continuous shooting include the aforementioned shutter priority mode, exposure bracket shooting mode, ISO bracket shooting mode, and the like.

When the shutter priority mode is set by the user setting unit 901, the F number is assumed to change during continuous shooting, so the shooting parameter to be monitored is the F number. Also, when the user setting unit 901 sets the exposure bracket shooting mode, the exposure value is assumed to change during continuous shooting, so the shooting parameter to be monitored is the exposure value. Also, when the ISO bracket shooting mode is set in the user setting unit 901, the ISO value is set as the shooting parameter to be monitored because it is assumed that the ISO value will change during continuous shooting.

The continuous shooting mode set by the user setting unit 901 is notified to the CPU 401 . The user setting unit 901 can be realized by, for example, a touch panel, push buttons, dials, or the like.

The imaging unit 902 includes an imaging element such as a CCD image sensor or a CMOS sensor that converts optical information sent from the optical unit 402 into an electrical signal, and converts the electrical signal obtained by the imaging element into a digital signal to obtain RAW data. is sent to the plane conversion unit 404 .

The imaging unit 902 also monitors the shooting parameters notified from the user setting unit 901 via the CPU 401 during continuous shooting, and sends information as to whether the shooting parameters have changed to the address setting unit 903 .

The address setting unit 903 sets addresses when the entropy coding unit 407 writes the coded data of each subband to the memory 410 .

In this embodiment, the address setting unit 903 uses preset information, such as the ratio of coded data of each subband statistically obtained in advance from a large amount of image data, in the setting of the first buffer memory. set the address based on

In this embodiment, under the control of the CPU 401, the address setting unit 903 sets the second buffer memory (to be described later) based on the notification of whether or not there is a change in shooting parameters during continuous shooting by the imaging unit 902. , and set the address as follows:

When the imaging unit 902 notifies that there is no change in shooting parameters during continuous shooting, the address setting unit 903 sets the size of the encoded data of each subband of the initial RAW data sent from the entropy encoding unit 407. , the storage address of the second buffer memory is set. That is, it is the same as the first embodiment.

Also, when the imaging unit 902 notifies that there is a change in shooting parameters during continuous shooting, the address setting unit 903 sets an address for the second buffer memory based on preset information. Note that the address setting unit 903 may be incorporated as one of the functions of the control program of the CPU 401 .

Next, the details of the operation of the imaging device 900 according to the second embodiment will be described based on the flowchart shown in FIG. In the operation of the imaging apparatus 900 according to the second embodiment, the same reference numerals are given to the same parts as those of the imaging apparatus according to the first embodiment, and the explanation will be simplified.

First, in S<b>1001 , the CPU 401 determines shooting parameters to be monitored during continuous shooting based on the continuous shooting mode input via the user setting unit 901 . Subsequently, in S701 to S703, the CPU 401 secures a first buffer memory and a second buffer memory in the memory 410, and instructs the address setting unit 903 to specify each sub-band in the first buffer memory. Allocate a region with a preset size. Then, the first RAW data is encoded and stored in the first buffer memory.

Next, in S<b>1002 , the CPU 401 acquires the RAW data of the second (or subsequent) image captured by the imaging unit 902 . Then, in subsequent S1003, the CPU 401 determines whether or not the shooting parameters when acquiring the RAW data of the second frame have changed from the shooting parameters when acquiring the first RAW data.

When determining that the shooting parameters have changed, the CPU 401 branches the process to S1004. In S1004, CPU 401 allocates the storage area (size) of each subband to the second buffer memory according to the same ratio as that of the first buffer memory, and instructs address setting section 903 to store each subband. The storage destination address of the encoded data is set to be the start address of the allocated area.

If it is determined that the shooting parameters do not change (fixed), the CPU 401 branches the process from S1003 to S704. This S704 is the same as in the first embodiment. That is, the CPU 401 controls the address setting unit 903, and based on the code amount (coded data size) of each subband of the first RAW data sent from the entropy coding unit 407, , an area having a size for storing the encoded data of each subband of the second and subsequent RAW data. Then, CPU 401 sets the storage destination address of the encoded data of each sub-band to address setting section 903 so as to be the head address of the allocated area.

Subsequently, in steps S705 to S711, the CPU 401 generates coded data for each of a plurality of subbands and stores coded data in the first and second buffer memories for acquisition of RAW data during continuous shooting. Move (write) to the recording medium 412 .

By using the imaging apparatus according to the second embodiment, in the case of continuous shooting in a mode in which the pattern is likely to change, the decrease in the number of continuous shots due to setting the buffer memory based on the first one is suppressed. can do Further, in the present embodiment, the setting of the second buffer memory is changed according to changes in shooting parameters during continuous shooting, but it may be changed based on the setting of the mode during continuous shooting. In this case, when the continuous shooting mode is set in any of the shutter priority mode, exposure bracket shooting mode, and ISO bracket shooting mode, the second buffer memory is set with statistical values, otherwise the first RAW data is stored. It is set based on the encoding result.

(Other examples)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

401 CPU 402 optical unit 403 imaging unit 404 plane transform unit 405 discrete wavelet transform unit 406 quantization unit 407 entropy coding unit 408, 903 address setting unit 409 Memory I/F 410 Memory 411 Recording processing unit 413 Recording medium 901 User setting unit

Claims (10)

An imaging device for recording image data obtained from imaging means on a recording medium,
an encoding means for generating encoded data for each subband by frequency-converting image data obtained by imaging to obtain a plurality of subbands, and quantizing and entropy-encoding each subband;
a memory functioning as a buffer for storing the encoded data obtained by the encoding means in the recording medium;
a control means for recording the encoded data obtained by the encoding means on the recording medium through the memory;
In the continuous shooting mode, the control means comprises: a first buffer memory for temporarily storing a predetermined number of coded data of the first image group from the beginning; securing in the memory a second buffer memory for temporarily storing encoded data of a second image group;
In the first buffer memory, an area having a size corresponding to a preset ratio is assigned to each subband, and encoded data of each subband obtained from the first image group is temporarily stored in each of the areas. store in
In the second buffer memory, an area having a size corresponding to the code amount ratio of each subband of the first image group is allocated to each subband, and each of the areas is obtained from the second image group. An imaging device characterized by temporarily storing coded data of each subband. 2. The imaging apparatus according to claim 1, wherein the first image group is the first image in the continuous shooting mode, and the second image group is the second and subsequent images in the continuous shooting mode. conversion means for converting the Bayer array RAW data obtained from the imaging means into four planes, each of which is composed of a single component;
3. The imaging apparatus according to claim 1, wherein the encoding means encodes each plane obtained by the transforming means. The frequency transform in the encoding means is a discrete wavelet transform,
4. The imaging apparatus according to any one of claims 1 to 3, wherein the size of each subband region in the first buffer memory is determined based on the area of the subband. The control means is
determining means for determining which of a first continuous shooting mode in which shooting parameters are fixed and a second continuous shooting mode in which shooting parameters change each time an image is taken is set;
When it is determined that the first continuous shooting mode has been set, the second buffer memory stores an area having a size corresponding to the code amount ratio of each subband of the first image group. temporarily storing the encoded data of each subband obtained from the second image group in each of the regions,
When it is determined that the second continuous shooting mode is set, an area having a size according to the same ratio as that of the first buffer memory is allocated to each subband in the second buffer memory, and 4. The imaging apparatus according to any one of claims 1 to 3, wherein the encoded data of each subband obtained from the second image group is temporarily stored in each. 6. The imaging apparatus according to claim 5, wherein the imaging parameter that changes each time imaging is performed in the second continuous shooting mode is an F-number. 6. The imaging apparatus according to claim 5, wherein the imaging parameter that changes each time imaging is performed in the second continuous shooting mode is an exposure value. 6. The imaging apparatus according to claim 5, wherein the imaging parameter that changes each time imaging is performed in the second continuous shooting mode is an ISO value. A control method for an imaging device for recording image data obtained from imaging means on a recording medium, comprising:
an encoding step of generating encoded data for each subband by frequency-converting image data obtained by imaging to obtain a plurality of subbands, and quantizing and entropy-encoding each subband;
a memory functioning as a buffer for storing encoded data obtained in the encoding step in the recording medium;
a control step of recording the encoded data obtained in the encoding step on the recording medium via the memory;
In the control step, in the continuous shooting mode, a first buffer memory for temporarily storing a predetermined number of coded data of the first image group from the beginning; securing in the memory a second buffer memory for temporarily storing encoded data of a second image group;
In the first buffer memory, an area having a size corresponding to a preset ratio is assigned to each subband, and encoded data of each subband obtained from the first image group is temporarily stored in each of the areas. store in
In the second buffer memory, an area having a size corresponding to the code amount ratio of each subband of the first image group is allocated to each subband, and each of the areas is obtained from the second image group. A control method for an imaging device characterized by temporarily storing coded data of each subband. A program that is read and executed by a computer to cause the computer to perform each step of the method according to claim 9.

Images