JP5671900B2 - Imaging apparatus, imaging method, and program - Google Patents

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program that continuously capture still images.

  In recent years, an imaging apparatus having a function of continuously capturing still images (continuous shooting) has been commercialized. In this imaging device, if the subject image moves fast, the continuous shooting interval is shortened, and if the subject image moves slowly, the continuous shooting interval is lengthened, so that continuous shooting without excess or deficiency suitable for the movement of the subject image is achieved. It becomes possible.

  Therefore, various imaging devices are disclosed that can change the continuous shooting speed (number of times of imaging per unit time during continuous shooting of still images) according to the movement of the subject image.

  For example, there has been disclosed a camera or the like that performs continuous shooting after detecting the motion of a subject image and determining a continuous shooting speed based on the detected motion (see, for example, Patent Documents 1 and 2).

  In addition, a moving image of a subject image is temporarily recorded, and after recording, a continuous image is obtained by selecting a still image from the recorded moving images at intervals suitable for the movement of the subject image. An imaging device is disclosed (for example, see Patent Document 3).

Japanese Patent Laid-Open No. 10-3110 JP 2007-134991 A JP 2001-42391 A

  However, since the cameras disclosed in Patent Documents 1 and 2 detect the movement of the subject image and perform continuous shooting after determining the continuous shooting speed, the time for detecting the movement of the subject image and the time for continuous shooting are used. A time lag occurs. If the movement of the subject image changes during this time lag, the continuous shooting speed may not necessarily be suitable for the movement of the subject image.

  In addition, the apparatus disclosed in Patent Document 3 requires a large-capacity buffer for temporarily storing moving images. In addition, since a process for selecting a still image is performed after the fact, it takes time to finally obtain a continuous shot image.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an imaging apparatus, an imaging method, and a program capable of acquiring a continuous shot image more efficiently at a continuous shooting speed more suitable for movement of a subject image. To do.

In order to achieve the above object, an imaging apparatus according to the first aspect of the present invention provides:
An imaging device capable of capturing a still image in parallel with capturing a moving image,
An imaging unit that captures an image;
A motion amount detection unit for detecting a motion amount of a subject image included in the image captured by the imaging unit;
A moving image encoding unit that encodes an image captured by the imaging unit as a moving image by inter-frame prediction using a motion amount of the subject image detected by the motion amount detection unit ;
In parallel with the encoding of the moving image by the moving image encoding unit, the unit time for continuous shooting of still images based on the amount of movement of the subject image at predetermined time intervals detected by the movement amount detection unit A calculation unit that calculates a continuous shooting speed that is the number of times per image is taken,
In parallel with the moving image encoding by the moving image encoding unit, a continuous image acquisition unit that acquires images captured by the imaging unit as a continuous image at the continuous shooting speed calculated by the calculation unit; ,
Is provided.

Further, the prior Kiren'utsushi image acquisition unit,
Changing the continuous shooting speed for acquiring the continuous shooting image to the continuous shooting speed calculated at predetermined time intervals by the calculation unit;
It is good as well.

In addition, the calculation unit
Based on a representative value of the amount of motion of the subject image detected by the motion amount detection unit, a continuous shooting speed for acquiring a continuous image in the continuous image acquisition unit is calculated.
It is good as well.

Further, the continuous image acquisition unit
An image captured at the highest continuous shooting speed by the imaging unit is temporarily stored as a continuous shooting image,
A process of acquiring a continuous shot image at the continuous shooting speed by selecting a still image according to the continuous shooting speed calculated by the calculation unit from the temporarily stored continuous shooting images. Repeat at a predetermined time interval,
It is good as well.

The moving image encoding by the moving image encoding unit is in MPEG format,
The encoded data is an I picture that is intra-frame encoded data, a P picture that is forward predictive encoded data, and a B picture that is bidirectional predictive encoded data.
It is good as well.
In this case, the calculation unit
The number of times per unit time during continuous shooting of still images based on the amount of motion of the subject image for each predetermined time interval that does not depend on the I picture, P picture, and B picture detected by the motion amount detection unit Calculate the continuous shooting speed,
It is good as well.

An imaging method according to the second aspect of the present invention includes:
An imaging method capable of capturing a still image in parallel with capturing a moving image,
An imaging process for capturing an image;
A motion amount detection step of detecting a motion amount of a subject image included in the image captured by the imaging step;
A moving image encoding step of encoding the image captured by the imaging step as a moving image by inter-frame prediction using the amount of motion of the subject image detected by the motion amount detection step ;
A unit time for continuous shooting of still images based on the amount of motion of the subject image at predetermined time intervals detected by the motion amount detection step in parallel with the motion image encoding by the motion image encoding step. A calculation step of calculating a continuous shooting speed that is the number of times per image is taken,
Concurrently with the encoding of the moving image by the moving image encoding step, the continuous image acquisition step of acquiring the image captured by the imaging step as a continuous image at the continuous shooting speed calculated by the calculation step; ,
Is provided.

A program according to the third aspect of the present invention is:
A computer that controls an imaging device capable of capturing a still image in parallel with capturing a moving image,
A motion amount detection unit that detects a motion amount of a subject image included in an image captured by an image capturing unit that captures an image;
A moving image encoding unit that encodes an image captured by the imaging unit as a moving image by inter-frame prediction using a motion amount of the subject image detected by the motion amount detection unit;
In parallel with the encoding of the moving image by the moving image encoding unit, the unit time for continuous shooting of still images based on the amount of movement of the subject image at predetermined time intervals detected by the movement amount detection unit A calculation unit that calculates a continuous shooting speed that is the number of times per image is taken,
In parallel with the encoding of the moving image by the moving image encoding unit, a continuous image acquisition unit that acquires an image captured by the imaging unit as a continuous image at the continuous shooting speed calculated by the calculation unit,
To function as.

  According to the present invention, the continuous shooting speed of a still image is calculated based on information about the movement of the subject image obtained when encoding a moving image performed in parallel with the continuous shooting. A continuous shot image can be acquired more efficiently at a continuous shooting speed more suitable for movement.

1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment of the present invention. FIG. 2A shows a still image captured at time t1. FIG. 2B is a diagram illustrating a still image captured at time t2. FIG. 2C shows a still image captured at time t3. 2 is a timing chart illustrating an outline of the operation of the imaging apparatus in FIG. 1. FIG. 2 is a schematic diagram for explaining an example of the timing of continuous shooting of still images in the imaging apparatus of FIG. 1. It is a schematic diagram for demonstrating an example of the timing of the continuous shooting of a still image in the imaging device which concerns on the 2nd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by the following embodiment and drawing.

(First embodiment)
First, a first embodiment of the present invention will be described.

  The imaging apparatus 1 according to the present embodiment is an apparatus that captures an image such as a digital camera or a digital video camera. The imaging device 1 has a function of continuously shooting still images. Further, the imaging apparatus 1 can continuously capture still images at an arbitrary timing while capturing a moving image.

  In order to compress a moving image, the imaging apparatus 1 performs encoding defined in MPEG format such as MPEG (Moving Picture Experts Group) -4.

  As illustrated in FIG. 1, the imaging apparatus 1 includes an operation unit 2, a display unit 3, an imaging unit 4, an A / D conversion unit 5, a signal processing unit 6, a control unit 7, and a recording unit 8.

  The operation unit 2 includes a shutter button, a recording button, a jog dial, a touch panel, and the like (all not shown). An operation by the user is input via the operation unit 2. The operation unit 2 outputs an operation input signal corresponding to the input operation. The output operation input signal is, for example, an operation input signal corresponding to recording start, recording end, continuous shooting start, continuous shooting end, or the like.

  The display unit 3 includes, for example, a liquid crystal display (not shown) and displays a moving image or a still image on the liquid crystal display.

  The imaging unit 4 includes an optical system and an image sensor (both not shown).

  The optical system includes a shutter, a lens, a focus lens, a diaphragm, and the like (all not shown). The light passing through the optical system is incident on the light receiving surface of the image sensor, whereby a subject image is formed on the light receiving surface of the image sensor.

  The image sensor includes an imaging element (not shown) such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The image sensor photoelectrically converts the formed subject image and outputs an analog image signal corresponding to the subject image.

  The A / D converter 5 converts the analog image signal output from the image sensor into a digital image signal and outputs the digital image signal.

  The signal processing unit 6 performs processing such as gradation correction and white balance adjustment on the digital image signal output from the A / D conversion unit 5.

  The control unit 7 includes a CPU and a memory (both not shown). When the CPU executes the program stored in the memory, the control unit 7 realizes the following functions.

  The control unit 7 comprehensively controls the imaging device 1 according to the operation input signal input to the operation unit 2. For example, the control unit 7 records the moving image according to the operation input signal corresponding to the start of recording, and ends the recording according to the operation input signal corresponding to the end of recording. The control unit 7 starts continuous shooting according to the operation input signal corresponding to the start of continuous shooting, and ends continuous shooting according to the operation input signal corresponding to the end of continuous shooting.

  Further, the control unit 7 controls the display of moving images and the like on the display unit 3. The control unit 7 controls the operation of the optical system and the image sensor of the imaging unit 4.

  The control unit 7 includes a moving image encoding unit 9, a calculation unit 10, and a continuous shot image acquisition unit 11.

  The moving image encoding unit 9 encodes an image based on the digital image signal output from the signal processing unit 6 as a moving image.

  The moving image encoding unit 9 encodes an image as a moving image by inter-frame prediction. Inter-frame prediction is a method of encoding a difference between an image (predicted image) predicted based on images at different time points and an actual image. In this method, in order to generate a predicted image, a motion vector (motion amount) of a subject image included in the image is obtained.

  For example, when the image captured at the time t1 shown in FIG. 2A is used as the reference image and the inter-frame prediction is performed on the image at the time t2 shown in FIG. 2B, the subject image 21 indicated by the arrow is displayed. A motion vector is detected as a motion amount. When the image captured at time t2 shown in FIG. 2B is used as a reference image and interframe prediction is performed on the image at time t3 shown in FIG. 2C, the subject image 21 indicated by the arrow is displayed. A motion vector is detected as a motion amount. As shown in FIGS. 2A to 2C, the movement of the subject image 21 is faster between the time point t1 and the time point t2 than between the time point t2 and the time point t3. The amount of movement is also increasing.

  In order to detect the amount of motion, the moving image encoding unit 9 includes a motion amount detection unit 12. The motion amount detection unit 12 performs, for example, block matching processing and the like to detect the motion amount of the subject image included in the image based on the digital image signal output from the signal processing unit 6.

  The moving image encoding unit 9 performs inter-frame prediction using the amount of motion detected by the motion amount detecting unit 12, and encodes the moving image by compressing the difference between the predicted image and the actual image. The moving image encoded by the moving image encoding unit 9 is output to the storage unit 8.

  In parallel with the moving image encoding by the moving image encoding unit 9, the calculation unit 10 is based on information (motion amount) on the motion of the subject image obtained at the time of encoding by the moving image encoding unit 9. A continuous shooting speed that is the number of times of image capturing per unit time during continuous shooting of images is calculated.

  The continuous shot image acquisition unit 11 is an image based on the digital image signal output from the signal processing unit 6 at the continuous shooting speed calculated by the calculation unit 10 in parallel with the encoding of the moving image by the moving image encoding unit 9. Is acquired as a continuous shot image. Note that the continuous shot image acquisition unit 11 may have a function of encoding an image as a still image. The still image encoded by the continuous shot image acquisition unit 11 is output to the storage unit 8.

  The recording unit 8 records a moving image, a still image, and the like output from the moving image encoding unit 9 and the continuous shot image acquisition unit 11.

  Next, the operation of the imaging apparatus 1 will be described with reference to the timing chart of FIG.

  When an operation input signal corresponding to the start of recording is input from the operation unit 2, the control unit 7 starts recording a moving image (time t4).

  When recording is started, the motion amount detection unit 12 of the moving image encoding unit 9 starts detecting the motion amount obtained by the inter-frame prediction. Furthermore, the moving image encoding unit 9 encodes an image as a moving image using the detected amount of motion. The encoded moving image is recorded in the recording unit 8. As a result, a moving image is recorded.

  These processes are continued until an operation input signal corresponding to the end of recording is input at time t7.

  When an operation input signal corresponding to the start of continuous shooting is input from the operation unit 2 while a moving image is being captured (time t4 to time t7), the control unit 7 starts continuous shooting of still images (time t5).

  When the continuous shooting is started, the calculation unit 10 calculates the continuous shooting speed based on the motion amount of the subject image detected by the motion amount detection unit 12 in parallel with the moving image encoding by the moving image encoding unit 9. calculate. Further, the continuous shot image acquisition unit 11 converts the digital image signal output from the signal processing unit 6 into the digital image signal at the continuous shooting speed calculated by the calculation unit 10 in parallel with the encoding of the moving image by the moving image encoding unit 9. A base image is acquired as a continuous shot image. The acquired continuous shot image is recorded in the recording unit 8. As a result, continuous shooting is performed.

  These processes are continued until an operation input signal corresponding to the end of continuous shooting is input at time t6.

  Since the motion amount detection unit 12 detects the motion amount in parallel with the encoding of the moving image, the continuous shot image acquisition unit 11 performs continuous shooting at a continuous shooting speed corresponding to the change in the motion amount between time t5 and time t6. Get an image.

  The timing of continuous shooting in this embodiment will be described in detail with reference to FIG.

  The moving image encoded by the moving image encoding unit 9 includes an I picture, a P picture, and a B picture corresponding to a frame in the moving image.

  The P picture is forward prediction encoded data that encodes only the difference changed from the previous frame in terms of time. A B picture is bi-directional predictive encoded data that encodes only the differences that have changed from temporally preceding and succeeding frames. P picture and B picture are encoded using the amount of motion. The I picture is intra-frame encoded data that is encoded without using prediction.

  The calculation unit 10 calculates the continuous shooting speed of still images based on the amount of motion obtained at predetermined time intervals. For example, the calculation unit 10 obtains the average value of the motion amount obtained by encoding each of the images from the encoded data A1 to the encoded data A2, and the encoding of each of the images from the encoded data A3 to the encoded data A4. The average value of the obtained motion amount, the average value of the motion amount obtained by encoding each image from the encoded data A5 to the encoded data A6, and the encoding of each image from the encoded data A7 to the encoded data A8 The average value of the amount of motion obtained in is acquired. And the calculation part 10 calculates the continuous shooting speed in each of the period P1, P2, P3, and P4 based on each acquired average value of the amount of motion. The continuous image acquisition unit 11 changes the continuous shooting speed to the continuous shooting speed calculated by the calculation unit 10.

  As described above, the continuous shooting speed of the still images is calculated based on the information on the movement of the subject image obtained when the moving images are encoded in parallel. For this reason, the time lag generated between the time for detecting the movement of the subject image and the time for performing continuous shooting can be minimized. In addition, a buffer for temporarily storing moving images for obtaining continuous shot images and a process for selecting continuous shot images after recording of moving images become unnecessary. As a result, a continuous shot image can be acquired more efficiently at a continuous shooting speed more suitable for the movement of the subject image.

  Further, according to the present embodiment, when inter-frame prediction is performed in moving image encoding, the amount of motion calculated in the inter-frame prediction is used for calculating the continuous shooting speed. In this way, the continuous shooting speed can be calculated most efficiently.

  Further, according to the present embodiment, the continuous shooting speed is updated at regular intervals based on the amount of movement, so that it is possible to quickly cope with the movement of the subject image.

  Further, according to the present embodiment, since the calculation unit 10 calculates the continuous shooting speed based on the average value of the movement amount of the subject image, the continuous shooting speed is not affected by the instantaneous fluctuation of the movement of the subject image. Can be calculated.

  Further, according to the present embodiment, continuous shot images can be efficiently acquired in parallel with the compression of moving images in the MPEG format.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.

  The configuration of the imaging apparatus 1 according to the present embodiment is the same as the configuration of the imaging apparatus 1 according to the first embodiment.

  As shown in FIG. 5, the continuous image acquisition unit 11 continuously captures still images at the highest continuous shooting speed and temporarily stores the continuous images.

  In parallel with this, for example, the calculation unit 10 acquires an average value of motion amounts obtained by encoding each image from the encoded data B1 to the encoded data B2. Then, the calculation unit 10 calculates the continuous shooting speed in the period P5 substantially at the same time as when the motion amount is obtained based on the acquired average value of the motion amount. The continuous image acquisition unit 11 selects a still image according to the continuous shooting speed calculated by the calculation unit 10 from the temporarily stored continuous shooting images, thereby continuously shooting images at the continuous shooting speed. To get.

  In this way, the continuous shot image acquisition unit 11 averages the motion amount obtained from the encoded data B3 to the encoded data B4, and the average value of the motion amount obtained from the encoded data B5 to the encoded data B6. The continuous shooting speed in each of the periods P6, P7, and P8 is calculated based on the motion amount calculated based on the average values of the motion amounts obtained from the encoded data B7 to the encoded data B8. The continuous-shot image acquisition unit 11 acquires a continuous-shot image at the continuous shooting speed by selecting a still image according to the calculated continuous shooting speed.

  In this way, the time lag generated between the time for detecting the movement of the subject image and the time for continuous shooting can be further shortened.

  Note that the calculation unit 10 may calculate the continuous shooting speed based on representative values such as an intermediate value and a mode value, without being limited to the average value of the motion amount obtained by encoding the moving image. .

  In the above embodiment, the case where the encoding format of the moving image is MPEG-4 has been described. For example, the encoding format of the moving image is MPEG-1, MPEG-2, H.264. 261, H.H. 263, H.M. H.264 or the like may be used.

  The control unit 7 may include two or more CPUs, and separate CPUs may execute recording of moving images and continuous shooting of still images. In this way, since the CPU is divided into a CPU for recording moving images and a CPU for continuous shooting of still images, it is easy to execute recording of moving images and continuous shooting of still images in parallel.

  In the above embodiment, the program to be executed is a computer-readable recording medium such as a flexible disk, a CD-ROM (Compact Disk Read-Only Memory), a DVD (Digital Versatile Disk), and an MO (Magneto-Optical Disk). It is also possible to configure a system that performs the above-described processing by recording and distributing the program and installing the program.

  Further, the program may be recorded on a disk device or the like included in a predetermined server device on a communication network such as the Internet, and may be downloaded, for example, superimposed on a carrier wave.

  Also, when the above functions are realized by sharing an OS (Operating System), or when the above functions are realized by cooperation between the OS and an application, only the part other than the OS is recorded on a medium and distributed. It may also be downloaded.

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 2 ... Operation part, 3 ... Display part, 4 ... Imaging part, 5 ... A / D conversion part, 6 ... Signal processing part, 7 ... Control part, 8 ... Recording part, 9 ... Moving image encoding , 10 ... calculation unit, 11 ... continuous shot image acquisition unit, 12 ... motion amount detection unit, 21 ... subject image

Claims (8)

An imaging device capable of capturing a still image in parallel with capturing a moving image,
An imaging unit that captures an image;
A motion amount detection unit for detecting a motion amount of a subject image included in the image captured by the imaging unit;
A moving image encoding unit that encodes an image captured by the imaging unit as a moving image by inter-frame prediction using a motion amount of the subject image detected by the motion amount detection unit ;
In parallel with the encoding of the moving image by the moving image encoding unit, the unit time for continuous shooting of still images based on the amount of movement of the subject image at predetermined time intervals detected by the movement amount detection unit A calculation unit that calculates a continuous shooting speed that is the number of times per image is taken,
In parallel with the moving image encoding by the moving image encoding unit, a continuous image acquisition unit that acquires images captured by the imaging unit as a continuous image at the continuous shooting speed calculated by the calculation unit; ,
An imaging apparatus comprising: Before Kiren'utsushi image acquisition unit,
Changing the continuous shooting speed for acquiring the continuous shooting image to the continuous shooting speed calculated at predetermined time intervals by the calculation unit;
The imaging apparatus according to claim 1 . The calculation unit includes:
Based on a representative value of the amount of motion of the subject image detected by the motion amount detection unit, a continuous shooting speed for acquiring a continuous image in the continuous image acquisition unit is calculated.
The imaging apparatus according to claim 1 or 2, wherein The continuous image acquisition unit
An image captured at the highest continuous shooting speed by the imaging unit is temporarily stored as a continuous shooting image,
A process of acquiring a continuous shot image at the continuous shooting speed by selecting a still image according to the continuous shooting speed calculated by the calculation unit from the temporarily stored continuous shooting images. Repeat at a predetermined time interval,
The imaging apparatus according to any one of claims 1 to 3, characterized in that. The moving image encoding by the moving image encoding unit is in MPEG format,
The encoded data is an I picture that is intra-frame encoded data, a P picture that is forward predictive encoded data, and a B picture that is bidirectional predictive encoded data.
The imaging apparatus according to any one of claims 1 to 4, characterized in that. The calculation unit includes:
The number of times per unit time during continuous shooting of still images based on the amount of motion of the subject image for each predetermined time interval that does not depend on the I picture, P picture, and B picture detected by the motion amount detection unit Calculate the continuous shooting speed,
The imaging apparatus according to claim 5. An imaging method capable of capturing a still image in parallel with capturing a moving image,
An imaging process for capturing an image;
A motion amount detection step of detecting a motion amount of a subject image included in the image captured by the imaging step;
A moving image encoding step of encoding the image captured by the imaging step as a moving image by inter-frame prediction using the amount of motion of the subject image detected by the motion amount detection step ;
A unit time for continuous shooting of still images based on the amount of motion of the subject image at predetermined time intervals detected by the motion amount detection step in parallel with the motion image encoding by the motion image encoding step. A calculation step of calculating a continuous shooting speed that is the number of times per image is taken,
Concurrently with the encoding of the moving image by the moving image encoding step, the continuous image acquisition step of acquiring the image captured by the imaging step as a continuous image at the continuous shooting speed calculated by the calculation step; ,
An imaging method comprising: A computer that controls an imaging device capable of capturing a still image in parallel with capturing a moving image,
A motion amount detection unit that detects a motion amount of a subject image included in an image captured by an image capturing unit that captures an image;
A moving image encoding unit that encodes an image captured by the imaging unit as a moving image by inter-frame prediction using a motion amount of the subject image detected by the motion amount detection unit;
In parallel with the encoding of the moving image by the moving image encoding unit, the unit time for continuous shooting of still images based on the amount of movement of the subject image at predetermined time intervals detected by the movement amount detection unit A calculation unit that calculates a continuous shooting speed that is the number of times per image is taken,
In parallel with the encoding of the moving image by the moving image encoding unit, a continuous image acquisition unit that acquires an image captured by the imaging unit as a continuous image at the continuous shooting speed calculated by the calculation unit,
Program to function as.

Images