JP4548041B2 - Imaging apparatus, imaging method, program, and recording medium - Google Patents

Description

  The present invention relates to an imaging device, an imaging method, a program, and a recording medium, and in particular, imaging that enables a moving image displayed when recording is instructed by a user to be easily recorded as a still image. The present invention relates to an apparatus, an imaging method, a program, and a recording medium.

  FIG. 1 is an example of a flowchart illustrating still image recording processing for recording a still image during recording of a moving image in a conventional camera system. This still image recording process is started, for example, when recording of a moving image is instructed by the user.

  In step S11, the camera system images the subject, records it as a moving image on a predetermined recording medium (records the moving image), and proceeds to step S12.

  In step S12, the camera system determines whether or not recording of a still image is instructed by the user. If it is determined that recording of the still image is not instructed by the user, the process returns to step S11 and the user records the still image. The moving image is recorded until recording is instructed.

  On the other hand, if it is determined in step S12 that the user has instructed recording of a still image, the process proceeds to step S13, and the camera system stops recording the moving image, and proceeds to step S14.

  In step S14, the camera system sweeps away (clears) the electric charge accumulated in the image sensor, starts imaging the subject, and proceeds to step S15. In step S15, the camera system determines whether or not the predetermined exposure time has elapsed. If it is determined that the predetermined exposure time has not elapsed, the camera system waits until it is determined that the predetermined exposure time has elapsed. .

  If it is determined in step S15 that the predetermined exposure time has elapsed, the process proceeds to step S16, and the camera system ends imaging of the subject. Then, the camera system records the captured image of the subject as a still image on a predetermined recording medium (records the still image), and returns to step S11. In step S11, the camera system restarts the recording of the moving image and repeats the above-described processing.

  As described above, in the still image recording process of FIG. 1, as described in step S <b> 13, when recording a still image, the recording of the moving image is temporarily interrupted, and the recording of the moving image is not interrupted. It has been difficult to respond to user demands such as recording still images.

  Further, in the still image recording process of FIG. 1, after it is determined in step S12 that the user has instructed recording of a still image, recording of moving images is stopped (step S14) until imaging of the subject is started in step S14. S13) and the process of clearing the electric charge accumulated in the image pickup device must be performed, and a time lag occurs after the still image recording is instructed until the still image is picked up. As a result, the user may miss a photo opportunity.

  FIG. 2 is an example of a flowchart illustrating still image recording processing for recording a still image during display of a moving image in a conventional camera system. This still image recording process is started, for example, when a user instructs to display a moving image.

  In step S31, the camera system captures the subject, displays it as a moving image, and proceeds to step S32. In step S32, the camera system determines whether or not recording of a still image is instructed by the user, as in step S12 of FIG. 1, and if it is determined that recording of the still image is not instructed by the user, Returning to S31, the moving image is displayed until recording of a still image is instructed by the user.

  On the other hand, if it is determined in step S32 that the user has instructed recording of a still image, the process proceeds to step S33, the camera system stops displaying the moving image, and proceeds to step S34. Note that the processing in steps S34 to S36 is the same as the processing in steps S14 to S16 in FIG.

  As described above, in the still image recording process of FIG. 2, as described in step S <b> 33, when a still image is recorded, the moving image display is temporarily interrupted, and the moving image display is not interrupted. It has been difficult to respond to user demands such as recording still images.

  Further, in the still image recording process of FIG. 2, after it is determined in step S32 that the user has instructed to record a still image, the reproduction of moving images is stopped (step S34) before the subject is imaged (step S34). S33) and the process of clearing the electric charge accumulated in the image sensor must be performed. As in the still image recording process of FIG. 1, the still image is captured after the still image recording is instructed. Time lag occurs. As a result, the user may miss a photo opportunity.

  Therefore, by recording a part of the image recorded as a moving image as a still image, the interruption of the recording of the moving image and the time lag from when the recording of the still image is commanded until the still image is captured are eliminated. There is a video camera device (see, for example, Patent Document 1).

This video camera device stores a captured image in a buffer memory and displays it as a moving image. Note that the buffer memory stores a predetermined number of images that are captured in time, and when more than the predetermined number of images are captured, the buffer memory newly captures images in order from the oldest captured image. Rewrite the image. The user instructs recording of a still image when a desired moving image is displayed while viewing the displayed moving image. When a user instructs the recording of a still image, the video camera device displays a plurality of temporally continuous images stored in the buffer memory and allows the user to select an image to be recorded as a still image. Then, the video camera device records the image selected by the user as a still image.
JP 2004-96783 A

  However, in the video camera device described in Patent Document 1, when a user instructs the recording of a still image, the user selects a still image from a plurality of temporally continuous images recorded in the buffer memory. It is necessary to select an image to be recorded as an image, that is, a moving image displayed when a still image recording command is issued. This selection by the user is to avoid a slight deviation between the moving image displayed at the time of recording the still image and the image actually recorded as a still image. is there.

  The present invention has been made in view of such a situation, and enables a moving image displayed when recording is instructed by a user to be easily recorded as a still image. .

An imaging apparatus according to the present invention includes an imaging unit that captures an image of a subject and outputs image data, an image processing unit for moving image that performs image processing for a moving image on image data output from the imaging unit, Based on the image data on which the image processing for moving images has been performed by the image processing unit for moving images, each of a display unit for displaying an image and a plurality of temporally continuous image data output from the imaging unit A plurality of storage means for storing, an accepting means for accepting an image recording command, and image data corresponding to an image displayed on the display means when the accepting means accepts an image recording command A selection unit that selects a storage unit from a plurality of storage units and a static image that performs image processing for a still image on image data stored in the storage unit selected by the selection unit. Image image processing means, still image encoding means for encoding image data processed for still images by still image processing means, and image data encoded by still image encoding means are recorded A display control unit that updates the image displayed on itself for each horizontal line, and the selection unit receives the image recording command when the reception unit receives the image recording command. When the number of horizontal lines of the updated image displayed on the display means is equal to or less than a predetermined number, the storage means for storing the image data corresponding to the image before the update is selected and displayed on the display means If the number of horizontal lines of the updated image that has been updated is greater than the predetermined number, the storage means for storing the image data corresponding to the updated image is selected. Characterized in that it.

  The moving image image processing means can perform conversion processing for converting the number of pixels of image data as image processing for moving images.

  The imaging apparatus further includes moving image encoding means for encoding image data that has been subjected to image processing for moving images by the moving image image processing means, and the recording control means is encoded by the moving image encoding means. Image data can be further recorded.

  The recording control unit can record the image data encoded by the moving image encoding unit and the image data encoded by the still image encoding unit on one recording medium.

  The storage capacity of the plurality of storage means should be determined based on the time from the start of image data output by the imaging means until the end of the display of the image based on the image data. Can do.

  The accepting unit further accepts a command for changing the mode of the imaging device to a recording mode for recording an image, and the imaging unit receives the number of pixels of the image data when the accepting unit accepts a command for setting the recording mode. Can be changed.

  The predetermined number may be half of the number of horizontal lines of the image.

  The imaging means includes a pixel imaging means for imaging a subject in pixel units, a conversion means for A / D (Analog / Digital) conversion of an image signal obtained as a result of imaging by the pixel imaging means in pixel units, and a conversion means. Image data obtained as a result of the conversion can be stored for each pixel, and pixel storage means for outputting the image data can be provided for each pixel.

  The pixel imaging unit, the conversion unit, and the pixel storage unit can be arranged two-dimensionally.

  The pixel imaging unit, the conversion unit, and the pixel storage unit can be arranged three-dimensionally.

  The image pickup means is a line sequential scanning type image pickup device, and the scanning speed of the image pickup device can be made faster than the frame rate of the image data.

The imaging method of the present invention includes an imaging step of imaging a subject and outputting image data, and an image processing step for moving image that performs image processing for a moving image on the image data output by the processing of the imaging step. And a display control step for displaying an image on the display means based on the image data on which the image processing for the moving image has been performed by the processing of the image processing step for moving image, and the temporal output output by the processing of the imaging step A storage control step of storing each of a plurality of continuous image data in a plurality of storage means, a reception step of receiving an image recording command, and an image recording command received by the processing of the receiving step, A selection step of selecting, from a plurality of storage means, storage means for storing image data corresponding to the image displayed on the display means; The still image processing step for performing still image processing on the image data stored in the storage means selected by the selection step processing and the still image processing step for processing still image processing. and encoding step for encoding to still image the image data which the image processing has been performed for, viewing including a recording control step for recording the encoded image data by the processing of the encoding step for still image, the processing of the display control step The image displayed on the display means is updated for each horizontal line, and the process of the selection control step is displayed on the display means when the recording command of the image is received by the process of the receiving step. If the number of horizontal lines in the updated image is less than or equal to the predetermined number, the image data corresponding to the image before the update Selecting the storage means to store, and storing the image data corresponding to the updated image when the number of horizontal lines of the updated image displayed on the display means is greater than the predetermined number The storage means is selected .

The program of the present invention includes an imaging step of imaging a subject and outputting image data, and an image processing step for moving image that performs image processing for moving images on the image data output by the processing of the imaging step. The display control step for displaying an image on the display means based on the image data on which the image processing for the moving image has been performed by the processing of the image processing step for moving image, and the time output by the processing of the imaging step A storage control step for storing each of a plurality of continuous image data in a plurality of storage means, a reception step for receiving an image recording command, and a display when an image recording command is received by the processing of the reception step A selection step of selecting storage means for storing image data corresponding to the image displayed on the means from a plurality of storage means; A still image processing step for performing image processing for a still image on the image data stored in the storage means selected by the processing of the selection step, and a processing of the still image by processing of the still image processing step. and encoding step for encoding to still image the image data which the image processing has been performed for, viewing including a recording control step for recording the encoded image data by the processing of the encoding step for still image, the processing of the display control step The image displayed on the display means is updated for each horizontal line, and the process of the selection control step is displayed on the display means when the recording command of the image is received by the process of the receiving step. If the number of horizontal lines in the updated image is less than or equal to the predetermined number, the image data corresponding to the image before the update When the number of horizontal lines of the updated image displayed on the display unit is larger than the predetermined number, the image data corresponding to the updated image is stored. The storage means is selected .

The program recorded on the recording medium of the present invention images an object and outputs image data, and performs image processing for moving images on image data output by the processing of the imaging step. The image processing step for moving image, the display control step for displaying the image on the display means based on the image data subjected to the image processing for moving image by the processing of the image processing step for moving image, and the processing of the imaging step A storage control step for storing each of a plurality of temporally continuous image data output by the plurality of storage means, a reception step for receiving a command for recording an image, and a command for recording an image by processing of the reception step. The storage means for storing the image data corresponding to the image displayed on the display means when received is selected from a plurality of storage means. A selection step for selecting, a still image processing step for performing still image processing on the image data stored in the storage means selected by the processing of the selection step, and a still image processing step seen including a still-image encoding step of image processing to encode the image data has been performed for the still image by the processing of, and a recording control step for recording the encoded image data by the processing of the encoding step for still images, The processing of the display control step updates the image displayed on the display means for each horizontal line, and the processing of the selection control step is performed when the image recording command is received by the processing of the receiving step. If the number of horizontal lines in the updated image displayed on the display means is less than or equal to the specified number, Select the storage means for storing the image data corresponding to the image, and if the number of horizontal lines of the updated image displayed on the display means is larger than the predetermined number, the updated image is supported The storage means for storing the image data to be selected is selected .

In the present invention, a subject is imaged, image data is output, and image processing for a moving image is performed on the image data. Based on the image data that has been subjected to image processing for the moving image , the image is updated for each horizontal line, and each of a plurality of temporally continuous image data is stored in a plurality of storage means. Then, an image recording command is received, and when the image recording command is received , if the number of horizontal lines of the updated image displayed on the display means is equal to or less than a predetermined number, If the storage means for storing the image data corresponding to the image is selected and the number of horizontal lines of the updated image displayed on the display means is greater than a predetermined number, the image corresponding to the updated image A storage unit for storing data is selected, and image processing for a still image is performed on the image data stored in the selected storage unit. The image data that has been subjected to image processing for the still image is encoded, and the encoded image data is recorded.

  According to the present invention, a moving image displayed when recording is instructed by a user can be easily recorded as a still image.

  Embodiments of the present invention will be described below. Correspondences between constituent elements described in the claims and specific examples in the embodiments of the present invention are exemplified as follows. This description is to confirm that specific examples supporting the invention described in the claims are described in the embodiments of the invention. Therefore, even if there are specific examples that are described in the embodiment of the invention but are not described here as corresponding to the configuration requirements, the specific examples are not included in the configuration. It does not mean that it does not correspond to a requirement. On the contrary, even if a specific example is described here as corresponding to a configuration requirement, this means that the specific example does not correspond to a configuration requirement other than the configuration requirement. not.

  Further, this description does not mean that all the inventions corresponding to the specific examples described in the embodiments of the invention are described in the claims. In other words, this description is an invention corresponding to the specific example described in the embodiment of the invention, and the existence of an invention not described in the claims of this application, that is, in the future, a divisional application will be made. Nor does it deny the existence of an invention added by amendment.

An imaging apparatus according to claim 1 is provided.
In an imaging device (for example, the digital camera 1 in FIG. 3) that images a subject and records image data of the subject,
An imaging means for imaging the subject and outputting the image data (for example, the imaging device 11 in FIG. 3);
Moving image processing means (for example, DSP 14 in FIG. 3) that performs image processing for moving images on the image data output from the imaging means;
Display means for displaying an image based on the image data that has been subjected to image processing for the moving image by the moving image image processing means (for example, the display unit 16 in FIG. 3);
A plurality of storage means (for example, buffer memories 21-1 to 21-3 in FIG. 3) for storing each of the plurality of temporally continuous image data output from the imaging means;
Receiving means for receiving the image recording command (for example, the operation input unit 25 in FIG. 3);
When a command for recording the image is received by the receiving unit, a storage unit that stores the image data corresponding to the image displayed on the display unit is selected from a plurality of the storage units. Selecting means (for example, the control unit 26 in FIG. 3);
Still image processing means (for example, DSP 23 in FIG. 3) that performs image processing for the still image on the image data stored in the storage means selected by the selection means;
Still image encoding means (for example, DSP 24 in FIG. 3) that encodes the image data that has been subjected to image processing for the still image by the still image processing means;
Recording control means (for example, the recording control unit 18 in FIG. 3) for recording the image data encoded by the still image encoding means ,
The display means updates the image displayed on itself for each horizontal line,
The selection means is updated when the number of horizontal lines of the updated image displayed on the display means is less than or equal to a predetermined number when the image recording command is received by the receiving means. When the storage means for storing the image data corresponding to the previous image is selected, and the number of horizontal lines of the updated image displayed on the display means is larger than the predetermined number, the updated image Is selected (for example, the process in step S95 or step S96 in FIG. 10).
It is characterized by that.

An imaging apparatus according to claim 2 is provided.
The moving image image processing means performs conversion processing (for example, filter processing) for converting the number of pixels of the image data as image processing for the moving image.

An imaging apparatus according to claim 3 is provided.
Moving image encoding means for encoding the image data that has been subjected to image processing for the moving image by the moving image image processing means (for example, DSP 17 in FIG. 3)
Further comprising
The recording control unit further records the image data encoded by the moving image encoding unit (for example, the process of step S58 in FIG. 6).
It is characterized by that.

The imaging device according to claim 4 is provided.
The recording control unit records the image data encoded by the moving image encoding unit and the image data encoded by the still image encoding unit on one recording medium (for example, FIG. 6). Processing in step S58)
It is characterized by that.

The imaging device according to claim 5 is:
The storage capacities of the plurality of storage means are the time from the start of the output of the image data by the imaging means until the end of the display of the image based on the image data (for example, the times t _{0 to} t in FIG. 5). _The storage capacity is determined based on ₅ ).

The imaging device according to claim 6 is:
The accepting means further accepts a command for changing the mode of the imaging device to a recording mode for recording the image (for example, the process of step S171 in FIG. 16),
The imaging unit changes the number of pixels of the image data when the command for switching to the recording mode is received by the receiving unit (for example, processing in step S173 in FIG. 16).
It is characterized by that.

The imaging device according to claim 8 is provided.
The imaging means includes
Pixel imaging means for imaging the subject in pixel units (for example, the photoelectric conversion unit 61 in FIG. 19);
Conversion means (for example, A / D conversion unit 62 in FIG. 19) for converting an image signal obtained as a result of imaging by the pixel imaging means in A / D (Analog / Digital) in units of pixels;
The image data obtained as a result of conversion by the conversion means is stored in pixel units, and pixel storage means (for example, the memory 63 in FIG. 19) for outputting the image data is provided for each pixel.

The imaging method according to claim 1 2,
In an imaging method of an imaging device (for example, the digital camera 1 in FIG. 3) that images a subject and records image data of the subject,
An imaging step (for example, step S51 in FIG. 6) for imaging the subject and outputting the image data;
A moving image processing step (for example, step S55 in FIG. 6) for performing image processing for moving images on the image data output by the imaging step processing;
A display control step for displaying an image on display means (for example, the display unit 16 in FIG. 3) based on the image data on which the image processing for the moving image has been performed by the processing of the image processing step for moving image ( For example, step S56 in FIG.
A storage control step (for example, step S71 in FIG. 7) that stores each of the plurality of temporally continuous image data output by the imaging step processing in a plurality of storage means (for example, the buffer memory 21 in FIG. 3). )When,
A receiving step (for example, step S72 in FIG. 7) for receiving the image recording command;
Storage means for storing the image data corresponding to the image displayed on the display means when a command for recording the image is received by the processing of the reception step, from among the plurality of storage means A selection step to select (for example, step S74 in FIG. 7);
A still image processing step (for example, step S75 in FIG. 7) for performing image processing for the still image on the image data stored in the storage means selected by the selection step; ,
A still image encoding step (for example, step S78 in FIG. 7) for encoding the image data on which the image processing for the still image has been performed by the processing of the still image processing step;
Recording control step of recording the image data encoded by the processing of the still image for encoding step (e.g., step S79 of FIG. 7) and the saw including a
The processing of the display control step updates the image displayed on the display means for each horizontal line,
In the process of the selection control step, the number of horizontal lines of the updated image displayed on the display means is less than or equal to a predetermined number when the image recording command is received by the process of the receiving step. If the storage means for storing the image data corresponding to the image before the update is selected, and the number of horizontal lines of the updated image displayed on the display means is greater than the predetermined number, The storage means for storing image data corresponding to the updated image is selected .

Program recorded in a recording medium according to the program and the first to fourth aspects according to claim 1 3,
In a program for causing a computer to perform a recording process of imaging a subject and recording image data of the subject.
An imaging step (for example, step S51 in FIG. 6) for imaging the subject and outputting the image data;
A moving image processing step (for example, step S55 in FIG. 6) for performing image processing for moving images on the image data output by the imaging step processing;
A display control step for displaying an image on display means (for example, the display unit 16 in FIG. 3) based on the image data on which the image processing for the moving image has been performed by the processing of the image processing step for moving image ( For example, step S56 in FIG.
A storage control step (for example, step S71 in FIG. 7) for storing each of the plurality of temporally continuous image data output by the imaging step processing in a plurality of storage means;
A receiving step (for example, step S72 in FIG. 7) for receiving the image recording command;
Storage means for storing the image data corresponding to the image displayed on the display means (for example, the buffer memory 21 in FIG. 3) when an instruction to record the image is accepted by the processing of the acceptance step A selection step (for example, step S74 in FIG. 7) for selecting from a plurality of the storage means,
A still image processing step (for example, step S75 in FIG. 7) for performing image processing for the still image on the image data stored in the storage means selected by the selection step; ,
A still image encoding step (for example, step S78 in FIG. 7) for encoding the image data on which the image processing for the still image has been performed by the processing of the still image processing step;
Recording control step of recording the image data encoded by the processing of the still image for encoding step (e.g., step S79 of FIG. 7) and the saw including a
The processing of the display control step updates the image displayed on the display means for each horizontal line,
In the process of the selection control step, the number of horizontal lines of the updated image displayed on the display means is less than or equal to a predetermined number when the image recording command is received by the process of the receiving step. If the storage means for storing the image data corresponding to the image before the update is selected, and the number of horizontal lines of the updated image displayed on the display means is greater than the predetermined number, The storage means for storing image data corresponding to the updated image is selected .

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings.

  FIG. 3 is a block diagram showing a functional configuration example of a digital camera to which the present invention is applied.

  The digital camera 1 in FIG. 3 captures a subject and displays an image obtained as a result as a moving image or a still image. The digital camera 1 records an image obtained as a result of imaging as a moving image or a still image.

  The digital camera 1 in FIG. 3 includes an image sensor 11, an A / D converter 12, a frame memory 13, DSPs (Digital Signal Processors) 14, 17, 23, and 24, selectors 15, 20, and 22, a display unit 16, The recording control unit 18 includes a recording unit 19, buffer memories 21-1 to 21-3, an operation input unit 25, and a control unit 26.

  Note that when there is no need to distinguish between the buffer memory 21-1 to the buffer memory 21-3, they are collectively referred to as a buffer memory 21.

  The image sensor 11 is composed of, for example, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) sensor, or the like. In the following description, the image sensor 11 is a CMOS sensor having pixels (for example, 3 million pixels) suitable for still images. The image sensor 11 captures an image of a subject via an optical system (lens) and a diaphragm (not shown), and supplies an image signal, which is an analog signal of a pixel suitable for a still image, to the A / D converter 12. That is, the image sensor 11 receives the reflected light from the subject, and A / D converts the image signal of the pixel suitable for the still image proportional to the number of electrons generated by photoelectrically converting the received reflected light. To the unit 12. Here, although not shown in FIG. 1, the digital camera 1 is provided with an AF (Auto Focus) circuit, an AE (Auto Exposure) circuit, and the like. The AF circuit controls the optical system to perform focus control, and the AE circuit controls the aperture to perform exposure control.

  The A / D conversion unit 12 performs A / D conversion on an image signal of a pixel suitable for a still image from the image sensor 11, and converts the image signal into digital data having a bit number of 10 bits or more, for example. . The A / D converter 12 supplies pixel data suitable for the converted still image to the frame memory 13 and the selector 20 as image data. Note that the A / D converter 12 may be formed on the imaging element 11 on-chip.

  The frame memory 13 stores image data for one frame of pixels suitable for a still image from the A / D converter 12 and supplies the stored image data to the DSP 14.

  The DSP 14 performs image processing for moving images on image data of pixels suitable for still images from the frame memory 13. Specifically, the DSP 14 performs filter processing for performing calculation for converting the number of pixels of image data suitable for still images into the number of pixels suitable for moving images, interpolation processing for color and sensitivity, and white balance. Processing for adjustment and RGB data conversion processing for converting image data into RGB data and the like are performed. Here, the DSP 14 performs, for example, 3 million pixel image data suitable for a still image from the frame memory 13 by filtering, and converts the image data of about 300,000 pixels (VGA (Video Graphics Array) size (640 × 480dot)) image data. The DSP 14 supplies image data of the number of pixels suitable for the moving image after the image processing for the moving image (hereinafter referred to as moving image data) to the selector 15 and the DSP 17.

  The selector 15 is based on a control signal (hereinafter referred to as a display control signal) for instructing display of a moving image or a still image supplied from the control unit 26. Image data of pixels suitable for the still image after image processing for the still image (hereinafter referred to as still image data) is selected and supplied to the display unit 16. The display unit 16 includes a display such as a liquid crystal or an organic EL (Electro Luminescent). The display unit 16 displays a moving image or a still image based on the moving image data or the still image data from the selector 15.

  The DSP 17 converts the moving image data of pixels suitable for the moving image from the DSP 14 into MPEG2 (Moving Picture Experts Group phase 4), MPEG4 (Moving Picture Experts Group phase 4), or H.264 (suitable for recording moving image data). H.264 / MPEG4 AVC or MPEG4 Part 10) and the like, and the encoded data is supplied to the recording control unit 18.

  The recording control unit 18 time-division multiplexes the moving image data of the pixels suitable for the moving image from the DSP 17 and the still image data of the pixels suitable for the still image from the DSP 24 to multiplex the moving image and the still image multiplexed data. Is supplied to the recording unit 19 for recording. For example, the recording control unit 18 packetizes each of the moving image data and the still image data by sequentially adding predetermined header information for each predetermined unit, and time-divides the moving image data and the still image data after packetization. Multiplexed and recorded in the recording unit 19 as multiplexed data. Details of a method for time-division multiplexing moving image data and still image data to be recorded in the recording unit 19 are disclosed in Japanese Patent Laid-Open No. 2000-188736.

  The recording unit 19 includes a magnetic tape, a DVD (Digital Versatile Disk), an EEPROM (Electronically Erasable and Programmable Read Only Memory), and the like, and records the multiplexed data from the recording control unit 18.

  In the digital camera 1, two recording units 19 may be provided, one of which records moving image data and the other records still image data. In this case, the recording control unit 18 does not need to time-division multiplex moving image data and still image data.

  The selector 20 is supplied with a control signal (hereinafter referred to as a storage selection signal) representing the buffer memory 21 that stores image data of pixels suitable for a still image from the A / D converter 12. The selector 20 selects the buffer memory 21 based on the storage selection signal, and supplies the selected buffer memory 21 with image data of pixels suitable for one frame of still image from the A / D converter 12. For example, the selector 20 supplies the image data from the A / D conversion unit 12 that is temporally continuous from the buffer memory 21-1 in order for each frame based on the storage selection signal. That is, each of the three frames of temporally continuous image data is supplied to the buffer memories 21-1 to 21-3.

  The buffer memory 21 stores image data of pixels suitable for a one-frame still image from the selector 20. The buffer memory 21 erases the stored image data and stores the image data from the selector 20 every time the image data is supplied from the selector 20. That is, the image data stored in the buffer memory 21 is updated every time image data is supplied from the selector 20. The buffer memory 21 supplies the stored image data to the selector 22.

  A control signal (hereinafter referred to as an output selection signal) representing the buffer memory 21 in which image data from the buffer memory 21 to be output from the selector 22 is stored is supplied to the selector 22 from the control unit 26. The selector 22 selects the buffer memory 21 based on the output selection signal, and supplies (outputs) image data of pixels suitable for the still image from the selected buffer memory 21 to the DSP 23.

  The DSP 23 performs image processing for the still image on the image data of the pixel suitable for the still image from the selector 22. Specifically, the DSP 23 performs color and sensitivity interpolation processing, white balance adjustment processing, RGB data conversion processing for converting image data into RGB data, and the like. The DSP 23 supplies image data of pixels suitable for the still image after image processing for the still image (hereinafter referred to as still image data) to the selector 15 and the DSP 24. Note that the type of image processing performed by the DSP 23 can be arbitrarily set by the user. For example, when the user does not set any type of image processing, the DSP 23 does not perform any processing on the image data of the pixel suitable for the still image, and selects the image data of the pixel suitable for the still image as it is. 15 and DSP 24.

  The DSP 24 encodes the still image data from the DSP 23 into a predetermined format conforming to JPEG (Joint Photographic Experts Group) or JPEG 2000 suitable for recording of still image data, and supplies the encoded data to the recording control unit 18. The format can be arbitrarily set by the user. For example, when the user does not set any format, that is, when the user records still image data that is not encoded, the DSP 24 supplies the recording control unit 18 without encoding the still image data.

  The operation input unit 25 receives operations of various buttons (not shown) from the user and supplies input signals corresponding to the operations to the control unit 26. For example, the user operates a shutter button (not shown) in order to give a command for recording a still image. The operation input unit 25 receives the operation (command) and supplies an input signal for commanding recording of a still image corresponding to the operation to the control unit 26. Further, the user operates a display button (not shown) in order to instruct display of a still image or a moving image. The operation input unit 25 receives the operation and supplies an input signal for instructing display of a still image or a moving image corresponding to the operation to the control unit 26.

  The control unit 26 controls each unit. For example, the control unit 26 supplies a storage selection signal to the selector 20. That is, the control unit 26 controls the selector 20 so as to supply the image data from the A / D conversion unit 12 to the buffer memory 21 represented by the storage selection signal. For example, first, the control unit 26 supplies a storage selection signal representing the buffer memory 21-1 to the selector 20, and then supplies a storage selection signal representing the buffer memory 21-2 to the selector 20. Then, the control unit 26 supplies a storage selection signal representing the buffer memory 21-3, and then supplies a storage selection signal representing the buffer memory 21-1 to the selector 20, and thereafter repeats the same processing.

  At this time, the selector 20 supplies the image data from the A / D converter 12 in the order of the buffer memory 21-1, the buffer memory 21-2, the buffer memory 21-3, the buffer memory 21-1,. .

  In addition, the control unit 26 records the still image by the operation input unit 25 from the buffer memories 21-1 to 21-3 in response to an input signal for instructing the recording of the still image from the operation input unit 25. When the instruction is received, the buffer memory 21 for storing image data corresponding to the moving image displayed on the display unit 16 is selected, and an output selection signal representing the selected buffer memory 21 is supplied to the selector 22. To do. That is, the control unit 26 controls the selector 22 so as to output the image data stored in the buffer memory 21 represented by the output selection signal.

  Further, the control unit 26 supplies the selector 15 with a display control signal for instructing the display of the moving image or the still image according to the input signal for instructing the display of the still image or the moving image from the operation input unit 25. To do. That is, the control unit 26 controls the selector 15 so as to display a moving image or a still image.

  FIG. 4 is a diagram for explaining the operation timing of the digital camera 1 of FIG. 3 when the user operates the display button to instruct the display of the moving image on the display unit 16. In FIG. 4, the user has not yet issued a still image recording command.

  In FIG. 4, the horizontal axis represents time. This also applies to FIGS. 5, 8, and 13 described later. From the top, the first row is the processing of the image sensor 11, the second row is the processing of the A / D converter 12, the third row is the processing of the frame memory 13 and the buffer memory 21, and the fourth row is the DSP 14. The fifth line represents the process of the display unit 16. In the following, the i-th frame (i is a positive number) is referred to as frame #i.

As shown in FIG. 4, the image sensor 11 captures a subject for a time t ₀ and outputs an image signal of a frame # 1 of a pixel suitable for a still image to the A / D converter 12 (A1). . The A / D conversion unit 12 performs an A / D conversion process for performing A / D conversion on an image signal output from the image sensor 11 after time t ₁ after the image sensor 11 starts capturing an object. The image data of frame # 1 of pixels suitable for the still image after A / D conversion is supplied to the frame memory 13 and the selector 20 (B1).

Here, the control unit 26 supplies a storage selection signal representing the buffer memory 21-1 to the selector 22. The selector 20 selects the buffer memory 21-1 based on the storage selection signal from the control unit 26, and the frame # 1 of the pixel suitable for the still image from the A / D conversion unit 12 is selected in the buffer memory 21-1. Supply image data. After time t ₂ after the A / D conversion unit 12 starts the A / D conversion process, the frame memory 13 stores the image data of the frame # 1 of the pixel suitable for the still image from the A / D conversion unit 12. And the stored image data is supplied to the DSP 14. At this time, the buffer memory 21-1 starts storing image data of pixels suitable for the still image of frame # 1 from the selector 20, and supplies the stored image data to the selector 22 (C1).

  Note that the buffer memories 21-2 and 21-3 also supply the stored image data to the selector 22. Here, in FIG. 4, the control unit 26 does not supply an output selection signal to the selector 22 because the user has not yet issued a still image recording command. Therefore, the selector 22 does not supply the image data supplied from all the buffer memories 21 to the selector 22.

After the time t ₃ after the frame memory 13 and the buffer memory 21-1 start recording the image data of the frame # 1, the DSP 14 sets the frame # 1 of the pixel suitable for the still image supplied from the frame memory 13. Image processing for a moving image with respect to the image data is started, and moving image data of frame # 1 of a pixel suitable for the moving image is supplied to the selector 15 (D1). In FIG. 4, since the display unit 16 displays a moving image, the control unit 26 supplies a display control signal for instructing the selector 15 to display the moving image. Based on the display control signal, the selector 15 supplies the display unit 16 with moving image data of the frame # 1 of pixels suitable for the moving image from the DSP 14.

After time t ₄ after the DSP 14 starts image processing for the moving image on the image data of frame # 1, the display unit 16 converts the moving image data of frame # 1 of the pixel suitable for the moving image from the selector 15 to the moving image data. Based on this, the display of the moving image is started, and the moving image of frame # 1 is displayed (E1). Display unit 16, after a time t ₅ from the start of display of the moving image frame # 1, and ends the display of the moving image of frame # 1.

As described above, the moving image of frame # 1 is displayed on the display unit 16. The imaging device 11, after supplying the image signal of frame # 1 to the A / D converter 12 during the time t _0, imaging an object, an image signal of frame # 2 of pixels suitable still images A / The data is output to the D converter 12 (A2). Then, the same processing (B2, C2, D2, E2) as frame # 1 is performed on frame # 2, and displayed on display unit 16 (E2). However, the image data of frame # 2 is stored not in the buffer memory 21-1, but in the buffer memory 21-2. Similar processing is performed on image data after frame # 3. However, the image data of frame # 3 is stored in the buffer memory 21-3 (C3), the image data of frame # 4 is stored in the buffer memory 21-1 (C4), and thereafter in the same order in the buffer memory 21. Image data is stored.

For example, when the moving image data of frame #i is recorded in the recording unit 19, the DSP 14 also supplies the DSP 17 with moving image data of the frame #i of a pixel suitable for the moving image. While the display unit 16 displays the image data of the frame #i, the DSP 17 encodes the moving image data of the frame #i having a pixel suitable for the moving image into a predetermined format suitable for the moving image. The recording control unit 18 causes the recording unit 19 to record the image data of frame #i. In FIG. 4, times t _{0 to} t ₅ differ depending on the performance of the digital camera 1 and the like.

  FIG. 5 is a diagram for explaining the operation timing of the digital camera 1 in FIG. 3 when the user operates the shutter button to instruct the recording of a still image. 4 that are the same as those in FIG. 4 are assigned the same reference numerals as in FIG.

  In FIG. 5, the first line from the top is the process of the image sensor 11, the second line is the process of the A / D converter 12, the third line is the process of the frame memory 13 and the buffer memory 21, and the fourth line. Represents the process of the DSP 14, the fifth line represents the process of the display unit 16, the sixth line represents the process of the DSPs 23 and 24, and the seventh line represents the process of the recording control unit 18.

In FIG. 5, at time t _A , the user operates the shutter button to instruct recording of a still image. When the user operates a shutter button for instructing recording of a still image at time t _A , the operation input unit 25 sends an input signal for instructing recording of a still image corresponding to the operation to the control unit. 26. The control unit 26 selects the buffer memory 21 that stores the image data corresponding to the moving image displayed on the display unit 16 at time t _A from the buffer memories 21-1 to 21-3. In the example of FIG. 5, since the moving image displayed on the display unit 16 at time t _A is the moving image of frame # 2 (E2), image data corresponding to the moving image of frame # 2 is stored. The buffer memory 21-2 is selected. The control unit 26 supplies an output selection signal representing the selected buffer memory 21-2 to the selector 22.

The selector 22 supplies the DSP 23 with image data of frame # 2 of pixels suitable for a still image from the buffer memory 21-2 based on an output selection signal representing the buffer memory 21-2. After a time t ₆ after the user issues a still image recording command, the DSP 23 starts image processing for the still image on the image data of frame # 2 of the pixel suitable for the still image from the selector 22, The still image data of frame # 2 after the image processing is supplied to the DSP 24. Then, the DSP 24 encodes the still image data of the frame # 2 of the pixel suitable for the still image from the DSP 23 into a predetermined format and supplies it to the recording control unit 18 (F2).

After time t ₇ after the DSP 23 starts image processing for the still image, the recording control unit 18 records the still image data of the frame # 2 of the pixel suitable for the still image from the DSP 24 in the recording unit 19. The recording unit 19 records the still image data of frame # 2 (G2).

Further, the control unit 26 stores the next frame of the image data of frame # 3 stored in the buffer memory 21 (buffer memory 21-3) at the time t _A (ie, the image data of frame # 4 and subsequent frames). The selector 20 is controlled so that it is not supplied to any of the buffer memories 21. That is, the control unit 26 supplies a storage selection signal representing the buffer memory 21-3 to the selector 20, and stores the image data of the frame # 3 of the pixel suitable for the still image in the buffer memory 21-3. A storage selection signal representing the buffer memory 21-1 is not supplied to the selector 20. Since the storage selection signal is not supplied from the control unit 26, the selector 20 does not supply image data from the A / D conversion unit 12 on and after the frame # 4 of the pixel suitable for the still image to all the buffer memories 21. Therefore, only the frame memory 13 stores the image data after frame # 4 of pixels suitable for still images (C′4).

  The control unit 26 supplies the storage selection signal to the selector 20 after the still image data of all frames # 2 is supplied from the selector 22 to the DSP 23. That is, after the still image data of all the frames # 2 is supplied to the DSP 23, the buffer memory 21 resumes storing the image data of the pixels suitable for the still image.

As described above, when a still image recording command is issued by the user, the frame #i _A (i _A displayed on the display unit 16 when the user commands recording (time t _A ). Still image data corresponding to a moving image is recorded in the recording unit 19.

  Here, the number of necessary buffer memories 21 will be described with reference to FIG.

As shown in FIG. 5, in the digital camera 1 (FIG. 3), the frame #i _A displayed on the display unit 16 is displayed when a user instructs to record a still image (time t _A ). Since the still image data corresponding to the moving image is recorded in the recording unit 19, the buffer memory 21 stores the image data corresponding to the moving image of frame #i _A displayed on the display unit 16. There is a need.

However, depending on the number of buffer memories 21 that store one frame of image data, it corresponds to the moving image of frame #i _A displayed on the display unit 16 when a user issues a still image recording command. The image data to be deleted may be deleted from the buffer memory 21 in some cases. That is, the buffer memories 21-1 to 21-3 sequentially store the image data for each frame, and when the image data is stored in all the buffer memories 21, the buffer memory in which the image data has already been stored. New image data is stored sequentially from the buffer memory 21 storing the oldest image data.

That is, since the buffer memory 21 is updated in order from the buffer memory 21 that stores the oldest image data, depending on the number of the buffer memories 21, the display may be performed when a user issues a still image recording command. In some cases, the buffer memory 21 storing the image data corresponding to the moving image of frame #i _A displayed on the unit 16 has already been updated and the image data has been deleted.

  In order to prevent such a case from occurring, the number f of the buffer memories 21 is determined by, for example, the manufacturer based on the following equation (1), and the digital camera 1 includes the number f of buffer memories. 21 is prepared.

f = (t ₁ + t ₂ + t ₃ + t ₄ + t ₅ ) / t ₀ (1)

  According to Expression (1), the time from when the imaging device 11 captures the subject and starts outputting the image signal of frame #i to when the display unit 16 finishes displaying the moving image of frame #i. The number of image signal frames that can be output by the image sensor 11 is the number f of the buffer memory 21. When the number f of the buffer memory 21 is a decimal number, the decimal part is rounded up.

For example, if t ₀ = 33 ms (output 30 frames of image signal per second), t ₁ = 10 ms, t ₂ = 15 ms, t ₃ = 10 ms, t ₄ = 25 ms, t ₅ = 33 ms, the number f is , 2.818, and so on. The number f after the decimal point is rounded up, so the number f of the buffer memory 21 is three.

  The number f of the buffer memories 21 described above is the minimum number of buffer memories 21 that are necessary, and the number of buffer memories 21 may be any number as long as it is f or more. Also, when one buffer memory 21 does not store one frame of image data but stores a plurality of frames of image data, the storage capacity of all the buffer memories 21 stores image data of f frames or more. The number of buffer memories 21 is not limited as long as the capacity is possible.

  Next, with reference to FIG. 6, moving image processing in which the digital camera 1 in FIG. 3 displays a moving image will be described. This moving image processing is started, for example, when the user operates a display button to command display of a moving image, and is performed for each frame. 6 is performed as image processing at the same time as the still image processing in FIG. That is, the digital camera 1 in FIG. 3 simultaneously performs both the moving image processing in FIG. 6 and the still image processing in FIG. 7 as image processing.

  In step S <b> 51, the image sensor 11 captures a subject, acquires the pixel value of each pixel of frame #i having the number of pixels suitable for a still image, and outputs the pixel value to the A / D converter 12. After the process of step S51, the process proceeds to step S52, and the A / D converter 12 A / D converts the image signal of the frame #i of the pixel suitable for the still image output from the image sensor 11, and the process proceeds to step S53. move on. In step S53, the A / D converter 12 supplies the frame memory 13 with the image data after A / D conversion of the frame #i of the pixel suitable for the still image, and stores it. The frame memory 13 supplies the stored image data of frame #i to the DSP 14. After the process of step S53, the process proceeds to step S54, and the A / D converter 12 supplies the selector 20 with image data of frame #i of the pixel suitable for the still image subjected to A / D conversion in step S52. Proceed to step S55.

  In step S55, the DSP 14 performs image processing for the moving image on the image data of frame #i of the pixel suitable for the still image from the frame memory 13, and the pixel suitable for the moving image after the image processing. Is supplied to the selector 15 and the DSP 17.

  In step S <b> 56, the selector 15 displays the moving image on the display unit 11 based on the moving image data of the frame #i of the pixel suitable for the moving image from the DSP 14. Specifically, the operation input unit 25 receives an operation of a display button for instructing a user to display a moving image, and supplies an input signal representing a moving image display command corresponding to the operation to the control unit 26. To do. The control unit 26 supplies a display control signal for instructing the selector 15 to display the moving image based on the input signal representing the moving image display command. The selector 15 displays a moving image based on the moving image data of the frame #i of the pixel suitable for the moving image from the DSP 14 on the display unit 11 according to a display control signal instructing the display of the moving image from the control unit 26. Let

  In step S57, the recording control unit 18 converts the moving image data of the frame #i of pixels suitable for the moving image from the DSP 17 into a predetermined format, supplies the converted data to the recording control unit 18, and proceeds to step S58. In step S58, the recording control unit 18 supplies the moving image data of the frame #i of the pixel suitable for the moving image from the DSP 17 to the recording unit 19 for recording, and ends the process.

  Note that the process of step S58 may be performed only when the recording of a moving image is instructed by the user.

  Next, still image processing in which the digital camera 1 stores a still image will be described with reference to FIG. This still image processing is started when image data of pixels suitable for the still image of frame #i is supplied to the selector 20 in step S54 of FIG. 6, and is performed for each frame.

  In step S71, the selector 20 causes the buffer memory 21 to store the image data of the frame #i of the pixel suitable for the still image supplied from the A / D conversion unit 12 in step S54 of FIG.

  Specifically, the control unit 26 supplies the selector 20 with a storage selection signal representing the buffer memory 21 that stores the image data of frame #i. For example, the control unit 26 selects a storage selection signal indicating selection of the buffer memory 21 in the order of the buffer memory 21-1, the buffer memory 21-2, the buffer memory 21-3, the buffer memory 21-1,. To supply. The selector 20 selects the buffer memory 21 based on the storage selection signal, and stores the image data in the selected buffer memory 21. The selector 20 does not store the image data in all the buffer memories 21 when the storage selection signal is not supplied from the control unit 26. All the buffer memories 21 supply the stored image data to the selector 22.

  After the process of step S71, the process proceeds to step S72, and the control unit 26 determines whether or not a still image recording is instructed. Specifically, the user views a moving image displayed on the display unit 16 in step S56 of FIG. 6 and when a desired moving image is displayed, the user presses a shutter button for instructing recording of a still image. Perform the operation. The operation input unit 25 supplies an input signal for instructing recording of a still image corresponding to the operation to the control unit 26. The control unit 26 determines whether or not the recording of the still image is instructed by determining whether or not an input signal for instructing recording of the still image is supplied from the operation input unit 25.

  If it is determined in step S72 that recording of a still image is not instructed, that is, if it is determined that an input signal for instructing recording of a still image is not supplied from the operation input unit 25, the processing ends.

  If it is determined in step S72 that recording of a still image has been commanded, that is, if it is determined that an input signal for instructing recording of a still image has been supplied from the operation input unit 25, the process proceeds to step S73, and the control unit 20 stops the supply of the storage selection signal to the selector 20, and the selector 20 stops storing the image data of the frame after the image data of the frame #i stored in the buffer memory 21.

After the processing of step S73, the process proceeds to step S74, and the control unit 26 displays the frame #i _A displayed on the display unit 16 when a still image recording is instructed (for example, time t _{A in} FIG. 5). The buffer memory 21 storing the image data corresponding to the moving image, that is, the image data of the frame #i _{A to be} recorded is selected.

Here, since the user performs an operation to instruct recording of a still image when a desired moving image is displayed while viewing the moving image displayed on the display unit 16, the user wants to record the still image as a still image. The image is a moving image of frame #i _A displayed on the display unit 16 when the recording of a still image is commanded. Accordingly, the control unit 26 selects the buffer memory 21 in which the image data corresponding to the moving image of frame #i _A displayed on the display unit 16 when the recording of the still image is instructed is stored.

Specifically, the control unit 26 stores the image data of the frame #i _A of the pixel suitable for the still image before the time t _A when the recording of the still image is instructed (t ₃ + t ₄ ). The memory 21 is selected. Then, the control unit 26 supplies an output selection signal representing the selected buffer memory 21 to the selector 22. Based on the output selection signal from the control unit 26, the selector 22 supplies the DSP 23 with image data of the frame #i _A of the pixel suitable for the still image from the buffer memory 21 represented by the output selection signal.

After the processing in step S74, the process proceeds to step S75, and the DSP 23 performs image processing for the still image on the image data of the frame #i _A of the pixel suitable for the still image from the selector 22, and the image processing. Still image data of a frame #i _{A having} a pixel suitable for a later still image is supplied to the selector 15 and the DSP 24.

In step S <b> 76, the selector 15 determines whether or not to display the still image of the frame #i _A of the pixel suitable for the still image from the DSP 23 on the display unit 16. Specifically, when the user performs a display button operation to instruct the display of a still image, the operation input unit 25 receives the operation, and an input signal that represents an instruction to display a still image corresponding to the operation Is supplied to the control unit 26. The control unit 26 supplies a display control signal for instructing the selector 15 to display a still image based on an input signal representing the still image display command. The selector 15 determines whether or not to display a still image on the display unit 11 by determining whether or not a display control signal for instructing display of a still image is supplied from the control unit 26.

In step S76, when it is determined that the display unit 16 displays a still image of the frame #i _A of a pixel suitable for a still image, that is, a display control signal for instructing display of the still image is supplied from the control unit 26. In the case where it is determined that the still image has been received, the process proceeds to step S77, where the selector 15 causes the display unit 16 to display the still image based on the still image data of the frame #i _A of the pixel suitable for the still image from the DSP 23. At this time, the display of the moving image of frame #i in step S56 of FIG. 6 on the display unit 16 is stopped for several seconds (for example, 2 seconds), and the still image of frame #i _A is displayed on the display unit 16 during that time. . The selector 15 supplies the moving image data of the frame #i from the DSP 14 and the still image data of the frame #i _A from the DSP 23 to the display unit 16, and the moving image of the frame #i and the frame #i _A It is also possible to display both of the still images.

After step S77, or in step S76, if it is determined not to display a still image on the display unit 16, that is, it is determined that the display control signal instructing display of the still image is not supplied from the control unit 26. In step S78, the DSP 24 converts the still image data of the frame #i _A of pixels suitable for the still image from the DSP 23 into a predetermined format and supplies it to the recording control unit 18.

After the processing in step S78, the process proceeds to step S79, and the recording control unit 18 records still image data of frame #i _A of pixels suitable for the still image from the DSP 24. At this time, the recording control unit 18 moves the moving image data of the frame #i suitable for the moving image supplied from the DSP 17 in step S58 of FIG. 6 and the frame #i _{A of} the pixel suitable for the still image from the DSP 24. The still image data is time-division multiplexed and recorded in the recording unit 19. When the recording of the still image data of frame #i _A in the recording unit 19 is completed, the control unit 26 resumes the storage of the image data stopped in step S73 in the buffer memory 21.

In the above-described still image processing of FIG. 7, image data of frame #i of pixels suitable for a still image is stored in the buffer memory 21, and when a still image recording is instructed, it is displayed on the display unit 16 at that time. Image data corresponding to the moving image of the current frame #i _A is read from the buffer memory 21, and still image data corresponding to the image data is recorded in the recording unit 19. As a result, the user can record the still image data of frame #i _A displayed on the display unit 16 when the user instructs to record a still image without missing a photo opportunity.

  In the still image processing of FIG. 7, image processing and encoding for a still image are performed on the image data corresponding to the image signal of the pixel suitable for the still image captured by the image sensor 11 in step S <b> 51 of FIG. 6. Since the still image data after the recording is recorded in the recording unit 19, still image data having a larger number of pixels than that of the moving image data, that is, high resolution can be recorded.

  On the other hand, in the video camera device described in Patent Document 1, an image recorded as a still image has the same number of pixels as an image recorded as a moving image, that is, about 300,000 pixels (VGA (640 × 480)), and still images have insufficient performance in terms of resolution.

As described above, in the digital camera 1 of FIG. 3, the moving image processing of FIG. 6 and the still image processing of FIG. 7 are performed at the same time. Therefore, while the moving image data of the frame #i is being recorded, the frame #i _{Even when} the still image data of _A is recorded, the recording of the moving image data of frame #i is not interrupted. Further, the moving image of frame #i can be continuously displayed on the display unit 16 without being frozen.

  FIG. 8 is another diagram for explaining the operation timing of the digital camera 1 in FIG. 3 when the user operates the shutter button to command recording of a still image. 4 and FIG. 5 are denoted by the same reference numerals as those in FIG. 4 and FIG.

  In FIG. 8, the first line from the top represents the process of the frame memory 13 and the buffer memory 21, the second line represents the process of the DSP 14, and the third line represents the process of the display unit 16. Note that the vertical axis of the processing of the display unit 16 in the third row represents the horizontal line (row) of the moving image to be displayed.

  That is, the display unit 16 in FIG. 8 updates (displays) the moving image displayed on itself for each horizontal line. Hereinafter, the j-th horizontal line (j is a positive number) of the moving image is referred to as a horizontal line #j. The number of horizontal lines in one frame is L (L is an even number). For convenience of explanation, the horizontal line number L is described as an even number here, but it may be an odd number. That is, the display unit 16 displays the horizontal lines # 1 to #L of the moving image of the frame #i.

In FIG. 8, the display unit 16 displays from the horizontal line #L of the moving image of one frame to the horizontal line # 1 in order during the time t ₅ . For example, the display unit 16 displays from the horizontal line #L of the moving image of the frame # 1 to the horizontal line # 1 in order during the time t ₅ (E′1). Thereafter, the display unit 16 updates the display from the horizontal line #L to the horizontal line # 1 in order from the horizontal line #L of the moving image of the frame # 2 during the time t ₅ . Therefore, for example, when the horizontal line #L of the moving image of frame # 2 is displayed, the moving image displayed on the display unit 16 is the horizontal line #L of the moving image of frame # 2 after the update, The horizontal lines # 1 to # L-1 of the moving image of the frame # 1 before the update are obtained.

As described above, in FIG. 8, two frames of moving images are mixed in the moving image displayed on the display unit 16. However, when the display of the moving image of one frame is completed, only the moving image of the one frame is displayed on the display unit 16. For example, at the time t _B when the display of the moving image of frame # 2 ends, only the moving image of frame # 2 is displayed on the display unit 16.

Here, when the recording of the still image is instructed by the user at time t _C and the display of the moving image of one frame is not completed at time t _C , the display unit 16 displays the frame # 2 before the update. Since the updated moving image of frame # 3 is displayed in a mixed manner, the control unit 26 determines whether the user has commanded the recording of the still image corresponding to the moving image of frame # 2 before the update, or the updated frame. It is necessary to determine whether the recording of the still image corresponding to the moving image # 3 is instructed. In other words, the control unit 26 stores a buffer that stores image data corresponding to a moving image displayed on the display unit 16 when a user instructs to record a still image based on a predetermined rule. It is necessary to select the memory 21. Hereinafter, the buffer memory 21 in which the image to be recorded selected by the control unit 26 is stored is referred to as a selected buffer memory 21.

  Therefore, a process in which the control unit 26 selects the selection buffer memory 21 based on the first to third rules will be described with reference to FIGS. 9 to 11. The rule for selecting the selection buffer memory 21 is not limited to the first to third rules.

  FIG. 9 is a diagram for explaining the first rule.

  In the upper graph of FIG. 9, the horizontal axis represents time, and the vertical axis represents a horizontal line (row) in which the display of the moving image is updated. In FIG. 9, the display unit 16 displays the moving images of frames # 2 to # 4 in order. Note that the time at which the display of the moving image of frame # 2 is set to 0. The lower block of FIG. 9 represents the selection buffer memory 21 selected by the control unit 26 based on the first rule.

In the first rule, as shown in FIG. 9, when the recording of the still image is instructed by the user (time t _C ), the number of horizontal lines larger than L / 2 (a majority) is displayed on the display unit 16. The buffer memory 21 in which the image data corresponding to the moving image of the pre-update frame t _C or the post-update frame (t _C +1) is stored is selected as the selection buffer memory 21. In addition, when the same number of horizontal lines of the moving image of the frame t _C before the update and the frame (t _C +1) after the update are displayed on the display unit 16, for example, it corresponds to the moving image of the frame t _C before the update. The buffer memory 21 in which the image data is stored is selected as the selection buffer memory 21.

For example, during the horizontal line #L of the moving image frame # 2 to the horizontal line # L / 2 is updated (from time 0 to time t _5/2), recording of a still image is instructed by the user If _{(0 ≦ t C ≦ t 5} /2), the control unit 26, as selected buffer memory 21, selects the buffer memory 21-1 image data corresponding to the moving image before the update frame # 1 is stored To do.

  That is, for example, when the horizontal line #L of the moving image of frame # 1 displayed on the display unit 16 is updated to the horizontal line #L of the moving image of frame # 2 (time 0), the horizontal line # 1 Through # L-1 have not been updated yet, the display unit 16 has the horizontal line #L of the moving image of the updated frame # 2 and the L / 2 of the frame # 1 which is the pre-updated frame. A large number of L-1 horizontal lines # 1 to # L-1 are displayed. Therefore, the control unit 26 selects the buffer memory 21-1 in which the image data corresponding to the moving image of the frame # 1 is stored as the selection buffer memory 21.

  When the horizontal line # (L-1) to the horizontal line # (L / 2-1) of the moving image of the frame # 1 is updated, similarly to when the horizontal line #L of the frame # 1 is updated. Since the moving image in which the number of horizontal lines larger than L / 2 is displayed is the moving image of frame # 1 before the update, the buffer memory 21-1 is selected. Also, when the horizontal line # L / 2 of the moving image of frame # 1 is updated, the moving image of frame # 1 before update and frame # 2 after update are displayed in L / 2 horizontal lines respectively. Therefore, since it is a moving image of frame # 1 before the update, the buffer memory 21-1 is selected.

Between until the (from time t _5/2 to time t ₅₎ horizontal line #L is updated from the horizontal line # L / 2 of the moving image frame # 2, the recording of a still image is instructed by the user If _{(t 5/2 <t C} ≦ t 5), the control unit 26, as selected buffer memory 21, buffer memory image data of the frame # 2 corresponding to the moving image of frame # 2 is stored 21- 2 is selected. That is, the moving image on which more horizontal lines than L / 2 are displayed is the updated moving image of frame # 2, so the buffer memory 21-2 is selected.

Furthermore, during to the (from time t ₅ to time 3t _5/2) horizontal line # L / 2 from the horizontal line #L of the moving image frame # 3 is displayed, recording of a still image is instructed by the user If _{(t 5 <t C ≦ 3t} 5/2), the control unit 26 selects the buffer memory 21, it updates the previous frame # 2 of the buffer memory 21-2 image data are stored corresponding to the moving picture Select. That is, since the moving image on which the horizontal line of L / 2 or more is displayed is the moving image of frame # 2 before the update, the buffer memory 21-2 is selected.

As described above, according to the first rule, when the recording of the still image is instructed by the user, the pre-update frame t _C or the post-update frame in which more horizontal lines than L / 2 are displayed. Since the buffer memory 21 in which the image data corresponding to the moving image of (t _C +1) is stored is selected as the selection buffer memory 21, a still image desired by the user can be recorded.

That is, the user instructs recording of a still image when a desired moving image is displayed while viewing the moving image displayed on the display unit 16. At this time, the displayed moving image may include a mixture of moving images of two frames before and after the update, but when the user displays many desired image horizontal lines, the desired moving image is displayed. Is displayed, and there is a high possibility of instructing recording of a still image. Therefore, according to the first rule, image data corresponding to the moving image of the pre-update frame t _C or the post-update frame (t _C +1) in which more horizontal lines than L / 2 are displayed is stored. When the selected buffer memory 21 is selected as the selected buffer memory 21, a still image corresponding to a moving image on which many horizontal lines desired by the user are displayed can be recorded.

  Next, still image processing in which the digital camera 1 stores a still image when the first rule is used will be described with reference to FIG. This still image processing is started when image data of pixels suitable for the still image of frame #i is supplied to the selector 20 in step S54 of FIG. 6, and is performed for each frame. Note that the still image processing of FIG. 10 is performed simultaneously with the moving image processing of FIG.

  Since the process of step S91 thru | or step S93 is the same as the process of step S71 thru | or S73 of FIG. 7, description is abbreviate | omitted.

In step S94, when the recording of the still image is instructed by the user (when the recording is instructed by the user) (time t _{C in} FIG. 8), the control unit 26 displays more horizontal lines than L / 2. Determine whether it has been updated. That is, between the time 0 to t _5/2 in FIG. 9, when the recording of a still image is instructed by the user _{(0 ≦ t c ≦ t 5} /2), the horizontal lines to be updated, the horizontal line #L From # L / 2, since the display of the number of horizontal lines greater than L / 2 has not been updated, it is determined that the display of the number of horizontal lines greater than L / 2 has not been updated. On the other hand, between times t _5/2 to t ₅ in FIG. 9, when the recording of a still image is instructed by the user _{(t 5/2 <t c} ≦ t 5), the horizontal lines to be updated, the horizontal line Since the display of #L to # 1 and the number of horizontal lines greater than L / 2 has been updated, it is determined that the display of the number of horizontal lines greater than L / 2 has been updated.

If it is determined in step S94 that the display of horizontal lines greater than L / 2 has been updated at the time of the still image recording command by the user, the process proceeds to step S95, and the control unit 26 updates the updated frame ( a buffer memory 21 for storing image data of a frame (t _C +1) corresponding to the moving image t _C +1), is selected as the selected buffer memory 21. That is, the image data corresponding to the updated moving image of the frame (t _C +1) in which the number of horizontal lines larger than L / 2 (a majority) is displayed on the display unit 16 when the user instructs to record a still image. Is selected as the selected buffer memory 21.

For example, between times t _5/2 to t ₅ in FIG. 9, when the recording of a still image is instructed by the user _{(t 5/2 <t c} ≦ t 5), the control unit 26, the frame after updating The buffer memory 21-2 storing the image data corresponding to the moving image # 2 is selected as the selection buffer memory 21.

On the other hand, if it is determined in step S94 that the display of the horizontal lines greater than L / 2 has not been updated when the user instructs to record a still image, the process proceeds to step S96, and the control unit 26 a buffer memory 21 for storing image data of a frame t _C corresponding to the moving image frame t _C, selects as the selected buffer memory 21. In other words, when the user instructs to record a still image, the buffer memory 21 stores image data corresponding to the moving image of the frame t _C before update in which the horizontal line of L / 2 or more is displayed on the display unit 16. Is selected as the selection buffer memory 21.

For example, between times 0 and t _5/2 in FIG. 9, when the recording of a still image is instructed by the user _{(0 ≦ t c ≦ t 5} /2), corresponding to the moving image pre-update frame # 1 The buffer memory 21-1 in which the image data to be stored is stored is selected as the selection buffer memory 21.

  After the processing of step S95 or S96, the control unit 26 supplies an output selection signal representing the selection buffer memory 21 to the selector 22. Based on the output selection signal from the control unit 26, the selector 22 supplies the DSP 23 with the image data from the buffer memory 21 indicated by the output selection signal, and the process proceeds to step S97.

  The processing in steps S97 through S101 is the same as that in steps S75 through S79 in FIG.

  FIG. 11 is a diagram for explaining the second rule.

  In FIG. 11, the display screen of the display unit 16 is divided into nine, which are regions 41-1 to 41-9. Hereinafter, when it is not necessary to distinguish the regions 41-1 to 41-9, they are collectively referred to as a region 41.

  For example, before starting the display of the moving image, the user selects the region 41 in which the important part of the image is estimated to be displayed from the regions 41-1 to 41-9 and sets it as the attention region. To do.

In the second rule, the control unit 26 corresponds to the moving image of the frame t _c ′ displayed in the area 41 set by the user when the recording of the still image is instructed by the user (time t _C ). The buffer memory 21 in which the image data to be stored is stored is selected as the selection buffer memory 21.

As described above, according to the second rule, the buffer memory 21 in which the image data corresponding to the moving image of the frame t _c ′ displayed in the region 41 where it is estimated that the important part of the image is displayed is stored. Since the selection buffer memory 21 is selected, a still image desired by the user can be recorded. That is, the user instructs recording of a still image when a desired moving image is displayed while viewing the moving image displayed on the display unit 16. At this time, the displayed moving image may include two frames of moving images before and after the update, but the user has a desired moving image in an area 41 where an important part of the image is displayed. When displayed, it is determined that a desired moving image is displayed, and there is a high possibility of commanding recording of a still image. Thus, in the second rule, the image data of the 'frame t _c corresponding to the moving image' frame t _c that important parts of the image is displayed in the area 41 is assumed to be displayed is stored buffer By selecting the memory 21 as the selection buffer memory 21, a still image corresponding to a moving image desired by the user can be recorded.

  Next, still image processing in which the digital camera 1 stores a still image when the second rule is used will be described with reference to FIG. This still image processing is started when image data of pixels suitable for the still image of frame #i is supplied to the selector 20 in step S54 of FIG. 6, and is performed for each frame. Note that the still image processing of FIG. 12 is performed simultaneously with the moving image processing of FIG. In addition, the user sets, as the attention area, the area 41 in which an important part of the image is estimated to be displayed.

  Since the process of step S111 thru | or step S113 is the same as the process of FIG.7 S71 thru | or S73, description is abbreviate | omitted.

In step S <b> 114, the control unit 26 is a region set as a region of interest among the region 41 (FIG. 11) of the display screen of the display unit 16 when the user instructs to record a still image (time t _C ). The buffer memory 21 in which image data corresponding to the moving image of the frame t _c ′ displayed in 41 is stored is selected as the selection buffer memory 21.

For example, when the area 41-3 is set as the attention area by the user, the buffer memory 21 in which the image data corresponding to the moving image of the frame t _c ′ displayed in the area 41-3 is selected is selected. The buffer memory 21 is selected.

  The control unit 26 supplies an output selection signal representing the selection buffer memory 21 to the selector 22. Based on the output selection signal from the control unit 26, the selector 22 supplies the image data from the buffer memory 21 represented by the output selection signal to the DSP 23, and proceeds to step S115.

  The processing from step S115 to step S119 is the same as that from step S75 to S79 in FIG.

  FIG. 13 is a diagram for explaining the third rule.

  In FIG. 13, the same components as those in FIG. 8 are denoted by the same reference numerals, and the description thereof will be omitted because it will be repeated.

According to the third rule, when a user instructs to record a still image, moving images of two frames, a pre-update frame t _C and a post-update frame (t _C +1), which are mixedly displayed on the display unit 16 are displayed. Two buffer memories 21 each storing corresponding image data are selected as the selection buffer memory 21. When the user instructs to record a still image, the moving image of the two frames, the frame t _C before the update and the frame (t _C +1) after the update, is not displayed on the display unit 16 in a mixed manner, that is, 1 When the update of the display of the moving image of one frame is completed and the user issues a command to record a still image (t _C = t _B ), the image data corresponding to the moving image of that one frame is stored. One buffer memory 21 to be selected is selected as the selection buffer memory 21.

As shown in FIG. 13, when the user instructs to record a still image at time t _C , the moving image displayed on the display unit 16 at that time includes a moving image of frame # 2 and a moving image of frame # 3. The image is mixed (E′2).

  Therefore, the control unit 26 selects the buffer memory 21-2 storing the moving image of frame # 2 and the buffer memory 21-3 storing the moving image of frame # 3 as the selection buffer memory 21. The output selection signals representing the buffer memories 21-1 and 21-2 are sequentially supplied to the selector 22. The selector 22 first supplies the DSP 23 with image data of frame # 2 of pixels suitable for a still image from the buffer memory 21-2.

After time t ₆ from the time when the recording of the still image is instructed by the user, the DSP 23 starts image processing for the still image on the image data of the frame # 2 of the pixel suitable for the still image from the selector 22, and the frame # 2 is supplied to the DSP 24 as still image data. Then, the DSP 24 encodes the still image data of the frame # 2 of the pixel suitable for the still image into a predetermined format and supplies it to the recording control unit 18 (F′2). Then, after time t ₇ after the DSP 23 starts image processing for the still image with respect to the image data of frame # 2, the recording control unit 18 records the still image data of frame # 2 of the pixel suitable for the still image. And the recording unit 19 records the still image data of frame # 2 (G′2).

  The selector 22 finishes supplying the image data of the frame # 2 of the pixel suitable for the still image from the buffer memory 21-2 to the DSP 23, and then selects the pixel suitable for the still image from the buffer memory 21-3. The image data of frame # 3 is supplied to the DSP 23. The DSP 23 ends the image processing for the still image for the image data of the frame # 2 of the pixel suitable for the still image from the selector 22, and then the image data of the frame # 3 of the pixel suitable for the still image from the selector 22 Image processing for a still image is started and supplied to the DSP 24 as still image data of frame # 3. Then, the DSP 24 encodes the still image data of the frame # 3 of the pixel suitable for the still image into a predetermined format, and supplies it to the recording control unit 18 (F′3).

  The recording control unit 18 starts recording the still image data of the frame # 3 of the pixel suitable for the still image from the DSP 24 after finishing the recording of the still image data of the frame # 2 of the pixel suitable for the still image, Still image data of frame # 3 of pixels suitable for a still image is recorded in the recording unit 19 (G′3).

As described above, according to the third rule, when the recording of the still image is instructed by the user, the pre-update frame t _C and the post-update frame (t _C) displayed mixedly on the display unit 16. Since the buffer memory 21 in which the image data corresponding to the moving images of the two frames +1) is stored is selected as the selection buffer memory 21, a still image desired by the user can be recorded. That is, the user instructs recording of a still image when a desired moving image is displayed while viewing the moving image displayed on the display unit 16. At this time, the displayed moving image may include a mixture of moving images of two frames, a frame t _C before update and a frame (t _C +1) after update. Therefore, according to the third rule, the buffer memory 21 in which the image data corresponding to the moving images of the two frames is stored is selected as the selection buffer memory 21, so that the possibility of the moving image desired by the user is obtained. A still image corresponding to both moving images of two frames can be recorded.

  Next, still image processing in which the digital camera 1 stores a still image when the third rule is used will be described with reference to FIG. This still image processing is started when image data of pixels suitable for the still image of frame #i is supplied to the selector 20 in step S54 of FIG. 6, and is performed for each frame. Note that the still image processing of FIG. 14 is performed simultaneously with the moving image processing of FIG.

  Since the process of step S131 thru | or step S133 is the same as the process of FIG.7 S71 thru | or S73, description is abbreviate | omitted.

In step S134, the control unit 26 displays two frames, a pre-update frame t _C and a post-update frame (t _C +1), which are displayed mixedly on the display unit 16 when a still image recording is instructed. Two buffer memories 21 each storing image data corresponding to a moving image are selected as selection buffer memories 21. Then, the control unit 26 supplies an output selection signal representing one of the two selection buffer memories 21 (for example, the buffer memory 21 storing the image data of the frame t _C before the update) to the selector 22. Based on the output selection signal from the control unit 26, the selector 22 supplies the DSP 23 with image data of pixels suitable for the still image from the buffer memory 21 represented by the output selection signal.

  The processing in steps S135 through S139 is the same as the processing in steps S75 through S79 in FIG.

After the process of step S139, the process proceeds to step S140, and the control unit 26 selects the other one of the two selected buffer memories 21 (for example, the buffer memory 21 that the output selection signal does not represent in step S134 (for example, the updated frame ( An output selection signal representing the buffer memory 21) storing the image data of t _C +1) is supplied to the selector 22. Then, based on the output selection signal from the control unit 26, from the buffer memory 21 represented by the output selection signal. The image data of the pixel suitable for the still image is supplied to the DSP 23. The DSP 23 performs image processing for the still image on the image data of the pixel suitable for the still image from the selector 22, and the image processing is performed. Still image data of pixels suitable for a later still image is supplied to the selector 15 and the DSP 24.

  The processing in steps S141 to S144 is the same as the processing in steps S76 to S79 in FIG.

  As described above, in the digital camera 1 of FIG. 3, the control unit 26 records the still image desired by the user by selecting the selection buffer memory 21 based on the first to third rules. it can.

  FIG. 15 is a flowchart for describing moving image processing based on the added pixel value performed by the digital camera 1 of FIG. In FIG. 15, the imaging device 11 does not supply the pixel value of each pixel to the A / D conversion unit 12, but for each predetermined region, an added pixel value obtained by adding the pixel values of the pixels included in the region is obtained. Supply (output) to the A / D converter 12.

  That is, in step S151, the image sensor 11 captures an image of a subject and outputs the added pixel value of frame #i to the A / D converter 12 as an image signal. As described above, the image sensor 11 outputs the added pixel value to the A / D converter 12, so that the image sensor 11 outputs the pixel value of each pixel to the A / D converter 12. The number of pixel values output from 11 can be reduced. Thereby, the current consumption of the image sensor 11 can be reduced. As a result, low power consumption of the digital camera 1 can be realized.

  The processing in steps S152 through S158 is the same as the processing in steps S52 through S58 in FIG. In the image processing in step S155, since the number of pixels of the image data is already small, the filter processing for converting the number of pixels may not be performed.

  FIG. 16 is another flowchart illustrating image processing performed by the digital camera 1 of FIG.

  In step S171, the control unit 26 determines whether or not a shutter button (not shown) for instructing recording of a still image is “half-pressed” by the user. Specifically, the user “half-presses” the shutter button when the operation mode of the digital camera 1 is set to a still image mode (recording mode) for recording a still image. Note that “half-pressing” the shutter button means pressing the shutter button to a predetermined position in the middle of the stroke of the shutter button. Further, pressing down to the end of the stroke of the shutter button is called “full press”, and when the user instructs to record a still image, the user presses the shutter button “full press”.

  If it is determined in step S171 that the shutter button for instructing recording of a still image is not “half-pressed” by the user, that is, the operation mode of the digital camera 1 is a normal mode in which recording of a still image is not performed. When it is determined that the digital camera 1 is, the digital camera 1 performs moving image processing based on the added pixel value of FIG. That is, when the operation mode of the digital camera 1 is the normal mode, the imaging device 11 supplies the added pixel value to the A / D conversion unit 12 and realizes low power consumption of the digital camera 1. Further, since the digital camera 1 does not perform the still image processing of FIG. 7, FIG. 10, FIG. 12, or FIG. The power consumption of the camera 1 can be reduced.

  In step S173, when it is determined that the shutter button for instructing recording of a still image is “half-pressed” by the user, the digital camera 1 sets the operation mode to the still image mode, and the moving image processing of FIG. And the still image processing of FIG. 7, FIG. 10, FIG. 12, or FIG. That is, when the operation mode of the digital camera 1 is the still image mode, the image sensor 11 outputs the pixel value of each pixel as an image signal. Therefore, the number of pixels of the image signal output from the image sensor 11 is the operation mode. Changes and increases compared to the normal mode.

  When the recording of the still image data is finished or the shutter button is canceled (cancelled), the digital camera 1 changes the operation mode from the still image mode to the normal mode.

  As described above, by switching the operation mode of the digital camera 1 to the normal mode or the still image mode, the still image is not recorded as compared with the case where the operation mode of the digital camera 1 cannot be switched. Power consumption can be reduced.

  17A and 17B show scanning speeds when the image pickup device 11 in FIG. 3 adopts a so-called rolling shutter or an electronic shutter system that performs a line-sequential (line-sequential) (line-scan) operation called a focal plane ( It is a figure explaining exposure speed.

  17A and 17B, the horizontal axis represents time, and the vertical axis represents a horizontal line (row) to be scanned (exposure). In FIGS. 17A and 17B, the image sensor 11 acquires the pixel value of each pixel of the 30 fps image.

  FIG. 17A shows an example of the scanning speed of the image sensor 11.

  In FIG. 17A, the time for imaging the subject and acquiring the pixel value of each pixel of the image of one frame, that is, the time for scanning from the horizontal lines #L to # 1 is about 1/30 second. That is, the image sensor 11 captures the subject, acquires the pixel value of each pixel of the image of one frame in about 1/30 second, and immediately after the acquisition of the pixel value of each pixel of the image of one frame ends. The pixel value of each pixel of the next frame image is acquired. By repeating this process, the image sensor 11 obtains an image of 30 fps.

  FIG. 17B shows another example of the scanning speed of the image sensor 11.

In FIG. 17B, the time to capture the subject and acquire the pixel value of each pixel of the image of one frame, that is, the time to scan from the horizontal line #L to # 1 is t _f seconds (which is shorter than _1/30 seconds). t _f <1/30). That is, the imaging device 11 captures the subject, acquires a pixel value of each pixel of an image of one frame at t _f seconds, 1/30 seconds after the start of the imaging, the image of the next frame The pixel value of each pixel is acquired. By repeating this process, the image sensor 11 obtains an image of 30 fps.

  As described above, in FIG. 17B, an image having the same frame rate of 30 fps as in FIG. 17A is captured, but the scanning time of the image sensor 11 is shortened. Thereby, it becomes possible to reduce the distortion of the image of the subject to be imaged.

  That is, in FIG. 17A and FIG. 17B, the image pickup device 11 employs an electronic shutter system that performs row-sequential operations, and thus images to be picked up have no simultaneity. That is, the image sensor 11 acquires an image for each horizontal line and does not acquire images of all horizontal lines at the same time. Thereby, when the imaging device 11 captures a moving subject, distortion may occur in the captured image.

  The distortion can be reduced by using a mechanical shutter to give the captured image a synchronism. However, when the subject is imaged and the acquired image is used as a moving image, the mechanical shutter is used. It is better not to. Therefore, in the digital camera 1 of FIG. 3, since the image acquired by the image sensor 11 is used not only as a still image but also as a moving image, it is preferable not to use a mechanical shutter.

  Therefore, in FIG. 17B, without using the mechanical shutter, the scanning speed of the image sensor 11 is increased compared to the frame rate, and images of all horizontal lines are taken almost simultaneously. Thereby, distortion of the photographed image can be reduced.

  FIG. 18 is a block diagram illustrating another functional configuration example of the digital camera 1.

  The same components as those in FIG. 3 are denoted by the same reference numerals as those in FIG.

  In the digital camera 1 of FIG. 18, an image sensor 51 is provided instead of the image sensor 11, the A / D converter 12, and the frame memory 13 of FIG. 3.

  The image sensor 51 images a subject for each pixel. Specifically, the image sensor 51 receives reflected light from the subject for each pixel and performs photoelectric conversion. And the image pick-up element 51 performs A / D conversion with respect to the image signal after photoelectric conversion for every pixel, and memorize | stores the image data after A / D conversion. The image sensor 51 supplies the image data after A / D conversion stored in all the pixels to the DSP 14 and the selector 20.

  FIG. 19 shows a configuration example of the image sensor 51 of FIG.

  The image sensor 51 includes, for each pixel, a photoelectric conversion unit 61 that photoelectrically converts reflected light from the subject in units of pixels, and an A / D that performs A / D conversion of the image signal that has been subjected to photoelectric conversion by the photoelectric conversion unit 61 in units of pixels. A D converter 62 and a memory 63 for storing image data after A / D conversion by the A / D converter 62 in units of pixels are provided.

  As described above, since the photoelectric conversion unit 61 and the A / D conversion unit 62 are provided for each pixel in the image sensor 51, the image sensor 51 does not operate in a row sequential manner to acquire an image. Images can be acquired simultaneously for all pixels. That is, the image picked up by the image pickup device 51 is synchronous, and it is possible to reduce subject distortion that occurs when picking up a moving subject.

  In the image sensor 51, the photoelectric conversion unit 61, the A / D conversion unit 62, and the memory 63 provided for each pixel may be two-dimensionally arranged, for example, stacked or different. You may make it arrange | position three-dimensionally by arrange | positioning on a surface.

  In addition, the digital camera 1 shown in FIG. 3 described above can be formed as a system LSI (Large Scale Integration) by forming predetermined parts or all parts on one chip.

  As described above, in the digital camera 1, the control unit 26 stores image data corresponding to the moving image displayed on the display unit 16 when the operation input unit 25 receives a still image recording command. The buffer memory 21 to be selected is selected from the three buffer memories 21, and an output selection signal representing the buffer memory 21 is supplied to the selector 22. The selector 22 supplies the image data stored in the buffer memory 21 represented by the output selection signal to the recording control unit 18 via the DSPs 23 and 24 and causes the recording unit 19 to record the image data. The moving image displayed when the recording is instructed can be easily recorded as a still image.

  Next, the series of processes described above can be performed by dedicated hardware or by software. When a series of processing is performed by software, a program constituting the software is installed in a general-purpose computer or the like.

  Therefore, FIG. 20 shows a configuration example of an embodiment of a computer in which a program for executing the series of processes described above is installed.

  The program can be recorded in advance on a hard disk 105 or a ROM 103 as a recording medium built in the computer.

  Alternatively, the program is stored temporarily on a removable recording medium 111 such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored permanently (recorded). Such a removable recording medium 111 can be provided as so-called package software.

  In addition to installing the program on the computer from the removable recording medium 111 as described above, the program can be transferred from a download site to a computer wirelessly via a digital satellite broadcasting satellite, a LAN (Local Area Network), The program can be transferred to a computer via a network such as the Internet, and the computer can receive the program transferred in this way by the communication unit 108 and install it in the built-in hard disk 105.

  The computer includes a CPU (Central Processing Unit) 102. An input / output interface 110 is connected to the CPU 102 via the bus 101, and the CPU 102 operates an input unit 107 including a keyboard, a mouse, a microphone, and the like by the user via the input / output interface 110. When a command is input as a result, the program stored in a ROM (Read Only Memory) 103 is executed accordingly. Alternatively, the CPU 102 also transfers from a program stored in the hard disk 105, a program transferred from a satellite or a network, received by the communication unit 108 and installed in the hard disk 105, or a removable recording medium 111 attached to the drive 109. The program read and installed in the hard disk 105 is loaded into a RAM (Random Access Memory) 104 and executed. Thereby, the CPU 102 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 102 outputs the processing result from the output unit 106 configured with an LCD (Liquid Crystal Display), a speaker, or the like, for example, via the input / output interface 110, or from the communication unit 108 as necessary. Transmission and further recording on the hard disk 105 are performed.

  Here, in the present specification, the processing steps for describing a program for causing the computer to perform various processes do not necessarily have to be processed in time series in the order described in the flowcharts, but in parallel or individually. This includes processing to be executed (for example, parallel processing or processing by an object).

  Further, the program may be processed by one computer or may be distributedly processed by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.

It is a figure which shows an example of the flowchart explaining the still image recording process which records a still image during the recording of a moving image in the conventional camera system. It is a figure which shows an example of the flowchart explaining the still image recording process which records a still image during the display of a moving image in the conventional camera system. It is a block diagram which shows the functional structural example of the digital camera to which this invention is applied. It is a figure explaining the timing of operation | movement of the digital camera of FIG. 3 when a user operates the display button and instruct | indicates the display of the moving image on a display part. It is a figure explaining the timing of operation | movement of the digital camera of FIG. 3 when a user operates the shutter button and commands recording of a still image. It is a flowchart explaining the moving image process in which the digital camera of FIG. 3 displays a moving image. 4 is a flowchart illustrating still image processing in which the digital camera 1 stores a still image. FIG. 6 is another diagram for explaining the timing of the operation of the digital camera in FIG. 3 when the user operates the shutter button to instruct the recording of a still image. It is a figure explaining a 1st rule. It is a flowchart explaining the still image process in which the digital camera at the time of using a 1st rule memorize | stores a still image. It is a figure explaining a 2nd rule. It is a flowchart explaining the still image process which the digital camera 1 at the time of using a 2nd rule memorize | stores a still image. It is a figure explaining a 3rd rule. It is a figure explaining the still image process in which the digital camera at the time of using a 3rd rule memorize | stores a still image. 4 is a flowchart for describing moving image processing based on an added pixel value performed by the digital camera of FIG. 3. 4 is another flowchart illustrating image processing performed by the digital camera in FIG. 3. FIG. 4 is a diagram illustrating a scanning speed when an electronic shutter system that performs row-sequential (line scanning) operation is employed in the image sensor of FIG. 3. It is a block diagram which shows the other functional structural example of a digital camera. It is a figure which shows the structural example of the image pick-up element of FIG. It is a block diagram which shows the structural example of one Embodiment of the computer to which this invention is applied.

Explanation of symbols

  1 Digital Camera, 11 Image Sensor, 12 A / D Converter, 13 Frame Memory, 14 DSP, 15 Selector, 16 Display Unit, 17 DSP, 18, Recording Control Unit, 19 Recording Unit, 20 Selector, 21 Buffer Memory, 22 Selector, 23 DSP, 24 DSP, 25 Operation input section, 26 Control section, 101 Bus, 102 CPU, 103 ROM, 104 RAM, 105 Hard disk, 106 Output section, 107 Input section, 108 Communication section, 109 Drive, 111 Removable recording Medium

Claims (14)

In an imaging device that images a subject and records image data of the subject,
Imaging means for imaging the subject and outputting the image data;
Moving image processing means for performing image processing for moving images on the image data output from the imaging means;
Display means for displaying an image based on the image data on which the image processing for the moving image has been performed by the image processing means for moving image;
A plurality of storage means for storing each of the plurality of temporally continuous image data output from the imaging means;
Receiving means for receiving an instruction to record the image;
When a command for recording the image is received by the receiving unit, a storage unit that stores the image data corresponding to the image displayed on the display unit is selected from a plurality of the storage units. A selection means;
Still image processing means for performing still image processing on the image data stored in the storage means selected by the selection means;
Still image encoding means for encoding the image data subjected to image processing for the still image by the still image processing means;
Recording control means for recording the image data encoded by the still image encoding means ,
The display means updates the image displayed on itself for each horizontal line,
The selection means is updated when the number of horizontal lines of the updated image displayed on the display means is less than or equal to a predetermined number when the image recording command is received by the receiving means. When the storage means for storing the image data corresponding to the previous image is selected, and the number of horizontal lines of the updated image displayed on the display means is larger than the predetermined number, the updated image An image pickup apparatus characterized by selecting the storage means for storing image data corresponding to . The imaging apparatus according to claim 1, wherein the moving image image processing unit performs conversion processing for converting the number of pixels of the image data as image processing for the moving image. A moving image encoding unit that encodes the image data on which the image processing for the moving image has been performed by the moving image image processing unit;
The imaging apparatus according to claim 1, wherein the recording control unit further records the image data encoded by the moving image encoding unit. The recording control unit records the image data encoded by the moving image encoding unit and the image data encoded by the still image encoding unit on a single recording medium. Item 4. The imaging device according to Item 3. The storage capacities of the plurality of storage means are storage capacities determined based on the time from the start of the output of the image data by the imaging means until the end of the display of the image based on the image data. The imaging apparatus according to claim 1. The accepting means further accepts a command for changing the mode of the imaging device to a recording mode for recording the image,
The imaging apparatus according to claim 1, wherein the imaging unit changes the number of pixels of the image data when an instruction for setting the recording mode is received by the receiving unit. The imaging apparatus according to claim 1 , wherein the predetermined number is ½ of the number of horizontal lines of the image. The imaging means includes
Pixel imaging means for imaging the subject in pixel units;
An image signal obtained as a result of being imaged by the pixel imaging means, A / D (Analog / Digital) conversion in pixel units,
The imaging apparatus according to claim 1, further comprising: a pixel storage unit that stores the image data obtained as a result of conversion by the conversion unit in units of pixels and outputs the image data for each pixel. The imaging apparatus according to claim 8 , wherein the pixel imaging unit, the conversion unit, and the pixel storage unit are two-dimensionally arranged. The imaging apparatus according to claim 8 , wherein the pixel imaging unit, the conversion unit, and the pixel storage unit are arranged three-dimensionally. The imaging means is a line sequential scanning type imaging device,
The imaging apparatus according to claim 1, wherein a scanning speed of the imaging device is faster than a frame rate of the image data. In an imaging method of an imaging apparatus that images a subject and records image data of the subject,
An imaging step of imaging the subject and outputting the image data;
A moving image processing step for performing image processing for moving images on the image data output by the imaging step;
A display control step of displaying an image on a display unit based on the image data on which the image processing for the moving image has been performed by the processing of the image processing step for moving image;
A storage control step of storing a plurality of temporally continuous image data output by the imaging step processing in a plurality of storage means;
An accepting step for accepting an instruction to record the image;
Storage means for storing the image data corresponding to the image displayed on the display means when a command for recording the image is received by the processing of the reception step, from among the plurality of storage means A selection step to select;
A still image processing step for performing image processing for the still image on the image data stored in the storage means selected by the selection step;
A still image encoding step for encoding the image data that has been subjected to image processing for the still image by the processing of the still image processing step;
Look including a recording control step of recording the image data encoded by the processing of the still image for encoding steps,
The processing of the display control step updates the image displayed on the display means for each horizontal line,
In the process of the selection control step, the number of horizontal lines of the updated image displayed on the display means is less than or equal to a predetermined number when the image recording command is received by the process of the receiving step. If the storage means for storing the image data corresponding to the image before the update is selected, and the number of horizontal lines of the updated image displayed on the display means is greater than the predetermined number, An imaging method comprising selecting the storage means for storing image data corresponding to the updated image. In a program for causing a computer to perform a recording process of imaging a subject and recording image data of the subject.
An imaging step of imaging the subject and outputting the image data;
A moving image processing step for performing image processing for moving images on the image data output by the imaging step;
A display control step of displaying an image on a display unit based on the image data on which the image processing for the moving image has been performed by the processing of the image processing step for moving image;
A storage control step of storing a plurality of temporally continuous image data output by the imaging step processing in a plurality of storage means;
An accepting step for accepting an instruction to record the image;
Storage means for storing the image data corresponding to the image displayed on the display means when a command for recording the image is received by the processing of the reception step, from among the plurality of storage means A selection step to select;
A still image processing step for performing image processing for the still image on the image data stored in the storage means selected by the selection step;
A still image encoding step for encoding the image data that has been subjected to image processing for the still image by the processing of the still image processing step;
Look including a recording control step of recording the image data encoded by the processing of the still image for encoding steps,
The processing of the display control step updates the image displayed on the display means for each horizontal line,
In the process of the selection control step, the number of horizontal lines of the updated image displayed on the display means is less than or equal to a predetermined number when the image recording command is received by the process of the receiving step. If the storage means for storing the image data corresponding to the image before the update is selected, and the number of horizontal lines of the updated image displayed on the display means is greater than the predetermined number, A program for selecting the storage means for storing image data corresponding to the updated image . In a recording medium in which a program for causing a computer to perform a recording process for imaging a subject and recording image data of the subject is recorded.
An imaging step of imaging the subject and outputting the image data;
A moving image processing step for performing image processing for moving images on the image data output by the imaging step;
A display control step of displaying an image on a display unit based on the image data on which the image processing for the moving image has been performed by the processing of the image processing step for moving image;
A storage control step of storing a plurality of temporally continuous image data output by the imaging step processing in a plurality of storage means;
An accepting step for accepting an instruction to record the image;
Storage means for storing the image data corresponding to the image displayed on the display means when a command for recording the image is received by the processing of the reception step, from among the plurality of storage means A selection step to select;
A still image processing step for performing image processing for the still image on the image data stored in the storage means selected by the selection step;
A still image encoding step for encoding the image data that has been subjected to image processing for the still image by the processing of the still image processing step;
Look including a recording control step of recording the image data encoded by the processing of the still image for encoding steps,
The processing of the display control step updates the image displayed on the display means for each horizontal line,
In the process of the selection control step, the number of horizontal lines of the updated image displayed on the display means is less than or equal to a predetermined number when the image recording command is received by the process of the receiving step. If the storage means for storing the image data corresponding to the image before the update is selected, and the number of horizontal lines of the updated image displayed on the display means is greater than the predetermined number, A recording medium recorded in a program, wherein the storage means for storing image data corresponding to the updated image is selected .

Images