JP6332963B2 - Image processing apparatus and image processing apparatus control method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing apparatus control method.

  Conventionally, an image processing apparatus having a so-called digest reproduction function for extracting and reproducing a part of an image from a moving image or a still image has been proposed. Patent Document 1 describes an information processing apparatus that extracts and reproduces an image showing a person designated by a user and an image showing a person having a high relationship with the designated person.

JP 2011-82915 A

  On the other hand, some movies and still images shot with a digital camera may be inappropriate for digest playback depending on the shooting conditions. Depending on the image selection conditions, images that are inappropriate for digest playback may be selected. There was a case.

  Accordingly, an object of the present invention is to select and reproduce an appropriate image when a part of the recorded image is extracted and reproduced.

In order to achieve the above object, an image processing apparatus according to the present invention provides :
An image processing apparatus having photographing means, in response to a still image recording instruction, a still image photographed according to the still image photographing instruction, a still image pre-shooting moving image photographed before photographing the still image, From the image recorded on the recording medium, recording control means for controlling the pre-shooting additional information indicating the vertical and horizontal postures at predetermined intervals of the image processing apparatus that is shooting the pre-shooting moving image to the recording medium, and the image recorded on the recording medium A reproduction control means for automatically selecting an image to be reproduced, and the reproduction control means is based on the pre-photographing additional information, and before and after taking a still image in which the vertical and horizontal postures of the image processing apparatus change during photographing. A moving image is determined, and control is performed so as not to select a moving image before still image shooting in which it is determined that the vertical and horizontal postures of the image processing apparatus have changed as a reproduction target .

  When a part of the recorded images is extracted and reproduced, an appropriate image can be selected and reproduced.

It is a block diagram which shows the structure of the image processing apparatus in this embodiment. It is a flowchart regarding the recording control in the moving image / still image recording mode in the present embodiment. It is a flowchart explaining the imaging | photography post-process in this embodiment. It is a figure which shows the example of an image displayed on the display part 104 at the time of review display. It is the schematic regarding the moving image and still image which are recorded in a moving image and still image recording mode. It is a figure which shows the additional information before imaging | photography instruction | indication. It is a flowchart regarding the reproduction | regeneration processing of the data recorded by the moving image / still image recording mode in this embodiment. It is a figure which shows the example of a display of the icon in reproduction | regeneration mode. It is a flowchart regarding the album playback processing in the album playback mode.

  Embodiments of the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating a configuration example of a camera 100 as an image processing apparatus. The camera 100 includes a first imaging unit 101, a second imaging unit 102, a display control unit 103, a display unit 104, an audio input unit 105, an audio processing unit 106, and an audio output unit 107. The camera 100 also includes an attitude detection unit 108, an image processing unit 109, an operation unit 110, a control unit 111, a memory 112, a recording / playback unit 113, a recording medium 114, a signal processing unit 115, and a communication unit 116. The internal bus 117 is used for transferring various data and commands between the blocks of the camera 100.

  The first imaging unit 101 and the second imaging unit 102 include, for example, a lens unit, a diaphragm, a shutter, and an imaging element. The first image pickup unit 101 and the second image pickup unit 102 are obtained by converting an optical image of a subject captured by a lens into an image signal using an image sensor such as a CCD sensor or a CMOS sensor by controlling a light amount by a diaphragm. The analog image signal is converted into a digital image signal. The converted digital image signal is temporarily stored in the memory 112.

  In the present embodiment, the first imaging unit 101 is used as a main camera for the user of the camera 100 to photograph a subject, and the second imaging unit 102 is a sub camera, opposite to the first imaging unit 101. It is assumed that the camera 100 is directed toward the user in the direction. The direction in which the second imaging unit 102 is installed is not limited to the direction opposite to the first imaging unit 101, and may be installed in a direction different from the first imaging unit 101.

  The display control unit 103 is a microcomputer that performs display control for displaying an image on the display unit 104, and performs a process of reading a digital image signal temporarily stored in the memory 112 and displaying the digital image signal on the display unit 104. Do. The display control unit 103 also performs processing for causing the display unit 104 to display image data included in a moving image file or a still image file read from the recording medium 114 by the recording / playback unit 113. The display unit 104 may be, for example, a liquid crystal panel or an organic EL panel mounted on the camera 100, or may be a display device (for example, a television, a monitor, a projector, or the like) different from the camera 100.

  The audio input unit 105 collects (collects) audio around the camera 100 using, for example, a built-in omnidirectional microphone, and digitally outputs the obtained analog audio signal using an A / D conversion unit (not shown). To obtain a digital audio signal. The digital audio signal acquired by the audio input unit 105 is transmitted to the audio processing unit 106.

  At the time of recording, the audio processing unit 106 performs processing such as level optimization processing, frequency limiting processing, noise reduction processing, and the like on the digital audio signal transmitted from the audio input unit 105, and the processed digital audio signal Is stored in the memory 112. If necessary, the audio processing unit 106 performs a process of compressing the processed digital audio signal using a known general audio compression method such as AC3 or AAC. Further, the audio processing unit 106 performs a process of decoding the compressed audio data included in the audio file or the moving image file read from the recording medium 114 by the recording / reproducing unit 113 during reproduction.

  The audio output unit 107 is, for example, a speaker or an audio output terminal (analog terminal / digital terminal). For example, if the audio output unit 107 is a speaker, the digital audio signal decoded by the audio processing unit 106 is converted into an analog audio signal and output to the outside as audio. When the audio output unit 107 is an audio output terminal, the digital audio signal decoded by the audio processing unit 106 is converted into an analog audio signal and output to an external device (external speaker or the like), or the digital audio signal is directly output to the external device ( Output to an optical component equipped with an optical digital terminal).

  The posture detection unit 108 is an angle sensor or a vertical / horizontal sensor for detecting the posture of the camera 100, and the control unit 111 is based on information about the posture of the camera 100 obtained by the posture detection unit 108. It is determined whether the position is horizontal or vertical. In this embodiment, the posture information is 0 degrees when the camera 100 is in the horizontal position (normal position), 90 degrees when the camera 100 is in the vertical position with the first surface, which is the right side when the camera 100 is in the horizontal position, In the case of the vertical position with the second surface, which is the left side surface of the camera 100 opposite to the first surface, facing down, the angle is set to −90 degrees.

  The image processing unit 109 performs image quality adjustment processing on the image signal stored in the memory 112 based on setting values set by the user such as white balance, color, and brightness, or setting values determined from image characteristics. It is a microcomputer equipped with a program for performing. In addition, the image processing unit 109 can reduce the number of pixels of the image indicated by the image signal to reduce the size (view angle) of the image, rotate the image, or perform color conversion. It is also possible to convert a certain image signal into YUV data and vice versa. The processed image signal is stored in the memory 112 again.

  Further, the image processing unit 109 uses the pattern matching or the like from the image signal converted into the YUV data as the subject information, the number of people in the image, the size and position of each face, the smile level, the person part. The brightness of the image is detected. Furthermore, the image processing unit 109 can also combine the image signal output from the first imaging unit 101 output at the same timing and the image signal output from the second imaging unit 102. The image processing unit 109 can also generate a menu screen, an icon, and the like, and further synthesize the image signal output from the first imaging unit 101 and the second imaging unit 102 with the menu screen, the icon, and the like. .

  The operation unit 110 includes, for example, a switch for inputting various operations related to image shooting / playback, such as a power button, a record button, a zoom adjustment button, an autofocus button, and a playback button. In addition, a menu display button, a determination button, other cursor keys, a pointing device, a touch panel, and the like are provided, and an operation signal is transmitted to the control unit 111 when the user operates these keys, buttons, and touch panel.

  The control unit 111 develops a control program for the camera 100 recorded in a ROM (not shown) in the memory 112 and controls each block of the camera 100. The memory 112 temporarily stores a control program and data as a work memory.

  The recording / reproducing unit 113 is a microcomputer equipped with a program for executing the following processes. When recording a moving image, the recording / reproducing unit 113 stores a moving image file together with the moving image data compressed by the signal processing unit 115, the audio data compressed by the audio processing unit 106, and various information such as the shooting date stored in the memory 112. As shown in FIG. The recording / playback unit 113 forms a data stream composed of compressed moving image data and audio data, sequentially records it on a recording medium 114, adds a file header, etc., and conforms to a file format such as FAT or exFAT. Record. The recording / playback unit 113 performs processing for recording the still image data stored in the memory 112 as a still image file on the recording medium 114 together with various types of information such as the shooting date when recording a still image. At the time of reproduction, the recording / reproducing unit 113 reads out a moving image file or a still image file recorded on the recording medium 114 according to the file format.

  The recording medium 114 may be a recording medium built in the camera 100 or a removable recording medium. The recording medium 114 includes all types of recording media such as a hard disk, an optical disk, a magneto-optical disk, a CD-R, a DVD-R, a magnetic tape, a nonvolatile semiconductor memory, and a flash memory. In the case of using removable recording media, the recording / playback unit 113 includes an interface for mounting these removable recording media.

  The signal processing unit 115 performs an encoding process according to a known encoding method such as MPEG or JPEG on the image signal captured by the first imaging unit 101 or the second imaging unit 102, and compresses the information amount. To do. In addition, the signal processing unit 115 extracts compressed moving image data and still image data from the moving image file and still image file read from the recording medium 114 by the recording / reproducing unit 113, and performs a decoding process.

  The communication unit 116 transmits and receives control signals, moving image files, still image files, various data, and the like between the camera 100 and the external device, and is connected to the external device by wire or wirelessly. Note that any communication method may be used. The internal bus 117 is used to transfer various data and commands between the units of the camera 100.

  Note that the processing of the display control unit 103, the audio processing unit 106, the image processing unit 109, the recording / playback unit 113, and the signal processing unit 115 is performed by the control unit 111 executing the above-described processing recorded in a ROM (not shown) or the like. The program may be loaded into the memory 112 and executed.

  Here, the still image recording operation of the camera 100 of the present embodiment will be described. In the control unit 111, the user operates the power button of the operation unit 110, and the mode changeover switch of the operation unit 110 is, for example, any mode such as “still image recording mode”, “moving image recording mode”, “reproduction mode”, and the like. This is confirmed by an instruction signal from the operation unit 110. If it is determined that the mode is the “still image recording mode”, the control unit 111 starts the following operation.

  The control unit 111 causes each block of the camera 100 to prepare for still image recording. The control unit 111 causes the memory 112 to store the image signals obtained from the first imaging unit 101 and the second imaging unit 102 until a shooting start instruction is input from the operation unit 110. Then, the control unit 111 controls the display control unit 103 so that the image signal stored in the memory 112 is read and an image is displayed on the display unit 104. The display control unit 103 may be controlled so that the image signal processed by the image processing unit 109 and stored in the memory 112 is read and an image is displayed on the display unit 104. In the present embodiment, the frame rate of the image signal output from the first imaging unit 101 or the second imaging unit 102 is 30 frames / second, but can be arbitrarily set. In the present embodiment, the size (number of pixels) of a still image to be recorded can be set to one size selected from a plurality of sizes.

  In the present embodiment, the camera 100 combines the image signal from the first imaging unit 101 and the image signal from the second imaging unit 102 that are output at the same timing in the image processing unit 109, It is possible to process as one image signal. The camera 100 can also process the image signal output from the first imaging unit 101 and the image signal output from the second imaging unit 102 separately.

  The control unit 111 determines whether an instruction to start shooting is input from the operation unit 110. When there is an instruction to start shooting, the control unit 111 stores the image signal obtained from the first imaging unit 101 or the second imaging unit 102 in the memory 112, and sets a set value for the image signal stored in the memory 112. The image processing unit 109 is controlled to perform image quality adjustment processing based on the above. In addition, the control unit 111 stores the image signal processed by the image processing unit 109 in the memory 112 and controls the signal processing unit 115 to generate the still image data by encoding the image signal stored in the memory 112. To do. Next, the control unit 111 stores the still image data generated by the signal processing unit 115 in the memory 112 again, and causes the recording / reproducing unit 113 to record the stored still image data on the recording medium 114 as a still image file. Control. When the above processing is completed, the control unit 111 returns each block to the still image recording preparation state again.

  Here, the moving image recording operation of the camera 100 of the present embodiment will be described. In the camera 100 according to the present embodiment, the user operates the power button of the operation unit 110, and the mode change switch of the operation unit 110 operates, for example, which mode is “movie recording mode”, “playback mode”, or the like. This is confirmed by an instruction signal from the unit 110. When it is determined that the video recording mode is set, the following operation is started.

  The control unit 111 causes each block of the camera 100 to prepare for moving image recording. The control unit 111 causes the memory 112 to sequentially store the image signals obtained from the first imaging unit 101 and the second imaging unit 102 until a shooting start instruction is input from the operation unit 110. Then, the control unit 111 controls the display control unit 103 so that the image signal stored in the memory 112 is read and an image is displayed on the display unit 104. The display control unit 103 may be controlled so that the image signal processed by the image processing unit 109 and stored in the memory 112 is read and an image is displayed on the display unit 104. In the present embodiment, it is assumed that the frame rate of the image signal output from the first imaging unit 101 or the second imaging unit 102 is 30 frames / second. In this embodiment, the size (number of pixels) of a moving image to be recorded can be set to one size selected from a plurality of sizes.

  Then, the control unit 111 determines whether an instruction to start shooting is input from the operation unit 110. When there is an instruction to start shooting, the control unit 111 stores the image signal obtained from the first imaging unit 101 or the second imaging unit 102 in the memory 112, and sets a set value for the image signal stored in the memory 112. The image processing unit 109 is controlled to perform image quality adjustment processing based on the above. While the moving image recording is continued, the control unit 111 causes the image processing unit 109 to sequentially process image signals output from the first imaging unit 101 and the second imaging unit 102 at 30 frames / second.

  In the present embodiment, the camera 100 combines the image signal from the first imaging unit 101 and the image signal from the second imaging unit 102 that are output at the same timing in the image processing unit 109, It is possible to process as one image signal. When the frame rates of the first imaging unit 101 and the second imaging unit 102 are different, for example, when the first imaging unit 101 is 30 frames / second and the second imaging unit 102 is 15 frames / second, the first of two frames The image signal of the second imaging unit 102 is combined with the image signal of the imaging unit 101. The camera 100 can also process the image signal output from the first image capturing unit 101 and the image signal output from the second image capturing unit 102 separately according to settings by the user or the like. Then, the control unit 111 sequentially stores the image signal processed by the image processing unit 109 in the memory 112.

  Next, the control unit 111 controls the signal processing unit 115 to sequentially compress the image signals of a plurality of frames stored in the memory 112 to generate moving image data. At this time, the control unit 111 controls the signal processing unit 115 so that each frame image is compression-coded as an intra-frame prediction encoded frame and an inter-frame prediction encoded frame. Then, the control unit 111 sequentially stores each frame image encoded by the signal processing unit 115 in the memory 112.

  On the other hand, when there is an instruction to start shooting, the control unit 111 controls each block so as to perform processing related to sound. The control unit 111 sequentially transfers the analog audio signal output from the audio input unit 105 to the audio processing unit 106, and controls the audio processing unit 106 to perform conversion into a digital signal, sound quality adjustment processing, and the like. When the audio compression is set, the control unit 111 controls the audio processing unit 106 so as to compress the audio signal using, for example, an AC3 or AAC audio compression method according to the setting. Then, the control unit 111 sequentially stores the audio data processed by the audio processing unit 106 in the memory 112.

  Next, the control unit 111 controls the recording / reproducing unit 113 so that the moving image data and audio data stored in the memory 112 are sequentially recorded on the recording medium 114. For example, 15 frames (0.5 seconds) of moving image data and 0.5 seconds of audio data are combined as one set to form a data stream to which various necessary information is added, and the recording medium 114 according to the file system. The control unit 111 controls the recording / reproducing unit 113 so as to record. The recording / reproducing unit 113 may form a data stream by combining 30 frames (one second) of moving image data and one second of audio data. The control unit 111 continues these operations until an instruction to stop moving image recording is given.

  When an instruction to stop shooting is input from the operation unit 110, the control unit 111 stops the processing of the image processing unit 109, and performs signal processing when the compression encoding of the image signal stored in the memory 112 is completed. The compression encoding process of the unit 115 is stopped. Then, the control unit 111 controls the recording / reproducing unit 113 to stop the operation after recording the compressed moving image data and audio data stored in the memory 112 to the recording medium 114 to the end. If necessary, the first frame of the moving image data of the moving image file and the image data of several frames from the beginning are transmitted to the signal processing unit 115 for decoding, and thumbnail image data obtained by thinning out the number of pixels of the decoded digital image signal is obtained. It may be generated and recorded in association with a moving image file. When this process ends, the control unit 111 returns each block to the ready state for moving image recording again.

  In this embodiment, the camera 100 has at least a normal recording mode and a moving image / still image recording mode as image data recording modes. The normal recording mode is a mode for receiving a moving image or still image recording instruction from a user or the like and recording moving image or still image data corresponding to the instruction on a recording medium. On the other hand, the moving image / still image recording mode is a mode for recording still image data corresponding to the recording instruction and moving image data corresponding to the recording instruction in response to one recording instruction from the user or the like. FIG. 2 is a flowchart showing recording control of the camera 100 in a state where the moving image / still image recording mode is set. 2 is executed by the control unit 111 controlling each unit of the camera 100. When the moving image / still image recording mode is set to be valid in the recording standby state and the moving image / still image recording mode is selected by an instruction from the operation unit 110, the processing of FIG. 2 is started.

  In step S <b> 201, the control unit 111 acquires additional information before an instruction to capture still image data (before an instruction to record). Additional information before the shooting instruction (before the recording instruction) to be acquired includes, for example, posture information of the camera 100 output from the posture detection unit 108, information on autofocus (AF) in the first imaging unit 101, information on zoom operation, and images. This is information on the subject output from the processing unit 109. Each item of the additional information before the photographing instruction will be described later. In this embodiment, the additional information before the shooting instruction is acquired in units of one second and stored in the memory 112, but the time interval for storing the additional information before the shooting instruction can be arbitrarily set. The time interval for storing the additional information before the photographing instruction may be, for example, 0.5 second unit or 2 second unit.

  In step S <b> 202, the control unit 111 acquires setting information for the second imaging unit 102 (sub camera) stored in the memory 112. In the sub-camera setting information, information indicating whether the current sub-camera shooting settings are valid or invalid, and an image acquired from the second imaging unit 102 are superimposed on an image acquired from the first imaging unit 101 (main camera). Display position and display size settings are included. The display position and display size of the image acquired from the second imaging unit 102 can be set by the user or the like via the operation unit 110. In step S203, the control unit 111 determines whether the shooting setting of the second imaging unit 102 (sub camera) is valid based on the acquired setting information of the sub camera. Validity / invalidity of the shooting setting of the sub camera can be set by the operation unit 110.

  If the shooting setting of the sub camera is enabled, the control unit 111 advances the process to step S204. In step S <b> 204, the control unit 111 combines an image signal acquired from the first imaging unit 101 and an image signal acquired from the second imaging unit 102 based on the sub camera setting information acquired in step S <b> 203. The image processing unit 109 is controlled so as to be generated. When the shooting setting of the sub camera is invalid (No in step S203), in step S205, the control unit 111 stores the image signal to be stored in the memory 112 from the image signal acquired from the first imaging unit 101. The image processing unit 109 is controlled so as to generate.

  In step S206, the control unit 111 sequentially stores the image signals generated in step S204 or S205 in the memory 112, and sequentially compresses the image signals of a plurality of frames stored in the memory 112 to generate moving image data. The processing unit 115 is controlled. For the audio signal, the control unit 111 controls the audio processing unit 106 to compress the audio data acquired from the audio input unit 105. The compressed moving image data and audio data are stored in the memory 112. In addition, the control unit 111 controls the image processing unit 109 to adjust the display size of the image signal generated in step S204 or step S205, and displays the adjusted image signal on the display unit 104. 103 is controlled.

  In step S207, the control unit 111 determines whether there is an instruction to change to a recording mode other than the moving image / still image recording mode, a playback mode, or the like via the operation unit 110. When the mode is changed (Yes in step S207), in step S208, the control unit 111 stops the image signal compression processing of the signal processing unit 115. In step S209, the control unit 111 deletes the moving image data and audio data stored in the memory 112 and the additional information before the shooting instruction, ends the processing of the moving image / still image recording mode, and processes the changed mode. Move on.

  If there is no mode change (No in step S207), in step S210, the control unit 111 includes the acquired additional information before the shooting instruction and the sub camera setting information stored in the memory 112. Determine if there are any changed items. Here, it may be determined whether or not all items of the additional information and the sub camera setting information before the shooting instruction that have been acquired have been changed, or whether or not only predetermined items have been changed. Good. In the present embodiment, the additional information before the shooting instruction acquired, the posture information of the camera 100, the sub camera setting information, whether the sub camera shooting settings are valid or invalid, and the second imaging unit 102 A change in the display position and display size of the image is determined.

  When there is a changed item (Yes in step S210), in step S211, the control unit 111 deletes the moving image data and audio data stored in the memory 112 and the additional information before the shooting instruction, and returns to the process of step S201. . For example, when the posture of the camera 100 changes from the horizontal position (0 degrees) to the vertical position (90 degrees) during shooting, the moving image data and audio data stored in the memory 112 and the additional information before the shooting instruction are deleted. In step S201, the control unit 111 newly acquires additional information before the shooting instruction. In the present embodiment, the control unit 111 determines that the posture information of the camera 100 has changed when the posture of the camera 100 has changed more than a predetermined value (for example, when it has changed by 90 degrees or more) during shooting of moving image data. .

  When the additional information before the shooting instruction or the setting information of the sub camera is changed, the data stored so far is erased because the setting change is recorded in the data. For example, when the orientation of the camera 100 is changed, moving image data in which the orientation of the image is changed (the image is rotated) is recorded, and when the display position of the image of the second imaging unit 102 is changed, the second imaging unit A state in which the display position 102 is changed is recorded in the moving image data. Since the moving image data in which the change of the setting is recorded in this way is difficult to see, if there is an item changed during the recording of the moving image data among the additional information before the shooting instruction and the setting information of the sub camera, the memory The data stored in 112 is erased.

  If there is no changed item in the additional information before the shooting instruction or the setting information of the sub camera (Yes in step S210), the control unit 111 presses the shutter switch (SW) 1 included in the operation unit 110 in step S212. Determine whether or not. In this embodiment, the shutter switch has a two-stage switch structure, and the first stage (half press) is SW1, and the second stage (full press) is SW2. When SW1 is pressed, the control unit 111 starts operations such as focusing processing, autofocus (AF) processing, automatic exposure processing, auto white balance (AWB) adjustment processing, AF auxiliary light emission processing, and the like in step S213. The first imaging unit 101 is instructed.

  Even when SW1 is pressed in step S212, the additional information before the shooting instruction is continuously stored in the memory 112. In the state where SW1 is pressed in step S212, the moving image data and audio data stored in the memory 112 and the additional information before the shooting instruction are stored even if the additional information before the shooting instruction or the setting information of the sub camera is changed. It will not be erased.

  Next, in step S214, the control unit 111 determines whether or not the shutter switch (SW) 2 has been pressed, that is, whether or not a still image shooting (recording) instruction has been issued. When SW2 is pressed (the shutter switch is fully pressed), in step S215, the control unit 111 controls the display control unit 103 to stop the image display on the display unit 104 of the camera 100. Further, the control unit 111 causes the signal processing unit 115 to temporarily stop the moving image compression processing and the storage processing of the compressed moving image data in the memory 112. Then, the control unit 111 controls the first imaging unit 101 so as to capture a still image corresponding to the timing when SW2 is pressed.

  At this time, the moving image data and audio data already stored in the memory 112 and additional information before the shooting instruction are held. When SW1 is not pressed in step S212, and when SW2 is not pressed in step S214, the control unit 111 continues to perform the processing from step S201.

  In the present embodiment, SW2 is composed of the same member as SW1, and different operations (step S213 and step S215) are assigned to the pressing operations of SW1 and SW2. Not limited to this, SW1 and SW2 are configured by predetermined buttons such as a touch panel, and processing when SW1 is pressed (step S213) and SW2 are pressed to perform still image shooting by selecting the predetermined button. The process (step S215) when it is made may be performed.

  In step S216, the control unit 111 performs post-shooting processing, and then ends this processing. The post-shooting process performed by the control unit 111 will be described with reference to FIG.

  In step S301, the control unit 111 compresses the still image data by compressing the image signal of the first imaging unit 101, which is the still image captured in step S215, at the timing when SW2 in step S214 is pressed. The signal processing unit 115 is controlled to generate. The control unit 111 records still image data processed by the signal processing unit 115, additional information before the shooting instruction acquired and stored in step S201 in FIG. 2, and attribute information of still image data described later as a still image file. The recording / reproducing unit 113 is controlled to record on the medium 114.

  In the present embodiment, the still image data generated in step S301 is an image signal from the first imaging unit 101 corresponding to the timing when SW2 is pressed. Not limited to this, still image data is generated and recorded by combining the image signal from the second imaging unit 102 (sub camera) corresponding to the timing at which SW2 is pressed with the image signal from the first imaging unit 101. May be. In addition to the still image data generated from the image signal from the first imaging unit 101, still image data of the image signal from the second imaging unit 102 may be generated and recorded.

  In step S302, the control unit 111 determines whether the recording time of the moving image data stored in the memory 112 before pressing SW2 is equal to or greater than t0. t0 is a predetermined value and can be set arbitrarily. When the recording time of the moving image data stored in the memory 112 before pressing SW2 is t0 or more, the control unit 111 records the moving image data before pressing SW2 on the recording medium 114 together with the audio data in step S303. In addition, the recording / reproducing unit 113 is controlled.

  In step S302, if the recording time of the moving image data before pressing SW2 is less than t0, in step S304, the control unit 111 obtains the moving image data and audio data before pressing SW2 and the additional information before shooting instruction. to erase.

  In step S305, the control unit 111 determines whether the shooting setting of the sub camera is valid or invalid. The camera 100 according to the present embodiment supports recording of still image data after still image shooting depending on whether the recording time of the moving image data stored in the memory 112 before the SW2 is pressed or whether the shooting setting of the sub camera is valid or invalid. The method of generating the moving image data to be generated is changed.

  When the shooting setting of the sub camera is valid, the image processing unit 109 generates stop motion moving image data in which each frame includes an image signal including an image signal recorded corresponding to the timing when the SW2 is pressed. .

  If the shooting setting of the sub camera is invalid and the recording time of the moving image data stored in the memory 112 before pressing SW2 is t0 or more and the stop motion moving image setting is enabled, the image The processing unit 109 generates stop motion video data. On the other hand, when the shooting setting of the sub camera is invalid and the recording time of the moving image data stored in the memory 112 before pressing SW2 is less than t0, or when the stop motion setting is invalid, the image processing unit 109 does not generate stop motion video data.

  In the moving image / still image recording mode of the present embodiment, the moving image data generated by the image processing unit 109 corresponding to the recording of the still image data is one of the following three. That is, the image processing unit 109 selects any one of moving image data including only moving image data before pressing SW2, moving image data including moving image data before pressing SW2 and stop motion moving image data, and moving image data including only stop motion moving image data. Generate video data.

  If the shooting setting of the sub camera is valid (Yes in step S305), in step S306, the control unit 111 controls the image processing unit 109 to generate an image signal for generating stop motion moving image data. When the still image data and the moving image data have different aspect ratios, the image processing unit 109 stops the still image data recorded corresponding to the timing when the SW2 is pressed so that the aspect ratio of the moving image data becomes the same. An image signal for motion video data is generated. For example, when the aspect ratio of the still image is 4: 3 and the aspect ratio of the moving image is 16: 9, the image processing unit 109 paints the left and right areas of the still image as black as an image signal for stop motion moving image data. Thus, an image signal with an aspect ratio of 16: 9 is generated.

  In step S307, the control unit 111 combines the image signal of the second imaging unit 102 with the image signal for the stop motion moving image data generated in step S306 and the image signal of the first imaging unit 101 corresponding to the timing of SW2 pressing. The image processing unit 109 is controlled to do so. The stop motion moving image data is generated using the combined image signal for stop motion moving image data obtained by combining the image signal for stop motion moving image data generated in step S306 and the image signal of the second imaging unit 102.

  In addition, an image signal obtained by combining the image signal of the first imaging unit 101 and the image signal of the second imaging unit 102 corresponding to the timing of pressing the SW2 is used for review display for review display on the display unit 104. This is a composite image signal. The image signal of the second imaging unit 102 used to generate the composite image signal for stop motion moving image data and the composite image signal for review display is sequentially captured by the second imaging unit during review display on the display unit 104 after taking a still image. 2 is an image signal acquired by the unit 102. In the present embodiment, the control unit 111 causes the second imaging unit 102 to continue acquiring image signals even after taking a still image in step S215.

  In step S308, the control unit 111 controls the image processing unit 109 to resize the composite image signal for review display generated in step S307, and performs display control to display the resized image signal on the display unit 104. The unit 103 is controlled. 4A shows an example of a review display displayed on the display unit 104. An image 401 is an image signal of the first imaging unit 101 corresponding to the timing of pressing SW2, and an image 402 is a second imaging unit. 102 is an image signal. The image 401 is a still image acquired from the first imaging unit 101, but the image 402 is synthesized with the image signals sequentially acquired by the second imaging unit 102 during the review display. In the area of the image 402 in the display unit 104, a moving image is displayed.

  In step S309, the control unit 111 controls the signal processing unit 115 to generate the stop motion moving image data by compressing the composite image signal for the stop motion moving image data generated in step S307.

  In step S310, the control unit 111 determines whether the required time of the stop motion moving image data generated in step S310 is equal to or longer than a predetermined time t1. The time t1 can be set arbitrarily. If the required time of the stop motion video data is not equal to or longer than the time t1, the control unit 111 repeats the processing from step S307 to step S309 until the required time of the stop motion video data reaches the time t1.

  In step S317, the control unit 111 records and reproduces the stop motion moving image data with the required time of time t1 on the recording medium 114 in a form to be added to the moving image data before pressing SW2 recorded in step S303. 113 is controlled, and the processing of FIG.

  If the shooting setting of the sub camera is invalid, in step S311, the control unit 111 determines whether the moving image data and audio data stored before pressing SW2 and the additional information before the shooting instruction are deleted by the processing in step S304. Determine whether or not. If the moving image data and audio data before the SW2 is pressed and the additional information before the shooting instruction are deleted, the processing ends without generating the moving image data after shooting the still image. When the recording time of the moving image data stored in the memory 112 before pressing SW2 is less than t0 and the shooting setting of the sub camera is invalid, only the shot still image data is recorded and corresponds to the still image data. Movie data to be recorded is not recorded.

  If the moving image data and audio data before the SW2 is pressed and the additional information before the shooting instruction are not deleted, the control unit 111 determines whether the setting of the stop motion moving image is valid (step S312). If the setting of the stop motion video is valid, the process of the control unit 111 proceeds to step S313. If the setting of the stop motion video is invalid, the processing is performed without generating the video data after shooting the still image. finish. The setting of the stop motion video can be arbitrarily set by the user via the operation unit 110.

  In step S313, the control unit 111 controls the image processing unit 109 to generate an image signal for stop motion moving image data (the aspect ratio is for moving images), similarly to the processing in step S306 described above. In step S314, the control unit 111 controls the image processing unit 109 to resize the image signal of the first imaging unit 101 at the timing of pressing SW2, that is, the still image data recorded in step S301. Further, the control unit 111 controls the display control unit 103 so that the still image data subjected to the ground size processing is displayed on the display unit 104 for review. FIG. 4B shows an example of an image signal displayed on the display unit 104. An image 401 is an image signal of the first imaging unit 101 corresponding to the timing of pressing SW2 (the still image recorded in step S301). Data).

  In step S315, the control unit 111 controls the signal processing unit 115 to compress the image signal for stop motion moving image data generated in step S313 and generate stop motion moving image data. In step S316, the control unit 111 determines whether the required time of the stop motion video data generated in step S315 is equal to or longer than a predetermined time t2. The time t2 can be set arbitrarily. If the time required for the stop motion video data is not equal to or longer than the time t2, the control unit 111 performs the process of step S315 until the time required for the stop motion video data reaches the time t2.

  In step S317, the control unit 111 controls the recording / reproducing unit 113 to record the stop motion moving image data whose required time is time t2 on the recording medium 114, and ends the process of FIG.

  In the present embodiment, the time t1 used in the determination in step S310 and the time t2 used in the determination in step S316 are set such that the time t1 is equal to or greater than the time t2. In the stop motion moving image generated in step S309, since the image of the second imaging unit 102 during review display is synthesized with the moving image, the facial expression of the user of the camera 100 during review display can be left. On the other hand, since the stop motion video generated in step S315 is a stop motion video consisting of only still images, the stop motion video generated in step S309 is configured to be recorded longer. It is not limited.

  FIG. 5 is a schematic diagram relating to moving images and still images to be recorded in the moving image / still image recording mode. FIG. 5A is a schematic diagram showing the timing (release) when SW2 is pressed and the data stored in the memory 112 in the moving image / still image recording mode. On October 10, 2013, the moving image data M1 (501) and the still image data P1 (502) are stored in the memory 112 in accordance with the release 1 shooting instruction at 15:16:20. Furthermore, the movie data M2 (503) and the still image data P2 (504) are issued at 15:30:45 at the release 2 shooting instruction, and the movie data M3 ( 505) and still image data P3 (506) are stored in the memory 112, respectively.

  FIG. 5B is a schematic diagram showing data recorded on the recording medium 114 in the present embodiment. After the SW2 is pressed, each data shown in FIG. 5A stored in the memory 112 is recorded on the recording medium 114 by the recording / playback unit 113 as a still image file or a moving image file through the processing of FIG. The In the present embodiment, a file including data recorded on the same day is stored in one directory 511 (directory name: 100_1010). The directory name and each file name are determined in accordance with DCF (Design rule for Camera File System). For movie files and still image files that are stored in the same directory and recorded in the movie / still image recording mode, the number used for the movie file name must be smaller than the number used for the still image file name. Each file name is determined. For moving image files and still image files recorded in the normal moving image recording mode or still image recording mode, the file names are determined in the order in which the files are recorded using the smallest number that is not used as the file name.

  In the present embodiment, moving image data recorded in the moving image / still image recording mode on the same day is stored as a series of moving image stream data in one moving image file in a predetermined directory. The moving image file 512 (moving image file name: MDG — 0001.MOV) stores moving image data corresponding to each of the three shootings in the moving image / still image recording mode performed on the same day as shown in FIG. A chapter number is set for each moving image data. The method for storing moving image data is not limited to the method described above, and for example, moving image data corresponding to each shooting may be stored in the directory as separate moving image files.

  In this embodiment, a file name including the character string “MDG” is assigned to a moving image file storing moving image data recorded in the moving image / still image recording mode. The moving image data recorded in the normal moving image recording mode other than the moving image / still image recording mode is a directory as one independent moving image file (the file name does not include the character string “MDG”) different from other moving image data. Stored in.

  Chapter 1 (514) indicates moving image data of only moving image data M1 stored in the memory 112 in response to the release 1 shooting instruction. Chapter 2 (515) indicates moving image data M2 stored in the memory 112 in response to the shooting instruction for release 2, and moving image data of stop motion moving image data generated from the still image data P2. Chapter 3 (516) indicates moving image data including only stop motion moving image data generated from the still image data P3 stored in the memory 112 in accordance with the release 3 shooting instruction.

  Chapter reference information regarding the moving image stream data is recorded in the header portion of the moving image file 512 or the like. Details of the chapter reference information will be described with reference to FIG. In the chapter reference information, the size of the moving image file (521) and the total number of frames of moving image stream data (522) are recorded. Furthermore, the set chapter number (523), the shooting time of each chapter (524), the top frame number of each chapter (525), and the number of frames of each chapter (526) are recorded. Furthermore, the display position (527) of the image of the second imaging unit 102 (sub camera) and the position (528) of stop motion moving image data in each chapter are recorded.

  The chapter number (523) is added each time SW2 is pressed in the moving image / still image recording mode and moving image data is recorded. The shooting date / time (524) of each moving image stream is the same as the shooting date / time of SW2 pressed in the moving image / still image recording mode, that is, the shooting date / time of the corresponding still image data recorded in the moving image / still image recording mode. In this way, by recording the shooting time of the video stream to be the same as the shooting date and time of the corresponding still image data, it is possible to easily recognize the moving image data recorded in the moving image / still image recording mode and the corresponding still image data. It becomes possible.

  The first frame number (525) of each chapter is the frame number of the first frame of each chapter in the entire moving image stream recorded in the moving image file 512. By using this, it is possible to easily find the beginning of each chapter. Become. The display position (527) of the image of the second imaging unit 102 is superimposed when the upper left coordinates of the image of the first imaging unit 101 are (X, Y) = (0, 0). And the horizontal width W and the vertical width H of the image of the second imaging unit 102. When the horizontal width W and the vertical width H of the image of the second imaging unit 102 are 0 as in chapter 1, the shooting setting of the sub camera is invalid and the image of the second imaging unit 102 is not superimposed. Is shown. Using the information of the image display position (527) of the second imaging unit 102, it is possible to trim and reproduce only the image of the second imaging unit 102.

  The position (528) of the stop motion moving image data indicates the frame number of the stop motion moving image portion when the first frame of each chapter is set to 0. In chapter 1, there is no stop motion video portion. In chapter 2, among the moving image data having 180 frames, the second half of the 90th frame to the 179th frame is a stop motion moving image. In chapter 3, all of the 180 frames of moving image data are stop motion moving images. By knowing the position of the stop motion moving image data in this way, it becomes easy to reproduce only the portion of the stop motion moving image in each moving image data or to reproduce only the portion other than the stop motion moving image.

  When moving image data corresponding to each still image is stored in the directory as separate moving image files, chapter reference information other than the chapter number (523) is stored in the header portion of each moving image file. .

  Still image files 517 (file name: IMG — 0002.JPG), 518 (IMG — 0003.JPG), and 519 (IMG — 0004.JPG) stored in the directory are still images shot according to the respective shooting instructions of releases 1, 2, and 3. Includes image data. Each still image file recorded in the moving image / still image recording mode includes still image data, attribute information of the still image data, and an addition before the shooting instruction stored in the memory 112 in the moving image / still image recording mode. Information is recorded in the header portion. In still image files including still image data recorded in the normal still image recording mode, attribute information of the still image data is recorded together with the still image data.

  In the present embodiment, moving image data recorded in the moving image / still image recording mode is stored in one moving image file as a series of moving image stream data, so that chapter reference information and chapters to be stored increase every time a still image is shot. This increases the capacity of the movie file. Therefore, if additional information before shooting instruction, which is information related to moving image data recorded in the moving image / still image recording mode, is stored in the header portion of a large-capacity file, it takes time to detect. For this reason, in the present embodiment, the additional information before the shooting instruction is recorded in the header portion of the corresponding still image data recorded in the moving image / still image recording mode, thereby detecting the additional information before the shooting instruction. It is possible to reduce the time required for

  The attribute information of the still image data specifically includes the manufacturer name and model name of the imaging device that recorded the still image data, the shooting date and time, the subject brightness that can determine overexposure and underexposure of the image, and focusing by autofocus Are the degree of focus indicating the accuracy of the image, setting information of the sub camera at the time of shooting, and the like. Also, a favorite level indicating the priority of the image given by the user, information on whether or not AF auxiliary light for focusing is emitted at the time of shooting, an arbitrary character string (event name) set by the user, etc. Information may be added and recorded as attribute information. Further, by analyzing the still image data by the image processing unit 109, information such as the number of persons included in the still image data, the number of persons, the position of the person, and the size of the face may be recorded as attribute information. Other shooting settings and thumbnail images are also recorded as attribute information of still image data.

  Here, the additional information before the photographing instruction will be described with reference to FIG. As shown in FIG. 6A, in the present embodiment, the additional information before the shooting instruction is recorded in units of one second. The time axis is set to 0 at the time of still image shooting when SW2 is pressed, and in this embodiment, the additional information before the shooting instruction is held from 3 seconds before the shooting of the still image, but the additional information before the shooting instruction is held. The time to perform can be arbitrarily set. For example, in the moving image / still image recording mode, the time for holding the additional information before the shooting instruction is determined in accordance with the time for which moving image data is recorded before still image shooting.

  The items recorded as additional information before the shooting instruction are the posture information of the camera 100, the information on the presence or absence of the AF operation in the first imaging unit 101, the information on the presence or absence of the zoom operation, and the person reflected in the main camera as the subject information. As an example, information on the degree of smile is shown (FIG. 6A).

  As described above, the posture information of the camera 100 is 0 degree when the camera 100 is in the horizontal position (normal position), and when the camera 100 is in the vertical position with the first surface, which is the right side when the camera 100 is in the horizontal position, facing down. At 90 degrees, the posture information of the camera 100 acquired from the posture detection unit 108 is recorded. The information on the presence or absence of the AF operation in the first imaging unit 101 indicates whether the focus lens, the diaphragm, or the like of the first imaging unit 101 is operating or stopped to perform the focusing process. The information on the presence / absence of the zoom operation is based on whether the zoom lens of the first image capturing unit 101 is operating or stopped to perform the optical zoom process, or the first image capturing unit 101 for performing the electronic zoom process. Indicates whether processing of the acquired image signal is being performed (during operation) or stopped. Information on the presence / absence of the AF operation in the first imaging unit 101 and information on the presence / absence of the zoom operation are acquired from the control unit 111.

  The degree of smile of a person is determined by detecting whether the person's face is detected from the image signal converted into YUV data using pattern matching or the like, and whether or not the subject person is smiling based on the degree of the mouth corner of the detected face. The value shown is detected by the image processing unit 109. In the present embodiment, the smile level of the person detected from the image signal obtained from the first image pickup unit 101 is recorded as additional information before the shooting instruction, but additionally detected from the image signal obtained from the second image pickup unit 102. The degree of smile of the person who made it may be recorded. In addition, as the smile level recorded in the additional information before the shooting instruction, the smile levels of a plurality of people may be recorded, or a value calculated in consideration of the smile levels of a plurality of people may be recorded. The smile level of the person in the present embodiment is high (well laughs well) according to the value obtained from the smile level of a predetermined number of people whose face area is larger than the angle of view of the image obtained from the first imaging unit 101. ), Medium (laughing to some extent), and low (not laughing very much).

  Information other than the items described above may be recorded as additional information before the shooting instruction. For example, by analyzing each frame constituting the moving image data by the image processing unit 109, the person included in the moving image data, the position of the person, the size of the face, and the like are recorded as additional information items before the shooting instruction. It may be.

  From FIG. 6A, it can be seen that the posture information of the camera 100 was held in the horizontal position (0 degree) until the still image was taken. Also, the AF operation was only performed 2 seconds before still image shooting, the zoom operation was only performed 3 seconds before, and the operation was stopped thereafter, and the recorded image was stable. I understand that. It can also be seen that the smile level of a person gradually increases from 3 seconds before still image shooting.

  Next, playback processing of data recorded in the moving image / still image recording mode will be described with reference to FIG. The processing in FIG. 7 is executed by the control unit 111 controlling each unit of the camera 100.

  In the playback mode, for still image files and moving image files recorded in the normal still image recording mode and moving image recording mode, still images, thumbnail images of moving images, etc. are sequentially reproduced in order of shooting date and time and displayed on the display unit 104. The When a thumbnail image of a moving image is displayed, when the thumbnail image is selected, reproduction of moving image data corresponding to the selected thumbnail image is started. Further, when a still image is displayed, the displayed still image can be enlarged / reduced or the like via the operation unit 110.

  On the other hand, the data recorded in the moving image / still image recording mode is reproduced by reproducing the recorded still image data or its thumbnail image in the reproduction mode, and the control unit 111 controls the display control unit 103. It is displayed on the display unit 104 (step S701). At this time, in the playback mode, the thumbnail image of the moving image data recorded in the moving image / still image recording mode is not displayed.

  In step S702, the control unit 111 searches for moving image data related to the displayed still image data. Movie data related to still image data refers to movie data (corresponding chapters) generated in response to recording of still image data in movie / still image recording mode, and movie / still image on the same day as the still image data. Video data (related chapters) recorded in the recording mode.

  First, the control unit 111 searches for a moving image file storing moving image data (related chapters) recorded in the moving image / still image recording mode on the same day as the displayed still image data. Specifically, the control unit 111 first searches for a still image file including the displayed still image data recorded on the recording medium 114 and a moving image file stored in the same directory. Then, the control unit 111 determines whether the file name of the moving image file stored in the same directory is a file name including the character string “MDG”. The still image file including the displayed still image data and the moving image file stored in the same directory and including the character string “MDG” in the file name are displayed on the same day as the displayed still image data. Stores moving image data recorded in the image recording mode.

  In this embodiment, the file name is used to determine whether the file is a moving image file recorded in the moving image / still image recording mode. However, the present invention is not limited to this, and the header portion of the moving image file and the like are shown in FIG. The determination may be made based on whether chapter reference information is recorded.

  When a moving image file storing moving image data recorded in the moving image / still image recording mode is recorded on the same day as the displayed still image data, the control unit 111 further supports recording of the displayed still image data. The moving image data (corresponding chapter) generated in this way is searched. Specifically, the control unit 111 uses the chapter reference information recorded in the header portion of the moving image file storing the moving image data recorded in the moving image / still image recording mode on the same day as the displayed still image data. To search for the corresponding chapter.

  In step S703, the control unit 111 determines whether moving image data related to the displayed still image data is recorded on the recording medium 114 based on the search result in step S702. If either the corresponding chapter of the displayed still image data or the related chapter of the displayed still image data is recorded on the recording medium 114, the control unit 111 advances the process to step S704. If neither the corresponding chapter of the displayed still image data nor the related chapter of the displayed still image data is recorded on the recording medium 114, the control unit 111 advances the processing to step S707.

  In step S704, the control unit 111 displays an icon or the like indicating that moving image data related to the displayed still image data is recorded on the display unit 104 based on the search result in step S702. The control unit 103 is controlled.

  FIG. 8 shows an example of icon display in step S704. When the related chapter of the displayed still image data is recorded on the recording medium 114, an icon (802 in FIG. 8) indicating the presence of the related chapter of the displayed still image data is displayed on the display unit 104. Further, when the corresponding chapter of the displayed still image data is recorded on the recording medium 114, an icon (803 in FIG. 8) indicating the presence of the displayed corresponding chapter of the still image data is displayed on the display unit 104. The The image processor 109 generates these icons and superimposes them on the still image data to be displayed (801 in FIG. 8). The control unit 111 controls the display control unit 103 so that the still image data on which the icon is superimposed is displayed on the display unit 104.

  FIG. 8A is a display example when the corresponding chapter of the displayed still image data and the related chapter of the displayed still image data are recorded on the recording medium 114. FIG. 8B is a display example when only the related chapter of the displayed still image data is recorded on the recording medium 114. FIG. 8C shows a display example when neither the corresponding chapter of the displayed still image data nor the related chapter of the displayed still image data is recorded on the recording medium 114.

  In step S705, the control unit 111 determines whether or not the user has instructed the reproduction of the moving image data corresponding to the icon displayed in step S704. In the present embodiment, each icon displayed in step S <b> 704 is configured to be selectable. When the user selects an icon, moving image data corresponding to the icon is reproduced and displayed on the display unit 104.

  For example, when an icon (802 in FIG. 8) indicating the presence of a related chapter of the displayed still image data is selected, the control unit 111 relates the displayed still image data, which is moving image data corresponding to the icon. The recording / reproducing unit 113 is controlled so as to reproduce the chapter. When an icon (803 in FIG. 8) indicating the presence of the corresponding chapter of the displayed still image data is selected, the control unit 111 corresponds to the displayed still image data that is moving image data corresponding to the icon. The recording / reproducing unit 113 is controlled so as to reproduce the chapter.

  When the reproduction instruction of the moving image data corresponding to the icon is made (Yes in step S705), in step S706, the control unit 111 reproduces the moving image data corresponding to the icon instructed to reproduce as described above. The recording / reproducing unit 113 is controlled. Then, after the reproduction of the moving image data is finished, the control unit 111 changes the display control unit 103 so that the display on the display unit 104 returns to the state where the icon indicating the moving image related to the still image data is displayed (step S704). Control.

  If it is determined in step S705 that no instruction to reproduce moving image data corresponding to the icon is given, the control unit 111 determines in step S707 whether an instruction to change still image data displayed on the display unit 104 has been issued. The user can issue an instruction to change still image data to be displayed via the operation unit 110. When receiving an instruction to change the still image data to be displayed (Yes in step S707), the control unit 111 changes the still image data to be displayed on the display control unit 103, and returns the process to step S701. When the instruction to change the still image data to be displayed has not been received (No in step S707), in step S708, the control unit 111 performs an instruction to change to a mode other than the playback mode or other processing via the operation unit 110. Determine if there are instructions. If there is no mode change or other processing instruction, the control unit 111 returns the process to step S704. If there is a mode change or other processing instruction, the control unit 111 ends this processing, and changes the mode. Make changes and other processing.

  For example, in the still image data P2 shown in FIG. 5, moving image data (related chapters) recorded in the moving image / still image recording mode on the same day as the still image data P2, and a moving image generated corresponding to the recording of the still image data P2. Both data (corresponding chapters) are recorded on the recording medium 114. When the still image data P2 is displayed on the display unit 104, as shown in FIG. 8A, an icon 802 and an icon 803 are displayed together with the still image data P2 (801 in FIG. 8). The related chapters of the still image data P2 are chapter 1 (514), chapter 2 (515), and chapter 3 (516) stored in the moving image file 512. The corresponding chapter of the still image data P2 is chapter 2 (515).

  When the icon 802 is selected, moving image data (chapter 1 (514), chapter 2 (515), chapter 3 (516)) stored in the moving image file 512, which is a related chapter of the still image data P2, is reproduced. The When the icon 803 is selected, chapter 2 (515), which is a corresponding chapter of the still image data P2, is reproduced.

  In the present embodiment, in step S704, when moving image data related to the displayed still image data is recorded on the recording medium 114, the control unit 111 performs processing to display each icon. Not limited to this, in step S704, the control unit 111 may perform control to change the icon to be displayed depending on the shooting setting of the sub camera when the displayed still image data is recorded in the moving image / still image recording mode. . The shooting settings of the sub camera when the displayed still image data is recorded in the movie / still image recording mode is recorded in the attribute information of the still image data stored in the still image file in which the still image data is stored. Has been.

  For example, in the still image data recorded by enabling the sub camera shooting setting in the moving image / still image recording mode, the control unit 111 indicates the presence of the corresponding chapter according to the corresponding chapter recorded in the recording medium 114. The display control unit 103 is controlled so as to be superimposed and displayed.

  On the other hand, the control unit 111 indicates the presence of the corresponding chapter even if the corresponding chapter is recorded on the recording medium 114 in the display of the still image data recorded with the shooting setting of the sub camera disabled in the moving image / still image recording mode. The display control unit 103 is controlled not to display the icon. That is, in the still image data recorded with the sub-camera shooting setting disabled, whether only the related chapter of the still image data is recorded or the corresponding chapter is recorded together with the related chapter, FIG. Only icons indicating the presence of related chapters are displayed as in ().

  In the control for changing the icon to be displayed, the sub camera image is combined when the sub camera image is not combined with the still image data recorded in the moving image / still image recording mode as in the present embodiment. This is because the corresponding chapter can be easily reproduced.

  Next, the album playback process will be described with reference to FIG. In the present embodiment, the camera 100 has an album playback mode as a playback mode in addition to the normal playback mode. In the album playback mode, playback is performed so that a comfortable image can be confirmed using at least one of moving image data recorded on the recording medium 114 in the moving image / still image recording mode and still image data recorded on the recording medium 114. Processing (digest reproduction processing) is performed.

  In the album playback mode, the video data recorded on the recording medium 114 in the video / still image recording mode or the still image data recorded on the recording medium 114 is excluded from data that is not suitable for album playback. Perform playback. Data that is not suitable for album playback is, for example, moving image data in which the camera posture is changed or AF operation is not stable due to additional information before the shooting instruction, still image data with low subject brightness, etc. It is. In the present embodiment, the data used in the album playback mode is recorded on the recording medium 114 in the recording mode including the moving image data recorded on the recording medium 114 in the moving image / still image recording mode and the moving image / still image recording mode. Selected from still image data.

  The process of FIG. 9 is started upon receiving an instruction to start the album playback mode via the operation unit 110. The processing in FIG. 9 is executed by the control unit 111 controlling each unit.

  In step S901, the control unit 111 selects the moving image data or the still image used for the album playback mode from the moving image data recorded on the recording medium 114 in the moving image / still image recording mode or the still image data recorded on the recording medium 114. Determine the conditions for extracting data. The extraction condition may be moving image data or still image data arbitrarily selected by the user, or may be determined using attribute information of still image data stored in the still image file or additional information before shooting instructions. Good. When using attribute information of still image data or additional information before shooting instructions, the extraction date includes the shooting date, the person being photographed, the event name added to the still image data or the moving image data, and the like. For example, based on the still image data displayed on the display unit 104 when an instruction to start the album playback mode is received, the shooting date of the still image data, the person included in the still image data, and the still image data are added. The extraction condition may be determined by the user selecting from the event name or the like.

  In step S902, the control unit 111 matches the extraction condition determined in step S901 from the moving image data recorded on the recording medium 114 in the moving image / still image recording mode or the still image data recorded on the recording medium 114. Detect data to be used. Whether or not the moving image data or the still image data matches the extraction condition determined in step S901 is determined based on the attribute information of the still image data stored in the still image file including the still image data recorded on the recording medium 114 or the shooting. Use additional information before giving instructions.

  For example, when the extraction condition determined in step S901 is “the shooting date is October 10, 2013”, the moving image data recorded in the moving image / still image recording mode stored in the directory 511 shown in FIG. All still image data is data that matches the extraction conditions. Specifically, in FIG. 5, chapter 1 (514), chapter 2 (515), and chapter 3 (516) are included in moving image data, and still image data P1, still image data P2, and still image data P3 are included in still image data. Is the data that matches the extraction conditions.

  Next, in step S902, the data detected based on the extraction condition in step S901 is distributed to album reproduction target data (step S905) and non-target data (step S906) in steps S903 and S904.

  In step S903, the control unit 111 determines whether the moving image data in the data detected in step S902 is moving image data in which the posture information of the camera 100 is changed from the additional information before the shooting instruction regarding each moving image data. To do. Like the additional information before the shooting instruction shown in FIG. 6B, the moving image data in which the posture information of the camera 100 has changed more than a predetermined (90 degrees or more) is set as non-target data for album reproduction (step S906). . Such moving image data is recorded, for example, when the posture information of the camera 100 changes while SW1 is pressed in step S212 of FIG. Video data with a camera orientation that changes during recording (camera orientation is rotated) is difficult to view because the orientation of the image changes during playback. exclude.

  The control unit 111 sets the non-target condition for album reproduction other than the change in the posture information of the camera 100 in the moving image data in which the posture information of the camera 100 has not changed and the still image data in the data detected in step S902. It is determined whether or not they match (step S904). Regarding the non-target conditions for album playback other than the change in the posture information of the camera 100, for example, regarding the moving image data, the AF operation or the zoom operation is not stable (FIGS. 6C and 6D), the person of the subject Are those with a low smile level (FIG. 6E). Furthermore, moving image data generated corresponding to still image data that emits AF auxiliary light for AF at the time of still image shooting may be a non-target condition for album reproduction. These determinations are made based on the attached information of the still image data corresponding to each moving image data and the additional information before the shooting instruction.

  Regarding the non-target conditions for album playback, for still image data, for example, the subject brightness is extremely high or low (overexposed or blackened), the focus level is low, or the favorite level is low Things. These determinations are made based on attribute information of each still image data.

  The control unit 111 performs the process of step S907 on the data (step S905) determined as the target data for album playback through the determinations of step S903 and step S904. In step S907, the control unit 111 converts the still image data recorded in the moving image / still image recording mode and the moving image data including the stop motion moving image data generated corresponding to the recording of the still image data into the album reproduction. Search the target data. Then, either the still image data recorded in the moving image / still image recording mode or the stop motion moving image portion of the moving image data generated corresponding to the recording of the still image data included in the target data of the album playback. , Exclude from album playback target data.

  For example, consider a case where the target data for album reproduction includes still image data P2 and chapter 2 (515) shown in FIG. When both the still image data P2 and the chapter 2 (515) are used for album reproduction, the still image data P2 and the stop motion video portion of the chapter 2 (515) generated using the still image data P2 are reproduced. Therefore, the scenes are duplicated and the reproduction becomes redundant.

  Therefore, either the still image data P2 or the stop motion video portion of chapter 2 (515) included in the target data for album playback is excluded from the target data for album playback. That is, only one of the still image data P2 or the moving image data of only the moving image data M2 portion of the chapter 2 (515) is included in the album reproduction target data. The stop motion moving image portion of chapter 2 (515) can be detected from the chapter reference information (528 in FIG. 5C) stored in the header portion of the moving image file 512 or the like.

  Also, consider a case where still image data P3 and chapter 3 (516) are included in the target data for album playback. Chapter 3 (516) consists only of stop motion moving image data generated from the chapter reference information using still image data P3. Therefore, either the still image data P3 or the chapter 3 (516) is excluded from the album reproduction target data. For example, when the image of the second imaging unit 102 is synthesized with the stop motion moving image portion of the chapter 3 (516), the still image data P3 without the synthesized image of the second imaging unit 102 is used as the album reproduction target data. exclude.

  In step S908, the control unit 111 selects moving image data or still image data to be used for album playback from the target data for album playback. You can select all the target data for album playback, or select a predetermined number of data from the target data for album playback based on additional information before shooting instructions and attribute information of still image data Alternatively, the data may be selected so that the reproduction time becomes a predetermined time. Examples of the arbitrary criteria include whether or not the person being photographed has a high smile level, whether or not a plurality of persons are included, and whether or not the favorite level is high.

  In step S909, the control unit 111 controls the signal processing unit 115 to generate album reproduction data for continuously reproducing the data selected in step S908, and displays the album reproduction data on the display unit 104 (preview display). The display control unit 103 is controlled. The order of data to be continuously reproduced can be arbitrarily set based on additional information before photographing and attribute information of still image data. While the preview is being displayed, it is possible to instruct the playback of the album playback data to be paused or restarted via the operation unit 110. Subsequently, in step S910, the control unit 111 causes the user to select whether or not the generated album reproduction data is recorded and stored in the recording medium 114. The control unit 111 controls the image processing unit 109 to generate a screen that allows the user to select whether or not to save the generated album playback data, and displays the generated screen on the display unit 104. 103 is controlled.

  When it is selected that the generated album reproduction data is recorded and stored in the recording medium 114, in step S911, the control unit 111 causes the recording / reproducing unit 113 to record and save the album reproduction data in the recording medium 114. To finish the album reproduction process. When the album reproduction data is not recorded and stored in the recording medium 114, the control unit 111 discards the album reproduction data and ends the album reproduction process.

  As described above, when a part of the recorded images is extracted and reproduced, an appropriate image can be selected and reproduced so that a comfortable image can be confirmed. In the present embodiment, moving image data in which the camera posture changes during shooting, moving image data inappropriate for album playback, or still image data is excluded from the album playback target. You can play an album.

DESCRIPTION OF SYMBOLS 101 1st imaging part 102 2nd imaging part 108 Posture detection part 111 Control part 113 Recording / reproducing part 114 Recording medium 115 Signal processing part

Claims (4)

An image processing apparatus having photographing means,
In response to the still image recording instruction, the still image shot in accordance with the still image shooting instruction, the pre-still image moving image shot before shooting the still image, and the still image shooting moving image before shooting Recording control means for controlling the pre-shooting additional information indicating the vertical and horizontal postures at predetermined intervals of the image processing apparatus to be recorded on a recording medium;
Replay control means for automatically selecting a replay target image from the images recorded on the recording medium,
The playback control means determines a pre-still image moving image in which the vertical and horizontal postures of the image processing device are changing during shooting based on the pre-shooting additional information, and the vertical and horizontal postures of the image processing device have changed. An image processing apparatus that performs control so that a determined moving image before still image shooting is not selected as a reproduction target . Before accepting the still image recording instruction, a pre-shooting process instruction for instructing execution of a pre-shooting process including at least one of a focusing process, an autofocus process, an automatic exposure process, and an auto white balance adjustment process is provided. Accepting means,
The recording control unit performs control so as not to record a moving image shot before the change when the vertical and horizontal postures of the image processing apparatus change before receiving the pre-shooting processing instruction, and After the reception, even if the vertical and horizontal postures of the image processing apparatus change, control is performed so that a moving image shot before the still image recording instruction is received is recorded together with the pre-shooting additional information. The image processing apparatus according to 1. It said reproduction control means, the image processing apparatus according to claim 1, wherein the controller controls so as not to select the video before the still image photographing operation of the focusing process is carried out as a reproduction target in shooting. A control method for an image processing apparatus, comprising:
In response to the still image recording instruction, the still image shot in accordance with the still image shooting instruction, the pre-still image moving image shot before shooting the still image, and the still image shooting moving image before shooting A recording control step for controlling the pre-shooting additional information indicating the vertical and horizontal postures at predetermined intervals of the image processing apparatus to be recorded on a recording medium;
A reproduction control step of automatically selecting an image to be reproduced from images recorded on the recording medium,
In the reproduction control step, based on the pre-shooting additional information, a moving image before shooting a still image in which the vertical / horizontal posture of the image processing device is changing during shooting is determined, and the vertical / horizontal posture of the image processing device is changed. A control method for an image processing apparatus, wherein control is performed so that a determined moving image before still image shooting is not selected as a reproduction target.

Images