JP6439563B2 - Imaging apparatus, imaging method, and program - Google Patents

Description

  The present technology relates to an imaging apparatus, an imaging method, and a program, and more particularly, to an imaging operation using a mechanical shutter.

JP 2012-182657 A JP 2012-129589 A

Patent Document 1 discloses a technique relating to an image pickup apparatus that uses both a so-called rolling shutter type electronic shutter control and a mechanical shutter.
Patent Document 2 discloses a technique for selecting an electronic shutter and a mechanical shutter.

In recent years, image sensors, which have become mainstream, are capable of smooth video playback, but have a different taste from video shot with traditional film cameras. In particular, a video shot at 24 fps, which has been often used for film movies, has a unique image quality (parasitism) and presence. In film, there is always a certain period of unexposed between video frames, which is one factor that produces a unique image quality.
Although there is a period in which no exposure is performed even in a recent moving image system, a smooth image can be obtained during the period not as long as the film. On the other hand, the image quality of the film age has been lost.

  In view of the above, an object of the present technology is to realize imaging capable of obtaining image data with a sense of image quality as in conventional film photography.

An imaging apparatus according to an embodiment of the present technology includes a mechanical shutter that is provided in an incident light path and performs a shutter operation, an imaging element unit that receives incident light via the mechanical shutter, and a mechanical shutter driving unit that drives the mechanical shutter. In the image sensor unit, the image sensor drive unit that executes an electronic shutter operation in which exposure is performed at different exposure timings for each part of the image sensor, and the mechanical shutter drive on the condition that it is moving image capturing A control unit configured to set a mechanical shutter operation to an effective shutter operation in which an open period for each frame period of the mechanical shutter is within a period in which all imaging pixels of the image sensor unit are exposed. .
The mechanical shutter operation here is an operation that opens and closes the mechanical shutter at high speed for each frame period, thereby defining the exposure period of the image sensor unit. With this mechanical shutter operation, a period of a certain length is not provided between the video frames, and moving image capturing with a unique image quality and realistic sensation like a movie by conventional film photography is executed. Like that.

In the imaging apparatus described above, it is conceivable that the control unit instructs the mechanical shutter driving unit to perform the mechanical shutter operation in response to setting the mechanical shutter operation to an effective shutter operation.
That is, when the effective shutter operation is determined, the mechanical shutter operation state is automatically set.

In the imaging apparatus described above, it is conceivable that the control unit performs recommended presentation control of the mechanical shutter operation in response to setting the mechanical shutter operation to an effective shutter operation.
That is, the user can select whether or not to perform the mechanical shutter operation by recommending the mechanical shutter operation.

In the imaging apparatus described above, it is conceivable that the control unit sets the mechanical shutter operation to an effective shutter operation on the condition that the moving image imaging mode is performed at a specific frame rate. For example, the specific frame rate is 24 fps or 23.976 fps.
For example, in the case of a specific frame rate, such as image data of 24P (24 frames / s progressive: 24 frames per second progressive scan), the video effect due to the mechanical shutter operation described above becomes significant. The condition is that the frame rate is a specific one.

In the above-described imaging apparatus, it is conceivable that the control unit sets the mechanical shutter operation to an effective shutter operation with the result of the battery check as one of the conditions.
That is, the mechanical shutter operation is set as an effective shutter operation when the remaining battery level is sufficient or when an external power source is used.

In the above-described imaging apparatus, it is conceivable that the control unit sets the mechanical shutter operation to an effective shutter operation on the condition of the sound collection environment check as one of the conditions.
That is, the mechanical shutter operation is set as an effective shutter operation when the operation sound of the mechanical shutter due to the mechanical shutter operation does not affect the sound of the captured content.

The above-described imaging apparatus further includes a signal processing unit that performs image processing of an imaging signal obtained from the imaging element unit to generate image data, and the control unit includes the mechanical shutter operation in the mechanical shutter driving unit. Instructing the processing setting corresponding to the mechanical shutter operation to the signal processing unit may be considered.
That is, signal processing corresponding to the mechanical shutter operation is automatically set.

In the imaging apparatus described above, when the control unit instructs the mechanical shutter driving unit to perform the mechanical shutter operation, it is conceivable that the control unit instructs the image data to include information corresponding to the mechanical shutter operation. .
The information corresponding to the mechanical shutter operation is information for executing appropriate processing at the time of reproducing image data captured by the mechanical shutter operation, for example. Based on this information, the image data reproducing device appropriately obtains a video effect obtained by taking an image by the mechanical shutter operation when reproducing the image data.

An imaging method according to the present technology includes a mechanical shutter that is provided in an incident light path and performs a shutter operation, an imaging element unit that receives incident light via the mechanical shutter, and a mechanical shutter driving unit that drives the mechanical shutter. An image pickup method for an image pickup apparatus, comprising: an image pickup element drive unit that executes an electronic shutter operation in which exposure is performed at different exposure timings for each part of the image pickup element in the image pickup element unit. Then, as one of the conditions for moving image capturing, the mechanical shutter operation is such that the mechanical shutter driving unit sets the open period for each frame period of the mechanical shutter within the period in which all the imaging pixels of the imaging element unit are exposed. Is set to an effective shutter operation.
As a result, it is possible to obtain image data provided with a period of a certain length that is not exposed between video frames.
The program according to the present technology is a program that causes the control unit of the imaging apparatus to execute the processing of the imaging method.

According to the present technology, image data having a certain length of time period not exposed between video frames can be obtained, and image data with a sense of image quality as in conventional film photography can be realized.
Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a block diagram of an imaging device of an embodiment of this art. It is explanatory drawing of charge reading of a rolling shutter system. It is explanatory drawing of the exposure period of a rolling shutter system. It is a block diagram of the noise cancellation circuit system of the audio | voice processing part of embodiment. It is explanatory drawing of the mechanical shutter operation | movement of embodiment. It is a flowchart of the shutter mode setting process of an embodiment. It is a flowchart of the shutter mode setting process of an embodiment. It is a detailed flowchart of the shutter mode setting process of an embodiment. It is a detailed flowchart of the shutter mode setting process of an embodiment. It is explanatory drawing of the influence of a frame feeling noise reduction. It is a block diagram of the reproducing | regenerating apparatus of embodiment. It is explanatory drawing of the video interpolation of embodiment. It is explanatory drawing of the black image insertion at the time of the video interpolation of embodiment. It is a flowchart of the content reproduction | regeneration processing of embodiment. It is a flowchart of the content reproduction | regeneration processing of embodiment.

Hereinafter, embodiments will be described in the following order.
<1. Configuration of Imaging Device>
<2. Mechanical shutter operation>
<3. Shutter mode setting process>
<4. Configuration and operation of playback device>
<5. Program and Storage Medium>
<6. Summary and Modification>

<1. Configuration of Imaging Device>
FIG. 1 shows a configuration example of an imaging apparatus 1 according to the embodiment. The imaging device 1 performs imaging of a subject in response to a user operation and is, for example, an imaging device capable of generating moving image data or still image data, or an imaging device dedicated to generating moving image data.
The imaging apparatus 1 displays and outputs image data obtained by the imaging operation, outputs and stores the image data in a storage medium, and transmits and outputs the data to an external device. As output image data, stream data of a so-called predetermined image format, file format data, RAW image data, and the like are assumed.

An imaging lens 10 is provided as a lens system for entering subject light. The imaging lens 10 is represented as one lens in FIG. 1, but actually includes a plurality of lenses such as a condenser lens, a focus lens, and a zoom lens.
Incident light that has passed through the imaging lens 10 passes through the aperture mechanism 11 and the mechanical shutter 12, reaches the imaging element unit 13, and is received.
Note that a structure in which incident light passes through the mechanical shutter 12 and then reaches the image sensor section 13 via the diaphragm mechanism 11 may be employed.

The lens driving unit 23 is shown as a mechanism for driving the focus lens and zoom lens in the imaging lens 10 in the optical axis direction.
The aperture drive unit 24 drives the aperture mechanism 11 to change its opening.
The lens control unit 25 controls the driving of the imaging lens 10 via the lens driving unit 23 and also drives the diaphragm mechanism 11 via the diaphragm driving unit 24 to control the diaphragm according to the subject luminance at the time of imaging. . The lens control unit 25 communicates with the control unit 20 and controls driving by the lens driving unit 25 and the diaphragm driving unit 24 in accordance with instructions from the control unit 20. The lens control unit 25 notifies the control unit 20 of lens information related to the focal length, the exit pupil distance, the focal position (focus position), and the like.

  The mechanical shutter 12 passes or blocks incident light to the image sensor unit 13. The mechanical shutter 12 is driven to open and close by a mechanical shutter drive unit 17. In the case of the present embodiment, the mechanical shutter 12 is of a type that can be opened and closed at a high speed that can be opened and closed every frame period of a moving image of, for example, about 24 fps and 30 fps.

The image pickup device section 13 includes an image pickup device formed by, for example, a CMOS (Complementary Metal Oxide Semiconductor) sensor and the like, and a read signal circuit for processing a signal read from the image pickup device (an image pickup device 40 in FIG. 2A described later, a read signal). Circuit 48).
In this image sensor section 13, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, etc. are executed in the readout signal circuit for the electrical signal obtained by photoelectric conversion in the image sensor, and further A / D ( Analog / Digital) conversion processing. The imaging signal as digital data is output to the signal processing unit 14 at the subsequent stage.
Charge accumulation and signal readout in the image sensor of the image sensor unit 13 are performed based on a timing signal from the image sensor driver 18. Under the control of the image sensor driving unit 18, an electronic shutter operation is performed in the image sensor unit 13.

The signal processing unit 14 is configured by, for example, a DSP (Digital Signal Processor). The signal processing unit 20 performs various types of signal processing on the digital signal (imaging signal) from the imaging element unit 13.
For example, the signal processing unit 14 performs video processing such as white balance and gamma processing, image quality correction processing such as inter-frame noise reduction, resolution conversion processing, display signal generation processing, and recording / communication coding processing (codec). And output image data (moving image or still image). The signal processing unit 14 also adds metadata to the image data.
Image data as a display signal is supplied to the display control unit 15. The image data for recording is supplied to the storage unit 19. The image data for communication is supplied to the communication unit 22.

The microphone 28 collects ambient sound (sound from the subject and environmental sound) during moving image capturing. The microphone 28 may be built in the imaging apparatus 1 or may be removable as an external microphone.
The audio signal S1 collected by the microphone 28 is amplified, A / D converted, noise canceled, etc. by the audio processing unit 30 and supplied to the signal processing unit 14 as audio data corresponding to the moving image data. . The signal processing unit 14 processes the audio data supplied from the audio processing unit 30 as audio data associated with the moving image when capturing a moving image.

  The microphone 29 is disposed in the vicinity of the mechanical shutter 12 inside the imaging device 1 and collects an operation sound (opening / closing sound) of the mechanical shutter 12. The audio signal S2 obtained by the microphone 29 is supplied to the audio processing unit 30 and used for noise cancellation processing.

The control unit 20 is configured by a microcomputer (arithmetic processing unit) including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, and the like.
When the CPU executes a program stored in a ROM, a flash memory, or the like, the entire imaging apparatus 1 is comprehensively controlled.
The RAM is used as a work area for various data processing of the CPU for temporary storage of data and programs.
ROM and flash memory (non-volatile memory) are used to store OS (Operating System) for the CPU to control each unit, content files such as image files, application programs for various operations, firmware, and the like. Used. For example, a program for executing shutter mode setting processing described later in this example is stored.

  Such a control unit 20 includes various signal processing instructions in the signal processing unit 14, an imaging operation and recording operation according to a user operation, a reproduction operation of a recorded image file, a shutter mode operation, zoom, focus, exposure adjustment, and the like. The operation of each necessary part is controlled for the camera operation, user interface operation, and the like.

The display unit 16 is a display unit that performs various displays for a user (imager or the like). For example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display formed on the housing of the imaging device 1 or the like. And a display driver for causing the display device to perform display. It may be formed using an LCD, an organic EL display, or the like in the form of a so-called viewfinder.
The display control unit 15 controls the display operation on the display unit 16. For example, the display control unit 15 causes the display unit 16 to execute various displays based on instructions from the control unit 20. For example, the display control unit 15 causes the display unit 16 to reproduce and display still images and moving images that have been imaged and recorded on a recording medium (a storage unit 19 described later), or to wait for release or (shutter operation) or to stand by for moving image capturing. A through image (subject monitoring image) is displayed as a moving image based on the captured image data of each frame captured inside. Further, the display control unit 15 causes various operation menus, icons, messages, etc., that is, display as a GUI (Graphical User Interface) to be executed on the screen of the display unit 16.

The operation unit 21 has an input function for inputting a user operation, and sends a signal corresponding to the input operation to the control unit 20.
The operation unit 21 is realized as, for example, various operators provided on the housing of the imaging apparatus 1 or a touch panel formed on the display unit 16.
As controls on the housing, a menu button, a determination button, a cross key, a cancel button, a zoom key, a slide key, a shutter button (release button), and the like are provided.
Various operations may be performed by touch panel operations using icons, menus, and the like displayed on the touch panel and the display unit 16.

The storage unit 19 includes, for example, a non-volatile memory, and stores image data (still image or moving image data file (content file)) generated by the signal processing unit 14, metadata about the image data file, thumbnail images, and the like. Function as a storage area.
Various forms of the storage unit 19 can be considered. For example, the storage unit 19 may be a flash memory built in the imaging apparatus 1, or a memory card (for example, a portable flash memory) that can be attached to and detached from the imaging apparatus 1 and a card recording / reproducing that performs recording / reproducing access to the memory card. The form by a part may be sufficient. Further, as a form built in the imaging apparatus 1, it may be realized as an HDD (Hard Disk Drive) or the like.

The communication unit 22 performs data communication with an external device in a wired or wireless manner. For example, communication of image data (an image data file as a still image or a moving image or moving image stream data) is performed between an external display device, a recording device, a reproducing device, an editing device, and the like.
Further, the communication unit 22 serves as a network communication unit, for example, performs communication through various networks such as the Internet, home network, and LAN (Local Area Network), and performs various data transmission / reception with servers, terminals, and the like on the network. May be.

The power supply unit 31 supplies an operating power supply voltage Vcc to each unit. The power supply unit 31 uses a loaded battery (primary battery or secondary battery) as a power supply. The power supply unit 31 can also obtain an operating power supply voltage Vcc using an external commercial AC power supply by connecting an AC adapter (not shown).
The control unit 20 performs on / off control of power supply to the power supply unit 31. In addition, the control unit 20 can detect whether or not the commercial AC power is being used via the AC adapter in the power supply unit 31 and can detect the remaining battery level when the battery is used.

In the imaging apparatus 1 according to the present embodiment, an electronic shutter operation in the imaging element unit 13 and a mechanical shutter operation using the mechanical shutter 12 are possible.
As the shutter mode, a shutter operation using only an electronic shutter operation and a shutter operation combining an electronic shutter operation and a mechanical shutter operation are possible.
An “electronic shutter mode” to be described later is a mode for performing a shutter operation only by an electronic shutter operation, and a “mechanical shutter mode” is a mode for performing a shutter operation by both an electronic shutter operation and a mechanical shutter operation.

Here, an electronic shutter operation, particularly a rolling shutter operation widely adopted in recent years will be described.
In general, as a shutter operation method for image photographing processing in an imaging apparatus, a shutter operation for controlling the start of exposure and the end of exposure from one direction of an image sensor surface is called a rolling shutter operation. For example, when the exposure start and the exposure end are controlled from the upper row to the lower row of the image sensor, the exposure time for each row is slightly shifted.
FIG. 2A shows an image sensor 40 using a CMOS image sensor used in the image sensor section 13 of the present embodiment.
The image sensor (CMOS image sensor) 40 includes a vertical scanning circuit 42, a horizontal scanning circuit 43, and a plurality of pixels 41 arranged in an array.

A horizontal reset line 44 and a horizontal selection line 45 are arranged in common for each pixel 41 arranged in the row direction (horizontal direction in the drawing), and for each pixel 41 arranged in the column direction (vertical direction in the drawing). Vertical signal lines 46 are arranged in common.
The vertical scanning circuit 42 sequentially selects the horizontal selection lines 45 to select the row of the pixels 41 from which charges are read. The vertical scanning circuit 42 can reset the exposure of the pixels 41 in units of rows by each horizontal reset 44.

Inside the pixel 41, electric charges are accumulated in the photodiode by an exposure process associated with subject imaging.
The charge accumulated in the photodiode of each pixel 41 is output to the vertical signal line 46 via the amplifier transistor and transfer transistor in the pixel 41. The signal charge output to each vertical signal line 46 is further supplied to the horizontal scanning circuit 43, and is supplied from the signal output line 47 to the readout signal circuit 48 as an imaging signal for one line.
Since each vertical pixel line 41 is commonly connected to the vertical signal line 46, one pixel at a time is read from the vertical signal line 46 in order to read out the signal of each pixel 41 independently. Only the signal for the pixel is output. That is, one row of pixels 41 is selected by the horizontal selection line 45, and the charge of each pixel 41 is read by each vertical signal line 46.
For example, as shown in FIG. 2B, a signal is first read out from each of the pixels 41d arranged in the bottom row, and then read out from the row of pixels 41c as shown in FIG. 2C. The read lines are changed sequentially.

Each of the pixels 41 starts exposure again immediately after the stored charge is read out. That is, the exposure process for the next image frame is started.
When the readout process is sequentially executed in units of rows in this way and the exposure process is started immediately after that, the exposure start time and the end time are different between the first row pixel 41 and the last row pixel 41, that is, the exposure time. Deviation occurs in (or exposure period).
In FIG. 2, only four rows of the pixels 41 a to 41 d are shown, but this shows only a part of the image sensor 40, and the actual image sensor 40 includes several hundred to several thousand rows, etc. A large number of rows are set, and reading is executed sequentially for each row.

  In the rolling shutter system, as described above, the pixels 41 in each row are selected by the horizontal selection line 45 when a predetermined time elapses after the horizontal reset line 44 resets the stored charge amount of the pixels 41 to zero. By reading out the accumulated charges, an imaging operation (electronic shutter operation) by the electronic shutter is realized.

With reference to FIG. 3, an example of the start and end timing of exposure of each row and an example of the charge readout timing will be described. FIG. 3 shows a time axis as a horizontal axis and a row as a vertical axis.
For example, in FIG. 3, the exposure start timing SL and the charge readout timing RL are time-shifted in units of rows (L1, L2,... Ln) as indicated by dotted lines.
That is, FIG. 3 shows the exposure period in each row of one frame, but in the pixel in the first row L1, exposure starts at time t10 and charge reading is performed at time t11.
For example, exposure of the pixels in the last row Ln starts at time t11, and charge reading is performed at time t12.
In this example, the exposure start and the exposure end (charge readout) of each row have the same timing shift in units of rows, so that the exposure times Tr of each row (L1, L2,... Ln) are all the same. It becomes.

  Such a rolling shutter system using a CMOS image sensor is widely used for low power consumption and high-speed reading. However, since the exposure timing at the top of the screen and the exposure timing at the bottom of the screen are different, the movement is fast. It is known that so-called rolling shutter distortion occurs on the subject, and when a light source that blinks at high speed enters the subject, it is not possible to capture images appropriately.

Next, noise cancellation processing in the audio processing unit 30 shown in FIG. 1 will be described.
In the case of the present embodiment, a circuit for canceling the operation sound of the mechanical shutter 12 from the sound collected during moving image capturing is mounted.
FIG. 4 shows a noise cancellation circuit system in the audio processing unit 30.
The audio signal (subject audio) S1 obtained by the microphone 28 is input from the terminal 51, amplified by the microphone amplifier 54, converted into digital audio data by the A / D converter 57, and then supplied to the subtractor 54. .
On the other hand, an audio signal (operation sound of the mechanical shutter 12) S2 obtained by the microphone 29 is input to the terminal 52. The audio signal S2 is amplified by the microphone amplifier 56, converted into digital audio data by the A / D converter 58, and then supplied to the adaptive filter 55. Then, the output of the adaptive filter 55 is supplied to the subtractor 54. The subtractor 54 subtracts the output of the adaptive filter 55 from the audio signal S1 to obtain an audio signal S1 ′ from the terminal 56. This is an audio signal in which the operation sound of the mechanical shutter 12 is canceled.

Here, it is assumed that the audio signal at the time point k is S1 (k) and S2 (k), and the output of the adaptive filter 55 is y (k). The output S1 ′ of the subtractor 54 is used for updating the filter coefficient of the adaptive filter 55 as the error component E (k) of the adaptive filter.
By subtracting the audio signal S1 (k) from the audio signal S1 (k), the audio signal S1 'canceling the shutter operation sound can be obtained.
The sound processing unit 30 can generate such sound data to be recorded together with the image data by performing such noise cancellation processing.

In this example, the mechanical shutter operation sound is obtained using the microphone 29. However, since the mechanical shutter operation sound is known in advance, an operation sound source may be prepared. That is, the shutter operation sound is stored in the memory as audio data, and the shutter operation sound stored in the memory is read out in synchronization with the operation timing of the mechanical shutter 12 and supplied to the adaptive filter 55. In this way, it is possible to cancel the shutter operation sound without using the microphone 29.

<2. Mechanical shutter operation>
In the present embodiment, at the time of moving image capturing, it is possible to perform a mechanical shutter operation that performs exposure during a period defined by the mechanical shutter 12 for each frame period. In particular, this mechanical shutter operation is an operation in which the mechanical shutter driving unit 17 sets the open period of each frame period of the mechanical shutter 12 within the period in which all the imaging pixels 41 of the imaging element unit 13 are exposed.

The mechanical shutter operation of the present embodiment will be described with reference to FIG.
FIG. 5 shows the vertical synchronization signal SV, the exposure period of the pixels 41 from the first row L1 to the last row Ln of the image sensor 40, the opening / closing operation of the mechanical shutter 12, and the timing of turning on / off the blinking light source. In addition, about the pixel 41 of each line L1-Ln, a mode that an electric charge is accumulate | stored by exposure is shown with the line which a waveform rises.
In the frame period defined by the vertical synchronization signal SV, the first row L1 is exposed during a period from time t0 to time t2, and charge reading is performed at time t2. The exposure period and readout timing of the next row L2 are slightly shifted. In the last row Ln, exposure is performed during a period from time t1 to time t3, and charge reading is performed at time t3.

For example, in such a case, as a mechanical shutter operation, an open period is set within the period from time t1 to time t2 when all the pixels 41 are in the exposure state, and this is performed for each frame period. That is, the mechanical shutter 12 is opened at a time ts after the time t1, and is closed at a time te before reaching the time t2.
In this manner, the mechanical shutter 12 is opened and closed during a period in which all the pixels 41 in the image sensor section 13 are exposed, and an image of one frame is acquired. Moving image capturing is performed by repeatedly reading the image in accordance with the interval of the vertical synchronization signal. In this case, since the mechanical shutter 12 is not exposed during the closed period, the captured moving image has a unique feeling of image quality (parasitism) and a sense of reality like an old film movie.

Further, in this case, rolling shutter distortion is eliminated. That is, even if the rolling shutter operation is performed in the image sensor section 13 as described above, the exposure start and end timings of all the pixels 41 are all the same, and the exposure timing shift for each row does not occur.
Further, in the rolling shutter system, when a blinking light source is imaged, there is a problem that the exposure amount of a subject with the same brightness varies depending on the exposure timing.
If the mechanical shutter 12 is not used when the blinking light source is turned on / off at the timing shown in FIG. 5, the subject image in the first row is affected by the light emission in the period TL1, and the subject image in the last row The subject image is affected by light emission in the period TL2. This changes the brightness at the top and bottom of the image. On the other hand, when the mechanical shutter operation is performed, a period that is affected by light emission is a period TL3 for all the pixels. Therefore, even if there is a flashing light source, there is no change in the exposure level in the image. That is, by performing the mechanical shutter operation and matching the start / end timing of exposure, the subject can be exposed with the same brightness even in the presence of a flashing light source that illuminates the subject.

<3. Shutter mode setting process>
In the present embodiment, moving image capturing by the mechanical shutter operation described above is executed under a predetermined condition. In the following, whether to perform imaging using both the mechanical shutter operation and the electronic shutter operation (mechanical shutter mode), or to perform imaging only with the normal electronic shutter operation (rolling shutter method) (electronic shutter mode) The shutter mode setting process for setting is described.

FIG. 6 shows an example of shutter mode setting processing performed by the control unit 20.
In the example of FIG. 6A, the control unit 20 performs imaging mode correspondence processing in step S101. Whether or not the imaging mode is a mode suitable for the above-described mechanical shutter operation, that is, an operation in which the open period of each frame period of the mechanical shutter 12 is within the period in which all the imaging pixels of the imaging element unit 13 are exposed. Is a process for determining. For example, it is checked whether or not the moving image capturing mode is set. Further, not only whether or not it is the moving image capturing mode, but it is also possible to check whether or not it is moving image capturing at a specific frame rate.
In addition, the control unit 20 performs battery handling processing in step S102. This is a process for determining whether or not the power supply state is suitable for the mechanical shutter operation. For example, it is checked whether or not sufficient power supply can be continued for a certain period of time.
In addition, the control unit 20 performs environmental sound handling processing in step S103. This is processing for determining whether or not the environmental sound to be collected and the state of the sound signal processing are suitable for the mechanical shutter operation. For example, a check is made as to whether the environment is greatly affected by the mechanical shutter operation sound, a check is made as to whether the noise cancellation function is effective, or the like.
Each of these is a condition determination for a mechanical shutter operation, and the mechanical shutter operation is set to an effective shutter operation based on the result of such a condition determination. That is, steps S101, S102, and S103 are performed as the effective shutter operation setting process in step S10.

  In step S104, the control unit performs shutter mode setting depending on whether the mechanical shutter operation is an effective shutter operation. That is, if the mechanical shutter operation is the effective shutter operation, the mechanical shutter operation mode is set, and when moving image capturing is started, the mechanical shutter operation as shown in FIG. 5 is performed. On the other hand, if the mechanical shutter operation is not an effective shutter operation, the electronic shutter operation mode is set, and the mechanical shutter operation as shown in FIG.

The example in FIG. 6B is obtained by adding the user handling process in step S110 to the contents of the effective shutter operation setting (S10). This is an example in which whether or not the user's intention desires a mechanical shutter operation is added to one of the conditions for an effective shutter operation.
The example of FIG. 6C is an example in which the order of step S110 is changed in the effective shutter operation setting (S10). In the case of FIG. 6B, the imaging mode handling process (S101), the user handling process (S110), the battery handling process (S102), and the environmental sound handling process (S103) are performed in this order. ), The battery handling process (S102), the environmental sound handling process (S103), and the user handling process (S110).
In the case of FIG. 6B, the battery handling process (S102) and the environmental sound handling process (S103) can be executed immediately before imaging only when the user intends to perform a mechanical shutter operation.
On the other hand, in the case of FIG. 6C, after the other conditions for setting the mechanical shutter operation as the effective shutter are satisfied, the effective shutter operation setting is finally performed depending on the user's intention.

The shutter mode setting process can be considered other than this.
For example, the shutter mode setting may be performed only according to the imaging mode compatible processing. That is, as shown in FIG. 7A, this is a processing example in which only steps S101 and S104 are performed.
The shutter mode setting may be performed according to the imaging mode handling process and the battery handling process. That is, this is an example of processing in steps S101 → S102 → S104 as shown in FIG. 7B.
The shutter mode setting may be performed according to the imaging mode handling process and the environmental sound handling process. In other words, as shown in FIG.
The shutter mode setting may be performed according to the imaging mode handling process and the user handling process. That is, this is a processing example of steps S101 → S110 → S104 as shown in FIG. 7D.
Furthermore, in the effective shutter operation setting (S10) of FIGS. 6A, 6B, 6C, 7A, 7B, 7C, and 7D, a modification may be considered in which the effective shutter determination is performed by checking other conditions. .

  More specific examples of the above processing will be described with reference to FIGS. In FIGS. 8 and 9, steps S101, S110, S102, S103, and S104 in FIG. 6B are illustrated in detail, and portions corresponding to these processes are surrounded by a one-dot chain line.

In the shutter mode setting process, the control unit 20 first performs the processes of steps S201 to S204 in FIG. 8 as the imaging mode correspondence process (S101).
In step S201, it is determined whether the current imaging mode is moving image imaging. In the still image capturing mode, although not shown, the shutter mode setting for still image capturing is performed.
In the moving image capturing mode, in step S202, the control unit 20 confirms whether or not the movie image capturing mode is selected as a mode in moving image capturing.
This is, for example, the frame rate selection at the time of motion imaging, and the cinematic imaging mode is, for example, a state in which 24P (24 fps progressive scan) is selected. Note that 24P is an example, and for example, 23.976 fps may be included in the movie-like imaging mode.

When the moving image capturing mode is selected and movie-like imaging such as 24P is selected, the control unit turns on the flag Fm in step S203.
Here, it is assumed that the flag Fm is a flag indicating whether or not the mechanical shutter operation is the effective shutter operation, and that the flag Fm is finally turned on is determined as the effective shutter operation.
If it is determined in step S202 that movie-like imaging has not been selected, the control unit 20 turns off the flag Fm in step S204.
Note that as the imaging mode handling process (S101), an example in which the flag Fm is turned on is also conceivable for moving image imaging.

Next, the control part 20 performs a user corresponding | compatible process (S110) as step S210-S213. First, in step S210, the control unit 20 checks whether or not the flag Fm is on. If the flag Fm is off, the user handling process (S110) is terminated.
When the flag Fm is on, the control unit 20 controls the presentation that recommends the mechanical shutter operation in step S211. For example, the control unit 20 instructs the display control unit 15 to display messages and operation images such as “Do you want to perform mechanical shutter operation?” And “Do you want to shoot more movie-like image quality” on the display unit 16? The user waits for “YES” or “NO” operation.
When the user performs the “NO” operation, the control unit 20 proceeds from step S212 to step S213, turns off the flag Fm, and ends the user handling process (S110).
When the user performs the “YES” operation, the user handling process (S110) is terminated without performing the flag process. That is, the flag Fm is kept on.
If the user operation cannot be detected even after waiting for a predetermined time, the user operation may be regarded as “YES” or “NO”.

Next, the control part 20 performs a battery corresponding | compatible process (S102) as step S220-S226 of FIG. First, in step S220, the control unit 20 checks whether or not the flag Fm is on. If the flag Fm is off, the battery handling process (S102) is terminated.
When the flag Fm is on, the control unit 20 confirms whether or not the shutter is automatically set in step S221.
This is an automatic setting state in which the user decides whether to perform the mechanical shutter operation according to the power supply state, or is selected to perform either the mechanical shutter operation or the electronic shutter operation regardless of the power supply state. It is a process of confirming whether the fixed setting state is present. These settings are assumed to be performed according to user operations.

If not in the automatic setting state, the control unit 20 confirms whether the setting selected in step S225 is the mechanical shutter operation or the electronic shutter operation. If the mechanical shutter operation is selected, the battery is used as it is. The response process (S102) is completed. That is, the flag Fm = on is maintained.
When the electronic shutter operation is selected as the fixed setting, the control unit 20 turns off the flag Fm in step S226 and ends the battery handling process (S102). Note that the fixed setting is a setting for selecting the mechanical shutter operation regardless of the power supply state, and the fixed setting for the electronic shutter may be omitted.

  When it is confirmed in step S221 that the automatic setting according to the power state is selected, the control unit 20 performs a battery check in step S222. In this case, the control unit 20 checks whether, for example, an AC adapter is connected to the power supply unit 31 to use an external power source, or whether the attached battery is used as a power source. Check if it exists.

If the control unit 20 determines that the external power supply is being used, the battery handling process (S102) ends from step S223. That is, the flag Fm = on is maintained.
On the other hand, if the internal battery is used, the process proceeds from step S223 to S224. If it is determined that the remaining battery level is insufficient, the flag Fm is turned off in step S226. If the remaining battery level is not insufficient, the flag Fm = on is maintained as it is, and the battery handling process (S102) is ended.

  Next, the control part 20 performs an environmental sound corresponding | compatible process (S103) as step S230-S234. First, in step S230, the control unit 20 confirms whether or not the flag Fm is on. If the flag Fm is off, the environmental sound handling process (S103) is terminated.

When the flag Fm is on, the control unit 20 checks the sound collection environment in step S231.
This environmental sound handling process is a process of determining whether or not the operating sound of the mechanical shutter is mixed in at a relatively large level as noise with respect to the sound of the captured video. If the mechanical shutter operation sound is mixed greatly, the mechanical shutter operation is not appropriate. Therefore, in step S231, it is first determined whether or not the environment is such that the mechanical shutter operation sound is collected significantly.
For example, it is determined whether the microphone 28 is a built-in microphone or a connected directional microphone. Alternatively, a directivity type check of a connected microphone may be performed. If the built-in microphone or the omnidirectional microphone is used, it can be determined that it is easy to pick up the mechanical shutter operation sound.
Further, the sound level collected by the microphone 28 may be confirmed. In a relatively noisy environment or an environment where the sound level from the subject is high, the mechanical shutter operation sound is easily masked.
These checks determine whether the sound collection environment is good.

If it is determined that the sound collection environment is good, the control unit 20 ends the environmental sound handling process (S103) from step S232. That is, the flag Fm = on is maintained.
If it is determined that the sound collection environment is not good, the control unit 20 proceeds from step S232 to step S233, and confirms whether or not the noise cancellation function (see FIG. 4) in the audio processing unit 30 is turned on. If it is on, the environmental sound corresponding process (S103) is terminated while maintaining the flag Fm = on.
If it is confirmed in step S233 that the noise cancellation function is off, the control unit 20 proceeds to step S234, turns off the flag Fm, and ends the environmental sound handling process (S103).
When the noise cancellation function is off, the flag Fm = on may be maintained after being automatically turned on.
In this example, the noise canceling function is provided. However, for example, when the imaging apparatus 1 does not have the noise canceling function as shown in FIG. 4, step S233 is not performed.
Further, when the noise cancellation function is always turned on, this step S103 itself may be unnecessary.

If the flag Fm = ON as a result of the processing so far, the control unit 20 satisfies the conditions and maintains the state where the mechanical shutter operation is the effective shutter operation.
Therefore, the control unit 20 performs shutter mode setting according to the flag Fm as the shutter mode setting process in step S104. That is, if the flag Fm = ON, the mechanical shutter mode is set (S240 → S241). If the flag Fm is off, the electronic shutter mode is set (S240 → S242).
The mechanical shutter mode is a mode in which the mechanical shutter drive unit 17 performs control so as to execute the opening / closing operation of the mechanical shutter 12 shown in FIG.

In addition, when setting the mechanical shutter mode in step S241, various settings corresponding to the mechanical shutter mode are also performed.
One of the video processes performed in the signal processing unit 14 is inter-frame noise reduction, which performs a process of weighting and adding the video of the past frame and the video of the current frame. For this reason, as shown in FIG. 10, the afterimage is not small and the sense of reality is reduced. In FIG. 10, the left side is a frame image (captured image) at times m, m + 1, and m + 2, and the right side is a frame image at each time subjected to inter-frame noise reduction. This is an example in which a subject moving from right to left on the screen is imaged, but the shaded portion remains as an afterimage from the previous frame.
In the present embodiment, in moving image capturing using a mechanical shutter operation, the purpose is to obtain a cinematic image quality and a sense of reality, so it is desirable that no afterimage is generated. Therefore, in step S241, the setting for turning off the inter-frame noise reduction is also performed.

In addition, there are monitors equipped with high frame rate reproduction using an interpolation technique even in a reproduction environment. As will be described in detail later, when movie-like image data is generated with 24P or the like, if high frame rate conversion is performed on the playback device side, a movie-like image quality and realistic sensation cannot be obtained. Therefore, the playback device side can detect that the image data is a movie-like image. Therefore, the signal processing unit 14 performs setting so that metadata indicating that the image is captured in the movie-like imaging mode is added to the image data.
It is desirable to perform the setting for adding such metadata in step S241.

  Further, for example, in moving image capturing in the mechanical shutter mode in which image capturing is performed at 24 fps, which is the frame rate of a film movie, it is appropriate to perform movie gamma setting, color setting, and the like in order to bring the image quality closer to a movie. Therefore, it is preferable that the control unit 20 performs gamma setting, color setting, and the like in step S241.

  Thereafter, when moving image capturing is started by a user operation, the process proceeds from step S250 to S252, and moving image capturing start control is performed. Thereafter, the moving image imaging is continued until the imaging is stopped. In the case of imaging after the mechanical shutter mode is set, in the signal processing unit 14, the inter-frame noise reduction is turned off according to the setting in step S241, and the signal processing is performed in the state of the gamma setting and color setting of the movie award. Is done. Further, metadata indicating that the image data is captured in the movie-like imaging mode is also added.

  Note that even after the shutter mode setting is performed in step S241 or S242, if the user has performed some mode operation or the like before the moving image capturing start operation, it is necessary to change the shutter mode setting as shown in FIG. The process from step S201 is performed.

Although FIGS. 8 and 9 described the example along FIG. 6B, the process corresponding to FIG. 6A may be a process in which step S100 is not performed as indicated by the broken line in FIG. Further, as processing corresponding to FIG. 6C, step S110 (S210 to S213) may be performed between steps S103 and S104.
Further, in the case of the processing examples of FIGS. 7A, 7B, 7C, and 7D, each process to be executed may be performed as shown in steps S101, S102, S103, and S110 of FIGS.

<4. Configuration and operation of playback device>
Next, a playback device 2 that plays back image data (image content) captured by the above-described imaging device 1 will be described.
FIG. 11 is a block diagram of a configuration example of the playback device 2. The playback device 2 includes a playback unit 61, a communication unit 62, a signal processing unit 63, a display control unit 64, a display unit 65, a control unit 66, an audio processing unit 68, and a speaker unit 69.

  The playback unit 61 is, for example, a device that plays back a portable recording medium such as an HDD, an optical disk, or a memory card. For example, it is possible to perform reproduction on a recording medium that has been imaged by the imaging apparatus 1 and recorded with image data. Alternatively, the image data received via the communication unit 62 can be recorded and reproduced.

  The communication unit 62 performs data communication with an external device by wire or wireless. For example, image data (an image data file as a still image or a moving image, a moving image stream data, or the like) is communicated between the imaging device 1 and the editing device. Further, the communication unit 22 serves as a network communication unit, for example, performs communication through various networks such as the Internet, home network, and LAN (Local Area Network), and performs various data transmission / reception with servers, terminals, and the like on the network. May be.

The signal processing unit 63 performs signal processing for reproduction display on the image data to be reproduced and displayed, that is, the image data read from the recording medium by the reproduction unit 61 and the image data received by the communication unit 62. For example, image data encoded for recording / reproduction and communication is decoded, color processing and gamma processing for the decoded image data, R, G, and B data separation are performed. Further, the signal processing unit 63 can perform high-quality reproduction of image data at a low frame rate, for example, by performing frame interpolation processing.
The signal processing unit 63 reads the metadata added to the image data, performs a predetermined process according to the information, and notifies the control unit 66 of the information.

The image data processed for reproduction display by the signal processing unit 63 is supplied to the display control unit 64. The display control unit 64 drives the display unit 65 based on the image data to display the reproduced video.
The display unit 65 is formed with a display device such as an LCD or an organic EL display, and a display driver that causes the display device to perform display. The display unit 65 may not be integrated with the playback device 2 but may be a separate device.

Audio data accompanying the image data is extracted by the signal processing unit 63 and supplied to the audio processing unit 68. The audio processing unit 68 performs various kinds of acoustic processing, volume adjustment, equalizing, D / A conversion, and the like on the audio data, and supplies them to the speaker unit 69.
The speaker unit 69 performs power amplifier amplification on the audio signal and outputs it to the speaker.

The operation unit 67 has an input function for inputting a user operation, and sends a signal corresponding to the operation input to the playback device 2 to the control unit 66.
The control unit 66 controls the entire playback device 2 to execute playback display according to the user's operation.

  In the reproducing apparatus 2, in particular, with respect to image data that has been imaged by the above-described mechanical shutter operation in the imaging apparatus 1, reproduction display that makes use of the characteristics is executed. Hereinafter, this point will be described.

2. Description of the Related Art In recent years, an apparatus that reproduces video is generally known that generates image data such as 120 fps and 240 fps by frame interpolation of image data such as 24 fps and 60 fps by frame interpolation technology, and reproduces it at a high frame rate. .
FIG. 12 schematically shows the state of frame interpolation for the original image data. In this case, an example in which an interpolated image frame is generated for frames (n), (n + 1), (n + 2)... Of image data, and interpolated image data having a three times higher frame rate is generated is illustrated. Yes. By displaying using this interpolated image data, a smooth reproduced video can be displayed even from image data at a low frame rate.

However, the above-described mechanical shutter operation is intended to obtain a feeling of image quality and a sense of reality like a film movie by prolonging a period during which no exposure is performed between frames. In that case, if the frame is interpolated and displayed as a smooth video, the target video effect cannot be obtained.
Therefore, the playback device 2 refers to the metadata of the image data and does not perform frame interpolation when it is determined that the image data is imaged by the mechanical shutter operation.

Even when frame interpolation is performed, a black image can be sandwiched between frames to make the image look more like a film.
FIG. 13 shows how a black image is inserted.
Assume that frame interpolation corresponding to 120P and 240P is performed on frames (n), (n + 1)... Of the original 24P image data.
When frame interpolation equivalent to 120P is performed, 4 frames are added to one frame period of 24P so that 5 frames are displayed. When frame interpolation equivalent to 240P is performed, nine frames are added to one frame period of 24P so that 10 frames are displayed.
In these cases, as shown in the figure, one or a plurality of interpolation frames in one frame period are set as a black image frame BK.
In the case of frame interpolation equivalent to 120P, the original frame image data is used as it is for the four interpolated images in the original one frame period, and one black image frame BK is inserted thereafter.
In the case of frame interpolation equivalent to 240P, the original frame image data is used as it is as eight interpolated images for 10 frames after interpolation in the original one frame period, and then two black image frames BK are inserted.
For example, by inserting the black image frame BK in this way, it is possible to display a film movie-like image quality feeling even when frame interpolation is performed.
In these cases, it is also possible to adjust the image quality of the film movie style by changing the number of black image frames BK to be inserted.

Thus, in order to cope with imaging by the mechanical shutter operation, in the playback apparatus 2, the control unit 66 performs processing as shown in FIG.
First, the processing of FIG. 14 will be described. This is the case of the playback apparatus 2 having a frame interpolation function.
When starting the reproduction display of the image data in response to a user operation or the like, the control unit 66 proceeds from step S301 to S302, and first confirms the metadata added to the image data to be reproduced and displayed. For example, processing such as extracting image data to be reproduced stored in the reproduction unit 61 or metadata of image data received by the communication unit 62 is performed.
These pieces of information are information added by the processing of the signal processing unit 14 of the imaging device 1 during imaging by the mechanical shutter operation. That is, the information is added to the image data based on the mechanical shutter mode setting in step S241 in FIG.
As specific information content,
Information indicating mechanical shutter mode imaging, information indicating frame interpolation inhibition, information instructing insertion of a black image, and the like are assumed.

The control unit 66 confirms whether or not such information is added in step S302, and branches the process in step S303.
When information indicating that the image is mechanical shutter mode imaging or information indicating that frame interpolation is prohibited is added to the image data, the process proceeds to step S304A, and control is performed to turn off the frame interpolation function in the signal processing unit 63. If these pieces of information are not added, the process of step S304A is not performed.
In step S305, the reproduction display operation by the reproduction unit 61 (or communication unit 62), the signal processing unit 63, the display control unit 64, and the audio processing unit 68 is started.
In this reproduction process, since frame interpolation processing is prohibited, for example, 24P image data is displayed as it is, and display that exhibits the desired video effect is performed.

The processing in FIG. 15 is for the playback apparatus 2 capable of inserting a black image frame.
Steps S301, S302, S303, and S305 are the same as those in FIG.
In this case, when the information indicating that the image is mechanical shutter mode imaging or the information for instructing the black image insertion is added to the image data, the control unit 66 proceeds to step S304B and involves the black image insertion in the signal processing unit 63. Control to turn on the frame interpolation function.
In step S305, the reproduction display operation by the reproduction unit 61 (or communication unit 62), the signal processing unit 63, the display control unit 64, and the audio processing unit 68 is started.
In this reproduction processing process, frame interpolation including black image insertion as shown in FIG. 13 is performed on the image data, and reproduction display based on this interpolation image data is performed. As a result, a display that exhibits the desired video effect is performed.

<5. Program and Storage Medium>
The program according to the present embodiment is a program that causes an arithmetic processing device such as a microcomputer to execute processing of the control unit 20 of the imaging device 1.
The program of the embodiment is a period in which all the imaging pixels of the image sensor unit 13 are exposed during the open period of each frame period of the mechanical shutter 12 by the mechanical shutter drive unit 17 on the condition that it is moving image imaging. This is a program for causing the control unit 20 to execute a process for setting an internal mechanical shutter operation to an effective shutter operation.
In other words, this program is a program that causes the information processing apparatus to execute the processing in any one of FIGS. 6A, 6B, 6C, 7A, 7B, 7C, and 7D, or a modified example thereof. More specifically, it is a program for executing the processing described with reference to FIGS.

With such a program, the mechanical shutter mode operation as the imaging apparatus 1 described above can be realized.
Such a program can be stored in advance in an HDD as a storage medium built in a device such as a computer device, a ROM in a microcomputer having a CPU, or the like. Alternatively, it can be stored (stored) temporarily or permanently in a removable storage medium such as a semiconductor memory, memory card, optical disk, magneto-optical disk, or magnetic disk. Such a removable storage medium can be provided as so-called package software.
Further, such a program can be installed from a removable storage medium to a personal computer or the like, or can be downloaded from a download site via a network such as a LAN or the Internet.

<6. Summary and Modification>
In the above embodiment, the following effects can be obtained.
The imaging apparatus 1 according to the embodiment includes a mechanical shutter 12 that is provided in an incident light path via a lens system and performs a shutter operation, and an imaging element unit 13 that receives incident light via the mechanical shutter 12. In addition, a mechanical shutter drive unit 17 that drives the mechanical shutter 12 and an image sensor drive unit 18 that performs a rolling shutter type electronic shutter operation in which exposure is performed at different exposure timings for each part of the image sensor in the image sensor unit 13. Have. Then, the control unit 20 is one of the conditions that it is moving image imaging, and the mechanical shutter driving unit 17 is in the open period for each frame period of the mechanical shutter 12 within the period in which all the imaging pixels of the image sensor unit 13 are exposed. Is set to an effective shutter operation (step S10).
Since the mechanical shutter operation of the present embodiment provides a certain length of time period between the image frames that is not exposed, a unique image quality feeling and realistic sensation like a movie by conventional film photography is provided. The moving image capturing is executed. At least in the case of moving image capturing, this mechanical shutter operation is set as an effective shutter operation, and movie-like image data can be generated by actually performing imaging in the mechanical shutter mode based on the effective shutter operation setting. .

  In addition, this mechanical shutter operation can eliminate image distortion that appears in the rolling shutter system, and can also eliminate a phenomenon such as a change in exposure amount in a frame when there is a flashing light source.

Further, the control unit 20 instructs the mechanical shutter driving unit 17 to perform the mechanical shutter operation in response to setting the mechanical shutter operation to the effective shutter operation (for example, the process of FIG. 6A).
That is, by automatically setting the mechanical shutter operation state when it is determined that the shutter operation is effective, the burden on the user operation can be reduced and the operation can be simplified.

Further, the control unit 20 performs recommended presentation control of the mechanical shutter operation in response to setting the mechanical shutter operation to the effective shutter operation (for example, the processes in FIGS. 6B and 6C).
That is, the user can select whether or not to perform the mechanical shutter operation by recommending the mechanical shutter operation. This makes it possible to select a mechanical shutter operation that reflects the user's intention.

Further, the control unit 20 sets the mechanical shutter operation to the effective shutter operation under one condition that the moving image capturing mode is performed at a specific frame rate (S202 in FIG. 8).
For example, in the case of a specific frame rate such as 24 fps or 23.976 fps, the above-described mechanical shutter operation is preferable, so that the video effect due to the mechanical shutter operation becomes remarkable when the specific frame rate is used. In some cases, a mechanical shutter operation can be made possible.
In the example of FIG. 8, the frame rate for movie-like imaging such as 24 fps or 23.976 fps is set as a condition for setting the mechanical shutter operation as an effective shutter operation, but other conditions such as 30P, 60i (60 fps interlaced scan) are used. An example where the frame rate is a condition for setting the mechanical shutter operation as an effective shutter operation is also conceivable.

Further, the control unit 20 sets the mechanical shutter operation to the effective shutter operation with the result of the battery check as one of the conditions (S102).
That is, the mechanical shutter operation is set as an effective shutter operation when the remaining battery level is sufficient or when an external power source is used. Since the mechanical shutter operation continues the high-speed opening / closing operation, the power consumption is relatively large. Therefore, by performing the mechanical shutter operation when the power supply state is sufficient, it is possible to stably perform imaging using the mechanical shutter operation.
In addition, taking into account the state of the power supply, it is possible to take a longer image by mechanical shutter operation.

Further, the control unit 20 sets the mechanical shutter operation to the effective shutter operation using the result of the sound collection environment check as one of the conditions (S103).
That is, when the operation sound of the mechanical shutter 12 does not affect the sound of image data to be captured, the mechanical shutter operation is set as an effective shutter operation. Since the operating sound of the mechanical shutter that opens and closes at a high speed becomes relatively large, when it is estimated that there is no influence of the operating sound, by performing the mechanical shutter operation, it is possible to perform imaging without reducing the sound quality.
If the operation sound of the mechanical shutter is a concern, it is also checked whether noise cancellation is possible (S233). This is because the mechanical shutter operation may be permitted because the sound quality of the image data is not deteriorated if the noise of the shutter operation can be suppressed by the noise canceling process. By doing so, the mechanical shutter operation in a quiet environment can be promoted.

  The signal processing unit 14 generates image data by performing signal processing of the imaging signal obtained from the imaging element unit 13, and the control unit 20 instructs the mechanical shutter driving unit 17 to perform a mechanical shutter operation. The signal processing unit 14 is instructed to set processing corresponding to the mechanical shutter operation (S241). That is, signal processing corresponding to the mechanical shutter operation is automatically set. This makes it possible for the photographer to use more easily by disabling inter-frame noise reduction and automatically setting gamma settings, color settings, and the like according to movie-like imaging without imposing a burden on the user.

Further, the control unit 20 instructs the mechanical shutter driving unit 17 to add information corresponding to the mechanical shutter operation to the image data when instructing the mechanical shutter operation (S241).
The information corresponding to the mechanical shutter operation is information for executing an appropriate process at the time of reproduction with image data captured by the mechanical shutter operation, for example. That is, information indicating that the image is mechanical shutter mode imaging, information indicating that frame interpolation is prohibited, information that instructs insertion of a black image, and the like.
With these pieces of information, it is possible to instruct the playback device 2 to perform processing during playback of the image data by the mechanical shutter operation. That is, the playback device 2 can provide a playback image in which the video effect by the mechanical shutter operation is exhibited.

In the reproduction apparatus 2 according to the embodiment, the control unit 66 may turn off the frame interpolation process or perform frame interpolation with black image frame insertion at the time of reproduction display based on information added to the image data. I have to.
As a result, a reproduction display that exhibits a film movie-like video effect intended by the photographer is executed.

Note that the configurations and processing examples of the imaging device 1 and the playback device 2 of the embodiment are merely examples. Various configuration examples and processing examples other than those described are assumed.
The processing of the control unit 66 described in the playback device 2 (the processing of FIG. 14 or 15) may be executed when the control unit 20 in the imaging device 1 plays back image data on the display unit 16.
The imaging device 1 may be an imaging device that can only capture moving images. In this case, the mechanical shutter operation is set as an effective shutter operation on the condition that a specific frame rate such as 24P is selected. It is possible.
Moreover, you may enable it to adjust the length of the open period of mechanical shutter operation | movement.
Further, during the operation imaging (period from the start to the end of imaging), the mechanical shutter operation may be performed only during a part of the period.

  Note that the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.

In addition, this technique can also take the following structures.
(1) a mechanical shutter provided in an incident light path and performing a shutter operation;
An image sensor that receives incident light via the mechanical shutter;
A mechanical shutter drive unit for driving the mechanical shutter;
In the image sensor unit, an image sensor drive unit that executes an electronic shutter operation in which exposure is performed at different exposure timings for each part of the image sensor,
As one of the conditions that it is moving image capturing, a mechanical shutter operation in which the mechanical shutter driving unit sets an open period for each frame period of the mechanical shutter within a period in which all imaging pixels of the image sensor unit are exposed. A control unit configured to set the effective shutter operation.
(2) The imaging device according to (1), wherein the control unit instructs the mechanical shutter driving unit to perform the mechanical shutter operation in response to setting the mechanical shutter operation to an effective shutter operation.
(3) The imaging device according to (1), wherein the control unit performs recommended presentation control of the mechanical shutter operation in response to setting the mechanical shutter operation to an effective shutter operation.
(4) The control unit sets the mechanical shutter operation as an effective shutter operation under one condition that the moving image capturing mode is performed at a specific frame rate. (1) to (3) Imaging device.
(5) The imaging device according to (4), wherein the specific frame rate is 24 fps or 23.976 fps.
(6) The imaging device according to any one of (1) to (5), wherein the control unit sets the mechanical shutter operation to an effective shutter operation based on a battery check result as one of the conditions.
(7) The imaging device according to any one of (1) to (6), wherein the control unit sets the mechanical shutter operation to an effective shutter operation based on a result of the sound collection environment check as one of conditions.
(8) It further includes a signal processing unit that performs image processing of the imaging signal obtained from the imaging element unit to generate image data,
When the control unit instructs the mechanical shutter driving unit to perform the mechanical shutter operation, the control unit instructs the signal processing unit to perform processing setting corresponding to the mechanical shutter operation. (1) to (7) The imaging device described in 1.
(9) When the control unit instructs the mechanical shutter drive unit to perform the mechanical shutter operation, the control unit instructs the image data to be added with information corresponding to the mechanical shutter operation. (1) to (8) The imaging device according to any one of the above.
(10) a mechanical shutter that is provided in the incident light path and performs a shutter operation;
An image sensor that receives incident light via the mechanical shutter;
A mechanical shutter drive unit for driving the mechanical shutter;
In the image sensor unit, an image sensor drive unit that executes an electronic shutter operation in which exposure is performed at different exposure timings for each part of the image sensor,
As an imaging method of an imaging apparatus equipped with
As one of the conditions that it is moving image capturing, a mechanical shutter operation in which the mechanical shutter driving unit sets an open period for each frame period of the mechanical shutter within a period in which all imaging pixels of the image sensor unit are exposed. Set the effective shutter operation.
(11) a mechanical shutter that is provided in the incident light path and performs a shutter operation;
An image sensor that receives incident light via the mechanical shutter;
A mechanical shutter drive unit for driving the mechanical shutter;
In the image sensor unit, an image sensor drive unit that executes an electronic shutter operation in which exposure is performed at different exposure timings for each part of the image sensor,
In the control unit of the imaging apparatus equipped with
As one of the conditions that it is moving image capturing, a mechanical shutter operation in which the mechanical shutter driving unit sets an open period for each frame period of the mechanical shutter within a period in which all imaging pixels of the image sensor unit are exposed. , A program that executes the process for setting the effective shutter operation.

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 2 ... Playback apparatus, 10 ... Imaging lens, 12 ... Mechanical shutter, 13 ... Imaging element part, 14 ... Signal processing part, 17 ... Mechanical shutter drive part, 18 ... Imaging element drive part, 20 ... Control part , 30 ... Audio processing unit

Claims (11)

A mechanical shutter provided in the incident light path and performing a shutter operation;
An image sensor that receives incident light via the mechanical shutter;
A mechanical shutter drive unit for driving the mechanical shutter;
In the image sensor unit, an image sensor drive unit that executes an electronic shutter operation in which exposure is performed at different exposure timings for each part of the image sensor,
As one of the conditions that it is moving image capturing, a mechanical shutter operation in which the mechanical shutter driving unit sets an open period for each frame period of the mechanical shutter within a period in which all imaging pixels of the image sensor unit are exposed. A control unit configured to set the effective shutter operation. The imaging apparatus according to claim 1, wherein the control unit instructs the mechanical shutter driving unit to perform the mechanical shutter operation in response to setting the mechanical shutter operation to an effective shutter operation. The imaging device according to claim 1, wherein the control unit performs recommended presentation control of the mechanical shutter operation in response to setting the mechanical shutter operation to an effective shutter operation. The control unit sets the mechanical shutter operation to an effective shutter operation on the condition that the moving image capturing mode is performed at a specific frame rate.
The imaging device according to any one of claims 1 to 3 . The imaging device according to claim 4, wherein the specific frame rate is 24 fps or 23.976 fps. The control unit sets the mechanical shutter operation to an effective shutter operation using the result of the battery check as one of the conditions.
The imaging device according to any one of claims 1 to 5 . The control unit sets the mechanical shutter operation to an effective shutter operation using the result of the sound collection environment check as one of the conditions.
The imaging device according to claim 1 . A signal processing unit that generates image data by performing signal processing of an imaging signal obtained from the imaging element unit;
When the control unit instructs the mechanical shutter driving unit to perform the mechanical shutter operation, the control unit instructs the signal processing unit to set processing corresponding to the mechanical shutter operation.
The imaging device according to claim 1 . Wherein, when instructing the mechanical shutter operation to the mechanical shutter driving unit, an instruction to the information corresponding to the mechanical shutter operation to the image data is added
The imaging device according to any one of claims 1 to 8 . A mechanical shutter provided in the incident light path and performing a shutter operation;
An image sensor that receives incident light via the mechanical shutter;
A mechanical shutter drive unit for driving the mechanical shutter;
In the image sensor unit, an image sensor drive unit that executes an electronic shutter operation in which exposure is performed at different exposure timings for each part of the image sensor,
As an imaging method of an imaging apparatus equipped with
As one of the conditions that it is moving image capturing, a mechanical shutter operation in which the mechanical shutter driving unit sets an open period for each frame period of the mechanical shutter within a period in which all imaging pixels of the image sensor unit are exposed. Set the effective shutter operation. A mechanical shutter provided in the incident light path and performing a shutter operation;
An image sensor that receives incident light via the mechanical shutter;
A mechanical shutter drive unit for driving the mechanical shutter;
In the image sensor unit, an image sensor drive unit that executes an electronic shutter operation in which exposure is performed at different exposure timings for each part of the image sensor,
In the control unit of the imaging apparatus equipped with
As one of the conditions that it is moving image capturing, a mechanical shutter operation in which the mechanical shutter driving unit sets an open period for each frame period of the mechanical shutter within a period in which all imaging pixels of the image sensor unit are exposed. , A program that executes the process for setting the effective shutter operation.

Images