JP6561428B2 - Electronic device, control method, and control program - Google Patents

Description

The present invention relates to an electronic device, a control method, and a control program.

  There has been proposed an electronic apparatus provided with an imaging element in which a back-illuminated imaging chip and a signal processing chip are laminated (hereinafter, this imaging element is referred to as a laminated imaging element) (for example, see Patent Document 1). The multilayer imaging element is laminated so that the back-illuminated imaging chip and the signal processing chip are connected via a micro bump for each block unit including a plurality of pixels.

JP 2006-49361 A

  However, there have not been many proposals for acquiring an image by capturing an image for each of a plurality of block units in a conventional imaging device including a multilayer imaging device, and the usability of an imaging device including a multilayer imaging device has not been sufficient.

  An object of an aspect of the present invention is to reduce power consumption and processing burden in an imaging apparatus.

According to the aspect of the present invention, a plurality of pixels that are arranged in a first direction and a second direction different from the first direction and that include a photoelectric conversion unit that converts light into charges, and a photoelectric conversion unit of the plurality of pixels An imaging region having a first wiring for transmitting a control signal for transferring the charge accumulated in the first and a second wiring for outputting a signal generated by the charges converted by the photoelectric conversion units of the plurality of pixels is first. Based on the shake amount calculated by the shake amount calculation unit, the shake amount calculation unit that calculates a plurality of image pickup devices arranged in one direction and the second direction, an imaging region to be driven among the image pickup elements The first imaging condition is set in the first imaging area among the plurality of imaging areas selected by the control unit to be selected and the control unit, and the second imaging that is different from the first imaging condition in the second imaging area among the plurality of imaging areas a setting unit for setting a condition, the electronic device provided is provided .
According to the aspect of the present invention, the amount of shake is calculated, and a plurality of the first direction and the second direction different from the first direction are arranged based on the calculated amount of shake, and light is converted into electric charges. generating a plurality of pixels including a photoelectric conversion unit, a first wiring for transmitting a control signal for transferring charges accumulated in the photoelectric conversion unit of a plurality of pixels, the charges converted by the photoelectric conversion unit of a plurality of pixels An imaging region having a second wiring that outputs the selected signal is selected by selecting an imaging region to be driven from among a plurality of imaging devices arranged in the first direction and the second direction . A control method including: setting a first imaging condition in a first imaging area among a plurality of imaging areas; and setting a second imaging condition different from the first imaging condition in a second imaging area among the plurality of imaging areas. Provided.
According to the aspect of the present invention, the amount of shake is calculated, and a plurality of the first direction and the second direction different from the first direction are arranged based on the calculated amount of shake, and light is converted into electric charges. generating a plurality of pixels including a photoelectric conversion unit, a first wiring for transmitting a control signal for transferring charges accumulated in the photoelectric conversion unit of a plurality of pixels, the charges converted by the photoelectric conversion unit of a plurality of pixels An imaging region having a second wiring that outputs the selected signal is selected by selecting an imaging region to be driven from among a plurality of imaging devices arranged in the first direction and the second direction . A first imaging condition is set in the first imaging area among the plurality of imaging areas, and a second imaging condition that is different from the first imaging condition is set in the second imaging area among the plurality of imaging areas. A control program is provided.

  According to the second aspect of the present invention, the imaging unit includes an imaging unit having an imaging element, and an image processing unit that performs image processing on an image signal output from the imaging unit, and the imaging unit calculates a camera shake amount. An image pickup apparatus comprising: an amount calculation unit; and a signal output unit that outputs, to the image processing unit, an image signal of an area corresponding to the shake amount calculated by the shake amount calculation unit among the image signals read from the image sensor Is provided.

  According to the third aspect of the present invention, there is provided a method for controlling an image pickup apparatus having an image pickup device, including calculating an amount of camera shake, and changing an area for controlling the drive of the image pickup device in accordance with the calculated amount of camera shake. Are provided.

  According to the fourth aspect of the present invention, the control device of the image pickup apparatus having the image pickup device controls the drive of the image pickup device in accordance with the shake amount calculation process for calculating the shake amount and the shake amount calculated in the shake amount calculation process. A control program for executing an area change process for changing an area to be performed is provided.

  According to the fifth aspect of the present invention, there is provided a method for controlling an image pickup apparatus having an image pickup device, wherein the amount of camera shake is calculated, and among the image signals read from the image pickup device, the calculated image shake amount is determined. And outputting an image signal of the region to the image processing unit.

  According to the sixth aspect of the present invention, in the control device of the image pickup apparatus having the image pickup device, the camera shake amount calculation processing for calculating the amount of camera shake and the image shake amount calculation processing among the image signals read from the image pickup device. There is provided a control program for executing signal output processing for outputting an image signal of an area corresponding to the calculated amount of camera shake to the image processing unit.

  According to the aspects of the present invention, it is possible to reduce power consumption and processing load in the imaging apparatus.

It is sectional drawing of a multilayer type image pick-up element. It is a figure explaining the pixel arrangement | sequence and unit group of an imaging chip. It is a circuit diagram corresponding to the unit group of an imaging chip. It is a block diagram which shows the functional structure of an image pick-up element. It is a block diagram which shows the structure of the imaging device which concerns on 1st Embodiment. It is a block diagram which shows the specific structure of the signal processing chip concerning 1st Embodiment. FIG. 6 is a functional block diagram of an image processing unit and a system control unit shown in FIG. 5. 3 is a flowchart illustrating an example of a method for controlling the imaging apparatus according to the first embodiment. It is a figure which shows the moving direction and moving amount | distance of a to-be-photographed object at the time of camera shake. It is a figure which shows the size of the image data in the imaging part which concerns on 1st Embodiment, an image process part, and a recording part. It is a block diagram which shows the structure of the imaging device which concerns on 2nd Embodiment. FIG. 12 is a functional block diagram of an image processing unit and a system control unit shown in FIG. 11. It is a flowchart which shows an example of the control method of the imaging device which concerns on 2nd Embodiment. It is a block diagram which shows the specific structure of the signal processing chip concerning 3rd Embodiment. 10 is a flowchart illustrating an example of a control method for an imaging apparatus according to a third embodiment. It is a figure which shows the size of the image data in the imaging part and image processing part which concern on 3rd Embodiment. It is a figure which shows the size of the image data in the imaging part and image processing part which concern on 4th Embodiment. It is a block diagram which shows the structure of the imaging device and electronic device which concern on 5th Embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. However, the present invention is not limited to this. Further, in the drawings, in order to describe the embodiment, the scale is appropriately changed and expressed, for example, partly enlarged or emphasized.

<First Embodiment>
FIG. 1 is a cross-sectional view of a multilayer image sensor. The multilayer image sensor 100 is described in Japanese Patent Application No. 2012-139026 filed earlier by the applicant of the present application. The imaging device 100 includes an imaging chip 113 that outputs a pixel signal corresponding to incident light, a signal processing chip 111 that processes the pixel signal, and a memory chip 112 that stores the pixel signal. The imaging chip 113, the signal processing chip 111, and the memory chip 112 are stacked, and are electrically connected to each other by a conductive bump 109 such as Cu.

  As shown in the figure, incident light is incident mainly in the positive direction of the Z-axis indicated by a white arrow. In the present embodiment, in the imaging chip 113, the surface on the side where incident light is incident is referred to as a back surface. Further, as shown in the coordinate axes, the left direction of the paper orthogonal to the Z axis is the X axis plus direction, and the front side of the paper orthogonal to the Z axis and the X axis is the Y axis plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

  An example of the imaging chip 113 is a back-illuminated MOS image sensor. The PD layer 106 is disposed on the back side of the wiring layer 108. The PD layer 106 includes a plurality of photodiodes (Photodiode; hereinafter referred to as PD) 104 that are two-dimensionally arranged and accumulate electric charges according to incident light, and a transistor 105 provided corresponding to the PD 104. .

  A color filter 102 is provided on the incident side of incident light in the PD layer 106 via a passivation film 103. The color filter 102 is a filter that passes a specific wavelength region of visible light. The color filter 102 has a plurality of types that transmit different wavelength regions, and has a specific arrangement corresponding to each of the PDs 104. The arrangement of the color filter 102 will be described later. A set of the color filter 102, the PD 104, and the transistor 105 forms one pixel.

  On the incident light incident side of the color filter 102, a microlens 101 is provided corresponding to each pixel. The microlens 101 condenses incident light toward the corresponding PD 104.

  The wiring layer 108 includes a wiring 107 that transmits the pixel signal from the PD layer 106 to the signal processing chip 111. The wiring 107 may be multilayer, and a passive element and an active element may be provided. A plurality of bumps 109 are disposed on the surface of the wiring layer 108. The plurality of bumps 109 are aligned with the plurality of bumps 109 provided on the opposing surface of the signal processing chip 111. Then, by pressing the imaging chip 113 and the signal processing chip 111, the aligned bumps 109 are joined and electrically connected.

  Similarly, a plurality of bumps 109 are disposed on the mutually facing surfaces of the signal processing chip 111 and the memory chip 112. These bumps 109 are aligned with each other. Then, when the signal processing chip 111 and the memory chip 112 are pressurized or the like, the aligned bumps 109 are joined and electrically connected.

  The bonding between the bumps 109 is not limited to Cu bump bonding by solid phase diffusion, and micro bump bonding by solder melting may be employed. Further, for example, about one bump 109 may be provided for one unit group described later. Therefore, the size of the bump 109 may be larger than the pitch of the PD 104. In addition, a bump larger than the bump 109 corresponding to the pixel region may be provided in a peripheral region other than the pixel region where the pixels are arranged (the pixel region 113A shown in FIG. 2).

  The signal processing chip 111 includes a TSV (Through-Silicon Via) 110 that connects circuits provided on the front surface and the back surface to each other. The TSV 110 is provided in the peripheral area. The TSV 110 may be provided in the peripheral area of the imaging chip 113 or the memory chip 112.

  FIG. 2 is a diagram for explaining a pixel array and a unit group of the imaging chip. FIG. 2 particularly shows a state where the imaging chip 113 is observed from the back side. An area where pixels are arranged in the imaging chip 113 is referred to as a pixel area 113A. In the pixel region 113A, 20 million or more pixels are arranged in a matrix. In the example shown in FIG. 2, 16 pixels of 4 pixels × 4 pixels adjacent to each other form one unit group 131. The grid lines in FIG. 2 indicate a concept in which adjacent pixels are grouped to form a unit group 131. The number of pixels forming the unit group 131 is not limited to this, and may be about 1000, for example, 32 pixels × 64 pixels, or more or less.

  As shown in the partially enlarged view of the pixel region 113A, the unit group 131 includes four so-called Bayer arrays including four pixels of green pixels Gb and Gr, a blue pixel B, and a red pixel R in the vertical and horizontal directions. The green pixel is a pixel having a green filter as the color filter 102, and receives light in the green wavelength band of incident light. Similarly, the blue pixel is a pixel having a blue filter as the color filter 102 and receives light in the blue wavelength band. The red pixel is a pixel having a red filter as the color filter 102 and receives light in the red wavelength band.

  FIG. 3 is a circuit diagram corresponding to the unit group of the imaging chip. In FIG. 3, a rectangle surrounded by a dotted line typically represents a circuit corresponding to one pixel. Note that at least some of the transistors described below correspond to the transistor 105 in FIG.

  As described above, the unit group 131 is formed of 16 pixels. The 16 PDs 104 corresponding to the respective pixels are each connected to the transfer transistor 302. The gate of each transfer transistor 302 is connected to a TX wiring 307 to which a transfer pulse is supplied. In the present embodiment, the TX wiring 307 is commonly connected to the 16 transfer transistors 302.

  The drain of each transfer transistor 302 is connected to the source of the corresponding reset transistor 303, and a so-called floating diffusion FD (charge detection unit) between the drain of the transfer transistor 302 and the source of each reset transistor 303 is connected to the amplifier transistor 304. Connected to the gate. The drain of each reset transistor 303 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. The gate of each reset transistor 303 is connected to a reset wiring 306 to which a reset pulse is supplied. In the present embodiment, the reset wiring 306 is commonly connected to the 16 reset transistors 303.

  The drain of each amplification transistor 304 is connected to a Vdd wiring 310 to which a power supply voltage is supplied. The source of each amplification transistor 304 is connected to the drain of each corresponding selection transistor 305. The gate of each selection transistor 305 is connected to a decoder wiring 308 to which a selection pulse is supplied. In the present embodiment, the decoder wiring 308 is provided independently for each of the 16 selection transistors 305. The source of each selection transistor 305 is connected to a common output wiring 309. The load current source 311 supplies current to the output wiring 309. That is, the output wiring 309 for the selection transistor 305 is formed by a source follower. Note that the load current source 311 may be provided on the imaging chip 113 side or may be provided on the signal processing chip 111 side.

  Here, the flow from the start of charge accumulation to pixel output after the end of accumulation will be described. A reset pulse is applied to the reset transistor 303 through the reset wiring 306. At the same time, a transfer pulse is applied to the transfer transistor 302 through the TX wiring 307. As a result, the potentials of the PD 104 and the floating diffusion FD are reset.

  When the application of the transfer pulse is canceled, the PD 104 converts the incident light to be received into electric charge and accumulates it. Thereafter, when the transfer pulse is applied again without the reset pulse being applied, the charge accumulated in the PD 104 is transferred to the floating diffusion FD. As a result, the potential of the floating diffusion FD changes from the reset potential to the signal potential after charge accumulation. When a selection pulse is applied to the selection transistor 305 through the decoder wiring 308, the change in the signal potential of the floating diffusion FD is transmitted to the output wiring 309 through the amplification transistor 304 and the selection transistor 305. By such an operation of the circuit, a pixel signal corresponding to the reset potential and the signal potential is output from the unit pixel to the output wiring 309.

  As shown in FIG. 3, in the present embodiment, the reset wiring 306 and the TX wiring 307 are common to the 16 pixels forming the unit group 131. That is, the reset pulse and the transfer pulse are simultaneously applied to all 16 pixels. Therefore, all the pixels forming the unit group 131 start charge accumulation at the same timing and end charge accumulation at the same timing. However, the pixel signal corresponding to the accumulated charge is selectively output to the output wiring 309 by sequentially applying the selection pulse to each selection transistor 305. In addition, the reset wiring 306, the TX wiring 307, and the output wiring 309 are provided separately for each unit group 131.

  By configuring the circuit with the unit group 131 as a reference in this way, the charge accumulation time can be controlled for each unit group 131. In other words, pixel signals with different charge accumulation times can be output between the unit groups 131. More specifically, while one unit group 131 performs charge accumulation once, the other unit group 131 repeats charge accumulation several times and outputs a pixel signal each time, so that Each frame for moving images can be output at a different frame rate between the unit groups 131.

  FIG. 4 is a block diagram illustrating a functional configuration of the image sensor. The analog multiplexer 411 sequentially selects the 16 PDs 104 that form the unit group 131. Then, the multiplexer 411 outputs each pixel signal of the 16 PDs 104 to the output wiring 309 provided corresponding to the unit group 131. The multiplexer 411 is formed on the imaging chip 113 together with the PD 104.

  The analog pixel signal output through the multiplexer 411 is amplified by an amplifier 412 formed in the signal processing chip 111. Then, the pixel signal amplified by the amplifier 412 is processed by a signal processing circuit 413 that performs correlated double sampling (CDS) / analog / digital (Analog / Digital) conversion, which is formed in the signal processing chip 111. Correlated double sampling signal processing is performed, and A / D conversion (conversion from an analog signal to a digital signal) is performed. The pixel signal is subjected to correlated double sampling signal processing in the signal processing circuit 413, whereby noise of the pixel signal is reduced. The A / D converted pixel signal is transferred to the demultiplexer 414 and stored in the pixel memory 415 corresponding to each pixel. The demultiplexer 414 and the pixel memory 415 are formed in the memory chip 112.

  The arithmetic circuit (camera shake amount calculation unit) 416 processes the pixel signal stored in the pixel memory 415 and passes it to the subsequent image processing unit. In the present embodiment, the arithmetic circuit 416 is provided in the signal processing chip 111. However, the arithmetic circuit 416 may be provided in the memory chip 112. Note that FIG. 4 shows connections for one unit group 131, but actually these exist for each unit group 131 and operate in parallel. However, the arithmetic circuit 416 may not exist for each unit group 131. For example, one arithmetic circuit 416 may perform sequential processing while sequentially referring to the values in the pixel memory 415 corresponding to each unit group 131.

  As described above, the output wiring 309 is provided corresponding to each of the unit groups 131. The image sensor 100 is formed by stacking an image pickup chip 113, a signal processing chip 111, and a memory chip 112. For this reason, by using the electrical connection between the chips using the bumps 109 for the output wirings 309, the wirings can be routed without enlarging each chip in the surface direction.

  Next, blocks set in the pixel region 113A (see FIG. 2) of the image sensor 100 will be described. In the present embodiment, the pixel region 113A of the image sensor 100 is divided into a plurality of blocks. The plurality of blocks are defined to include at least one unit group 131 per block. In each block, pixels included in each block are controlled by different control parameters. That is, pixel signals having different control parameters are acquired for a pixel group included in a block and a pixel group included in another block. Examples of the control parameter include a charge accumulation time or accumulation count, a frame rate, a gain, a thinning rate, the number of addition rows or addition columns to which pixel signals are added, the number of digitization bits, and the like. Furthermore, the control parameter may be a parameter in image processing after obtaining an image signal from a pixel.

  Here, the charge accumulation time refers to the time from when PD 104 starts to accumulate charge until it ends. The number of times of charge accumulation refers to the number of times the PD 104 accumulates charges per unit time. The frame rate is a value representing the number of frames processed (displayed or recorded) per unit time in a moving image. The unit of the frame rate is represented by fps (Frames Per Second). The higher the frame rate, the smoother the movement of the subject (that is, the object to be imaged) in the moving image.

  The gain means a gain factor (amplification factor) of the amplifier 412. By changing this gain, the ISO sensitivity can be changed. This ISO sensitivity is a photographic film standard established by ISO and represents how much light the photographic film can record. However, generally, ISO sensitivity is also used when expressing the sensitivity of the image sensor 100. In this case, the ISO sensitivity is a value representing the ability of the image sensor 100 to capture light. Increasing the gain improves the ISO sensitivity. For example, when the gain is doubled, the electrical signal (pixel signal) is also doubled, and the brightness is appropriate even when the amount of incident light is half. However, when the gain is increased, noise included in the electric signal is also amplified, so that noise increases.

  The thinning-out rate is the ratio of the number of pixels that do not read out pixel signals with respect to the total number of pixels in a predetermined area. For example, when the thinning rate of a predetermined area is 0, it means that pixel signals are read from all pixels in the predetermined area. Further, when the thinning rate of the predetermined area is 0.5, it means that the pixel signal is read from half of the pixels in the predetermined area. Specifically, when the unit group 131 is a Bayer array, it is read as a pixel from which pixel signals are alternately read out every other unit of the Bayer array in the vertical direction, that is, every two pixels (two rows) of the pixel unit. Pixels that are not output are set. It should be noted that the resolution of the image is reduced when the pixel signal readout is thinned out. However, since 20 million or more pixels are arranged in the image sensor 100, for example, even if thinning is performed at a thinning rate of 0.5, an image can be displayed with 10 million or more pixels. For this reason, it is considered that the reduction in resolution is not a concern for the user (photographer).

  Further, the number of added rows refers to the number of vertical pixels (number of rows) to be added when pixel signals of pixels adjacent in the vertical direction are added. Further, the number of added columns refers to the number of horizontal pixels (number of columns) to be added when pixel signals of pixels adjacent in the horizontal direction are added. Such addition processing is performed in the arithmetic circuit 416, for example. The arithmetic circuit 416 performs the process of adding the pixel signals of a predetermined number of pixels adjacent in the vertical direction or the horizontal direction, thereby obtaining the same effect as the process of reading out the pixel signals by thinning out at a predetermined thinning rate. In the addition process described above, the average value may be calculated by dividing the added value by the number of rows or columns added by the arithmetic circuit 416.

  The number of bits for digitization refers to the number of bits when the signal processing circuit 413 converts an analog signal into a digital signal in A / D conversion. As the number of bits of the digital signal increases, brightness, color change, and the like are expressed in more detail.

  In the present embodiment, the accumulation condition refers to a condition related to charge accumulation in the image sensor 100. Specifically, the accumulation condition refers to the charge accumulation time or number of accumulations, the frame rate, and the gain among the control parameters described above. Since the frame rate can change according to the charge accumulation time and the number of times of accumulation, the frame rate is included in the accumulation condition. Further, the amount of light for proper exposure changes according to the gain, and the charge accumulation time or number of times of accumulation can also change according to the amount of light for proper exposure. For this reason, the gain is included in the accumulation condition.

  The imaging condition is a condition related to imaging of a subject. Specifically, the imaging condition refers to a control parameter including the above accumulation condition. The imaging conditions include control parameters (for example, charge accumulation time or accumulation count, frame rate, gain) for controlling the image sensor 100, as well as control parameters (for controlling reading of signals from the image sensor 100). For example, a thinning rate), a control parameter for processing a signal from the image sensor 100 (for example, the number of addition rows or addition columns for adding pixel signals, the number of digitization bits, the image processing unit 30 described later performs image processing) Is also included.

  FIG. 5 is a block diagram illustrating a configuration of the imaging apparatus according to the first embodiment. An imaging apparatus 1 illustrated in FIG. 5 includes, for example, devices such as a digital camera, a smartphone, a mobile phone, and a personal computer having an imaging function. As illustrated in FIG. 5, the imaging device 1 includes a lens unit 10, an imaging unit 20, an image processing unit 30, a work memory 40, a display unit 50, an operation unit 55, a recording unit 60, and a system control unit 70. The lens unit 10 is an imaging optical system composed of a plurality of lens groups. The lens unit 10 guides the light flux from the subject to the imaging unit 20. The lens unit 10 may be integrated with the imaging device 1 or may be an interchangeable lens that can be attached to and detached from the imaging device 1. Further, the lens unit 10 may have a built-in focus lens or a zoom lens.

  The imaging unit 20 includes an imaging element 100 and a drive unit 21. The drive unit 21 is a control circuit that controls driving of the image sensor 100 in accordance with an instruction from the system control unit 70. Here, the drive unit 21 controls the timing (or timing cycle) at which the reset pulse and the transfer pulse are applied to the reset transistor 303 and the transfer transistor 302, respectively, thereby reducing the charge accumulation time or accumulation count as a control parameter. Control. In addition, the drive unit 21 controls the frame rate by controlling the timing (or timing cycle) at which the reset pulse, the transfer pulse, and the selection pulse are applied to the reset transistor 303, the transfer transistor 302, and the selection transistor 305, respectively. To do. The drive unit 21 controls the thinning rate by setting pixels to which the reset pulse, the transfer pulse, and the selection pulse are applied.

  In addition, the drive unit 21 controls the ISO sensitivity of the image sensor 100 by controlling the gain (also referred to as gain factor or amplification factor) of the amplifier 412. In addition, the driving unit 21 sends an instruction to the arithmetic circuit 416 to set the number of added rows or the number of added columns to which the pixel signals are added. Further, the drive unit 21 sets the number of bits for digitization by sending an instruction to the signal processing circuit 413. Furthermore, the drive unit 21 sets a block in the pixel region 113 </ b> A of the image sensor 100. In this way, the drive unit 21 functions as an image sensor control unit that causes the image sensor 100 to capture an image under different image capturing conditions for each of a plurality of blocks and output a pixel signal. The system control unit 70 instructs the drive unit 21 on the block position, shape, range, and the like.

  The image sensor 100 delivers the pixel signal from the image sensor 100 to the image processing unit 30. The image processing unit 30 uses the work memory 40 as a work space, performs various image processing on the RAW data including pixel signals of each pixel, and generates image data in a predetermined file format (for example, JPEG format). . In this specification, “image data” is data constituting an image (still image, moving image, live view image) captured by the image capturing unit 20, and before image processing is performed in the image processing unit 30. Data (that is, RAW data) and data after image processing is performed. In the present specification, “RAW data” refers to image data before image processing is performed in the image processing unit 30. In the present specification, “image data” may be referred to as “image signal”.

  The image processing unit 30 executes various image processes. For example, the image processing unit 30 generates an RGB image signal by performing color signal processing (color tone correction) on a signal obtained by the Bayer array. The image processing unit 30 performs image processing such as white balance adjustment, sharpness adjustment, gamma correction, and gradation adjustment on the RGB image signal. Further, the image processing unit 30 performs a process of compressing in a predetermined compression format (JPEG format, MPEG format, etc.) as necessary. The image processing unit 30 outputs the generated image data to the recording unit 60. In addition, the image processing unit 30 outputs the generated image data to the display unit 50.

  Parameters that are referred to when the image processing unit 30 performs image processing are also included in the control parameters (imaging conditions). For example, parameters such as color signal processing (tone correction), white balance adjustment, gradation adjustment, and compression rate are included in the control parameters. The signal read from the image sensor 100 changes according to the charge accumulation time, and the parameters referred to when performing image processing also change according to the change in the signal. The image processing unit 30 sets different control parameters for each block, and executes image processing such as color signal processing based on these control parameters.

  The image processing unit 30 extracts frames at predetermined timings from a plurality of frames obtained in time series from the imaging unit 20. Alternatively, the image processing unit 30 discards frames at a predetermined timing among a plurality of frames obtained in time series from the imaging unit 20. Thereby, since the amount of data can be reduced, it is possible to reduce the load of subsequent processing. Further, the image processing unit 30 calculates one or a plurality of frames to be interpolated between the frames based on a plurality of frames obtained in time series from the imaging unit 20. Then, the image processing unit 30 adds the calculated one or more frames between the frames. As a result, a moving image with smoother motion can be reproduced during moving image reproduction. Moreover, although the drive part 21 is comprised so that the thinning-out rate may be controlled, it is not restricted to such a structure. For example, the drive unit 21 reads pixel signals from all pixels, but the image processing unit 30 or the arithmetic circuit 416 controls the thinning rate by discarding predetermined pixel signals out of the read pixel signals. Good.

  In the present embodiment, in addition to the above-described processing, the image processing unit 30 performs processing for calculating a camera shake amount (motion amount due to camera shake) by analysis processing of image data read from the image sensor 100. That is, the image processing unit 30 calculates the amount of camera shake based on the moving direction and the moving amount of the subject in a plurality of frames (at least the current frame and a frame before the current frame) obtained in time series from the imaging unit 20. (See, for example, FIG. 9). Then, the image processing unit 30 outputs camera shake amount information indicating the calculated camera shake amount to the system control unit 70.

  The work memory 40 temporarily stores image data and the like when image processing by the image processing unit 30 is performed. The display unit 50 is configured by a liquid crystal display panel, for example. The display unit 50 displays images (still images, moving images, live view images) and various types of information captured by the imaging unit 20. The display unit 50 includes a touch panel 51 as shown in FIG. The touch panel 51 is formed on the display screen of the display unit 50. The touch panel 51 outputs a signal indicating the position touched by the user to the system control unit 70 when the user selects an image or the like.

  The operation unit 55 is a release switch, a moving image switch, various operation switches, and the like operated by a user. The operation unit 55 outputs a signal corresponding to the operation by the user to the system control unit 70. The recording unit 60 has a card slot into which a storage medium such as a memory card can be mounted. The recording unit 60 stores the image data and various data generated by the image processing unit 30 in a recording medium mounted in the card slot. The recording unit 60 has an internal memory. The recording unit 60 can also record the image data and various data generated by the image processing unit 30 in the internal memory.

  The system control unit 70 controls the overall processing and operation of the imaging apparatus 1. The system control unit 70 has a CPU (Central Processing Unit). In the present embodiment, the system control unit 70 divides the imaging surface (pixel area 113A) of the imaging device 100 (imaging chip 113) into a plurality of blocks, and different charge accumulation times (or charge accumulation times) and frame rates among the blocks. Get an image with a gain. Therefore, the system control unit 70 instructs the drive unit 21 about the block position, shape, range, and accumulation condition for each block. In addition, the system control unit 70 causes an image to be acquired with a different thinning rate between blocks, the number of addition rows or addition columns to which pixel signals are added, and the number of digitization bits. For this reason, the system control unit 70 instructs the drive unit 21 on the imaging conditions for each block (the thinning rate, the number of added rows or columns to which pixel signals are added, and the number of digitization bits). Further, the image processing unit 30 executes image processing under imaging conditions (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate) that are different between blocks. Therefore, the system control unit 70 instructs the image processing unit 30 on the imaging conditions for each block (control parameters such as color signal processing, white balance adjustment, gradation adjustment, and compression rate).

  Further, the system control unit 70 causes the recording unit 60 to record the image data generated by the image processing unit 30. Further, the system control unit 70 causes the display unit 50 to display an image by causing the display unit 50 to output the image data generated in the image processing unit 30. Further, the system control unit 70 causes the display unit 50 to display an image by reading out the image data recorded in the recording unit 60 and causing the display unit 50 to output the image data. The images displayed on the display unit 50 include still images, moving images, and live view images. Here, the live view image is an image displayed on the display unit 50 by sequentially outputting the image data generated by the image processing unit 30 to the display unit 50. The live view image is used for the user to confirm the image of the subject imaged by the imaging unit 20. The live view image is also called a through image or a preview image.

  In the present embodiment, the system control unit 70 performs a process of selecting an area for driving and controlling the image sensor 100 according to the camera shake amount indicated by the camera shake amount information from the image processing unit 30 in addition to the above-described process. Then, the system control unit 70 outputs an instruction signal designating the selected area to the drive unit 21. Based on the instruction signal from the system control unit 70, the drive unit 21 reads out the pixel signal only in the region in the pixel region 113A, and outputs image data (RAW data) including the pixel signal of each pixel in the region. Forward to 30. An area in the pixel area 113A that the system control unit 70 selects according to the amount of camera shake is referred to as a “transfer area”.

  FIG. 6 is a block diagram showing a specific configuration of the signal processing chip according to the first embodiment. In addition to the signal processing chip 111, FIG. 6 also shows the imaging chip 113, the memory chip 112, the image processing unit 30, and the system control unit 70.

  The signal processing chip 111 has a function of the driving unit 21. The signal processing chip 111 includes a sensor control unit 441, a block control unit 442, a synchronization control unit 443, and a signal control unit 444 as shared control functions. The signal processing chip 111 includes a sensor control unit 441, a block control unit 442, a synchronization control unit 443, and a control unit (drive control unit) 420 that performs overall control of the signal control unit 444. The control unit 420 converts an instruction from the system control unit 70 into a control signal that can be executed by each of the control units 441 to 444 and delivers the control signal to each.

  Based on the control signal from the control unit 420, the sensor control unit 441 performs control to send control pulses (reset pulse, transfer pulse, selection pulse) related to charge accumulation and charge readout of each pixel to the imaging chip 113. Do. Specifically, the sensor control unit 441 controls the start and end of charge accumulation by sending a reset pulse and a transfer pulse to the target pixel. In addition, the sensor control unit 441 outputs a pixel signal to the output wiring 309 by sending a selection pulse to the readout pixel of the pixel signal.

  Based on the control signal from the control unit 420, the block control unit 442 performs control to send a specific pulse for specifying the unit group 131 to be driven and controlled to the imaging chip 113. As described above, the pixel region 113A of the image sensor 100 is divided into a plurality of blocks. The plurality of blocks include at least one unit group 131 per block. Pixels included in the same block start charge accumulation at the same timing and end charge accumulation at the same timing. Therefore, the block control unit 442 blocks one or a plurality of unit groups 131 by sending specific pulses to the target unit group 131 based on a control signal from the control unit 420. The transfer pulse and the reset pulse received by each pixel via the TX wiring 307 and the reset wiring 306 are the logical product of each pulse sent by the sensor control unit 441 and a specific pulse sent by the block control unit 442. In this manner, the block control unit 442 controls each region as an independent block, thereby realizing charge accumulation control for each region.

  The synchronization control unit 443 performs control to send the synchronization signal to the imaging chip 113 based on the control signal from the control unit 420. Each pulse (reset pulse, transfer pulse, selection pulse) becomes active in the imaging chip 113 in synchronization with the synchronization signal. For example, the synchronization control unit 443 adjusts the synchronization signal, thereby realizing random control, thinning control, and the like in which only specific pixels of pixels belonging to the same unit group 131 are subject to drive control.

  The signal control unit 444 controls the gain (gain factor, amplification factor) of the amplifier 412 based on the control signal from the control unit 420. Further, the signal control unit 444 performs timing control on the A / D converter 413b based on the control signal from the control unit 420. The pixel signal output via the output wiring 309 is input to the A / D converter 413b via the amplifier 412 and the CDS circuit 413a. The A / D converter 413b converts the input pixel signal into a digital signal under the control of the signal control unit 444. The pixel signal converted into the digital signal is delivered to the demultiplexer 414. Then, the pixel signal is stored as a pixel value of digital data in the pixel memory 415 corresponding to each pixel by the demultiplexer 414.

  The signal processing chip 111 has a timing memory 430. Based on an instruction from the system control unit 70, the control unit 420 repeats block division information on which unit group 131 is combined to form a block, and how many times the formed blocks accumulate charge. (That is, the charge accumulation timing) is stored in the timing memory 430. The timing memory 430 is configured by a flash RAM, for example. The controller 420 performs charge accumulation control in units of blocks by referring to block classification information and accumulation count information stored in the timing memory 430.

  As described above, the system control unit 70 selects a transfer area for driving and controlling the image sensor 100 according to the amount of camera shake. Then, the system control unit 70 outputs an instruction signal designating the selected transfer area to the control unit 420. Based on the instruction signal from the system control unit 70, the control unit 420 performs control such that a plurality of unit groups are combined to form a block corresponding to the transfer area.

  FIG. 7 is a functional block diagram of the image processing unit and the system control unit shown in FIG. As illustrated in FIG. 7, the image processing unit 30 includes an image generation unit 31 and a camera shake detection unit (a camera shake amount calculation unit) 32. The image generation unit 31 generates image data by performing various kinds of image processing on the RAW data including pixel signals of each pixel output from the imaging unit 20. The camera shake detection unit 32 detects a motion vector based on the movement direction and the movement amount of the subject due to camera shake in a plurality of frames obtained in time series from the imaging unit 20. The motion vector represents a reference frame and motion from the frame as a vector. Then, the camera shake detection unit 32 predicts the position of the subject in the next frame from the detected motion vector. The camera shake detection unit 32 calculates (detects) the amount of subject displacement between the current frame and the next frame as a camera shake amount. Then, the camera shake detection unit 32 outputs camera shake amount information indicating the calculated camera shake amount to the system control unit 70.

  The system control unit 70 includes a display control unit 71 and an imaging control unit (drive control unit) 72. The display control unit 71 performs control to output the image data generated by the image generation unit 31 to the display unit 50 and display an image (still image, moving image, live view image) on the display screen of the display unit 50. The imaging control unit 72 performs drive control of the imaging element 100. In other words, the imaging control unit 72 outputs an instruction signal for instructing imaging conditions such as appropriate exposure to the driving unit 21. Further, the imaging control unit 72 selects a transfer area according to the camera shake amount indicated by the camera shake amount information from the camera shake detection unit 32, and outputs an instruction signal designating the selected transfer area to the drive unit 21. The drive unit 21 sets a transfer area in the image sensor 100 based on an instruction signal from the imaging control unit 72. Further, the drive unit 21 drives and controls the transfer area of the image sensor 100 in accordance with the imaging condition instructed by the instruction signal from the imaging control unit 72.

  In the system control unit 70, the display control unit 71 and the imaging control unit 72 are realized by the CPU executing processing based on the control program.

  Next, the photographing operation according to the first embodiment will be described. FIG. 8 is a flowchart illustrating an example of a control method of the imaging apparatus according to the first embodiment. FIG. 9 is a diagram showing the moving direction and moving amount of the subject during camera shake, where (A) shows the position of the subject during normal shooting, and (B) shows the position of the subject during camera shake shooting. ing.

  In the process shown in FIG. 8, when the power of the imaging apparatus 1 is turned on by the user, the imaging control unit 72 starts capturing a live view image (step S1). In addition, the display control unit 71 displays the live view image captured by the imaging unit 20 on the display screen of the display unit 50 (step S2). At this time, since the live view image does not need to be a smooth image of the movement of the subject, the imaging control unit 72 drives and controls the imaging element 100 so as to capture an image at a low frame rate, for example. In addition, when the image pickup apparatus 1 is turned on, the image pickup control unit 72 performs drive control on all the pixel regions 113A of the image pickup element 100, for example.

  The image processing unit 30 receives image data of a live view image captured by the imaging unit 20. The image generation unit 31 performs various image processing on the received image data. Further, the image generation unit 31 creates motion vector evaluation values in a plurality of frames obtained in time series. In addition, the camera shake detection unit 32 detects a motion vector indicating the moving direction and the moving amount of the subject due to the camera shake based on the evaluation value of the motion vector.

  Here, the camera shake detection unit 32 detects a motion vector using a background subject among a plurality of subjects. This is because in the case of a movable subject such as a person, it is difficult for the camera shake detection unit 32 to determine whether the subject is moving or whether the subject is moving due to camera shake. For this reason, the camera shake detection unit 32 compares a plurality of frames obtained in time series to detect a moving subject. Further, the camera shake detection unit 32 detects a human subject using a face detection function described in, for example, Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940). In addition to face detection, the camera shake detection unit 32 detects a human subject included in image data as described in, for example, Japanese Patent Application Laid-Open No. 2010-16621 (US2010 / 0002940). Is detected. The camera shake detection unit 32 recognizes subjects other than the moving subject detected as described above as background subjects.

  In the example shown in FIG. 9, a human subject O1 and a background subject including a building subject O2 are captured as live view images. The camera shake detection unit 32 detects background subjects including a human subject O1 and a building subject O2 in a frame before the current frame (the frame of the image shown in FIG. 9A). Further, the camera shake detection unit 32 detects background subjects including the human subject O1 and the building subject O2 in the current frame (the frame of the image shown in FIG. 9B).

  Then, the camera shake detection unit 32 detects a motion vector by specifying the motion (change in position) of the background subject between a plurality of frames. In the example shown in FIG. 9, the area 201 in the live view image in (A) has moved to the position of the area 202 in the live view image in (B) due to the hand shake of the user. The camera shake detection unit 32 extracts the edge portion of the building subject O2 in the previous frame and also extracts the edge portion of the building subject O2 in the current frame. The camera shake detection unit 32 detects the motion vector 203 by specifying the moving direction and the moving amount of the edge portion of the building subject O2.

  The camera shake detection unit 32 predicts the position of the subject in the next frame from the detected motion vector. The camera shake detection unit 32 calculates (detects) the amount of subject displacement between the current frame and the next frame as a camera shake amount. Then, the camera shake detection unit 32 outputs camera shake amount information indicating the calculated camera shake amount to the system control unit 70. Such processing for calculating the amount of camera shake by the camera shake detector 32 is executed for each frame. However, in order to reduce the processing load and power consumption of the camera shake detection unit 32, a process for calculating the camera shake amount may be executed every several frames instead of every frame.

  The imaging control unit 72 acquires camera shake amount information from the camera shake detection unit 32 (step S3). Then, the imaging control unit 72 selects a transfer area for each frame according to the camera shake amount indicated by the camera shake amount information, and outputs an instruction signal designating the selected transfer area to the drive unit 21 (step S4). The drive unit 21 sets a transfer area in the image sensor 100 for each frame based on an instruction signal from the imaging control unit 72. In addition, the drive unit 21 controls the transfer area of the image sensor 100. Thus, the process of selecting the area for driving and controlling the transfer area is executed for each frame. However, in order to reduce the processing load and power consumption of the imaging control unit 72 and the drive unit 21, a process for selecting an area for driving and controlling the transfer area is performed every several frames instead of every frame. Also good.

  As described above, the imaging control unit 72 sets a transfer area corresponding to the amount of camera shake calculated by the camera shake detection unit 32, and performs control so that the subject is imaged using only the transfer area. FIG. 10 is a diagram illustrating image data sizes in the imaging unit, the image processing unit, and the recording unit according to the first embodiment. As shown in FIG. 10A1, in general, in electronic camera shake correction, the system control unit 70 performs control so as to perform imaging in the pixel region 113A of the imaging device 100. In the image processing unit 30, the camera shake detection unit 32 calculates a camera shake amount based on the image data. Further, the image generation unit 31 extracts (trims) only image data of the region 200 of the recording image size (viewing angle for recording on the recording unit 60) from the image data of the pixel region 113A based on the amount of camera shake. . After that, as shown in FIG. 10A 2, the image data of the recording image size area 200 is output from the image processing unit 30 to the recording unit 60 and recorded in the recording unit 60. Further, the image data of the recording image size area 200 is output from the image processing unit 30 to the display unit 50 and displayed on the display unit 50.

  On the other hand, in this embodiment, as shown in FIG. 10B1, the imaging control unit 72 sets a transfer area (area of the recorded image size in FIG. 10) 200 according to the amount of camera shake calculated by the camera shake detection unit 32. Then, control is performed so that the subject is imaged using only the transfer area 200. Accordingly, in the image processing unit 30, the image generation unit 31 performs image processing only on the image data in the transfer area 200. Then, as shown in FIG. 10B 2, the image data in the transfer area 200 is output from the image processing unit 30 to the recording unit 60 and recorded in the recording unit 60. The image data in the transfer area 200 is output from the image processing unit 30 to the display unit 50 and displayed on the display unit 50.

Thereafter, the imaging control unit 72 determines whether or not recording of a still image is started by confirming whether or not the release switch has been pressed by the user (step S5). When it is determined that the recording of the still image is started, the imaging control unit 72 performs a process of capturing a still image by the imaging unit 20 and recording a still image in the recording unit 60 (step S6). Note that in response to the half-pressing operation of the release switch, the imaging control unit 72 performs AF (Automatic
Prepare for shooting such as Focusing and AE (Automatic Exposure). Then, following the half-press operation of the release switch, in response to the full-press operation of the release switch, the imaging control unit 72 determines an imaging condition that provides proper exposure, and Control is performed to cause the drive unit 21 to perform imaging. Thereafter, the imaging control unit 72 determines whether or not an operation for turning off the power by the user has been performed (step S8). If the imaging control unit 72 determines that an operation to turn off the power is not performed, the imaging control unit 72 repeatedly executes the processes of steps S1 to S4. Thus, as shown in FIGS. 10B1 and 10B2, only the transfer area of the image sensor 100 is driven and controlled by the drive unit 21, and only the image data in the transfer area 200 is imaged by the image processing unit 30 in the image processing unit 30. Processing is performed. If the imaging control unit 72 determines that an operation to turn off the power by the user has been performed, the imaging control unit 72 ends the process.

  When it is determined that the recording of the still image has not started, the imaging control unit 72 determines whether the recording of the moving image has been started by confirming whether the moving image switch has been pressed (step S7). ). When it is determined that the recording of the moving image has started, the imaging control unit 72 starts capturing the moving image by the imaging unit 20 and recording the moving image in the recording unit 60 (step S11). Further, the display control unit 71 displays the moving image shot by the imaging unit 20 on the display screen of the display unit 50 (step S12). At this time, the moving image is preferably an image in which the movement of the subject is smoother than the live view image. For this reason, the imaging control unit 72 controls the drive of the imaging device 100 so that an image is captured at a higher frame rate than when the live view image is captured.

  Even during the shooting of a moving image, the camera shake detection unit 32 calculates the amount of camera shake based on each frame of the moving image. Then, the camera shake detection unit 32 outputs camera shake amount information indicating the calculated camera shake amount to the system control unit 70. Such processing for calculating the amount of camera shake by the camera shake detector 32 is executed for each frame. However, in order to reduce the processing load and power consumption of the camera shake detection unit 32, a process for calculating the camera shake amount may be executed every several frames instead of every frame.

  The imaging control unit 72 acquires camera shake amount information from the camera shake detection unit 32 (step S13). Then, the imaging control unit 72 selects a transfer area for each frame according to the camera shake amount indicated by the camera shake amount information, and outputs an instruction signal designating the selected transfer area to the drive unit 21 (step S14). The drive unit 21 sets a transfer area in the image sensor 100 for each frame based on an instruction signal from the imaging control unit 72. In addition, the drive unit 21 controls the transfer area of the image sensor 100. Thus, the process of selecting the area for driving and controlling the transfer area is executed for each frame. However, in order to reduce the processing load and power consumption of the imaging control unit 72 and the drive unit 21, a process for selecting an area for driving and controlling the transfer area is performed every several frames instead of every frame. Also good.

  Thereafter, the imaging control unit 72 determines whether or not the recording of the moving image has ended (step S15). If the imaging control unit 72 determines that the recording of the moving image has not been completed, the imaging control unit 72 repeatedly executes the processes of steps S11 to S14. Thus, as shown in FIGS. 10B1 and 10B2, only the transfer area of the image sensor 100 is driven and controlled by the drive unit 21, and only the image data in the transfer area 200 is imaged by the image processing unit 30 in the image processing unit 30. Processing is performed. If the imaging control unit 72 determines that the recording of the moving image has ended, the imaging control unit 72 determines that an operation to turn off the power by the user has been performed (step S8). When it is determined that the user has not performed an operation to turn off the power, the imaging control unit 72 returns to the process of step S1 and repeatedly executes the processes of steps S1 to S4. If the imaging control unit 72 determines that an operation to turn off the power by the user has been performed, the imaging control unit 72 ends the process.

  As described above, according to the first embodiment, the camera shake amount calculation unit 32 that calculates the amount of camera shake and the image sensor 100 are driven and controlled according to the camera shake amount calculated by the camera shake amount calculation unit 32. An imaging control unit 72 that selects an area (transfer area 200 shown in FIG. 10) for driving and controlling the element 100. According to such a configuration, it is possible to reduce power consumption and processing load in the imaging apparatus 1. That is, in general electronic image stabilization, imaging is performed in the pixel region 113A of the image sensor 100, and various image processing is performed on the image data in the pixel region 113A in the image processing unit 30. Thereafter, the image data of the pixel area 113A subjected to the image processing in the image processing unit 30 is trimmed into the image data of the area of the recording image size. Therefore, the image processing unit 30 performs image processing on extra image data other than the transfer area (recorded image size area) 200 actually recorded in the recording unit 60. Specifically, image data of about 10% of the upper, lower, left, and right areas of the transfer area 200 shown in FIG. 10 is processed as extra image data. For this reason, power consumption increases in the imaging unit 20 and the image processing unit 30. Further, the processing load in the image processing unit 30 increases, and high-speed image processing becomes difficult. On the other hand, in the first embodiment, the image processing unit 30 performs image processing only on the transfer area (recorded image size area) 200. Therefore, camera shake correction can be performed, power consumption in the imaging unit 20 and the image processing unit 30 is reduced, and a processing burden on the image processing unit 30 is also reduced.

  Further, in the first embodiment described above, the imaging control unit 72 outputs the image signal of the selected area of the imaging element 100, so that it is possible to reduce power consumption and processing load in the imaging unit 20. In addition, since the imaging control unit 72 selects an area for driving and controlling the imaging element 100 for each frame, the imaging control unit 72 can select an area for driving and controlling the imaging element 100 with high accuracy in accordance with the movement of the subject.

  In the first embodiment described above, the camera shake amount calculation unit 32 calculates the camera shake amount by analyzing the image signal read from the image sensor 100. Therefore, the camera shake amount is calculated without using a special mechanism. be able to. Therefore, the cost of the imaging device 1 does not increase. Further, in the first embodiment described above, the camera shake amount calculation unit 32 calculates the camera shake amount based on the moving direction and the movement amount of the subject in the image signal obtained in time series, and thus calculates an accurate camera shake amount. Can do. In the first embodiment described above, the camera shake amount calculation unit 32 calculates the camera shake amount based on the moving direction and the moving amount of the background subject (for example, the building subject O2 shown in FIG. 9). It is possible to avoid calculating an erroneous amount of camera shake based on the movement of the camera. In the first embodiment described above, the camera shake amount calculation unit 32 is provided in the image processing unit 30 that performs image processing of an image signal. Therefore, the camera shake amount is calculated by a process similar to or substantially similar to the electronic camera shake correction. Can be realized.

Second Embodiment
In the first embodiment described above, the camera shake detection unit 32 is configured to calculate the camera shake amount based on a plurality of frames. On the other hand, in the second embodiment, the system control unit 70 calculates the amount of camera shake based on the acceleration information from the acceleration sensor. Then, the system control unit 70 selects a transfer area based on the calculated camera shake amount.

  FIG. 11 is a block diagram illustrating a configuration of an imaging apparatus according to the second embodiment. As shown in FIG. 11, the imaging device 1 </ b> A according to the second embodiment is provided with an acceleration sensor 65. The acceleration sensor 65 is a sensor that detects acceleration (here, angular acceleration) of the sensor itself. As the acceleration sensor 65, a three-axis acceleration sensor capable of detecting acceleration in three directions of the XYZ axes is used. The acceleration sensor 65 outputs acceleration information representing the detected acceleration to the system control unit 70. Further, the image processing unit 30A performs various types of image processing in the same manner as the image processing unit 30 illustrated in FIG. 5, but does not perform processing for calculating the amount of camera shake. In addition to the process executed by the system control unit 70 shown in FIG. 5, the system control unit 70 </ b> A performs a process of calculating a camera shake amount based on the acceleration indicated by the acceleration information from the acceleration sensor 65. In addition, the structure of the lens part 10, the imaging part 20, the image process part 30, the work memory 40, the display part 50, the operation part 55, and the recording part 60 among the imaging devices 1A shown in FIG. 11 is shown in FIG. The configuration is the same. Accordingly, the same components are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 12 is a functional block diagram of the image processing unit and the system control unit shown in FIG. As shown in FIG. 12, the image processing unit 30 </ b> A in the second embodiment includes only the image generation unit 31. That is, unlike the image processing unit 30 illustrated in FIG. 7, the image processing unit 30 </ b> A is not provided with the camera shake detection unit 32. Further, unlike the system control unit 70 shown in FIG. 7, the system control unit 70 </ b> A in the second embodiment includes a camera shake correction unit (a camera shake amount calculation unit) 73 in addition to the display control unit 71 and the imaging control unit 72. It is out. The camera shake correction unit 73 performs a process of calculating a camera shake amount based on the acceleration indicated by the acceleration information from the acceleration sensor 65. Specifically, the camera shake correction unit 73 specifies the direction of camera shake based on the direction of acceleration indicated by the acceleration information. In addition, the camera shake correction unit 73 calculates a camera shake amount corresponding to the acceleration value indicated by the acceleration information. In the present embodiment, the imaging control unit 72 outputs an instruction signal for designating a transfer area to the drive unit 21 based on the camera shake amount calculated by the camera shake correction unit 73. The drive unit 21 sets a transfer area based on an instruction signal from the imaging control unit 72. Then, the drive unit 21 drives and controls the transfer area.

  In the configuration illustrated in FIG. 11, the configuration of the image generation unit 31, the display control unit 71, and the imaging control unit 72 is the same as the configuration illustrated in FIG. Accordingly, the same components are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 13 is a flowchart illustrating an example of a control method of the imaging apparatus according to the second embodiment. In the process shown in FIG. 13, when the power of the imaging apparatus 1 is turned on by the user, the camera shake correction unit 73 acquires acceleration information from the acceleration sensor 65 (step S21). Further, the imaging control unit 72 starts capturing a live view image (step S22). The display control unit 71 displays the live view image captured by the imaging unit 20 on the display screen of the display unit 50 (step S23). Note that the processes in steps S22 and S23 correspond to the processes in steps S1 and S2 shown in FIG.

  The camera shake correction unit 73 calculates a camera shake amount based on the acceleration information from the acceleration sensor 65 (step S24). That is, the camera shake correction unit 73 specifies the direction of camera shake based on the direction of acceleration indicated by the acceleration information. Further, the camera shake correction unit 73 predicts the position of the subject in the next frame based on the acceleration value indicated by the acceleration information. Then, the camera shake correction unit 73 calculates a camera shake amount corresponding to the predicted position of the subject. Then, the imaging control unit 72 selects a transfer area for each frame according to the amount of camera shake calculated by the camera shake correction unit 73, and outputs an instruction signal designating the selected transfer area to the drive unit 21 (step S25). The drive unit 21 sets a transfer area in the image sensor 100 for each frame based on an instruction signal from the imaging control unit 72. In addition, the drive unit 21 controls the transfer area of the image sensor 100. Thus, the process of selecting the area for driving and controlling the transfer area is executed for each frame. However, in order to reduce the processing load and power consumption of the imaging control unit 72 and the drive unit 21, a process for selecting an area for driving and controlling the transfer area is performed every several frames instead of every frame. Also good.

  Thereafter, the imaging control unit 72 determines whether or not recording of a still image is started by confirming whether or not the release switch has been pressed by the user (step S26). If it is determined that recording of a still image has started, the imaging control unit 72 performs still image shooting by the imaging unit 20 and recording of the still image in the recording unit 60 (step S27). Note that the processing in steps S26 and S27 corresponds to the processing in steps S5 and S6 shown in FIG. Thereafter, the imaging control unit 72 determines whether or not an operation for turning off the power by the user has been performed (step S29). If the imaging control unit 72 determines that an operation to turn off the power is not performed, the imaging control unit 72 repeatedly executes the processes of steps S21 to S25. Accordingly, only the transfer area of the image sensor 100 is driven and controlled by the drive unit 21, and the image processing unit 30 performs image processing only on the image data of the transfer area 200 (see FIG. 10 in the first embodiment). . If the imaging control unit 72 determines that an operation to turn off the power by the user has been performed, the imaging control unit 72 ends the process.

  When it is determined that the recording of the still image has not started, the imaging control unit 72 determines whether the recording of the moving image has been started by confirming whether the moving image switch has been pressed (step S28). ). When it is determined that the recording of the moving image has started, the camera shake correction unit 73 acquires acceleration information from the acceleration sensor 65 (step S30). Further, the imaging control unit 72 starts capturing a moving image by the imaging unit 20 and recording a moving image in the recording unit 60 (step S31). Further, the display control unit 71 displays the moving image shot by the imaging unit 20 on the display screen of the display unit 50 (step S32). Note that the processes in steps S28, S31, and S32 correspond to the processes in steps S7, S11, and S12 shown in FIG.

  The camera shake correction unit 73 calculates a camera shake amount based on the acceleration information from the acceleration sensor 65 (step S33). Then, the imaging control unit 72 selects a transfer area for each frame according to the amount of camera shake calculated by the camera shake correction unit 73, and outputs an instruction signal designating the selected transfer area to the drive unit 21 (step S34). The drive unit 21 sets a transfer area in the image sensor 100 for each frame based on an instruction signal from the imaging control unit 72. In addition, the drive unit 21 controls the transfer area of the image sensor 100. Thus, the process of selecting the area for driving and controlling the transfer area is executed for each frame. However, in order to reduce the processing load and power consumption of the imaging control unit 72 and the drive unit 21, a process for selecting an area for driving and controlling the transfer area is performed every several frames instead of every frame. Also good.

  Thereafter, the imaging control unit 72 determines whether or not the recording of the moving image has ended (step S35). If the imaging control unit 72 determines that the recording of the moving image has not been completed, the imaging control unit 72 repeatedly executes the processes of steps S30 to S34. Accordingly, only the transfer area of the image sensor 100 is driven and controlled by the drive unit 21, and the image processing unit 30 performs image processing only on the image data in the transfer area 200 (see FIG. 10). Then, when it is determined that the recording of the moving image has been completed, the imaging control unit 72 determines that an operation for turning off the power by the user has been performed (step S29). When it is determined that the user has not performed an operation to turn off the power, the imaging control unit 72 returns to the process of step S21 and repeatedly executes the processes of steps S21 to S25. If the imaging control unit 72 determines that an operation to turn off the power by the user has been performed, the imaging control unit 72 ends the process.

  As described above, according to the second embodiment, similarly to the first embodiment described above, it is possible to reduce power consumption and processing load in the imaging apparatus 1. In the second embodiment described above, the acceleration sensor 65 that detects acceleration is provided, and the camera shake amount calculation unit 32 calculates the camera shake amount based on the acceleration from the acceleration sensor 65. Can be calculated.

  In the second embodiment described above, the system control unit 70 uses the camera shake amount calculated based on the acceleration information only for selecting the transfer area. However, the system control unit 70 may use the camera shake amount calculated based on the acceleration information for optical camera shake correction. For example, the system control unit 70 calculates the amount of camera shake based on the acceleration information from the acceleration sensor 65. Then, the system control unit 70 corrects image shift due to camera shake by moving some or all of the lenses of the lens unit 10 based on the amount of camera shake. Alternatively, the system control unit 70 corrects image shift due to camera shake by moving the image sensor 100 itself based on the amount of camera shake. Alternatively, the system control unit 70 corrects image shift due to camera shake by moving some or all of the lenses of the lens unit 10 and the image sensor 100 itself.

  When moving the lens of the lens unit 10, the system control unit 70 outputs a control signal corresponding to the amount of camera shake to the lens unit 10. The lens unit 10 drives a driving device (not shown) based on a control signal from the system control unit 70, thereby performing control to move a part or all of the lens by the direction and movement amount indicated by the control signal. Do. When moving the image sensor 100, the system control unit 70 outputs a control signal corresponding to the amount of camera shake to the image capturing unit 20. The imaging unit 20 controls the movement of the imaging element 100 by the direction and the movement amount indicated by the control signal by driving a driving device (not shown) based on the control signal from the system control unit 70.

  In such a configuration, the lens of the lens unit 10 and the image sensor 100 itself move in accordance with the amount of camera shake, so that image shift due to camera shake is corrected. However, there are cases in which image shift due to camera shake cannot be compensated only by optical camera shake correction. For example, the camera shake speed is fast and it is difficult to move the lens or the like by following the camera shake speed. In this case, the system control unit 70 selects a transfer area corresponding to an image shift that cannot be compensated by optical camera shake correction.

  Specifically, the camera shake correction unit 73 calculates the camera shake amount based on the acceleration information from the acceleration sensor 65. Then, the camera shake correction unit 73 determines whether or not the calculated camera shake amount is equal to or greater than a predetermined threshold value. When the camera shake correction unit 73 determines that the camera shake amount is equal to or greater than the predetermined threshold value, the camera shake correction unit 73 determines a value within the predetermined threshold value as the first camera shake amount. Then, the camera shake correction unit 73 outputs a control signal corresponding to the first camera shake amount to the lens unit 10 and the imaging unit 20. The lens unit 10 and the imaging unit 20 move the lens and the imaging device 100 itself based on a control signal from the camera shake correction unit 73.

  Further, the camera shake correction unit 73 determines a value obtained by subtracting the first camera shake amount from the camera shake amount calculated based on the acceleration information as the second camera shake amount. Then, the imaging control unit 72 selects a transfer area corresponding to the second camera shake amount. The imaging control unit 72 outputs an instruction signal designating the selected transfer area to the drive unit 21. The drive unit 21 performs drive control in the transfer area based on the instruction signal from the camera shake correction unit 73. Note that the transfer area selected by the imaging control unit 72 based on the second camera shake amount is preferably a larger area than the area of the recorded image size. This is because it is difficult for the imaging control unit 72 to predict the transfer area of the recorded image size in accordance with the image shift in consideration of the movement amount of the lens and the imaging element 100 by the optical camera shake correction. Even with such a configuration, it is possible to reduce power consumption and processing load in the imaging apparatus 1.

  Note that when the image pickup control unit 72 cannot compensate for the image shift due to the camera shake only by selecting the transfer area based on the camera shake amount, the image shift may be corrected by the optical camera shake correction.

<Third Embodiment>
In the first embodiment described above, the camera shake detection unit 32 is configured to calculate the camera shake amount based on a plurality of frames. In the second embodiment described above, the system control unit 70 calculates the amount of camera shake based on the acceleration information from the acceleration sensor. On the other hand, in the third embodiment, the arithmetic circuit 416 calculates a camera shake amount based on a plurality of frames, or the arithmetic circuit 416 calculates a camera shake amount based on acceleration information from the acceleration sensor 65. Then, the arithmetic circuit 416 selects a transfer area based on the calculated camera shake amount.

  FIG. 14 is a block diagram showing a specific configuration of the signal processing chip according to the third embodiment. In the signal processing chip 111 illustrated in FIG. 14, the arithmetic circuit 416 reads out image data of each pixel (RAW data before image processing by the image processing unit 30) from the pixel memory 415. Then, the arithmetic circuit 416 calculates a camera shake amount based on a plurality of image data (frames). Alternatively, the arithmetic circuit 416 is connected to the acceleration sensor 65 and acquires acceleration information from the acceleration sensor 65. Then, the arithmetic circuit 416 calculates the amount of camera shake based on the acceleration information from the acceleration sensor 65. In addition, the arithmetic circuit 416 selects a transfer area based on the calculated camera shake amount. Then, the arithmetic circuit 416 outputs a control signal designating the selected transfer area to the control unit 420. The configuration shown in FIG. 14 is the same as the configuration shown in FIG. 6 except that the acceleration sensor 65 is provided. Accordingly, the same components are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 15 is a flowchart illustrating an example of a method for controlling the imaging apparatus according to the third embodiment. FIG. 15M illustrates a case where the arithmetic circuit 416 calculates the camera shake amount based on the image data, and FIG. 15N illustrates a case where the arithmetic circuit 416 calculates the camera shake amount based on the acceleration information. Before the arithmetic circuit 416 executes the processes of FIGS. 15M and 15N, the imaging control unit 72 outputs an instruction signal to the control unit 420, whereby imaging by the imaging element 100 is started.

  The process shown in FIG. 15M will be described. As shown in FIG. 15M, the arithmetic circuit 416 reads out image data (RAW data) of each pixel stored in the pixel memory 415 for each frame (step S41M). Then, the arithmetic circuit 416 calculates a camera shake amount based on a plurality of frames read out from the pixel memory 415 in time series (step S42M). As the processing (camera shake amount calculation processing) of step S42M, the same processing as the camera shake amount calculation processing (camera shake amount calculation processing described in the first embodiment) by the camera shake detection unit 32 can be used. is there. That is, the arithmetic circuit 416 detects the motion vector based on the moving direction and moving amount of the subject (edge portion of the background subject) due to camera shake in a plurality of frames obtained in time series. Then, the arithmetic circuit 416 predicts the position of the subject in the next frame from the detected motion vector. The arithmetic circuit 416 calculates (detects) the shift amount of the subject between the current frame and the next frame as a camera shake amount. Then, the arithmetic circuit 416 outputs a control signal indicating the calculated camera shake amount to the control unit 420. Processing for calculating the amount of camera shake by the arithmetic circuit 416 is executed for each frame. However, in order to reduce the processing load and power consumption of the arithmetic circuit 416, the processing for calculating the camera shake amount may be executed every several frames instead of every frame.

  In the first embodiment described above, the camera shake detection unit 32 calculates a motion vector based on the image data image-processed by the image generation unit 31. On the other hand, in the third embodiment, the arithmetic circuit 416 calculates a motion vector based on image data (that is, RAW data) before image processing by the image generation unit 31. The RAW data is single-color image data in a state where image processing such as color signal processing and gamma correction is not performed. As described above, the arithmetic circuit 416 extracts only the edge portion of the background subject in the image data and determines how much the edge portion has moved, thereby calculating the motion vector of the subject. Therefore, the motion vector can be calculated regardless of the color component of the image data. For this reason, the arithmetic circuit 416 can calculate a motion vector based on image data (that is, RAW data) before image processing by the image generation unit 31.

  The arithmetic circuit 416 outputs a control signal designating the transfer area corresponding to the camera shake amount calculated in step S42M to the control unit 420 (step S43M). The control unit 420 converts the control signal from the arithmetic circuit 416 into a control signal that can be executed by the sensor control unit 441, the block control unit 442, and the synchronization control unit 443, and the sensor control unit 441, the block control unit 442, And output to the synchronization control unit 443.

  As described above, the sensor control unit 441 controls the start and end of charge accumulation by sending a reset pulse and a transfer pulse to each pixel in the transfer region based on a control signal from the control unit 420. . In addition, the sensor control unit 441 outputs a pixel signal to the output wiring 309 by sending a selection pulse to each pixel in the transfer area. Further, the block control unit 442 sends a specific pulse for specifying the unit group 131 of the transfer area to the imaging chip 113 based on the control signal from the control unit 420. Thereby, charge accumulation control is executed for each pixel in the transfer region in the pixel region 113A, and pixel signals are read from each pixel in the transfer region.

  Next, the process shown in FIG. As shown in FIG. 15N, the arithmetic circuit 416 acquires acceleration information from the acceleration sensor 65 (step S41N). Then, the arithmetic circuit 416 calculates a camera shake amount based on the acceleration information from the acceleration sensor 65 (step S42N). As the processing (camera shake amount calculation processing) of step S42N, the same processing as the camera shake amount calculation processing (camera shake amount calculation processing described in the second embodiment) by the camera shake correction unit 73 can be used. is there. That is, the arithmetic circuit 416 specifies the direction of camera shake based on the direction of acceleration indicated by the acceleration information. Further, the camera shake correction unit 73 predicts the position of the subject in the next frame based on the acceleration value indicated by the acceleration information. Then, the camera shake correction unit 73 calculates a camera shake amount corresponding to the predicted position of the subject. Processing for calculating the amount of camera shake by the arithmetic circuit 416 is executed for each frame. However, in order to reduce the processing load and power consumption of the arithmetic circuit 416, the processing for calculating the camera shake amount may be executed every several frames instead of every frame.

  The arithmetic circuit 416 outputs a control signal designating a transfer area corresponding to the camera shake amount calculated in step S42N to the control unit 420 (step S43N). The control unit 420 outputs a control signal to the sensor control unit 441 and the block control unit 442 based on the control signal from the arithmetic circuit 416. The sensor control unit 441 and the block control unit 442 perform charge accumulation control for each pixel in the transfer region in the pixel region 113A, and perform control so that pixel signals are read from each pixel in the transfer region.

  FIG. 16 is a diagram illustrating the size of image data in the imaging unit and the image processing unit according to the third embodiment. As shown in FIG. 16C1, the signal processing chip 111 (that is, the control unit 420, the sensor control unit 441, and the block control unit 442) responds to the camera shake amount calculated by the signal processing chip 111 (that is, the arithmetic circuit 416). The transfer area (recorded image size area in FIG. 16) 200 is set in the pixel area 113 </ b> A of the imaging chip 113. Then, the imaging chip 113 performs control so that the subject is imaged using only the transfer area 200. Further, when the signal processing chip 111 receives the image data of the transfer area 200 from the imaging chip 113, the signal processing chip 111 performs predetermined signal processing (amplification by the amplifier 412, CDS processing by the CDS 413a, A / D by the A / D 413b). Conversion process). Then, the signal processing chip 111 stores the image data in the pixel memory 415 of the memory chip 112.

  Thereafter, as shown in FIG. 16C 2, the image data (RAW data) of each pixel in the transfer area stored in the pixel memory 415 is transferred to the image processing unit 30 according to control by the control unit 420. In the image processing unit 30, the image generation unit 31 performs various types of image processing on the image data in the transfer area. The image processing unit 30 outputs the image processed image data to the recording unit 60. As a result, the image data in the transfer area (recorded image size area) is recorded in the recording unit 60. Further, the image processing unit 30 outputs the image processed image data to the display unit 50. Thereby, the image data of the transfer area is displayed on the display screen of the display unit 50.

  As described above, according to the third embodiment, similarly to the first embodiment and the second embodiment described above, it is possible to reduce power consumption and processing burden in the imaging apparatus 1. Further, since it is not necessary to calculate the amount of camera shake in the image processing unit 30 or the system control unit 70, the processing of the image processing unit 30 or the system control unit 70 can be reduced.

  In the third embodiment described above, the image sensor 100, the drive control unit (control unit 420, sensor control unit 441, block control unit 442, synchronization control unit 443), and camera shake amount calculation unit (arithmetic circuit 416) are integrated. It is configured. That is, the imaging chip 113, the signal processing chip 111, and the imaging chip 113 are stacked, and the amount of camera shake is calculated in the arithmetic circuit 416 of the signal processing chip 111. Therefore, a configuration in which a transfer area is selected according to the amount of camera shake can be realized simply by incorporating a chip in which the imaging element 100 and the like are integrated into the imaging apparatus 1.

  In the third embodiment described above, the signal processing chip 111 uses the camera shake amount calculated based on the acceleration information only for selecting the transfer area. However, the signal processing chip 111 may use the camera shake amount calculated based on the acceleration information for optical camera shake correction. For example, the control unit 420 of the signal processing chip 111 corrects image shift due to camera shake by moving some or all lenses of the lens unit 10 based on the camera shake amount calculated by the arithmetic circuit 416 based on the acceleration information. To do. Alternatively, the control unit 420 corrects image shift due to camera shake by moving the image sensor 100 itself based on the amount of camera shake. Alternatively, the control unit 420 corrects image shift due to camera shake by moving some or all of the lenses of the lens unit 10 and the image sensor 100 itself.

  Even in such a configuration, as described in the above-described second embodiment, there may be a case where image shift due to camera shake cannot be compensated only by optical camera shake correction. In this case, the control unit 420 selects a transfer area corresponding to an image shift that cannot be compensated by optical camera shake correction.

  Specifically, the control unit 420 determines whether or not the camera shake amount calculated by the arithmetic circuit 416 is greater than or equal to a predetermined threshold value. When determining that the amount of camera shake is equal to or greater than the predetermined threshold, the control unit 420 determines a value within the predetermined threshold as the first camera shake amount. Then, the control unit 420 outputs a control signal corresponding to the first camera shake amount to the lens unit 10 and the imaging unit 20. The lens unit 10 and the imaging unit 20 move the lens and the imaging device 100 itself based on a control signal from the camera shake correction unit 73.

  Further, the control unit 420 determines a value obtained by subtracting the first camera shake amount from the camera shake amount calculated based on the acceleration information as the second camera shake amount. Then, the control unit 420 selects a transfer area corresponding to the second camera shake amount. The control unit 420 outputs a control signal designating the selected transfer area to the sensor control unit 441 and the block control unit 442. The sensor control unit 441 and the block control unit 442 perform drive control in the transfer area based on a control signal from the control unit 420. Note that the transfer area selected by the control unit 420 based on the second camera shake amount is preferably an area larger than the area of the recorded image size. This is because it is difficult for the control unit 420 to predict the transfer area of the recorded image size in accordance with the image shift in consideration of the movement amount of the lens and the image sensor 100 due to the optical camera shake correction. Even with such a configuration, it is possible to reduce power consumption and processing load in the imaging apparatus 1.

  Note that when the control unit 420 cannot compensate for image shift due to camera shake only by selecting a transfer area based on the amount of camera shake, the image shift may be corrected by optical camera shake correction.

<Fourth embodiment>
In the first embodiment, the second embodiment, and the third embodiment described above, the transfer area for driving and controlling the image sensor 100 is changed according to the amount of camera shake. On the other hand, in the fourth embodiment, the drive unit 21 performs drive control on all the pixel regions 113A of the image sensor 100. In addition, the drive unit 21 outputs image data of an area corresponding to the amount of camera shake among the image data read from the image sensor 100 to the image processing unit 30.

  FIG. 17 is a diagram illustrating the size of image data in the imaging unit and the image processing unit according to the fourth embodiment. As shown in FIG. 17 (d1), the drive unit 21 (the control unit 420, the sensor control unit 441, the block control unit 442, and the synchronization control unit 443) does not divide the pixel region 113A of the image sensor 100, and Drive control is performed on the pixel region 113A. Then, the drive unit 21 reads a pixel signal from each pixel in the pixel region 113 </ b> A of the image sensor 100. On the other hand, as in the third embodiment described above, the arithmetic circuit 416 calculates a camera shake amount based on a plurality of frames. Alternatively, similarly to the third embodiment described above, the arithmetic circuit 416 calculates the amount of camera shake based on the acceleration information from the acceleration sensor 65. Then, the arithmetic circuit 416 outputs a control signal indicating the calculated camera shake amount to the control unit 420.

  As shown in FIG. 17 (d2), the control unit (signal output unit) 420 includes a transfer area (recorded image size area) corresponding to the amount of camera shake among the image data (RAW data) stored in the pixel memory 415. The image processing unit 30 outputs 200 pieces of image data for each frame. In the image processing unit 30, the image generation unit 31 performs various types of image processing on the image data in the transfer area. The image processing unit 30 outputs the image processed image data to the recording unit 60. As a result, the image data in the transfer area (recorded image size area) is recorded in the recording unit 60. Further, the image processing unit 30 outputs the image processed image data to the display unit 50. Thereby, the image data of the transfer area is displayed on the display screen of the display unit 50.

  As described above, according to the fourth embodiment, the imaging unit 21 including the imaging device 100 and the image processing unit 30 that performs image processing on the image signal output from the imaging unit 21 are provided. The unit 21 includes a camera shake amount calculation unit (arithmetic circuit 416) that calculates a camera shake amount, and a region 200 corresponding to the camera shake amount calculated by the camera shake amount calculation unit 416 among the image signals read from the image sensor 100. A signal output unit (control unit 420) that outputs an image signal to the image processing unit 30. According to such a configuration, since the image processing unit 30 only needs to perform image processing on the transfer area 200, power consumption and processing load on the image processing unit 30 can be reduced.

  In the fourth embodiment described above, the signal output unit 420 outputs the image signal of the region 200 corresponding to the camera shake amount for each frame to the image processing unit. Therefore, the signal output unit 420 accurately responds to the camera shake amount according to the movement of the subject. Area 200 can be selected. In the fourth embodiment described above, the acceleration sensor 65 that detects acceleration is provided, and the camera shake amount calculation unit 416 calculates the camera shake amount based on the acceleration from the acceleration sensor 65. According to such a configuration, an accurate camera shake amount can be calculated using the acceleration sensor 65. Further, according to the above-described fourth embodiment, the camera shake amount calculation unit 416 calculates the camera shake amount by the analysis processing of the image signal read from the image sensor 100. Therefore, the camera shake amount is calculated without using a special mechanism. Can be calculated. Therefore, the cost of the imaging device 1 does not increase.

<Fifth Embodiment>
In the fifth embodiment, the imaging device 1A in the above-described second embodiment is separated into an imaging device 2A and an electronic device 2B.

  FIG. 18 is a block diagram illustrating configurations of an imaging apparatus and an electronic apparatus according to the fifth embodiment. In the configuration shown in FIG. 18, the imaging device 2A is a device that images a subject. The imaging apparatus 2A includes a lens unit 10, an imaging unit 20, an image processing unit 30A, a work memory 40, an operation unit 55, a recording unit 60, an acceleration sensor 65, and a first system control unit 75. In the imaging apparatus 2A, the configurations of the lens unit 10, the imaging unit 20, the image processing unit 30A, the work memory 40, the operation unit 55, the recording unit 60, and the acceleration sensor 65 are the same as those illustrated in FIG. is there. Accordingly, the same components are denoted by the same reference numerals, and redundant description is omitted.

  The electronic device 2B is a device that displays images (still images, moving images, live view images). The electronic device 2B includes a display unit 50 and a second system control unit 75. The configuration of the display unit 50 in the electronic device 2B is the same as the configuration illustrated in FIG. Accordingly, the same components are denoted by the same reference numerals, and redundant description is omitted.

  The first system control unit 75 includes a first communication unit 75A. The second system control unit 76 has a second communication unit 76B. The first communication unit 75A and the second communication unit 76B transmit and receive signals to each other in a wired or wireless manner. The first system control unit 75 has a configuration corresponding to, for example, the imaging control unit 72 and the camera shake correction unit 73 in the configuration illustrated in FIG. 11. Moreover, the 2nd system control part 76 has only the structure corresponded to the display control part 71 among the structures shown in FIG.

  The configuration shown in FIG. 11 (display control unit 71, control unit 72, and selection unit 73) may be provided in either the first system control unit 75 or the second system control unit 76. 11 may be provided in the first system control unit 75 or the second system control unit 76, and a part of the configuration shown in FIG. 11 is provided in the first system control unit 75. A configuration other than a part of the configuration illustrated in FIG.

  Note that the imaging device 2A includes, for example, a digital camera, a smartphone, a mobile phone, and a personal computer that have an imaging function and a communication function, and the electronic device 2B includes, for example, a smartphone, a mobile phone that has a display function and a communication function, It is composed of a portable terminal such as a portable personal computer.

  The first system control unit 75 shown in FIG. 18 is realized by a CPU (not shown) executing processing based on a control program. Further, the second system control unit 76 shown in FIG. 18 is realized by a CPU (not shown) executing processing based on a control program.

  In the configuration shown in FIG. 18, the image processing unit 30 and the first system control unit 75 may be configured integrally. In this case, a system control unit having one or more CPUs performs the processing based on the control program, thereby taking on the function of the image processing unit 30 and the function of the first system control unit 75.

  Moreover, you may isolate | separate the imaging device 1 in above-described 1st Embodiment into an imaging device and an electronic device similarly to the separation method shown in FIG. 18, or substantially the same.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various modifications or improvements can be added to the above-described embodiment without departing from the spirit of the present invention. In addition, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements, and omitted forms are also included in the technical scope of the present invention. In addition, the configurations of the above-described embodiments and modifications can be applied in appropriate combinations.

  In the first embodiment described above (see FIG. 8), the camera shake detection unit 32 may perform the process of calculating the camera shake amount only when shooting a live view image or only when shooting a moving image. In addition, the imaging control unit 72 executes processing for acquiring camera shake amount information (steps S3 and S13) and processing for specifying a transfer area (steps S4 and S14) only when shooting a live view image or only when shooting a moving image. You may make it do. That is, the imaging control unit 72 may execute only the processes of steps S3 and S4, or may execute only the processes of steps S13 and S14.

  In the second embodiment (see FIG. 13), the camera shake correction unit 73 and the imaging control unit 72 calculate the camera shake amount (steps S24 and S33) and specify the transfer area (steps S25 and S34). ) May be executed only when shooting a live view image or only when shooting a moving image. That is, the imaging control unit 72 may execute only the processes of steps S24 and S25, or may execute only the processes of steps S33 and S34.

  In the first embodiment described above, the camera shake detection unit 32 executes a process for calculating the camera shake amount for each frame only during the shooting of the moving image, and calculates the camera shake amount during the shooting of the live view image. The process may be executed every several frames. In the second embodiment described above, the camera shake correction unit 73 performs a process for calculating the camera shake amount for each frame only during the shooting of the moving image, and calculates the camera shake amount during the shooting of the live view image. The process may be executed every several frames. According to such a configuration, it is possible to reduce the processing load and power consumption of the camera shake detection unit 32 and the camera shake correction unit 73 during shooting of the live view image.

  In the first and second embodiments described above, the imaging control unit 72 outputs an instruction signal for designating a transfer area to the drive unit 21 depending on the frame rate of the live view image or the frame rate of the moving image. In some cases, the image sensor 100 may read out signals for several frames before the drive unit 21 controls the transfer area. In such a case, the camera shake detection unit 32 and the camera shake correction unit 73 predict the position of the subject in a frame several frames after the current frame. The camera shake detection unit 32 and the camera shake correction unit 73 calculate (detect) a subject shift amount between the current frame and a frame several frames after the current frame as a camera shake amount. Then, the camera shake detection unit 32 and the camera shake correction unit 73 output camera shake amount information indicating the calculated camera shake amount to the system control unit 70.

  In the third embodiment described above, the arithmetic circuit 416 calculates a camera shake amount from a plurality of frames (see FIG. 15M) and calculates a camera shake amount from acceleration information (FIG. 15N). For example). In this case, the arithmetic circuit 416 can calculate the camera shake amount with higher accuracy by calculating the camera shake amount based on the two processes.

  In each of the embodiments described above, for example, the imaging control unit 72 or the like may specify the transfer area start position coordinates, the transfer area horizontal size, the transfer area vertical size, and the transfer area vertical size. Is specified. In the second embodiment, the third embodiment, and the fourth embodiment described above, the acceleration sensor 65 that detects acceleration (angular acceleration) is provided. However, a gyro that detects angular velocity may be provided. Also in this case, the camera shake correction unit 73 and the arithmetic circuit 416 can calculate the camera shake amount based on the angular velocity information indicating the angular velocity detected by the gyro. Further, although a 3-axis acceleration sensor is used as the acceleration sensor 65, a 5-axis acceleration sensor or the like may be used.

  In the first embodiment described above, the camera shake detection unit 32 calculates a motion vector based on data (such as luminance values) of R, G, and B components in a plurality of image data obtained in time series. Alternatively, the motion vector may be calculated based on only one or two of R, G, and B data.

  In each of the above-described embodiments, the transfer area is a recording image size area (that is, an angle of view for recording on the recording unit 60), but the transfer area may be an area larger than the recording image size. Good.

  Further, in each of the above-described embodiments, the arrangement of the color filters 102 is a Bayer arrangement, but an arrangement other than this arrangement may be used. Further, the number of pixels forming the unit group 131 only needs to include at least one pixel. In addition, the block only needs to include at least one pixel. Therefore, it is possible to execute imaging under different imaging conditions for each pixel.

  In each embodiment described above, part or all of the configuration of the drive unit 21 may be mounted on the imaging chip 113, or part or all of the configuration may be mounted on the signal processing chip 111. Further, a part of the configuration of the image processing unit 30 may be mounted on the imaging chip 113 or the signal processing chip 111. Further, a part of the configuration of the system control unit 70 may be mounted on the imaging chip 113 or the signal processing chip 111.

  In each of the above-described embodiments, the case where the size of the block area is set in advance has been described. However, the user may set the size of the block area.

  DESCRIPTION OF SYMBOLS 1,1A, 2A ... Imaging device, 2B ... Electronic device, 20 ... Imaging part, 30, 30A ... Image processing part, 31 ... Image generation part, 32 ... Camera shake detection part (hand-shake amount calculation part), 50 ... Display part, 65 ... Acceleration sensor, 70, 70A ... System controller, 75 ... First system controller, 76 ... Second system controller, 71 ... Display controller, 72 ... Imaging controller (drive controller), 73 ... Camera shake correction Unit (camera shake amount calculation unit), 100 ... imaging element, 200 ... transfer area, 415 ... pixel memory, 416 ... arithmetic circuit (hand shake amount calculation unit), 420 ... control unit (drive control unit), 441 ... sensor control unit, 442: Block control unit, 443: Synchronization control unit

Claims (10)

A plurality arranged in a second direction different from the first direction as the first direction, and a plurality of pixels including a photoelectric conversion unit that converts light into electric charge, the electric charge accumulated in the photoelectric conversion unit of the plurality of pixels first wiring and the second the first direction is the imaging area having wiring and a for outputting the signals generated by the charges converted by the photoelectric conversion unit of the plurality of pixels for sending a control signal for transferring And a plurality of image sensors arranged in the second direction,
A shake amount calculation unit for calculating a shake amount;
Based on the shake amount calculated by the shake amount calculation unit, a control unit that selects the imaging region to be driven among the imaging elements;
A first imaging condition is set in a first imaging area among the plurality of imaging areas selected by the control unit, and a second imaging condition different from the first imaging condition is set in a second imaging area among the plurality of imaging areas. A setting section for setting
Electronic equipment comprising.   The electronic device according to claim 1, wherein the control unit outputs an image signal of the selected imaging region.   The electronic device according to claim 1, wherein the control unit selects the imaging region to be driven for each frame. It has an acceleration sensor that detects acceleration,
The shake amount calculation unit, the electronic device according to any one of claims 1 to 3 for calculating the shake amount on the basis of the acceleration from the acceleration sensor. The shake amount calculation unit, the electronic device according to any one of claims 1 to 3 for calculating the deflection amount by the analysis processing of the image signal read from the imaging device. The electronic apparatus according to claim 5, wherein the shake amount calculation unit calculates the shake amount based on a moving direction and a moving amount of a subject in an image signal obtained in time series. The electronic apparatus according to claim 6, wherein the shake amount calculation unit calculates the shake amount based on a movement direction and a movement amount of a background subject. The electronic apparatus according the shake amount calculation unit, in any one of claims 1 to 7 provided in the image processing unit for performing image processing of the image signal. Calculating the amount of deflection,
Based on the calculated shake amount, a plurality of pixels arranged in a first direction and a second direction different from the first direction and including a photoelectric conversion unit that converts light into charges, and the plurality of pixels second wiring that outputs a first wiring for transmitting a control signal for transferring charge accumulated in the photoelectric conversion unit of the pixel, the signal generated by the conversion charges in the photoelectric conversion unit of the plurality of pixels And selecting the imaging region to be driven from among a plurality of imaging devices arranged in the first direction and the second direction, and a plurality of the imaging regions selected by the selection. A first imaging condition is set in the first imaging area, and a second imaging condition different from the first imaging condition is set in the second imaging area among the plurality of imaging areas . Calculating the amount of deflection,
Based on the calculated shake amount, a plurality of pixels arranged in a first direction and a second direction different from the first direction and including a photoelectric conversion unit that converts light into charges, and the plurality of pixels second wiring that outputs a first wiring for transmitting a control signal for transferring charge accumulated in the photoelectric conversion unit of the pixel, the signal generated by the conversion charges in the photoelectric conversion unit of the plurality of pixels And selecting the imaging region to be driven from among a plurality of imaging devices arranged in the first direction and the second direction, and a plurality of the imaging regions selected by the selection. A first imaging condition is set in the first imaging area, and a second imaging condition that is different from the first imaging condition is set in the second imaging area among the plurality of imaging areas . Control program.