JP7263028B2 - Imaging device and its control method - Google Patents

Description

The present invention relates to an imaging device and its control method.

2. Description of the Related Art There are imaging apparatuses that perform correction processing on image data read out from an imaging sensor using a correction circuit provided in the imaging sensor.

The imaging apparatus of Patent Document 1 executes processing on image data acquired by the imaging sensor in a received light data processing unit provided in the imaging sensor.

JP 2018-46324 A

In order to perform correction processing using different parameters according to the characteristics of the image sensor and the operation mode of the image sensor, mounting a large-capacity memory that stores multiple parameters on the image sensor chip would increase the chip size. I have concerns.

An object of the present invention is to enable the image sensor chip to perform correction processing according to the characteristics of the image sensor and the operation mode of the image sensor, and to suppress the expansion of the memory size provided in the image sensor chip. .

The present invention is an imaging device having a sensor chip having a pixel portion in which a plurality of unit pixels are arranged in a matrix and an image processing chip, wherein the sensor chip receives correction data from the image processing chip. and means for performing correction processing on image data read out from the pixel portion, wherein the image data in a unit corresponding to the readout processing unit from the pixel portion is subjected to the first correction means for executing the correction processing using the correction data received through the reception means of the above to obtain corrected image data; and first transmission means for transmitting the corrected image data to the image processing chip. The image processing chip includes second receiving means for receiving the corrected image data from the sensor chip, processing means for performing image processing on the corrected image data received by the second receiving means, and the correcting means. and second transmitting means for transmitting correction data used in the above to the sensor chip.

According to the imaging device of the present invention, the imaging sensor chip can perform correction processing according to the characteristics of the imaging sensor and the operation mode of the imaging device, and it is possible to suppress the expansion of the memory size provided in the imaging sensor chip. becomes.

It is a block diagram explaining a structure of an imaging device. 3 is a schematic diagram showing the arrangement of unit pixels in the pixel portion 101. FIG. It is a figure explaining packetization of image data. 4 is a flow chart showing the operation of the imaging device; 4 is a timing chart showing transmission timings of correction data;

<Example 1>
The first embodiment of the present invention will be described in detail below.

FIG. 1 is a block diagram illustrating the configuration of an imaging device according to the present invention. The imaging device 10 includes an imaging sensor 100 , a control circuit 110 , a ROM 120 , an operation section 130 , a power switch 140 and an imaging optical system 150 .

The imaging sensor 100 is a sensor chip that generates an image signal based on light received via an optical system. The image sensor 100 includes a pixel unit 101, a correction unit 102, a generation unit 103, a transmission unit 104, a reception unit 105, and a RAM .

The pixel unit 101 includes a plurality of unit pixels that acquire pixel data of each pixel based on the intensity of received light. In the pixel portion 101, a plurality of unit pixels are arranged in a matrix. A surface composed of a plurality of unit pixels of the pixel portion 101 is defined as "one screen", and image data composed of pixel data of unit pixels for one screen is defined as image data for one screen.

FIG. 2 is a schematic diagram showing the arrangement of unit pixels in the pixel portion 101. As shown in FIG. A unit of a plurality of unit pixels arranged in the vertical direction among the plurality of unit pixels of the pixel portion 101 is called a column. A unit of a plurality of unit pixels arranged in the horizontal direction among the plurality of unit pixels of the pixel portion 101 is called a row. The pixel unit 101 has a structure in which a plurality of rows (M rows) are simultaneously selected from among a plurality of unit pixels, and pixel data of the unit pixels included in the plurality of selected rows are read out via vertical output lines. . Here, pixel data of unit pixels included in a plurality of rows output from the pixel unit 101 is called block image data.

The correction unit 102 performs correction processing on the block image data obtained from the pixel unit 101 using correction data obtained from the RAM 106 .

For example, it is assumed that the correction unit 102 performs column offset noise correction. The offset superimposed on the pixel data of the block image data has a different value for each column to which the unit pixel belongs. Striped noise may occur in the image data due to the superimposition of different offsets for each column. Column offset noise correction is processing for correcting the difference in offset amount between columns and suppressing the occurrence of striped noise. Specifically, the correction unit 102 subtracts the offset correction value from the pixel data of the unit pixels of the plurality of rows obtained from the pixel unit 101 using the correction data indicating the column offset correction value for each column obtained from the RAM 106. Execute the processing to be performed.

For the image signal corrected here, the image processing unit 111 of the control circuit 110 subsequently performs processing other than the column offset noise correction. Therefore, the word length of the image signal per unit pixel is not rounded, and is transmitted as it is longer than the word length of the image signal per unit pixel read by the pixel unit 101 .

The generating unit 103 adds additional information such as a header and a footer to the block image data (corrected image data) corrected by the correcting unit 102, and packetizes the image signal.

FIG. 3 is a schematic diagram showing packetization of block image data executed by the generation unit 103. As shown in FIG. In FIG. 3, both M and N are integers, M is greater than N, and M is an integer multiple of N. In FIG. As described above, the pixel unit 101 reads pixel data for every M rows and outputs it as block image data. Further, the correction unit 102 executes correction processing (column offset noise correction processing) on the block image data. FIG. 3A is a schematic diagram showing block image data. The block image data is data in which pixel data of unit pixels for M rows are put together.

The generation unit 103 divides the corrected block image data into N rows, and adds a header (H) and a footer (F) to each divided image data (transmission image data). The header and footer are data added for purposes such as marking symbol alignment of image signals. Note that N rows are the processing units of the processing unit 113 of the image processing unit 111 . FIG. 3B is a schematic diagram showing transmission image data obtained by dividing block image data.

By matching the size of the image data to be transmitted from the imaging sensor 100 to the image processing unit 111 in advance to the processing unit of the processing unit 113 of the image processing unit 111, the processing unit 113 can process the received image data as it is. can. That is, even if the readout unit of the image sensor 100 exceeds the processing unit of the image processing unit, the image processing unit can process the image signal without being aware of it.

Note that N is preferably an integer greater than 1. Rather than packetizing each line and adding a header or footer to each line, division into every N lines can reduce the overhead for the header and footer.

The transmission unit 104 outputs the packetized transmission image data to the image processing unit 111 .

The receiving unit 105 receives correction data from the image processing unit 111 and outputs it to the RAM 106 .

The RAM 106 stores the correction data from the reception unit 105 and outputs the correction data to the correction unit 102 in synchronization with readout of image signals from the pixel unit 101 according to instructions from a timing control circuit (not shown) or the like.

The control circuit 110 is a control circuit chip that controls the operation of the imaging device 10 as a whole. The control circuit 110 is also an image processing chip that processes image data acquired from the imaging sensor 100 . The control circuit 110 includes an image processing unit 111 that processes image data acquired from the imaging sensor 100 , a system control unit 117 , an operation reception unit 118 , and a power control unit 119 . The control circuit 110 is a chip different from the imaging sensor 100 . Note that each functional block of the control circuit 110 and the electronic circuit may be provided over a plurality of chips.

The image processing unit 111 is a processing circuit that processes image data acquired from the imaging sensor 100 . Image processing unit 111 includes receiving unit 112 , processing unit 113 , determining unit 114 , RAM 115 , and transmitting unit 116 .

The receiving unit 112 receives packetized transmission image data from the imaging sensor 100 and outputs the data to the processing unit 113 .

The processing unit 113 analyzes additional information such as headers and footers of packetized transmission image data, extracts pixel data of each unit pixel of the pixel unit 101, and performs predetermined data processing. The predetermined data processing is processing performed on pixel data for N rows. Moreover, the predetermined data processing includes processing different from the correction processing executed by the correction unit 102 . For example, predetermined data processing includes enhancement processing, color correction processing, and noise reduction processing. In addition, the processing unit 113 generates image data for one screen using a plurality of pieces of processed transmission image data.

The determination unit 114 determines correction data to be used by the correction unit 102 . The determination unit 114 determines correction data by analyzing the image after the processing of the processing unit 113 has been performed in advance.

The determination unit 114 determines correction data using sample image data obtained in the adjustment process of the imaging device 10 . The sample image data is, for example, image data obtained by imaging a subject, such as a white board, from which uniform pixel data are expected to be obtained. At this time, the image data is acquired from the imaging sensor 100 without performing correction processing by the correction unit 102 . The determination unit 114 analyzes the noise component of the obtained sample image data, and calculates the offset value superimposed for each column of unit pixels. The determination unit 114 calculates column offset correction values from the obtained offset values. The determination unit 114 stores the determined offset correction value in the ROM 120 .

The RAM 115 is a volatile storage medium that stores correction data for the transmission unit 116 expanded from the ROM 120 to output to the imaging sensor 100 .

The transmission unit 116 transmits the correction data read from the RAM 115 to the imaging sensor 100 when the imaging apparatus 10 is activated and when the operation mode is changed.

A system control unit 117 is a processor for controlling the operation of the imaging device 10 . A system control unit 117 controls each functional block of the control circuit 110 and the operation of the imaging sensor 100 . The system control unit 117 controls correction data determination processing, correction data transmission processing, imaging execution instruction, operation mode setting processing, and the like.

The operation reception unit 118 receives input of an operation signal input from the operation unit 130, which will be described later. Operation accepting portion 118 outputs information corresponding to the input operation signal to system control portion 117 . The system control unit 117 executes imaging instructions, operation mode change processing, and the like based on the input information.

The power control unit 119 receives input of a release signal from the operation unit 130, which will be described later. Operation accepting portion 118 outputs to system control portion 117 that the release signal has been input.

The ROM 120 is a nonvolatile storage medium that stores correction data used by the correction unit 102 of the image sensor 100 .

The operation unit 130 is an operation member used by the user to control the operation of the imaging device 10 . The operation unit 130 includes a release button that can be operated by the user for instructing imaging, a menu button for setting imaging parameters, a selection button, and the like. For example, a release signal is output to the operation reception unit 118 in response to the user pressing the release button.

The power switch 140 is a changeover switch that can be operated by the user so that the user can switch the imaging device 10 on and off. When the user switches the power switch 140 to the ON state, a power ON signal is output to the power control unit 119 . When the user switches the power switch 140 to the off state, a power off signal is output to the power control unit 119 . When the power SW 140 is on, the output of the signal line connected to the power control unit 119 is High, and when the power SW 140 is off, the output of the signal line connected to the power control unit 119 is Low. may be

FIG. 4 is a flowchart showing an operation in which correction data is transmitted from the image processing unit 111 to the imaging sensor 100 and correction processing is performed.

In response to the power switch 140 being turned on, the power control unit 119 supplies power to the system control unit 117, and processing is started.

In S<b>401 , the system control unit 117 expands correction data from the ROM 120 to the RAM 115 .

In S402, the system control unit 117 sets the operation mode. Immediately after the power is turned on, the system control unit 117 sets the operation mode that was set immediately before the power was turned off last time. Note that when an operation mode change instruction is input via the operation unit 130, the operation mode is set according to the input instruction.

In S<b>403 , the system control unit 117 reads correction data corresponding to the set operation mode among the correction data developed in the RAM 115 and transmits the read correction data to the imaging sensor 100 . Correction data used in the live view (LV) imaging mode is assumed to be correction data Cd1. Correction data Cd2 is correction data used in the still image capturing mode. A change in imaging mode includes a change in parameters such as ISO sensitivity, exposure, and aperture. The corresponding correction data Cd may be changed according to the change in ISO sensitivity.

In S404, the RAM 106 of the image sensor 100 stores the received correction data.

In S405, the system control unit 117 determines whether or not the operation mode of the imaging device 10 has been changed. Execution of the change of the operation mode is instructed based on a user instruction input via the operation unit 130 . In the case of the LV imaging mode, the system control unit 117 can also change to an operation mode with different imaging parameters such as ISO sensitivity, aperture, and exposure by analyzing the acquired image.

Further, it is assumed that the release button of the operation unit 130 is a button that can be operated in two stages, and that it can output a first release signal for instructing imaging preparation and a second release signal for instructing execution of imaging. In this case, it is determined that an instruction to change the operation mode from the LV imaging mode to the still image imaging mode has been issued in response to the input of the first release signal.

If there is an operation mode change, the process returns to S402 to execute the corresponding operation mode setting process. If there is no change in operation mode, the process proceeds to S406.

In S406, the system control unit 117 determines whether or not an imaging instruction has been input. When the operation reception unit 118 receives the release signal from the operation unit 130, the system control unit 117 determines that an imaging instruction has been input. If it is determined that an imaging instruction has been input, the process proceeds to S407. Otherwise, the process proceeds to S411.

In S407, the system control unit 117 causes the imaging sensor 100 and the image processing unit 111 to perform imaging processing.

In step S<b>408 , the correcting unit 102 performs correction processing using the correction data on each block image data composed of pixel data of unit pixels for M rows read from the pixel unit 101 . The process proceeds to S409.

In step S<b>409 , the generation unit 103 acquires transmission image data obtained by dividing the block image data processed by the correction unit 102 into every N rows, and outputs the transmission image data to the image processing unit 111 of the control circuit 110 via the transmission unit 104 .

In S410, the processing unit 113 of the image processing unit 111 executes image processing on the received transmission image data. Further, the processing unit 113 performs image processing on a plurality of pieces of transmission image data, generates image data for one screen, and outputs the image data to an external storage medium (not shown). Assume that the external storage medium is a storage medium such as an SD card inserted in the imaging device 10 .

In S411, it is determined whether or not there is an instruction to turn off the imaging device 10 (power off). If there is no drop instruction, the process returns to S405. When there is an instruction to drop, the operation is terminated.

FIG. 5 is a schematic diagram showing the progress of correction data stored in the RAM 106 according to the operation mode. For example, assume that the imaging apparatus 10 is activated at time t0 and starts operating in the LV operation mode. At this time, the correction data Cd1 corresponding to the LV operation mode is transmitted from the control circuit 110 and stored in the RAM 106 at time t1. The correction data Cd1 is correction data on the premise of LV imaging.

Assume that a first release signal is input from the release button of the operation unit 130 at time t2. At this time, it is determined in S405 that a change from the LV imaging mode to the still image imaging mode has been instructed, and setting to the still image imaging mode is executed in S402. At time t3, correction data Cd2 corresponding to the still image operation mode is transmitted from the control circuit 110 and stored in the RAM 106. FIG.

At time t4, the processing of S407 to S410 is executed in response to input of the second release signal from the release button of the operation unit .

It is also assumed that when the user stops operating the release button of the operation unit 130, the operation mode is changed from the still image imaging mode to the LV imaging mode. At time t5 when the still image capturing process is completed, the operation mode is set to the LV operation mode, and the correction data Cd1 is stored in the RAM 106 at time t6.

According to the imaging apparatus described above, the correction process can be performed in the imaging sensor according to the readout processing unit of the imaging sensor. Even if the image processing unit does not support correction processing according to the read processing unit of the imaging sensor, correction processing can be performed without replacing or recreating the image processing unit.

Note that in the case where the pixel unit 101 of the image sensor 100 has a plurality of readout modes and is in a mode in which data is read out in units of N rows instead of in units of M rows, the correction unit 102 of the image sensor 100 does not perform correction processing and performs N rows as they are. Send packets line by line. Then, the processing unit 113 of the image processing unit 111 may perform processing corresponding to the correction processing in the correction unit 102 of the imaging sensor 100 .

As described above, according to the present embodiment, the imaging sensor chip can perform correction processing according to the characteristics of the imaging sensor and the operation mode of the imaging device, and the expansion of the memory size provided in the imaging sensor chip can be suppressed. becomes. In addition, correction processing can be performed within the image sensor according to the readout processing unit of the image sensor. Even if the image processing unit does not support correction processing according to the read processing unit of the imaging sensor, correction processing can be performed without replacing or recreating the image processing unit.

REFERENCE SIGNS LIST 10 imaging device 100 imaging sensor 101 pixel unit 102 correction unit 103 generation unit 104 transmission unit 105 reception unit 106 RAM
110 control circuit 111 image processing unit 112 reception unit 113 processing unit 114 determination unit 115 RAM
116 transmission unit 117 system control unit 120 ROM

Claims (10)

An imaging device having a sensor chip having a pixel portion in which a plurality of unit pixels are arranged in a matrix and an image processing chip ,
The sensor chip is
a first receiving means for receiving correction data from the image processing chip;
means for performing correction processing on the image data read out from the pixel portion, receiving the image data in a unit corresponding to the readout processing unit from the pixel portion through the first receiving means; a correction means for executing the correction process using the corrected data to obtain corrected image data ;
a first transmitting means for transmitting the corrected image data to the image processing chip;
The image processing chip is
a second receiving means for receiving the corrected image data from the sensor chip ;
a processing means for performing image processing on the corrected image data received by the second receiving means ;
and second transmission means for transmitting correction data used by the correction means to the sensor chip. The sensor chip comprises first storage means for storing the correction data received via the first reception means,
2. The imaging apparatus according to claim 1, wherein said image processing chip comprises second storage means for storing said correction data to be transmitted via said second transmission means. further comprising third storage means, which is non-volatile storage means connected to the image processing chip;
3. The image pickup apparatus according to claim 2, wherein the correction data stored in the third storage means is expanded from the third storage means to the second storage means when the image pickup apparatus is started. The image processing chip further comprises determining means for determining the correction data,
The determination means determines the correction data based on image data obtained without correction by the correction means, and stores the determined correction data in the third storage means. The imaging device according to claim 3. setting means for setting an operation mode of the imaging device to one of a plurality of operation modes;
The second transmission means transmits, to the sensor chip, correction data corresponding to the operation mode set by the setting means, among the correction data corresponding to the plurality of operation modes, to the sensor chip. is changed, correction data corresponding to the changed operation mode is transmitted to the sensor chip. A control method for an imaging device having a sensor chip having a pixel portion in which a plurality of unit pixels are arranged in a matrix and an image processing chip,
a first receiving step in which the sensor chip receives correction data from the image processing chip;
a step of performing a correction process on the image data read out from the pixel portion by the sensor chip, wherein the first receiving step is performed on the image data in a unit corresponding to a readout processing unit from the pixel portion; a correction step of performing the correction process using the correction data received in to obtain corrected image data ;
a first transmission step in which the sensor chip transmits the corrected image data to the image processing chip;
a second receiving step in which the image processing chip receives the corrected image data from the sensor chip;
a processing step of performing image processing on the corrected image data received by the image processing chip in the second receiving step ;
and a second transmission step of transmitting correction data used in the correction step from the image processing chip to the sensor chip. a first storage step in which the sensor chip stores the correction data received in the first reception step in a first storage means;
7. The method of controlling an image pickup apparatus according to claim 6, further comprising: a second storage step of storing the correction data transmitted by the second transmission step, in a second storage unit, in the image processing chip. . wherein the image processing chip develops the correction data from a non-volatile third storage means connected to the image processing chip to the second storage means when the imaging device is activated. 8. The method for controlling an imaging device according to claim 7. further comprising a determining step in which the image processing chip determines the correction data;
The determination step determines the correction data based on image data obtained without performing correction in the correction step, and stores the determined correction data in the third storage means. The control method of the imaging device according to claim 8 . A setting step of setting the operation mode of the imaging device to one of a plurality of operation modes,
In the second transmission step, among the correction data corresponding to the plurality of operation modes, the correction data corresponding to the operation mode set by the setting step is transmitted to the sensor chip, and the operation mode of the imaging device is selected. 10. The method of controlling an image pickup apparatus according to claim 6, further comprising transmitting correction data corresponding to the changed operation mode to the sensor chip when the operation mode is changed.

Images