JP7714640B2 - Imaging device and fixed imaging element - Google Patents

Description

The present invention relates to an imaging device and a solid-state imaging element equipped with a light-receiving element array.

Conventionally, the solid-state imaging element described in Patent Document 1 (JP Patent Publication No. 2022-164387) has been known as a solid-state imaging element such as a CMOS used in imaging devices such as digital cameras.

As shown in Figure 11, this solid-state imaging device 500 has a photodetector array 501 in which photodetectors 502, each including a photoelectric conversion element, are arranged two-dimensionally in an array and connected to pixel drive wiring 503 and vertical signal lines 504. It also has peripheral circuits such as a column-parallel signal processing circuit 505, an output circuit 506, a timing control circuit 507, a horizontal scanning circuit 508, and a vertical scanning circuit 509.

In this solid-state imaging element 500, when the light-receiving element array 501 is exposed to light during imaging using an imaging device (digital camera, etc.) equipped with the solid-state imaging element 500, signals output from each of the multiple light-receiving elements 502 that make up the light-receiving element array 501 are output to the outside via vertical signal lines 504, column-parallel signal processing circuits 505, output circuits 506, etc.

In recent years, there has been a demand for power savings in solid-state imaging elements 500 and imaging devices equipped with solid-state imaging elements 500. However, with this solid-state imaging element 500 and imaging devices, all signals output from the multiple light-receiving elements 502 included in the light-receiving element array 501 are output to the outside (a so-called full-amount readout configuration). As a result, the amount of signal output within and to the outside of the solid-state imaging element 500 is large, and the solid-state imaging element 500 requires power consumption according to this signal amount. As a result, power savings have not been sufficient in the above-mentioned solid-state imaging element 500 or imaging devices equipped with this solid-state imaging element 500.

Japanese Patent Application Publication No. 2022-164387

Therefore, an objective of the present invention is to provide an imaging device and solid-state imaging element that can reduce power consumption.

The imaging device according to the present invention comprises:
a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements;
The image forming apparatus further includes a compression unit that generates a compressed signal for each light receiving element group by compressing signals output from the plurality of light receiving elements of the light receiving element group.

Moreover, the imaging device
The image sensor may include a solid-state image sensor having the light receiving element array and the compression section.

Moreover, in the imaging device,
In the light receiving element group, when the light receiving element array is exposed, the amount of exposure of at least one light receiving element of the light receiving element group may be different from the amount of exposure of the other light receiving elements of the light receiving element group.

In addition, in the imaging device,
In the light receiving element group, the difference in the amount of exposure may be caused by a difference in exposure time of each light receiving element.

In addition, in the imaging device,
The compressed signal may include a weighted average or an average value of signal values included in the signals output from the light receiving elements of the light receiving element group.

Moreover, the imaging device
The image processing device may further include a reconstruction processing unit that reconstructs the compressed signal generated by the compression unit into pixel signals of pixels in an output image that correspond to the compressed signal.

In addition, in the imaging device,
The reconstruction processing unit may reconstruct the compressed signal into the pixel signal based on pre-compression information included in the compressed signal.

In addition, in the imaging device,
Each light receiving element of the light receiving element group has a photoelectric conversion element,
The reconstruction processing unit may reconstruct the compressed signal into the pixel signal by calculating an expected value of the signal value that can be obtained if the photoelectric conversion element of each light receiving element has a capacity that will not overflow due to the exposure when there are light receiving elements in the light receiving element group whose photoelectric conversion elements overflow and light receiving elements whose photoelectric conversion elements do not overflow when the light receiving element array is exposed to light.

In addition, in the imaging device,
the plurality of light receiving elements included in the light receiving element group include a reference light receiving element that is a light receiving element serving as a reference, and non-reference light receiving elements other than the reference light receiving element,
The compression unit may generate the compressed signal based on the difference between a reference signal value, which is a signal value included in the signal output from the reference light receiving element, and a non-reference signal value, which is a signal value included in the signal output from the non-reference light receiving element.

In addition, in the imaging device,
The compression unit is configured to:
When there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold, the compressed signal is generated including a weighted average or an average of the signal values output from each light receiving element of the light receiving element group;
When there is a first signal value which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value which is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal may be generated by deleting the signal output from each of the light-receiving elements that corresponds to the second signal value, and adding position information of the non-reference light-receiving element that output the first signal value or the second signal value relative to the reference light-receiving element to the signal output from the reference light-receiving element.

In addition, in the imaging device,
The compression unit is configured to:
When there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold, the compressed signal is generated including a weighted average or an average of the signal values output from each light receiving element of the light receiving element group;
When there is a first signal value which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value which is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal may be generated including a weighted average or average value of the reference signal value and the first signal value, and position information of the non-reference light receiving element that output the first signal value or the second signal value relative to the reference light receiving element.

Further, the solid-state imaging device according to the present invention comprises:
a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements;
The image forming apparatus further includes a compression unit that generates a compressed signal for each light receiving element group by compressing signals output from the plurality of light receiving elements of the light receiving element group.

FIG. 1 is a functional block diagram of an imaging device according to the first embodiment. FIG. 2 is a diagram showing the configuration of a solid-state image sensor included in the imaging device. FIG. 3 is a diagram showing the configuration of the light receiving element groups included in the solid-state imaging device. FIG. 4 is a timing chart of the light receiving element group. FIG. 5 is a diagram for explaining the processing in the compression unit and the processing in the reconstruction processing unit when there is no light receiving element in a light receiving element group with an overflowing photoelectric conversion element. FIG. 6 is a diagram for explaining the processing in the compression unit and the processing in the reconstruction processing unit when there is a light receiving element in a light receiving element group where the photoelectric conversion element has overflowed. FIG. 7 is a diagram showing the configuration of a solid-state imaging device according to the second embodiment. FIG. 8 is a diagram for explaining the processing in the compression unit and the processing in the reconstruction processing unit when there are only non-reference light receiving elements in a light receiving element group whose pixel values differ from the pixel value of the reference light receiving element by less than a predetermined threshold. FIG. 9 is a diagram for explaining the processing in the compression unit and the processing in the reconstruction processing unit when there is a non-reference light receiving element in a light receiving element group whose pixel value differs from the pixel value of the reference light receiving element by a predetermined threshold or more. FIG. 10 is a diagram showing the processing in the compression unit and the processing in the reconstruction processing unit when an image is formed on a light receiving element array having a plurality of light receiving element groups. FIG. 11 is a diagram showing the configuration of a conventional solid-state imaging device.

The imaging device according to this embodiment includes:
a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements;
The image forming apparatus further includes a compression unit that generates a compressed signal for each light receiving element group by compressing signals output from the plurality of light receiving elements of the light receiving element group.

In this way, the compression unit generates a compressed signal for each light-receiving element group and outputs each of these compressed signals (i.e., signals with reduced signal volume), thereby reducing power consumption in the configuration downstream of the compression unit. Note that signal compression in the compression unit according to the present invention is a process that results in a signal with a smaller signal volume than a signal generated by simple addition (pixel addition, etc.) of the signals output from each light-receiving element.

The imaging device is
The image sensor may include a solid-state image sensor having the light receiving element array and the compression section.

In addition, in the imaging device,
In the light receiving element group, when the light receiving element array is exposed, the amount of exposure of at least one light receiving element of the light receiving element group may be different from the amount of exposure of the other light receiving elements of the light receiving element group.

This configuration makes it possible to reduce the signal volume while still including information from each signal output from light-receiving elements with different exposure levels.

in this case,
In the light receiving element group, the difference in the amount of exposure may be caused by a difference in exposure time of each light receiving element.

By adjusting the exposure time of the light receiving elements in this way to create differences in the amount of exposure of the light receiving elements within a light receiving element group, it is possible to change the amount of exposure of the light receiving elements through control.

In addition, in the imaging device,
The compressed signal may include a weighted average or an average value of signal values included in the signals output from the light receiving elements of the light receiving element group.

Moreover, the imaging device
The image processing device may further include a reconstruction processing unit that reconstructs the compressed signal generated by the compression unit into pixel signals of pixels in an output image that correspond to the compressed signal.

With this configuration, pixel signals are reconstructed downstream of the compression unit and output, improving the image quality of the resulting image.

In addition, in the imaging device,
The reconstruction processing unit may reconstruct the compressed signal into the pixel signal based on pre-compression information included in the compressed signal.

In addition, in the imaging device,
Each light receiving element of the light receiving element group has a photoelectric conversion element,
The reconstruction processing unit may reconstruct the compressed signal into the pixel signal by calculating an expected value of the signal value that can be obtained if the photoelectric conversion element of each light receiving element has a capacity that will not overflow due to the exposure when there are light receiving elements in the light receiving element group whose photoelectric conversion elements overflow and light receiving elements whose photoelectric conversion elements do not overflow when the light receiving element array is exposed to light.

This configuration makes it possible to obtain images with a dynamic range larger than the dynamic range defined by each light-receiving element, i.e., high-dynamic-range imaging is possible.

In addition, in the imaging device,
the plurality of light receiving elements included in the light receiving element group include a reference light receiving element that is a light receiving element serving as a reference, and non-reference light receiving elements other than the reference light receiving element,
The compression unit may generate the compressed signal based on the difference between a reference signal value, which is a signal value included in the signal output from the reference light receiving element, and a non-reference signal value, which is a signal value included in the signal output from the non-reference light receiving element.

In this way, by compressing the signal based on the difference between the reference signal value and non-reference signal value for each position of the light receiving element, the difference in signal amount for each position can be reconstructed in the reconstruction processing unit, thereby ensuring resolution in the output image.

in particular,
The compression unit is configured to:
When there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold, the compressed signal is generated including a weighted average or an average of the signal values output from each light receiving element of the light receiving element group;
When there is a first signal value which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value which is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal may be generated by deleting the signal output from each of the light-receiving elements that corresponds to the second signal value, and adding position information of the non-reference light-receiving element that output the first signal value or the second signal value relative to the reference light-receiving element to the signal output from the reference light-receiving element.

In this way, by constructing a compressed signal using the signal output from the reference light-receiving element and the positional information added to it, the signal volume can be effectively reduced compared to when all signals output from each light-receiving element in a light-receiving element group are output to a downstream configuration.

Further, the compression unit may be configured to:
When there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold, the compressed signal is generated including a weighted average or an average of the signal values output from each light receiving element of the light receiving element group;
When there is a first signal value which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value which is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal may be generated including a weighted average or average value of the reference signal value and the first signal value, and position information of the non-reference light receiving element that output the first signal value or the second signal value relative to the reference light receiving element.

Moreover, the solid-state imaging device according to this embodiment has
a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements;
The image forming apparatus further includes a compression unit that generates a compressed signal for each light receiving element group by compressing signals output from the plurality of light receiving elements of the light receiving element group.

In this way, the compression unit generates a compressed signal for each light-receiving element group and outputs each of these compressed signals (i.e., signals with reduced signal volume), thereby reducing power consumption in the configuration downstream of the compression unit.

The first embodiment of the present invention will now be described with reference to the accompanying drawings.

As shown in FIGS. 1 and 2 , the imaging device 100 according to the first embodiment includes a light receiving element array 2 having a plurality of light receiving element groups 20, each having a plurality of light receiving elements 21, and at least one compression unit 10 that generates a compressed signal S2 for each light receiving element group 20 by compressing the signal S1 output from the plurality of light receiving elements 21 in the light receiving element group 20. The imaging device 100 of this embodiment includes a number of compression units 10 corresponding to the number of light receiving element groups 20, and each compression unit 10 generates a compressed signal S2 for each light receiving element group 20 by compressing the signal S1 output from the plurality of light receiving elements 21 in the corresponding light receiving element group 20. This imaging device 100 is capable of capturing images of objects such as digital cameras, smartphones, and tablet devices, and the imaging device 100 of this embodiment is, for example, a smartphone.

Specifically, as shown in FIG. 1, the imaging device 100 includes an imaging unit 101 and a control unit 102. The imaging device 100 of this embodiment also includes a non-volatile memory 103, a working memory 104, an operation unit 105, a display unit 106, a recording medium 107, a connection unit 108, a short-range wireless communication unit 109, a public network connection unit 110, a microphone 111, and a speaker 112.

Under the control of the control unit 102, the imaging unit 101 converts the image of the object formed by the optical system of the imaging unit 101 into an electrical signal, then performs noise reduction processing and outputs the digital data as image data for the output image. Specifically, the imaging unit 101 has an optical system composed of at least one optical element such as a lens, and a solid-state imaging element 1 that converts the image of the object (image object) formed through the optical system into an electrical signal (i.e., captures the image).

The solid-state imaging device 1 is incorporated into, for example, a smartphone, digital camera, etc., and is used to capture an image of an object. In this embodiment, the solid-state imaging device 1 is, for example, a CMOS image sensor.

Specifically, as shown in Figure 2, the solid-state imaging device 1 has a photodetector array 2, a first circuit section 3 and a second circuit section 4 connected to the photodetector array 2, and a signal processing section 5 to which the second circuit section 4 is connected. The photodetector array 2, first circuit section 3, second circuit section 4, and signal processing section 5 are arranged on the same semiconductor substrate or on multiple electrically connected semiconductor substrates.

The light receiving element array 2 has a plurality of light receiving element groups 20 arranged two-dimensionally in a matrix. The light receiving element array 2 also has a plurality of row signal lines 25 arranged in rows and each extending in the row direction (left-right direction in FIG. 2) for the plurality of light receiving element groups 20 arranged in a matrix, and a plurality of column signal lines 26 arranged in columns and each extending in the column direction (up-down direction in FIG. 2). Each of these plurality of row signal lines 25 is connected to the first circuit unit 3, and each of the plurality of column signal lines 26 is connected to the second circuit unit 4.

As shown in FIG. 3, each of the multiple light receiving element groups 20 has multiple light receiving elements (pixels) 21. In the light receiving element group 20 of this embodiment, the multiple light receiving elements 21 are adjacent to one another. Specifically, the multiple light receiving elements 21 included in the light receiving element group 20 are arranged adjacent to each other in the row and column directions. In this embodiment, each light receiving element group 20 has four light receiving elements 21 (first light receiving element 21a, second light receiving element 21b, third light receiving element 21c, and fourth light receiving element 21d) arranged in two rows and two columns (i.e., a 2x2 array). Each light receiving element group 20 also has an amplifier 22 to which each of the light receiving elements 21a, 21b, 21c, and 21d is connected.

In Figure 2, to make it easier to understand the configuration of the photodetector array 2, the multiple photodetector groups 20 are shown with a gap between adjacent photodetector groups 20 in the row direction and between adjacent photodetector groups 20 in the column direction. However, in reality, the photodetector array 2 has multiple photodetector groups 20 arranged in a matrix with adjacent photodetector groups 20 in the row direction adjacent to each other and adjacent photodetector groups 20 in the column direction adjacent to each other.

In addition, in Figure 3, to make it easier to understand the configuration of the light receiving element group 20, the light receiving elements 21a, 21b, 21c, and 21d are shown with a gap between adjacent light receiving elements 21 in the row direction and between adjacent light receiving elements 21 in the column direction. However, in reality, in the light receiving element group 20, multiple light receiving elements 21 are arranged in a matrix with adjacent light receiving elements 21 in the row direction adjacent to each other and adjacent light receiving elements 21 in the column direction adjacent to each other.

Each of the multiple light receiving elements 21 photoelectrically converts input light (incident light) and accumulates an amount of charge internally corresponding to the amount of input light, outputting this accumulated charge as an electrical signal. In this embodiment, the range (signal range) of the signal (pixel value) S1 that can be output from each light receiving element 21 is 0 to 500. Each light receiving element 21 also has a photoelectric conversion element (photodiode in this embodiment) 211.

Each of the multiple row signal lines 25 transmits the reset signal RES, the first to fourth charge transfer signals TX1 to TX4, and the select signal SEL output from the first circuit unit 3 to each of the connected light receiving element groups 20.

Here, the reset signal RES is a signal for resetting the charge of each light-receiving element 21a, 21b, 22c, and 21d, the first charge transfer signal TX1 is a signal for turning the first light-receiving element 21a ON/OFF, the second charge transfer signal TX2 is a signal for turning the second light-receiving element 21b ON/OFF, the third charge transfer signal TX3 is a signal for turning the third light-receiving element 21c ON/OFF, and the fourth charge transfer signal TX4 is a signal for turning the fourth light-receiving element 21d ON/OFF.

Each of the multiple column signal lines 26 is connected to each light receiving element group 20 in the corresponding column and transmits the compression signal S2 output from each light receiving element group 20 to the second circuit unit 4.

The first circuit unit 3 outputs a reset signal RES, first to fourth charge transfer signals TX1 to TX4, and a select signal SEL to each light receiving element group 20, thereby controlling the light receiving element array 2 and controlling the compression of the signals S1a, S1b, S1c, and S1d output from each light receiving element 21 for each light receiving element group 20. Note that hereinafter, the signals S1a, S1b, S1c, and S1d output from each light receiving element 21a, 21b, 22c, and 21d in the light receiving element group 20 may also be referred to simply as signal S1.

The second circuit unit 4 performs signal processing such as reading out the compressed signal S2 from each light receiving element group 20, removing noise, and A/D (Analog/Digital) converting each read-out compressed signal S2.

The signal processing unit 5 performs various processes on each compressed signal S2 output from the second circuit unit 4. In this embodiment, the signal processing unit 5 is an ISP (Image Signal Processor) within the sensor.

In the solid-state imaging device 1 configured as described above, signals such as those shown in Figure 4 are sent from the first circuit unit 3 to each light-receiving element group 20, and the signals S1a, S1b, S1c, and S1d output from each light-receiving element 21a, 21b, 21c, and 21d in the light-receiving element group 20 are compressed to generate a compressed signal S2.

Specifically, in each light-receiving element group 20, at time t1, the reset signal turns ON and the select signal turns OFF. At this time, the first to fourth charge transfer signals TX1 to TX4 turn ON, and the charges in each of the light-receiving elements 21a, 21b, 21c, and 21d are reset.

Next, at time t2, the first charge transfer signal is turned OFF, and after a predetermined interval, at time t3, the second charge transfer signal is turned OFF, at time t4, the third charge transfer signal is turned OFF, and at time t5, the third charge transfer signal is turned OFF.

Due to this difference in the OFF time (exposure time), when the photosensitive element array 2 is exposed during photography or the like in the imaging device 100, the amount of exposure of at least one photosensitive element 21 in the photosensitive element group 20 will be different from the amount of exposure of the other photosensitive elements 21 in that photosensitive element group 20. In the imaging device 100 of this embodiment, the amount of exposure of each of the photosensitive elements 21a, 21b, 21c, and 21d in the photosensitive element group 20 is different.

Next, at time t6, the reset signal turns ON and the select signal turns OFF, and then at time t9, the reset signal turns OFF and the select signal turns ON.

Then, at times t7 and t8 between times t6 and t9, the first to fourth charge transfer signals TX1 to TX4 are simultaneously turned ON and OFF. As a result, the charges accumulated while each of the light-receiving elements 21a, 21b, 21c, and 21d is OFF (more specifically, the first light-receiving element 21a during the first accumulation time T1 (time t2 to time t7), the second light-receiving element 21b during the second accumulation time T2 (time t3 to time t7), the third light-receiving element 21c during the third accumulation time T3 (time t4 to time t7), and the fourth light-receiving element 21d during the fourth accumulation time T4 (time t5 to time t7)) are simultaneously output as electrical signals to the column signal lines 26 connected to the light-receiving element groups 20 including each of these light-receiving elements 21a, 21b, 21c, and 21d, and as a result, the signals S1a, S1b, S1c, and S1d output from each of the light-receiving elements 21a, 21b, 21c, and 21d are compressed to become a compressed signal S2. The relationship between the accumulation times T1 to T4 in the imaging device 100 of this embodiment is T2 = (3/4) T1, T3 = (2/4) T1, and T4 = (1/4) T1.

The compressed signal S2 generated for each light receiving element group 20 contains the average value of the pixel values (signal values) contained in the signal S1 output from each light receiving element 21a, 21b, 21c, and 21d of the light receiving element group 20.

For example, when focusing on one light receiving element group 20A, if the exposure amounts of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20A are within the signal range, as shown in FIG. 5, when the pixel value included in the signal S1a output from the first light receiving element 21a is 400, the pixel value included in the signal S1b output from the second light receiving element 21b is 300, the pixel value included in the signal S1c output from the third light receiving element 21c is 200, and the pixel value included in the signal S1d output from the fourth light receiving element 21d is 100, the pixel value (average value) included in the compressed signal S2 output from the light receiving element group 20A is
(400 + 300 + 200 + 100) / 4 = 250
In this embodiment, the "average value of the signal" refers to an unweighted sum of the signal.

Furthermore, when the exposure amount of some of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20A exceeds the signal range, for example, as shown in FIG. 6, when the pixel value included in the signal S1a output from the first light receiving element 21a is 500, the pixel value included in the signal S1b output from the second light receiving element 21b is 500, the pixel value included in the signal S1c output from the third light receiving element 21c is 500, and the pixel value included in the signal S1d output from the fourth light receiving element 21d is 300, the pixel value (average value) included in the compressed signal S2 output from the light receiving element group 20A is
(500 + 500 + 500 + 300) / 4 = 450
This becomes:

As described above, in the solid-state imaging device 1 of this embodiment, the above-mentioned compression unit 10 that generates, for each light-receiving element group 20, compressed signals S2 by compressing the signals S1a, S1b, S1c, and S1d output from the multiple light-receiving elements 21a, 21b, 21c, and 21d included in the light-receiving element group 20, is composed of the light-receiving element group 20 and the first circuit unit 3 that controls the light-receiving element group 20. In other words, the solid-state imaging device 1 of this embodiment has multiple compression units 10 (the number corresponding to the number of light-receiving element groups 20).

The control unit 102 controls each unit of the imaging device 100 in accordance with input signals and programs. This control unit 102 has a reconstruction processing unit 102A that reconstructs the compressed signal S2 output from the solid-state imaging element 1 into a pixel signal S3. Note that the configuration is not limited to one in which the control unit 102 controls the entire imaging device 100. It is also possible for the entire imaging device 100 to be controlled by multiple pieces of hardware sharing the processing.

The reconstruction processing unit 102A reconstructs the compressed signal S2 (compressed signal corresponding to each light receiving element group 20) output from the solid-state imaging device 1 into a pixel signal S3 of the pixel in the output image corresponding to the compressed signal S2, based on the pre-compression information contained in the compressed signal S2.

The control unit 102, which has this reconstruction processing unit 102A, reconstructs each compressed signal S2 output from the solid-state imaging device 1 to generate a pixel signal S3, thereby generating an output image (image data). The control unit 102 outputs (displays, etc.) this output image externally via the display unit 106.

Specifically, when the photoelectric conversion elements 211 of the photoreceptor array 20 do not overflow when the photoreceptor array 2 is exposed, the reconstruction processing unit 102A reconstructs the compressed signal S2 into a pixel signal S3 based on the pixel values included in the compressed signal S2 and the accumulation times T1, T2, T3, and T4 of the photoreceptors 21a, 21b, 21c, and 21d. Furthermore, when the photoreceptor array 2 is exposed and there are photoreceptors 21 in the photoreceptor group 20 whose photoelectric conversion elements 211 overflow and those whose photoelectric conversion elements 211 do not overflow, the reconstruction processing unit 102A reconstructs the compressed signal S2 into a pixel signal S3 so as to include predicted pixel values (signal values) that can be obtained if the photoelectric conversion elements 211 of each photoreceptor 21 have a capacity that will not overflow when exposed.

More specifically, for example, as shown in FIG. 5, when the pixel value contained in the compressed signal S2 output from the light receiving element group 20A is 250, the reconstruction processing unit 102A performs the following calculation, where x is the pixel value contained in the pixel signal S3, and T4 = (1/4)T1, T3 = (2/4)T1, and T2 = (3/4)T1.
{x+(3/4)x+(2/4)x+(1/4)x}/4=250
x=400
Then, the reconstruction processing unit 102A reconstructs the pixel signal S3 from the compressed signal S2 so that the pixel value x included in the pixel signal S3 becomes 400.

Furthermore, for example, as shown in FIG. 6, when the pixel value included in the compressed signal S2 output from the light receiving element group 20A is 450 (i.e., when there are light receiving elements 21 whose photoelectric conversion elements 211 have overflowed and light receiving elements 21 whose photoelectric conversion elements 211 have not overflowed in the light receiving element group 20), the reconstruction processing unit 102A performs the following calculation, using x1 as the pixel value included in the signal S1 obtained from the light receiving element 21d at the fourth accumulation time T4.
(500+500+500+x1)/4=450
x1 = 300
Then, because the pixel value x1 included in the signal S1d output from the light receiving element 21d during the fourth accumulation time T4 is 300, the reconstruction processing unit 102A converts the signal S1d (pixel value 300) into a value equivalent to the signal S1a output from the light receiving element 21a during the first accumulation time T1. Specifically, because the first accumulation time T1 is four times the fourth accumulation time T4, 300×4=1200. As a result, the reconstruction processing unit 102A reconstructs the pixel signal S3 so that the pixel value (predicted value) x2 included in the pixel signal S3 becomes 1200.

As such, with the reconstruction processing unit 102A of the imaging device 100 of this embodiment, even if light of an amount such that each of the light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20 has a value (pixel value) greater than the signal range is incident (i.e., even if there is a light receiving element 21 in the light receiving element group 20 whose photoelectric conversion element 211 overflows), an output image is generated without blown-out highlights. In other words, the imaging device 100 of this embodiment is capable of High Dynamic Range (HDR) imaging with a signal range of 0 to 2000.

The non-volatile memory 103 is electrically erasable and recordable non-volatile memory. In this embodiment, the non-volatile memory 103 stores the OS (operating system), which is the basic software executed by the control unit 102, and applications that work with the OS to realize applied functions. The working memory 104 is used as image display memory for the display unit 106 and as a working area for the control unit 102.

The operation unit 105 is used by a user or the like to input instructions to the imaging device 100. In this embodiment, the operation unit 105 includes a power button for turning the imaging device 100 on and off, and a touch panel formed on the display unit 106.

The display unit 106 displays the output image (image data) and displays text for operation.

The recording medium 107 records the image data output from the imaging unit 101.

The connection unit 108 is an interface for connecting to external devices. The imaging device 100 exchanges data with external devices via this connection unit 108.

The short-range wireless communication unit 109 is a communication unit for performing short-range wireless communication. The short-range wireless communication unit 109 is composed of an antenna for wireless communication, a modulation/demodulation circuit for processing wireless signals, and a communication controller.

The public network connection unit 110 is an interface for conducting public wireless communications. The imaging device 100 communicates with other devices via this public network connection unit 110. At this time, the control unit 102 realizes the call by inputting and outputting audio signals via the microphone 111 and speaker 112. In this embodiment, the public network connection unit 110 is an antenna, and the control unit 102 connects to the public network via this antenna.

The imaging device 100 described above includes a photoreceptor array 2 having a plurality of photoreceptor groups 20, each having a plurality of photoreceptors 21, and a compression unit 10 that generates a compressed signal S2 for each photoreceptor group 20 by compressing the signal S1 output from the plurality of photoreceptors 21 in the photoreceptor group 20. In this way, the compression unit 10 generates a compressed signal S2 for each photoreceptor group 20 and outputs each of these compressed signals S2 (i.e., signals with reduced signal volume), thereby reducing power consumption in the components downstream of the compression unit 10 (such as the control unit 102 in this embodiment).

The imaging device 100 of this embodiment also includes a solid-state imaging element 1 having a light-receiving element array 2 and a compression unit 10. With this configuration, the compression unit 10 in the solid-state imaging element 1 generates a compressed signal S2 for each light-receiving element group 20, and outputs each of these compressed signals S2 (i.e., signals with reduced signal volume), thereby reducing power consumption in the configuration downstream of the solid-state imaging element 1 (more specifically, the compression unit 10).

The imaging device 100 of this embodiment also includes a reconstruction processing unit 102A that reconstructs the compressed signal S2 generated by the compression unit 10 into a pixel signal S3 of the pixel in the output image that corresponds to the compressed signal S2, based on the pre-compression information contained in the compressed signal S2. As a result, the pixel signal S3 is reconstructed downstream of the compression unit 10, and this pixel signal S3 is output to a display unit 106 or the like, thereby improving the image quality of the image displayed on the display unit 106 or the like (i.e., the image obtained from the display unit 106).

Furthermore, in the imaging device 100 of this embodiment, when the light receiving element array 2 is exposed, the amount of exposure of at least one light receiving element 21 in the light receiving element group 20 is different from the amount of exposure of the other light receiving elements 21 in the light receiving element group 20. With this configuration, the signal amount can be reduced while including information on the signals S1a, S1b, S1c, and S1d output from the light receiving elements 21a, 21b, 21c, and 21d with different exposure amounts in each compressed signal S2.

In addition, in the imaging device 100 of this embodiment, the differences in the amount of exposure for each of the light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20 are caused by differences in the exposure time (charge accumulation time) of each of the light receiving elements 21a, 21b, 21c, and 21d. By adjusting the exposure time for each of the light receiving elements 21a, 21b, 21c, and 21d in this manner to cause differences in the amount of exposure for each of the light receiving elements 21a, 21b, 21c, and 21d within the light receiving element group 20, it is possible to change the amount of exposure for each of the light receiving elements 21a, 21b, 21c, and 21d by controlling (in this embodiment, by control by the first circuit unit 3).

Furthermore, in the imaging device 100 of this embodiment, each compression unit 10 generates a compressed signal S2 for each corresponding light receiving element group 20, the compressed signal S2 including the average value of the pixel values (signal values) included in the signals S1a, S1b, S1c, and S1d output from each light receiving element 21a, 21b, 21c, and 21d of the light receiving element group 20. A weighted average value may be used instead of the average value.

In addition, in the imaging device 100 of this embodiment, each light receiving element 21 in the light receiving element group 20 has a photoelectric conversion element 211. When the light receiving element array 2 is exposed to light and some light receiving elements 211 in the light receiving element group 20 overflow and some do not, the reconstruction processing unit 102A calculates the predicted signal value that can be obtained if the photoelectric conversion element 211 of each light receiving element 21 has a capacity that will not overflow due to exposure, and reconstructs the compressed signal S2 into a pixel signal S3. This makes it possible to obtain an output image (image data) with a dynamic range larger than the dynamic range defined by each light receiving element 21, i.e., enables imaging with a high dynamic range.

Furthermore, in the imaging device 100 of this embodiment, even if the light receiving element array 2 has N light receiving elements 21, there are m light receiving element groups 20, so the number (or number of times) of signals S2 read from the light receiving element array 2 is limited to N/m, where N>m.

This reduces the amount of data read from the light-receiving element array 2 compared to a configuration in which the signals of each light-receiving element are read out individually from the light-receiving element array. As a result, the amount of data transferred from the solid-state imaging element 1 to the control unit 102, etc. is reduced, enabling the imaging device 100 to achieve both high image quality, low power consumption, and a high frame rate.

In addition, the imaging device 100 of this embodiment can omit the operation of reading pixels multiple times, such as in dual conversion gain (DCG), making it possible to achieve high image quality, low power consumption, and a high frame rate.

In the image capturing device 100 of this embodiment as described above, the signals S1a, S1b, S1c, and S1d output from the multiple light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20 are compressed to become a compressed signal S2, and the reconstruction processing unit 102A reconstructs the compressed signal S2, which contains information about the signals S1a, S1b, S1c, and S1d from the multiple light receiving elements 21a, 21b, 21c, and 21d, into a pixel signal S3 corresponding to one pixel in the captured image, thereby generating a captured image. In other words, the image capturing device 100 of this embodiment generates an output image with a number of pixels corresponding to the number of light receiving element groups 20.

Next, a second embodiment of the present invention will be described with reference to Figures 7 to 10. The same reference numerals will be used for components similar to those in the first embodiment, and components that differ from the first embodiment will be described in detail.

As shown in FIG. 7 , the imaging device 100 according to the second embodiment includes a light receiving element array 2A having a plurality of light receiving element groups 20, each having a plurality of light receiving elements 21, and at least one compression unit 10A that generates, for each light receiving element group 20, a compressed signal S2 by compressing the signal S1 output from the plurality of light receiving elements 21 in the light receiving element group 20. The imaging device 100 of this embodiment includes one compression unit 10A, and this one compression unit 10A generates, for each light receiving element group 20, a compressed signal S2 by compressing the signal S1 output from the plurality of light receiving elements 21 in the light receiving element group 20.

Specifically, the imaging device 100 includes an imaging unit 101 and a control unit 102. Similar to the first embodiment, the imaging device 100 of this embodiment also includes a non-volatile memory 103, a working memory 104, an operation unit 105, a display unit 106, a recording medium 107, a connection unit 108, a short-range wireless communication unit 109, a public network connection unit 110, a microphone 111, and a speaker 112.

The imaging unit 101 includes an optical system composed of at least one optical element, and a solid-state imaging element 1A that converts the image of an object (image object) formed through the optical system into an electrical signal (i.e., captures the image).

The solid-state imaging device 1A has a light-receiving element array 2A in which multiple light-receiving elements 21 are arranged in a matrix, a first circuit unit 3A and a second circuit unit 4A connected to the light-receiving element array 2A, and a signal processing unit 5A to which the second circuit unit 4A is connected.

In the light receiving element array 2A of this embodiment, multiple row signal lines 25A are arranged for each row of the light receiving elements 21 arranged in a matrix, and multiple column signal lines 26A are arranged for each column of the light receiving elements 21 arranged in a matrix.

The first circuit unit 3A controls the light receiving element array 2, while the second circuit unit 4A performs signal processing such as reading out the signal S1 from each light receiving element 21, removing noise, and A/D converting each read-out signal S1.

The signal processing unit 5A is a component that performs various processes on the compressed signal S2 output from the second circuit unit 4, and also performs signal processing (compression processing) to compress the signal S1 output from each light receiving element 21 for each light receiving element group 20. In other words, in the imaging device 100 of this embodiment, at least a portion of the signal processing unit 5A constitutes the compression unit 10A.

When the first light receiving element 21a is used as the reference light receiving element and the second to fourth light receiving elements 21b to 21d are used as non-reference light receiving elements among the multiple light receiving elements (in this embodiment, the first light receiving element 21a, second light receiving element 21b, third light receiving element 21c, and fourth light receiving element 21d) included in the light receiving element group 20 (see FIG. 10), the signal processing unit 5A generates a compressed signal S2 based on the difference between the reference signal value Sv1a, which is the pixel value (signal value) included in the signal S1a output from the reference light receiving element 21a, and the non-reference signal values Sv1b, Sv1c, and Sv1d, which are pixel values (signal values) included in the signals S1b, S1c, and S1d output from the non-reference light receiving elements 21b, 21c, and 21d.

In the solid-state imaging device 1A of this embodiment, when one signal processing unit 5A (compression unit 10A) processes a signal S1 from each light receiving element 21 of the light receiving element array 2, it processes the signal S1 for each light receiving element group 20, treating multiple adjacent light receiving elements (in this embodiment, light receiving elements arranged in a 2x2 matrix) 21a, 21b, 21c, and 21d as light receiving element groups 20, to generate a compressed signal S2, and outputs the compressed signal S2 generated for each light receiving element group 20 to a downstream configuration (such as the control unit 102).

In more detail, as shown in FIG. 8, the signal processing unit 5A (compression unit 10A) calculates whether the difference between the non-reference signal values (pixel values) Sv1b, Sv1c, Sv1d included in the signals S1b, S1c, S1d output from the non-reference light receiving elements 21b, 21c, 21d of the light receiving element group 20 and the reference signal value (pixel value) Sv1a included in the signal S1a output from the reference light receiving element 21a is equal to or greater than a predetermined threshold value (in this embodiment, the difference is calculated by multiplying the difference by 1/2 by 1/2). When the signal range of 1 is 0 to 500, and only non-reference signal values (first signal values) Sv1b, Sv1c, and Sv1d exist for which the pixel value difference is less than 25 to 75, a compressed signal S2 is generated (first compression process) including the average value of the pixel values Sv1a, Sv1b, Sv1c, and Sv1d included in the signals S1a, S1b, S1c, and S1d output from the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20. The predetermined threshold is set approximately 10% of the maximum value of the signal range of the light receiving element 21. The first compression process in the signal processing unit 5A of this embodiment is so-called binning. The compressed signal S2 of this embodiment includes the average value of the pixel values Sv1a, Sv1b, Sv1c, and Sv1d and position information 3b111. In this embodiment, the position information of the compressed signal S2 generated by the first compression process in the signal processing unit 5A (compression unit 10) is a fixed value (3b111).

Furthermore, as shown in FIG. 9, when there is a non-reference signal value (first signal value) Sv1c whose difference from the reference signal value Sv1a is less than a predetermined threshold, and non-reference signal values (second signal values) Sv1b, Sv1d whose difference from the reference signal value Sv1a is equal to or greater than a predetermined threshold, the signal processing unit 5A (compression unit 10A) deletes the signals S1b, S1d corresponding to the second signal values Sv1b, Sv1d from the signals output from the light receiving elements 21a, 21b, 21c, 21d. At the same time, the pixel value Sv1a included in the signal S1a output by the reference light receiving element 21a is set to the average value of the pixel value Sv1a and the first signal value Sv1c (100 in the example shown in FIG. 9), and position information (symbol 3b010 of the compressed signal S2 in the example shown in FIG. 9) of the non-reference light receiving element 21c that output the signal S1c including the first signal value Sv1c relative to the reference light receiving element 21a is added to generate a compressed signal S2 (second compression process).

The position information for the second compression process (symbol 3b010 of compressed signal S2 in Figure 9) is actually 3-bit data, and in the example shown in Figure 9, it is "010." In this three-digit number, the first digit (the leftmost digit) indicates the position of the second light-receiving element 21b relative to the reference light-receiving element (first light-receiving element) 21a, the second digit indicates the position of the third light-receiving element 21c relative to the reference light-receiving element 21a, and the third digit indicates the position of the fourth light-receiving element 21d relative to the reference light-receiving element 21a. The pixel values contained in the signals S1b, S1c, and S1d from the light-receiving elements 21b, 21c, and 21d corresponding to each position are 1 when the first signal value is reached, and 0 when the second signal value is reached.

The control unit 102 has a reconstruction processing unit 102B, which reconstructs, from the compressed signal S2 output from the solid-state imaging device 1A, pixel signals S3a, S3b, S3c, and S3d of the pixels corresponding to each of the light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20 that correspond to the compressed signal S2.

Specifically, the reconstruction processing unit 102B reconstructs the compressed signal S2 (pixel value 4, position information 3b111: see Figure 8) generated by the first compression process, for example, as follows:

For the compressed signal S2 output from the upper right light receiving element group 20 in the light receiving element array 2A in FIG. 10, the reconstruction processing unit 102B, as also shown in FIG. 8, sets the pixel value of the pixel signal S3a of the pixel corresponding to the reference light receiving element 21a to pixel value 4 of the compressed signal S2, and sets the pixel values of the pixel signals S3b, S3c, S3d of the pixels corresponding to each of the non-reference light receiving elements 21b, 21c, 21d to pixel value 4 of the pixel signal S3a of the pixel corresponding to the reference light receiving element 21a, thereby reconstructing the compressed signal S2 into each pixel signal S3a, S3b, S3c, S3d.

Furthermore, the reconstruction processing unit 102B reconstructs the compressed signal S2 (pixel value 100, position information 3b010: see Figure 9) generated by the second compression process, for example, as follows:

For the compressed signal S2 output from the upper left light receiving element group 20 in the light receiving element array 2 in Figure 10, the reconstruction processing unit 102B, as also shown in Figure 9, sets the pixel value of the pixel signal S3a of the pixel corresponding to the reference light receiving element 21a to the pixel value 100 of the compressed signal S2, sets the pixel value of the pixel signal S3c of the pixel corresponding to the non-reference light receiving elements 21b, 21c, and 21d, which has position information (3b010) set to 1, to the pixel value 100 of the pixel corresponding to the reference light receiving element 21a, and sets the pixel values of the pixel signals S3b and S3d of the pixel corresponding to the pixel with position information (3b010) set to 0, to the pixel value 4 of the adjacent pixel in the row direction (in the example shown in Figure 10, the pixel adjacent to the right), thereby reconstructing the compressed signal S2 into each pixel signal S3a, S3b, S3c, and S3d. Furthermore, the reconstruction processing unit 102B may set each pixel value of the pixel signals S3b, S3d of a pixel whose position information (3b010) is 0 to the weighted average or average value of the pixel value 100 of the compressed signal S2 and the pixel value 4 of the compressed signal S2 output from the adjacent light receiving element group 20 in the row direction.

In the imaging device 100 described above, as shown in FIG. 10, when a vertical line image (the area that appears white compared to other areas in FIG. 10) is formed at the left end of the light receiving element array 2 during exposure, the signal processing unit 5A (compression unit 10A) performs a first compression process on the light receiving element groups 20 arranged vertically on the right side of the light receiving element array 2, and performs a second compression process on the light receiving element groups 20 arranged vertically on the left side of the light receiving element array 2. In this way, a compressed signal S2 generated for each light receiving element group 20 is output from the solid-state imaging device 1A.

Next, the reconstruction processing unit 102B of the control unit 102 reconstructs the compressed signal S2 generated by the first compression process and the compressed signal S2 generated by the second compression process using different methods (see Figures 8 and 9), thereby generating a high-resolution image with little blur as image data.

The imaging device 100 according to this embodiment includes a light receiving element array 2A having a plurality of light receiving element groups 20, each having a plurality of light receiving elements 21, and a signal processing unit 5A (compression unit 10A) that generates a compressed signal S2 for each light receiving element group 20 by compressing the signal S1 output from the plurality of light receiving elements 21 in the light receiving element group 20. In this way, the signal processing unit 5A generates a compressed signal S2 for each light receiving element group 20 and outputs each of these compressed signals S2 (i.e., signals with reduced signal volume), thereby reducing power consumption in the configuration downstream of the signal processing unit 5A.

In addition, in the imaging device 100 of this embodiment, the multiple light receiving elements 21 included in the light receiving element group 20 include a reference light receiving element 21a, which is the reference light receiving element 21, and non-reference light receiving elements 21b, 21c, and 21d other than the reference light receiving element 21a. The signal processing unit 5A (compression unit 10A) generates a compressed signal S2 based on the difference between a reference signal value Sv1a, which is a pixel value (signal value) included in the signal S1a output from the reference light receiving element 21a, and non-reference signal values Sv1b, Sv1c, and Sv1d, which are pixel values (signal values) included in the signals S1b, S1c, and S1d output from the non-reference light receiving elements 21b, 21c, and 21d.

In this way, by compressing the signals S1a, S1b, S1c, and S1d output from each light-receiving element 21a, 21b, 21c, and 21d based on the difference between the reference signal value (pixel value) Sv1a and the non-reference signal values (pixel values) Sv1b, Sv1c, and Sv1d for each position of the non-reference light-receiving elements 21b, 21c, and 21d relative to the reference light-receiving element 21a, the reconstruction processing unit 102B can reconstruct the difference in signal amount for each position, thereby ensuring resolution in the output image. In other words, a high-resolution image with little blur is obtained.

In addition, in the imaging device 100 of this embodiment, when the signals S1b, S1c, and S1d output from the non-reference light-receiving elements 21b, 21c, and 21d of the light-receiving element group 20 contain only non-reference signal values Sv1b, Sv1c, and Sv1d whose difference from the reference signal value Sv1a is less than a predetermined threshold, the signal processing unit 5A (compression unit 10A) generates a compressed signal S2 containing the average value of the pixel values (signal values) Sv1a, Sv1b, Sv1c, and Sv1d included in the signals S1a, S1b, S1c, and S1d output from each light-receiving element 21a, 21b, 21c, and 21d of the light-receiving element group 20. Note that a weighted average value may be used instead of the average value.

Furthermore, when there is a first signal value Sv1c, which is a non-reference signal value whose difference from the reference signal value Sv1a is less than a predetermined threshold, and second signal values Sv1b, Sv1d, which are non-reference signal values whose difference from the reference signal value Sv1a is equal to or greater than a predetermined threshold, the signal processing unit 5A (compression unit 10A) deletes the signals S1b, S1d corresponding to the second signal values Sv1b, Sv1d from the signals S1a, S1b, S1c, S1d output from each of the light receiving elements 21a, 21b, 21c, 21d, and adds position information (3b010 in this embodiment) of the non-reference light receiving element 21c that output the first signal value Sv1c relative to the reference light receiving element 21a to the signal S1a output from the reference light receiving element 21a, thereby generating a compressed signal S2.

In this way, by constructing a compressed signal S2 using the signal S1a (more specifically, a signal based on the signal S1a) output from the reference light-receiving element 21a and the positional information (3b010, etc.) added to it, the signal volume is suitably reduced compared to when the signals S1a, S1b, S1c, and S1d output from each light-receiving element 21a, 21b, 21c, and 21d in the light-receiving element group 20 are all output to a downstream configuration.

Furthermore, in the imaging device 100 of this embodiment, even if the light receiving element array 2 has N light receiving elements 21, there are m light receiving element groups 20, so the number (or number of times) of signals S2 output from the solid-state imaging element 1A is limited to N/m, where N>m.

This reduces the signal volume (data volume) of the signal S2 output from the solid-state imaging device 1A compared to a configuration in which signals read out from each light-receiving element in the light-receiving element array are output individually from the solid-state imaging element. Furthermore, by reconstructing N pixels from information on m pixels + α in the configuration downstream of the solid-state imaging device 1A (such as the reconstruction processing unit 102B), the imaging device 100 can achieve both high image quality, low power consumption, and a high frame rate.

In the imaging device 100 of this embodiment as described above, the signals S1a, S1b, S1c, and S1d output from the multiple light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20 are compressed to become a compressed signal S2, and the reconstruction processing unit 102A reconstructs this compressed signal S2 into pixel signals S3a, S3b, S3c, and S3d of pixels corresponding to each light receiving element 21a, 21b, 21c, and 21d in the captured image, thereby generating a captured image. In other words, the imaging device 100 of this embodiment generates an output image with a number of pixels corresponding to the number of light receiving elements 21.

The imaging device 100 and solid-state imaging elements 1, 1A of the present invention are not limited to the first and second embodiments described above, and various modifications can of course be made without departing from the spirit of the present invention. For example, the configuration of one embodiment can be added to the configuration of another embodiment, or part of the configuration of one embodiment can be replaced with the configuration of another embodiment. Furthermore, part of the configuration of one embodiment can be deleted.

In the first and second embodiments described above, the light receiving element group 20 has four light receiving elements 21a, 21b, 21c, and 21d arranged in a 2x2 matrix, but is not limited to this configuration. The light receiving element group 20 may have a plurality (two or more) of light receiving elements 21. It is preferable that the plurality of light receiving elements 21 in the light receiving element group 20 be arranged in a 2x2, 3x3, 4x4, or other matrix such that the number of light receiving elements 21 arranged in the row direction is the same as the number of light receiving elements 21 arranged in the column direction (i.e., in a square matrix).

In addition, in the light receiving element group 20 of the second embodiment described above, the reference light receiving element is arranged in the first row, first column position (the upper left position in the examples of Figures 8 and 9) in the matrix arrangement (2 x 2 arrangement) shown in Figures 8 and 9, but it may also be arranged in another position.

In addition, in the imaging device 100 of the first and second embodiments described above, the solid-state imaging element 1, 1A has a compression unit 10, and the reconstruction processing units 102A, 102B are located in a different position (control unit 102) from the solid-state imaging element 1, 1A, but this configuration is not limited. The solid-state imaging element 1, 1A may have both the compression unit 10 and the reconstruction processing units 102A, 102B.

Furthermore, in each of the imaging devices 100 of the first and second embodiments described above, the pixel value (average value) included in the compressed signal S2 is a value derived by so-called simple averaging, but this configuration is not limited to this. The average value (binning) may also be a value derived by weighted averaging.

In addition, in the second embodiment described above, the position information in the second compression process that generates the compressed signal S2 is a signal that indicates the position of the light receiving elements 21b, 21c, and 21d that output the first signal value relative to the reference light receiving element 21a, but this configuration is not limited to this. The position information in the second compression process may be a signal that indicates the position of the light receiving elements 21b, 21c, and 21d that output the second signal value relative to the reference light receiving element 21a, and the position information may be used to identify the position of the light receiving elements 21b, 21c, and 21d that output the first signal value relative to the reference light receiving element 21a.

In addition, in the above second embodiment, when the non-reference signal values contained in the signals output from the non-reference light receiving elements 21b, 21c, and 21d of the light receiving element group 20 include a first signal value and a second signal value, the signal processing unit (compression unit) 5A deletes the signal corresponding to the second signal value from the signals output from each light receiving element 21a, 21b, 21c, and 21d, and generates a compressed signal S2 by adding position information relative to the reference light receiving element 21a of the non-reference light receiving elements 21b, 21c, and 21d that output the first signal value or the second signal value to the signal output from the reference light receiving element 21a, but this configuration is not limited to this. For example, when the non-reference signal values included in the signals output from the non-reference light-receiving elements 21b, 21c, and 21d of the light-receiving element group 20 include a first signal value and a second signal value, the signal processing unit (compression unit) 5A may be configured to generate a compressed signal S2 that includes a weighted average or average value of the reference signal value and the first signal value, and position information of the non-reference light-receiving elements 21b, 21c, and 21d that output the first signal value or the second signal value relative to the reference light-receiving element 21a.

In order to express the present invention, the present invention has been appropriately and sufficiently described above through embodiments with reference to the drawings. However, it should be recognized that those skilled in the art could easily modify and/or improve the above-described embodiments. Therefore, unless modifications or improvements made by those skilled in the art deviate from the scope of the claims set forth in the claims, such modifications or improvements are deemed to be encompassed within the scope of the claims.

1, 1A... solid-state imaging element, 2, 2A... light-receiving element array, 20... light-receiving element group, 21... light-receiving element, 21a... first light-receiving element, reference light-receiving element, 21b... second light-receiving element, non-reference light-receiving element, 21c ...third light receiving element, non-reference light receiving element, 21d...fourth light receiving element, non-reference light receiving element, 211...photoelectric conversion element, 22...amplifier, 25, 25A...row signal line, 26, 26A...column signal line, 3, 3A...first circuit unit, 4, 4A...second circuit unit, 5...signal processing unit, 5A...signal processing unit (compression unit), 10, 10A...compression unit, 100...imaging device, 101...imaging unit, 102...control unit, 102A, 102B...reconstruction processing unit, 103...non-volatile memory, 104...working memory, 105...operation unit, 106...display unit, 107...recording medium, 108...connection unit, 109...short-range wireless communication unit, 110...public network connection unit, 111...microphone, 112...speaker, 500...solid-state imaging element, 501...light receiving element array, 502...light receiving element, 503 ...pixel drive wiring, 504...vertical signal line, 505...column-parallel signal processing circuit, 506...output circuit, 507...timing control circuit, 508...horizontal scanning circuit, 509...vertical scanning circuit, RES...reset signal, S1, S1a, S1b, S1c, S1d...signal, S2...compression signal, S3, S3a, S3b, S3c, S3d...pixel signal, SEL...select signal, Sv1a...reference signal value, Sv1b, Sv1d...non-reference signal value, second signal value, Sv1c...non-reference signal value, first signal value, t1-t9...time, T1...first accumulation time, T2...second accumulation time, T3...third accumulation time, T4...fourth accumulation time, TX1...first charge transfer signal, TX2...second charge transfer signal, TX3...third charge transfer signal, TX4...fourth charge transfer signal

Claims (12)

a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements;
a compression unit that generates a compressed signal for each light-receiving element group by compressing signals output from the plurality of light-receiving elements of the light-receiving element group;
a reconstruction processing unit that reconstructs the compressed signal generated by the compression unit into pixel signals of pixels in an output image that correspond to the compressed signal ,
In the light receiving element group, when the light receiving element array is exposed, the amount of exposure of at least one light receiving element of the light receiving element group is different from the amount of exposure of the other light receiving elements of the light receiving element group,
the compressed signal includes a weighted average or an average of signal values included in the signals output from the light receiving elements of the light receiving element group,
Each light receiving element of the light receiving element group has a photoelectric conversion element,
The reconstruction processing unit, when there are light receiving elements in the light receiving element group whose photoelectric conversion elements overflow and light receiving elements whose photoelectric conversion elements do not overflow when the light receiving element array is exposed, reconstructs the compressed signal into the pixel signal by calculating an expected value of the signal value that can be obtained if the photoelectric conversion element of each light receiving element has a capacity that will not overflow due to the exposure . The imaging device of claim 1, comprising a solid-state imaging element having the light receiving element array and the compression unit. 3. The imaging device according to claim 1 , wherein the difference in the amount of exposure in the light receiving element group is caused by a difference in exposure time of each light receiving element. The imaging device according to claim 1 , wherein the reconstruction processing unit reconstructs the compressed signal into the pixel signal based on pre-compression information included in the compressed signal. the plurality of light receiving elements included in the light receiving element group include a reference light receiving element that is a light receiving element serving as a reference, and non-reference light receiving elements other than the reference light receiving element,
the compression unit generates the compressed signal based on a difference between a reference signal value, which is a signal value included in the signal output from the reference light-receiving element, and a non-reference signal value, which is a signal value included in the signal output from the non-reference light-receiving element;
The compression unit is configured to:
a first compression process for generating the compressed signal including a weighted average or average value of the signal values output from each light receiving element of the light receiving element group when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold value;
2. The imaging device according to claim 1, further comprising: a first signal value that is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold; and a second signal value that is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold; and a second compression process that deletes the signal output from each of the light receiving elements that corresponds to the second signal value; and adds position information of the non-reference light receiving element that output the first signal value or the second signal value relative to the reference light receiving element to the signal output from the reference light receiving element, thereby generating the compressed signal . the plurality of light receiving elements included in the light receiving element group include a reference light receiving element that is a light receiving element serving as a reference, and non-reference light receiving elements other than the reference light receiving element,
the compression unit generates the compressed signal based on a difference between a reference signal value, which is a signal value included in the signal output from the reference light-receiving element, and a non-reference signal value, which is a signal value included in the signal output from the non-reference light-receiving element;
The compression unit is configured to:
a first compression process for generating the compressed signal including a weighted average or average value of the signal values output from each light receiving element of the light receiving element group when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold value;
2. The imaging device according to claim 1, wherein, when there is a first signal value that is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value that is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, a second compression process is performed to generate the compressed signal including a weighted average or average value of the reference signal value and the first signal value, and position information of the non-reference light receiving element that output the first signal value or the second signal value relative to the reference light receiving element. 7. The imaging device according to claim 5, wherein the predetermined threshold is set approximately to 10% of a maximum value of a signal range of a light receiving element included in the light receiving element group. 7. The imaging device according to claim 5, wherein the compressed signal generated by the first compression process further includes position information that is a fixed value. The reconstruction processing unit performs the following on the compressed signal generated by the first compression processing:
7. The imaging device according to claim 5, wherein the compressed signal is reconstructed into the pixel signal by making pixel values of pixel signals of pixels corresponding to the reference light receiving element the same as pixel values of the compressed signal, and making pixel values of pixel signals of pixels corresponding to each of the non-reference light receiving elements the same as pixel values of pixel signals of pixels corresponding to the reference light receiving element. The reconstruction processing unit performs the following on the compressed signal generated by the second compression processing:
7. The imaging device according to claim 5, wherein the compressed signal is reconstructed into the pixel signal by making a pixel value of a pixel signal of a pixel corresponding to the reference light receiving element the same as a pixel value of the compressed signal, making a pixel value of a pixel signal of a pixel corresponding to the non-reference light receiving element, the pixel value of a pixel whose position information is 1, the same as a pixel value of a pixel corresponding to the reference light receiving element, and making a pixel value of a pixel signal of a pixel whose position information is 0 the same as a pixel value of a pixel adjacent to the pixel in the row direction. The reconstruction processing unit performs the following on the compressed signal generated by the second compression processing:
7. The imaging device according to claim 5, wherein the compressed signal is reconstructed into the pixel signal by setting a pixel value of a pixel signal of a pixel corresponding to the reference light receiving element to the same pixel value of the compressed signal, setting a pixel value of a pixel signal of a pixel corresponding to the non-reference light receiving element, among the pixels corresponding to the non-reference light receiving element, whose position information is 1, to the same pixel value of a pixel corresponding to the reference light receiving element, and setting each pixel value of a pixel signal of a pixel whose position information is 0 to a weighted average or average of the pixel value of the compressed signal and the pixel values of compressed signals output from light receiving element groups adjacent to the pixel in the row direction. a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements;
a compression unit that generates a compressed signal for each light-receiving element group by compressing signals output from the plurality of light-receiving elements of the light-receiving element group;
a reconstruction processing unit that reconstructs the compressed signal generated by the compression unit into pixel signals of pixels in an output image that correspond to the compressed signal ,
In the light receiving element group, when the light receiving element array is exposed, the amount of exposure of at least one light receiving element of the light receiving element group is different from the amount of exposure of the other light receiving elements of the light receiving element group,
the compressed signal includes a weighted average or an average of signal values included in the signals output from the light receiving elements of the light receiving element group,
Each light receiving element of the light receiving element group has a photoelectric conversion element,
The reconstruction processing unit, when there are photoreceptors in the photoreceptor group whose photoelectric conversion elements overflow and those whose photoelectric conversion elements do not overflow when the photoreceptor array is exposed to light, reconstructs the compressed signal into the pixel signal by calculating an expected value of the signal value that can be obtained if the photoelectric conversion element of each photoreceptor has a capacity that will not overflow due to the exposure .