JP7689018B2 - Image Generation Device - Google Patents

Description

This disclosure relates to an image generating device that generates an image of a subject.

In recent years, systems have been proposed that use cameras to capture images of a person's face and determine the person's health, alertness, and concentration in real time. The light sources used to capture images of the subject are classified as visible light and invisible light, but many invisible light sources are used to capture images with stable brightness that are not affected by ambient light.

One of the issues when capturing an image by shining invisible light on a subject and receiving the reflected light is saturation of pixel signals due to specular reflection. For example, if specular reflection occurs on part of the subject during imaging, saturation occurs due to the excessive amount of light received, and some of the image information of the subject is lost. Loss of some of the image information of the subject leads to a decrease in accuracy when determining a person's health state, alertness, concentration state, etc.

Non-Patent Document 1 proposes a technology for capturing images using a stereo camera system. In the stereo camera system, an object is captured using two cameras placed at a distance from one another. With this technology, even if one of the cameras is unable to obtain the necessary information due to the effects of specular reflection, an image of the object can be generated using information from the other camera.

"Improvement of glasses reflection problem in pupil detection using stereo-calibrated cameras," by Hisahisa Ishida, Kiyotake Fukumoto, and Yoshinobu Ebisawa, ITE Annual Convention 2011

However, in Non-Patent Document 1, a mechanism is required to simultaneously capture an image of a subject using two cameras installed at separate locations and to mutually complement the two captured images, which poses the problem of the device becoming larger.

Therefore, this disclosure provides an image generating device that suppresses the effects of specular reflection and easily generates images.

The image generating device of the present disclosure includes a first light-emitting unit that irradiates light toward a subject, a second light-emitting unit that irradiates light toward the subject, a light-receiving unit that receives, at a plurality of pixels arranged in a two-dimensional form, each of the reflected lights generated when the light irradiated by the first light-emitting unit and the second light-emitting unit is reflected by the subject, a drive control unit that controls an emission period of the first light-emitting unit, an emission period of the second light-emitting unit, and a light-receiving period of the light-receiving unit, a saturation level determination unit that determines whether or not a first pixel signal and a second pixel signal output from the pixel of the light-receiving unit in response to the irradiation of light by the first light-emitting unit and the second light-emitting unit exceed a saturation level, and a control unit that determines whether or not a first pixel signal and a second pixel signal output from the pixel of the light-receiving unit exceed a saturation level based on a result of the determination by the saturation level determination unit. and a combining unit that combines the first pixel signal and the second pixel signal to generate a combined signal, wherein the first light-emitting unit and the second light-emitting unit are disposed apart from each other, and the drive control unit generates, in the same frame, a first light-emitting pulse that causes the first light-emitting unit to emit light and a second light-emitting pulse that causes the second light-emitting unit to emit light, at different timings, and generates, at different timings, a first light-receiving pulse that causes reflected light generated when light irradiated by the first light-emitting unit is reflected by the subject, and a second light-receiving pulse that causes the reflected light generated when light irradiated by the second light-emitting unit is reflected by the subject, and the light-receiving unit generates a first light-receiving pulse that causes the first light-receiving unit to receive the reflected light, the first pixel signal including information on an amount of received light received in accordance with the second light receiving pulse and the second pixel signal including information on a second amount of received light received in accordance with the second light receiving pulse, the saturation level determination unit compares the first amount of received light, the second amount of received light, and the saturation level, and the combining unit (1) combines the first pixel signal and the second pixel signal with a first combining coefficient when the first amount of received light and the second amount of received light are both smaller than the saturation level, (2) combines the first pixel signal and the second pixel signal with a second combining coefficient when only the first amount of received light is greater than the saturation level, and (3) combines the first pixel signal and the second pixel signal with a third combining coefficient when only the second amount of received light is greater than the saturation level. The light receiving unit includes a correction unit that combines an element signal and the second pixel signal, and further corrects the first light receiving amount information or the second light receiving amount information, wherein the drive control unit controls a first light receiving period of the light receiving unit corresponding to the irradiation of light by the first light emitting unit and a second light receiving period of the light receiving unit corresponding to the irradiation of light by the second light emitting unit to be different from each other, the correction unit corrects the first light receiving amount information or the second light receiving amount information based on the amount of light received resulting from the difference between the first light receiving period and the second light receiving period, and the combining unit combines the first pixel signal and the second pixel signal using the first light receiving amount information or the second light receiving amount information corrected by the correction unit.
The image generating device of the present disclosure includes a first light-emitting unit that irradiates light toward a subject, a second light-emitting unit that irradiates light toward the subject, a light-receiving unit that receives, at a plurality of pixels arranged in a two-dimensional form, each of reflected lights generated when the light irradiated by the first light-emitting unit and the second light-emitting unit is reflected by the subject, and a drive control unit that controls an emission period of the first light-emitting unit, an emission period of the second light-emitting unit, and a light-receiving period of the light-receiving unit. A saturation level determination unit that determines whether a first pixel signal and a second pixel signal output from the pixel of the light-receiving unit in response to the irradiation of light by the first light-emitting unit and the second light-emitting unit exceed a saturation level, and a combination unit that generates a combined signal by combining the first pixel signal and the second pixel signal based on a determination result of the saturation level determination unit, wherein the first light-emitting unit and the second light-emitting unit are arranged apart from each other, and the drive control unit controls the first light-emitting unit to emit light and the second light-emitting unit to emit light in the same frame. the first light-receiving pulse is generated at a different timing from the first light-emitting pulse for receiving the reflected light generated by the light irradiated by the first light-emitting unit being reflected by the subject, and the second light-receiving pulse is generated at a different timing from the first light-receiving pulse for receiving the reflected light generated by the light irradiated by the second light-emitting unit being reflected by the subject; the light-receiving unit outputs the first pixel signal including information on a first amount of received light received in accordance with the first light-receiving pulse, and outputs the second pixel signal including information on a second amount of received light received in accordance with the second light-receiving pulse; the saturation level determination unit compares the first amount of received light, the second amount of received light, and the saturation level; the combining unit (1) combines the first pixel signal and the second pixel signal with a first combining coefficient when the first amount of received light and the second amount of received light are both smaller than the saturation level, and (2) combines the first pixel signal and the second pixel signal with a second combining coefficient when only the first amount of received light is larger than the saturation level;
(3) when only the second amount of received light is greater than the saturation level, the first pixel signal and the second pixel signal are combined by a third combination coefficient;
a correction unit that corrects the information on the first amount of received light or the information on the second amount of received light,
the drive control unit controls a first light-receiving start time from a start of irradiation of the first light-emitting unit to a start of light-receiving by the light-receiving unit to be different from a second light-receiving start time from a start of irradiation of the second light-emitting unit to a start of light-receiving by the light-receiving unit;
the correction unit corrects information about the first amount of received light or information about the second amount of received light based on an amount of received light caused by a difference between the first light-receiving start time and the second light-receiving start time;
The combining unit combines the first pixel signal and the second pixel signal using information on the first amount of received light or information on the second amount of received light corrected by the correction unit.
Image generating device.

The image generating device disclosed herein can easily generate images by suppressing the effects of specular reflection.

FIG. 2 is a diagram showing an example of a space in which an image generating device is provided. FIG. 13 is a diagram showing a state in which specular reflection occurs on a part of a subject. 1 is a diagram showing an image generating device and a subject according to a first embodiment as viewed from above. 1 is a diagram showing an image generating device according to a first embodiment as viewed from a subject side. 1 is a block diagram of an image generating device according to a first embodiment; 4 is a time chart showing the operation of the image generating device according to the first embodiment; 3 is a diagram showing a schematic diagram of pixel information etc. acquired by a light receiving unit of the image generating device according to the first embodiment; 4 is a diagram showing pixel signals output from a light receiving unit of the image generating device according to the first embodiment; 11 is a table showing the determination results of a saturation level determination section and the synthesis process in a synthesis section. 10A to 10C are diagrams illustrating another example of the arrangement of the first light-emitting unit, the second light-emitting unit, and the light-receiving unit of the image generating device according to the first embodiment. FIG. 11 is a block diagram of an image generating device according to a second embodiment. 10 is a time chart showing the operation of an image generating device according to a second embodiment. 13 is a time chart showing the operation of the image generating device according to the first modification of the second embodiment. 13 is a time chart showing the operation of an image generating device according to a second modification of the second embodiment. 13 is a time chart showing the operation of the image generating device according to the third modification of the second embodiment. FIG. 11 is a block diagram of an image generating device according to a third embodiment. 13 is a time chart showing the operation of an image generating device according to a third embodiment. 11 is a table showing the determination results of a saturation level determination section and the synthesis process in a synthesis section. 13 is a time chart showing the operation of an image generating device according to a modification of the third embodiment. 11 is a table showing the determination results of a saturation level determination section and the synthesis process in a synthesis section. FIG. 13 is a block diagram of an image generating device according to a fourth embodiment. 13 is a time chart showing the operation of an image generating device according to a fourth embodiment. 13 is a time chart showing the operation of an image generating device according to a modification of the fourth embodiment. 13 is a time chart showing the operation of an image generating device according to a fifth embodiment. FIG. 13 is a diagram showing another example of the operation of the image generating device according to the fifth embodiment. FIG. 13 is a diagram showing another example of the operation of the image generating device according to the fifth embodiment. FIG. 13 is a diagram showing another example of the operation of the image generating device according to the fifth embodiment.

(Foundational knowledge of the present disclosure)
The knowledge on which the present disclosure is based will be described with reference to Figures 1 and 2. The knowledge on which the present disclosure is based includes new knowledge not found in the prior art and conventional knowledge on which the new knowledge is based.

Figure 1 shows an example of a space 8 in which an image generating device is installed.

As shown in FIG. 1, the image generating device is installed in a predetermined space 8, such as a vehicle interior. The image generating device is a device that captures an image of a subject 9, that is, a driver, and generates an image, and is used in a system that judges the driver's health condition, etc. The image generating device includes a light source that emits light and a camera that receives the light. In such an image generating device, the light emitted from the light source may be specularly reflected by part of the subject.

Figure 2 shows how specular reflection occurs on a part of the subject 9. For example, if the subject 9 is wearing glasses, light emitted from a light source may be specularly reflected by the lenses or frames of the glasses. The figure shows an example in which light from four light sources is specularly reflected on the lenses of the glasses.

As shown in Figure 2, when specular reflection occurs on the glasses of subject 9, saturation occurs due to an excessive amount of received light, and image information around the eyes of subject 9 is lost. In this case, it is difficult to completely prevent saturation from occurring, even if the light output of the light source is reduced, for example. Furthermore, if the light output of the light source is reduced too much, the brightness decreases and an image of subject 9 cannot be accurately captured. If an image of subject 9 cannot be accurately captured in this way, it becomes impossible to correctly judge the health state, wakefulness, concentration state, etc. of subject 9.

In contrast, the image generating device of this embodiment has the following configuration to suppress the effects of specular reflection and generate images easily.

The following embodiments are specifically described with reference to the drawings. Each of the embodiments described below shows a specific example of the present disclosure. The numerical values, shapes, materials, components, the arrangement and connection of the components, steps, and the order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in an independent claim showing an embodiment of the present disclosure are described as optional components. The embodiment of the present disclosure is not limited to the current independent claim, but may be expressed by other independent claims.

Note that each figure is a schematic diagram and is not necessarily a precise illustration. In addition, in each figure, the same reference numerals are used for substantially the same configurations, and duplicate explanations may be omitted or simplified.

(Embodiment 1)
[Image Generation Device]
The image generating device 1 according to the first embodiment will be described with reference to FIGS.

Figure 3 is a diagram of the image generating device 1 and the subject 9 according to the first embodiment, viewed from above. Figure 4 is a diagram of the image generating device 1, viewed from the subject 9 side. Figure 5 is a block diagram of the image generating device 1.

As shown in Figures 3 and 4, the image generating device 1 includes a first light emitting unit 10 and a second light emitting unit 20 that emit light, a light receiving unit 30 that receives the light, and a substrate 40. As shown in Figure 5, the image generating device 1 further includes a drive control unit 50 that controls the first light emitting unit 10, the second light emitting unit 20, and the light receiving unit 30, a saturation level determination unit 60, and a synthesis unit 70.

The substrate 40 is rectangular, measuring, for example, 50 mm in length and 150 mm in width. The substrate 40 is provided with a first light-emitting unit 10, a second light-emitting unit 20, a light-receiving unit 30, a drive control unit 50, a saturation level determination unit 60, and a synthesis unit 70. The drive control unit 50, the saturation level determination unit 60, and the synthesis unit 70 may be provided on an external circuit board different from the substrate 40.

The first light-emitting unit 10 and the second light-emitting unit 20 are light sources that irradiate light to the subject 9. The irradiation light L1, L2 irradiated from the first light-emitting unit 10 and the second light-emitting unit 20 is, for example, infrared light or laser light. Each of the first light-emitting unit 10 and the second light-emitting unit 20 may be composed of multiple light sources (for example, light-emitting elements). FIG. 4 shows an example in which the first light-emitting unit 10 is composed of two light sources 10a and 10b, and the second light-emitting unit 20 is composed of two light sources 20a and 20b. In addition, each of the first light-emitting unit 10 and the second light-emitting unit 20 may have a diffusion plate for diffusing and irradiating the emitted light.

The first light-emitting unit 10 and the second light-emitting unit 20 are provided at positions where they do not overlap when viewed from the subject 9, i.e., at different positions when viewed from the subject 9. Specifically, the first light-emitting unit 10 and the second light-emitting unit 20 are disposed on either side of the light-receiving unit 30, sandwiching the light-receiving unit 30, when viewed from a direction perpendicular to the substrate 40. In other words, the first light-emitting unit 10 and the second light-emitting unit 20 are disposed spaced apart from each other on the substrate 40.

The light receiving unit 30 is disposed between the first light emitting unit 10 and the second light emitting unit 20. The light receiving unit 30 receives reflected light R1 and R2, which are generated when the light emitted by the first light emitting unit 10 and the second light emitting unit 20 is irradiated on the subject 9 and reflected by the light receiving unit 30. Specifically, the light receiving unit 30 receives reflected light R1 generated when the irradiation light L1 irradiated by the first light emitting unit 10 is reflected on the subject 9, and also receives reflected light R2 generated when the irradiation light L2 irradiated by the second light emitting unit 20 is reflected on the subject 9. The light receiving unit 30 is composed of a plurality of pixels 31 arranged two-dimensionally. As the light receiving unit 30, for example, a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor is used.

The drive control unit 50 controls the first light emission period Tg1 of the first light emitter 10, the second light emission period Tg2 of the second light emitter 20, and the first light reception period Tr1 and the second light reception period Tr2 of the light receiver 30. The light emission periods and light reception periods will be described later.

The drive control unit 50 generates, at different timings, a first light emission pulse SG1 that causes the first light emitter 10 to emit light and a second light emission pulse SG2 that causes the second light emitter 20 to emit light in the same frame. The drive control unit 50 also generates, at different timings, a first light reception pulse RG1 that causes the reflected light R1 reflected by the subject 9 to be received and a second light reception pulse RG2 that causes the reflected light R2 reflected by the subject 9 to be received.

Figure 6 is a time chart showing the operation of the image generating device 1. Figure 6 shows the operation of the image generating device 1 in one frame.

As shown in FIG. 6, the drive control unit 50 generates a first light emission pulse SG1 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG1 output from the drive control unit 50.

The drive control unit 50 also generates a second light emission pulse SG2 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG1. The second light emission unit 20 irradiates the illumination light L2 toward the subject 9 in accordance with the second light emission pulse SG2 output from the drive control unit 50.

The irradiation lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg1 by the first light emission pulse SG1 and the second light emission period Tg2 by the second light emission pulse SG2 so that they are equal.

The drive control unit 50 generates a first light receiving pulse RG1 that controls the light receiving unit 30. The reflected light R1 that is irradiated from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 according to the first light receiving pulse RG1 output from the drive control unit 50, and outputs pixel information related to the first amount of received light RL1.

The drive control unit 50 also generates a second light receiving pulse RG2 that controls the light receiving unit 30 at a timing different from that of the first light receiving pulse RG1. Reflected light R2 emitted from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG2 output from the drive control unit 50, and outputs pixel information related to the second amount of received light RL2.

The reflected light R1 and R2 are pulse-wave shaped lights having a predetermined time width. The drive control unit 50 controls the first light receiving start time ts1 from the start of light irradiation by the first light emitter 10 to the start of light reception by the light receiver 30 to be equal to the second light receiving start time ts2 from the start of light irradiation by the second light emitter 20 to the start of light reception by the light receiver 30. The drive control unit 50 also controls the first light receiving period Tr1 of the light receiver 30 corresponding to the irradiation of light by the first light emitter 10 to the second light receiving period Tr2 of the light receiver 30 corresponding to the irradiation of light by the second light emitter 20 to be equal.

Figure 7 is a diagram that shows a schematic of pixel information acquired by the light receiving unit 30 of the image generating device 1. (a) of Figure 7 shows how specular reflection occurs on one part of the subject 9 due to the light emitted by the first light emitting unit 10. (b) of Figure 7 shows how specular reflection occurs on another part of the subject 9 due to the light emitted by the second light emitting unit 20. Note that (a) and (b) of Figure 7 show separate pixel information obtained from one frame, rather than two images generated from two frames. (c) of Figure 7 shows one image generated by combining two pieces of pixel information.

As described above, the first light-emitting unit 10 and the second light-emitting unit 20 are provided at different positions and emit light at different times. Therefore, the reflected light R1 and R2 reflected at the same position on the subject 9 are incident on different addresses on the light-receiving unit 30, i.e., different pixels 31.

Therefore, as shown in FIG. 7A, even if specular reflection occurs on a part of the subject 9 due to the light emitted by the first light-emitting unit 10, it is possible to obtain pixel information that could not be obtained by the first light-emitting unit 10 using the light emitted by the second light-emitting unit 20. Also, as shown in FIG. 7B, even if specular reflection occurs on another part of the subject 9 due to the light emitted by the second light-emitting unit 20, it is possible to obtain pixel information that could not be obtained by the second light-emitting unit 20 using the light emitted by the first light-emitting unit 10. In other words, even if specular reflection occurs on a part of the subject 9 due to the light emitted by one of the light-emitting units, an image without specular reflection can be generated as shown in FIG. 7C by combining the pixel information shown in FIG. 7A and FIG. 7B.

Below, we will explain the specific configuration for generating an image based on the reflected light R1 and R2 received by the light receiving unit 30.

Figure 8 is a diagram showing pixel signals output from the light receiving unit 30 of the image generating device 1. The light receiving unit 30 receives reflected light R1, R2 from each of a plurality of pixels 31 arranged two-dimensionally, and outputs a first pixel signal s1 containing information on the first amount of received light RL1, and a second pixel signal s2 containing information on the second amount of received light RL2. For ease of understanding, the following description focuses on one pixel 31.

The light receiving unit 30 has multiple packets for reading out the signal charge stored in the pixel 31. Figure 8 shows an example in which the light receiving unit 30 has three packets.

The light receiving unit 30 reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in response to the emission of the first light emitter 10 into a first packet p1, and outputs a first pixel signal s1 including information on the first amount of received light RL1 corresponding to the signal charge. The light receiving unit 30 also reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in response to the emission of the second light emitter 20 into a second packet p2, and outputs a second pixel signal s2 including information on the second amount of received light RL2 corresponding to the signal charge.

The first pixel signal s1 and the second pixel signal s2 output from the light receiving unit 30 are vertically and horizontally transferred, and are then converted from analog data to digital data, and are then output to the saturation level determination unit 60. Note that the conversion from analog data to digital data does not need to be performed by the light receiving unit 30, and may be performed outside the light receiving unit 30.

The saturation level determination unit 60 shown in FIG. 5 determines whether the first amount of received light RL1 and the second amount of received light RL2 exceed a predetermined level. The predetermined level is set in advance based on the saturation level TH when the signal charge accumulated in the pixel 31 is saturated. For example, the predetermined level is set to 90% or more and 100% or less when the signal charge accumulated in the pixel 31 is saturated. Hereinafter, the predetermined level may be referred to as the saturation level TH.

For example, the saturation level determination unit 60 determines whether the first pixel signal s1 output from the pixel of the light receiving unit 30 in response to the irradiation of light from the first light emitting unit 10 exceeds the saturation level TH. The saturation level determination unit 60 also determines whether the second pixel signal s2 output from the pixel of the light receiving unit 30 in response to the irradiation of light from the second light emitting unit 20 exceeds the saturation level TH. The saturation level TH is set to one and the same value, but is not limited to this. The saturation level TH may be set to different values when comparing with the first pixel signal s1 and when comparing with the second pixel signal s2. The determination result determined by the saturation level determination unit 60 is output to the synthesis unit 70.

Figure 9 is a table showing the judgment results of the saturation level judgment unit 60 and the synthesis process in the synthesis unit 70.

The synthesis unit 70 synthesizes the first pixel signal s1 and the second pixel signal s2 based on the judgment result of the saturation level judgment unit 60 to generate a synthetic signal s10.

For example, if saturation does not occur in either the first pixel signal s1 or the second pixel signal s2, the synthesis unit 70 generates the synthesis signal s10 as follows:

As shown in FIG. 9 (1), when the first amount of received light RL1 and the second amount of received light RL2 are both smaller than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a first combination coefficient. The first combination coefficient is a coefficient that combines the first amount of received light RL1 as half and the second amount of received light RL2 as half. In other words, when the first amount of received light RL1 and the second amount of received light RL2 are both smaller than the saturation level TH, the combiner 70 uses half each of the first pixel signal s1 and the second pixel signal s2 to generate a combined signal s10.

Also, for example, when saturation occurs in at least one of the first pixel signal s1 and the second pixel signal s2, i.e., when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal greater than a predetermined level, the synthesis unit 70 generates the synthesis signal s10 without using the pixel signal greater than the predetermined level.

Specifically, when only the first amount of received light RL1 is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with the second combination coefficient. The second combination coefficient is a coefficient for combining the first amount of received light RL1 as 0. In other words, when only the first amount of received light RL1 is greater than the saturation level TH, the combiner 70 generates a combined signal s10 using the second pixel signal s2 without using the first pixel signal s1 (see (2) in FIG. 9).

When only the second amount of received light RL2 is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a third combination coefficient. The third combination coefficient is a coefficient for combining the signals while regarding the second amount of received light RL2 as 0. In other words, when only the second amount of received light RL2 is greater than the saturation level TH, the combiner 70 generates a combined signal s10 using the first pixel signal s1 without using the second pixel signal s2 (see (3) in FIG. 9).

In this way, in the image generating device 1, when the irradiation light L1 irradiated by the first light-emitting unit 10 is specularly reflected by the subject 9, the composite signal s10 is generated using information on the second amount of received light RL2 that is not affected by specular reflection, without using information on the first amount of received light RL1. Also, in the image generating device 1, when the irradiation light L2 irradiated by the second light-emitting unit 20 is specularly reflected by the subject 9, the composite signal s10 is generated using the first amount of received light RL1 that is not affected by specular reflection, without using information on the second amount of received light RL2.

When an excessive amount of light is irradiated onto the subject 9, the first amount of received light RL1 and the second amount of received light RL2 both become greater than the saturation level TH. In this case, the synthesis unit 70 may output an abnormal value. By setting an abnormal value, it is possible to notify that an abnormality has occurred in the image.

In this way, the synthesis unit 70 synthesizes the multiple pixel signals output from the pixels 31 based on the determination result of the saturation level determination unit 60, and outputs a synthesis signal s10. The image generating device 1 generates an image of the subject 9 based on the multiple synthesis signals s10 output from the multiple pixels 31.

According to this image generating device 1, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a predetermined level, the composite signal s10 is generated without using the pixel signal that is greater than the predetermined level. This makes it possible to easily generate an image by suppressing the effects of specular reflection.

In the above, an example is shown in which four light sources are arranged outside the light receiving unit 30, but the arrangement of the first light emitting unit 10 and the second light emitting unit 20 is not limited to this.

Figure 10 is a diagram showing another example of the arrangement of the first light-emitting unit 10, the second light-emitting unit 20, and the light-receiving unit 30 of the image generating device 1.

(a) of FIG. 10 shows an example in which the first light-emitting unit 10 and the second light-emitting unit 20 are arranged on both left-right outer sides of the light-receiving unit 30. (b) of FIG. 10 shows an example in which the first light-emitting unit 10 and the second light-emitting unit 20 are arranged on both top-bottom outer sides of the light-receiving unit 30. (c) of FIG. 10 shows an example in which the first light-emitting unit 10 and the second light-emitting unit 20 are arranged on both diagonal outer sides of the light-receiving unit 30. The first light-emitting unit 10 and the second light-emitting unit 20 may be arranged in any manner as long as they are spaced apart.

(Embodiment 2)
[Image Generation Device]
An image generating device 1A according to a second embodiment will be described with reference to Fig. 11 and Fig. 12. In the second embodiment, an example will be described in which the image generating device 1A further includes a correction unit 80 and a distance calculation unit 90. Descriptions of the same configuration as in the first embodiment may be omitted.

The image generating device shown below not only generates images, but also has the function of measuring distance. The image generating device shown below may be used as a distance measuring device.

Figure 11 is a block diagram of an image generating device 1A according to the second embodiment. Figure 12 is a time chart showing the operation of the image generating device 1A. Figure 12 shows the operation of the image generating device 1A in one frame.

As shown in FIG. 11, the image generating device 1A of the second embodiment also includes a first light emitting unit 10, a second light emitting unit 20, a light receiving unit 30, a drive control unit 50, a saturation level determination unit 60, and a synthesis unit 70. The image generating device 1A also includes a correction unit 80 that corrects the information on the amount of received light, and a distance calculation unit 90 that calculates the distance from the image generating device 1A to the subject 9.

As shown in FIG. 12, the drive control unit 50 generates a first light emission pulse SG3 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG3 output from the drive control unit 50.

The drive control unit 50 also generates a second light emission pulse SG4 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG3. The second light emission unit 20 irradiates the illumination light L2 toward the subject 9 in accordance with the second light emission pulse SG4 output from the drive control unit 50.

The irradiation lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg3 by the first light emission pulse SG3 and the second light emission period Tg4 by the second light emission pulse SG4 to be equal.

The drive control unit 50 generates a first light receiving pulse RG3 that controls the light receiving unit 30. The reflected light R1 emitted from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 according to the first light receiving pulse RG3 output from the drive control unit 50, and outputs pixel information related to the first amount of received light RL3.

The drive control unit 50 also generates a second light receiving pulse RG4 that controls the light receiving unit 30 at a timing different from that of the first light receiving pulse RG3. The reflected light R2 that is irradiated from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG4 output from the drive control unit 50, and outputs pixel information related to the second amount of received light RL4.

The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light reception start time ts3 from the start of light irradiation by the first light emitter 10 to the start of light reception by the light receiver 30, and the second light reception start time ts4 from the start of light irradiation by the second light emitter 20 to the start of light reception by the light receiver 30, so that they are equal. The drive control unit 50 also controls the first light reception period Tr3 of the light receiver 30 corresponding to the irradiation of light by the first light emitter 10 and the second light reception period Tr4 of the light receiver 30 corresponding to the irradiation of light by the second light emitter 20, so that they are different. Specifically, the drive control unit 50 controls the first light reception period Tr3 to be 1/2 of the second light reception period Tr4.

Below, we will explain the specific configuration for generating an image based on the reflected light R1 and R2 received by the light receiving unit 30 and for determining the distance to the subject 9.

The saturation level determination unit 60 shown in FIG. 11 determines whether the first amount of received light RL3 and the second amount of received light RL4 exceed a predetermined level. The determination result determined by the saturation level determination unit 60 is output to the correction unit 80.

The correction unit 80 corrects the information on the first amount of received light RL3, for example, when it is determined that the pixel signal indicating the information on the second amount of received light RL4 is greater than the saturation level TH. The reason for correcting the information on the first amount of received light RL3 is that the first light receiving period Tr3 is different from the second light receiving period Tr4. When performing the correction, the correction unit 80 corrects the information on the first amount of received light RL3 based on the amount of received light caused by the difference between the first light receiving period Tr3 and the second light receiving period Tr4. For example, the correction unit 80 corrects the information on the first amount of received light RL3 using the ratio of the amount of received light of the surrounding pixels that is smaller than the saturation level TH (second amount of received light of the surrounding pixels/first amount of received light of the surrounding pixels). The amount of received light of the surrounding pixels may be the amount of received light of the pixel closest to the pixel 31, or may be the average value of the amount of received light of multiple pixels located around the pixel 31.

The information on the first amount of received light RL3 after correction by the correction unit 80 is expressed by the following (Equation 1).

The correction unit 80 outputs information on the corrected first received light amount RL3 to the synthesis unit 70 and the distance calculation unit 90.

The synthesis unit 70 synthesizes the first pixel signal s1 and the second pixel signal s2 using information on the first amount of received light RL3 after correction by the correction unit 80. Specifically, when only the second amount of received light RL4 is greater than the saturation level TH, the synthesis unit 70 generates a synthesis signal s10 using the first pixel signal s1 including information on the corrected first amount of received light RL3 without using the second pixel signal s2 including information on the second amount of received light RL4.

When the first amount of received light RL3 and the second amount of received light RL4 are both smaller than the saturation level TH, and when only the first amount of received light RL3 is greater than the saturation level TH, the synthesis unit 70 generates the synthesis signal s10 in the same manner as in embodiment 1.

Next, we will explain the calculation process performed by the distance calculation unit 90.

The distance calculation unit 90 can calculate the distance to the subject 9 when the first amount of received light RL3 and the second amount of received light RL4 are both smaller than the saturation level TH. For example, the distance calculation unit 90 can calculate the distance to the subject 9 using the first amount of received light RL3 and the second amount of received light RL4. If the distance to the subject 9 is L, the light emission pulse width is Tn, and the speed of light is c (approximately 299,792,458 m/s), the distance L is expressed by the following (Equation 2).

In this image generating device 1A, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a predetermined level (saturation level TH), the composite signal s10 is generated without using the pixel signal that is greater than the predetermined level. This makes it possible to easily generate an image by suppressing the effects of specular reflection. Furthermore, with the image generating device 1A, when the first amount of received light RL3 and the second amount of received light RL4 are both smaller than the saturation level TH, the distance L to the subject 9 can be easily measured.

[First Modification of the Second Embodiment]
An image generating device 1B according to a first modification of the second embodiment will be described with reference to Fig. 13. In the first modification of the second embodiment, an example will be described in which the first light-receiving start time ts5 is delayed from the first light-receiving start time ts3 of the second embodiment.

Figure 13 is a time chart showing the operation of image generating device 1B. Figure 13 shows the operation of image generating device 1B in one frame.

As shown in FIG. 13, the drive control unit 50 generates a first light emission pulse SG5 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG5 output from the drive control unit 50.

The drive control unit 50 also generates a second light emission pulse SG6 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG5. The second light emission unit 20 irradiates the illumination light L2 toward the subject 9 in accordance with the second light emission pulse SG6 output from the drive control unit 50.

The irradiation lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg5 by the first light emission pulse SG5 and the second light emission period Tg6 by the second light emission pulse SG6 to be equal.

The drive control unit 50 generates a first light receiving pulse RG5 that controls the light receiving unit 30. The reflected light R1 emitted from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 according to the first light receiving pulse RG5 output from the drive control unit 50, and outputs pixel information related to the first amount of received light RL5.

The drive control unit 50 also generates a second light receiving pulse RG6 that controls the light receiving unit 30 at a different timing than the first light receiving pulse RG5. Reflected light R2 emitted from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG6 output from the drive control unit 50, and outputs pixel information related to the second amount of received light RL6.

The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light-receiving start time ts5 from the start of light irradiation by the first light-emitting unit 10 to the start of light reception by the light-receiving unit 30 so that it is different from the second light-receiving start time ts6 from the start of light irradiation by the second light-emitting unit 20 to the start of light reception by the light-receiving unit 30. Specifically, the drive control unit 50 controls the first light-receiving start time ts5 to be the sum of the second light-receiving start time ts6 and the first light-emitting period Tg5. The drive control unit 50 also controls the first light-receiving period Tr5 of the light-receiving unit 30 corresponding to the irradiation of light by the first light-emitting unit 10 and the second light-receiving period Tr6 of the light-receiving unit 30 corresponding to the irradiation of light by the second light-emitting unit 20 so that they are different. Specifically, the drive control unit 50 controls the first light receiving period Tr5 to be half the second light receiving period Tr6.

The saturation level determination unit 60 determines whether the first amount of received light RL5 and the second amount of received light RL6 exceed a predetermined level. The result of the determination made by the saturation level determination unit 60 is output to the correction unit 80.

The correction unit 80 corrects the information on the first amount of received light RL5, for example, when it is determined that the pixel signal indicating the information on the second amount of received light RL6 is greater than the saturation level TH. The information on the first amount of received light RL5 is corrected because the first light-receiving start time ts5 is different from the second light-receiving start time ts6. When making the correction, the correction unit 80 corrects the information on the first amount of received light RL5 based on the amount of received light caused by the difference between the first light-receiving start time ts5 and the second light-receiving start time ts6. For example, the correction unit 80 corrects the information on the first amount of received light RL5 using the ratio of the amount of received light of the surrounding pixels that is smaller than the saturation level TH (second amount of received light of the surrounding pixels/first amount of received light of the surrounding pixels).

The information on the first amount of received light RL5 after correction by the correction unit 80 is expressed by the following (Equation 3).

The correction unit 80 outputs information on the corrected first received light amount RL5 to the synthesis unit 70 and the distance calculation unit 90.

The synthesis unit 70 synthesizes the first pixel signal s1 and the second pixel signal s2 using information on the first amount of received light RL5 after correction by the correction unit 80. Specifically, when only the second amount of received light RL6 is greater than the saturation level TH, the synthesis unit 70 generates a synthesis signal s10 using the first pixel signal s1 including information on the corrected first amount of received light RL5 without using the second pixel signal s2 including information on the second amount of received light RL6.

When the first amount of received light RL5 and the second amount of received light RL6 are both smaller than the saturation level TH, or when only the first amount of received light RL5 is greater than the saturation level TH, the composite signal s10 is generated in the same manner as in embodiment 1.

The distance calculation unit 90 can calculate the distance to the subject 9 when the first amount of received light RL5 and the second amount of received light RL6 are both smaller than the saturation level TH. For example, the distance calculation unit 90 can calculate the distance to the subject 9 using the first amount of received light RL5 and the second amount of received light RL6. If the distance to the subject 9 is L, the light emission pulse width is Tn, and the speed of light is c, then the distance L is expressed by the following (Equation 4).

In this image generating device 1B, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a predetermined level (saturation level TH), a composite signal s10 is generated without using a pixel signal that is greater than the predetermined level. This makes it possible to easily generate an image by suppressing the effects of specular reflection. Furthermore, with the image generating device 1B, when the first amount of received light RL5 and the second amount of received light RL6 are both smaller than the saturation level TH, the distance L to the subject 9 can be easily measured.

[Modification 2 of the Second Embodiment]
An image generating device 1C according to a second modification of the second embodiment will be described with reference to Fig. 14. In the second modification of the second embodiment, an example in which the second light receiving period Tr8 is set to 1/2 of the first light receiving period Tr7 will be described.

Figure 14 is a time chart showing the operation of image generating device 1C. Figure 14 shows the operation of image generating device 1C in one frame.

As shown in FIG. 14, the drive control unit 50 generates a first light emission pulse SG7 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG7 output from the drive control unit 50.

The drive control unit 50 also generates a second light emission pulse SG8 that controls the second light emission unit 20 at a timing different from the first light emission pulse SG7. The second light emission unit 20 irradiates the illumination light L2 toward the subject 9 in accordance with the second light emission pulse SG8 output from the drive control unit 50.

The irradiation lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg7 by the first light emission pulse SG7 and the second light emission period Tg8 by the second light emission pulse SG8 so that they are equal.

The drive control unit 50 generates a first light receiving pulse RG7 that controls the light receiving unit 30. The reflected light R1 emitted from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 according to the first light receiving pulse RG7 output from the drive control unit 50, and outputs pixel information related to the first amount of received light RL7.

The drive control unit 50 also generates a second light receiving pulse RG8 that controls the light receiving unit 30 at a different timing than the first light receiving pulse RG7. The reflected light R2 that is irradiated from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG8 output from the drive control unit 50, and outputs pixel information related to the second amount of received light RL8.

The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light reception start time ts7 from the start of light irradiation by the first light emitter 10 to the start of light reception by the light receiver 30 to be equal to the second light reception start time ts8 from the start of light irradiation by the second light emitter 20 to the start of light reception by the light receiver 30. The drive control unit 50 also controls the first light reception period Tr7 of the light receiver 30 corresponding to the light irradiation of the first light emitter 10 and the second light reception period Tr8 of the light receiver 30 corresponding to the light irradiation of the second light emitter 20 to be different. Specifically, the drive control unit 50 controls the second light reception period Tr8 to be 1/2 of the first light reception period Tr7.

The saturation level determination unit 60 determines whether the first amount of received light RL7 and the second amount of received light RL8 exceed a predetermined level. The result of the determination made by the saturation level determination unit 60 is output to the correction unit 80.

For example, the correction unit 80 corrects the information on the second amount of received light RL8 when it is determined that the pixel signal indicating the information on the first amount of received light RL7 is greater than the saturation level TH. The reason for correcting the information on the second amount of received light RL8 is that the second light receiving period Tr8 is different from the first light receiving period Tr7. When making the correction, the correction unit 80 corrects the information on the second amount of received light RL8 based on the amount of received light resulting from the difference between the first light receiving period Tr7 and the second light receiving period Tr8. For example, the correction unit 80 corrects the information on the second amount of received light RL8 using the ratio of the amounts of received light of the surrounding pixels that is smaller than the saturation level TH (first amount of received light of the surrounding pixel/second amount of received light of the surrounding pixel).

The information on the second amount of received light RL8 after correction by the correction unit 80 is expressed by the following (Equation 5).

The correction unit 80 outputs information on the corrected second received light amount RL8 to the synthesis unit 70 and the distance calculation unit 90.

The synthesis unit 70 synthesizes the first pixel signal s1 and the second pixel signal s2 using information on the second amount of received light RL8 after correction by the correction unit 80. Specifically, when only the first amount of received light RL7 is greater than the saturation level TH, the synthesis unit 70 generates a synthesis signal s10 using the second pixel signal s2 including information on the corrected second amount of received light RL8 without using the first pixel signal s1 including information on the first amount of received light RL7.

When the first amount of received light RL7 and the second amount of received light RL8 are both smaller than the saturation level TH, and when only the second amount of received light RL8 is greater than the saturation level TH, the composite signal s10 is generated in the same manner as in embodiment 1.

The distance calculation unit 90 can calculate the distance to the subject 9 when the first amount of received light RL7 and the second amount of received light RL8 are both smaller than the saturation level TH. For example, the distance calculation unit 90 can calculate the distance to the subject 9 using the first amount of received light RL7 and the second amount of received light RL8. If the distance to the subject 9 is L, the light emission pulse width is Tn, and the speed of light is c, then the distance L is expressed by the following (Equation 6).

In this image generating device 1C, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a predetermined level (saturation level TH), a composite signal s10 is generated without using a pixel signal that is greater than the predetermined level. This makes it possible to easily generate an image by suppressing the effects of specular reflection. Furthermore, with the image generating device 1C, when the first amount of received light RL7 and the second amount of received light RL8 are both smaller than the saturation level TH, the distance L to the subject 9 can be easily measured.

[Third Modification of the Second Embodiment]
An image generating device 1D according to a third modification of the second embodiment will be described with reference to Fig. 15. In the third modification of the second embodiment, an example in which the second light-receiving start time ts10 is delayed from the second light-receiving start time ts8 of the second modification will be described.

Figure 15 is a time chart showing the operation of image generating device 1D. Figure 15 shows the operation of image generating device 1D in one frame.

As shown in FIG. 15, the drive control unit 50 generates a first light emission pulse SG9 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG9 output from the drive control unit 50.

The drive control unit 50 also generates a second light emission pulse SG10 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG9. The second light emission unit 20 irradiates the illumination light L2 toward the subject 9 in accordance with the second light emission pulse SG10 output from the drive control unit 50.

The irradiation lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg9 caused by the first light emission pulse SG9 and the second light emission period Tg10 caused by the second light emission pulse SG10 so that they are equal.

The drive control unit 50 generates a first light receiving pulse RG9 that controls the light receiving unit 30. The reflected light R1 emitted from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 according to the first light receiving pulse RG9 output from the drive control unit 50, and outputs pixel information related to the first amount of received light RL9.

The drive control unit 50 also generates a second light receiving pulse RG10 that controls the light receiving unit 30 at a timing different from that of the first light receiving pulse RG9. Reflected light R2 emitted from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG10 output from the drive control unit 50, and outputs pixel information related to the second amount of received light RL10.

The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the second light-receiving start time ts10, which is from the start of light irradiation by the second light-emitting unit 20 to the start of light reception by the light-receiving unit 30, so that it is different from the first light-receiving start time ts9. Specifically, the drive control unit 50 controls the second light-receiving start time ts10 to be the sum of the first light-receiving start time ts9 and the second light-emitting period Tg10. The drive control unit 50 also controls the first light-receiving period Tr9 of the light-receiving unit 30 corresponding to the irradiation of light by the first light-emitting unit 10 and the second light-receiving period Tr10 of the light-receiving unit 30 corresponding to the irradiation of light by the second light-emitting unit 20 so that they are different. Specifically, the drive control unit 50 controls the second light-receiving period Tr10 to be 1/2 of the first light-receiving period Tr9.

The saturation level determination unit 60 determines whether the first amount of received light RL9 and the second amount of received light RL10 exceed a predetermined level. The result of the determination made by the saturation level determination unit 60 is output to the correction unit 80.

For example, the correction unit 80 corrects the information on the second amount of received light RL10 when it is determined that the pixel signal indicating the information on the first amount of received light RL9 is greater than the saturation level TH. The reason for correcting the information on the second amount of received light RL10 is that the first light receiving start time ts9 and the second light receiving start time ts10 are different, and the first light receiving period Tr9 and the second light receiving period Tr10 are different. When performing the correction, the correction unit 80 corrects the information on the second amount of received light RL10 based on the amount of received light caused by the difference between the first light receiving start time ts9 and the second light receiving start time ts10, and the first light receiving period Tr9 and the second light receiving period Tr10. For example, the correction unit 80 corrects the information on the second amount of received light RL10 using the ratio of the amount of received light of the surrounding pixels that is smaller than the saturation level TH (first amount of received light of the surrounding pixels/second amount of received light of the surrounding pixels).

The information on the second amount of received light RL10 after correction by the correction unit 80 is expressed by the following (Equation 7).

The correction unit 80 outputs information on the corrected second received light amount RL10 to the synthesis unit 70 and the distance calculation unit 90.

The synthesis unit 70 synthesizes the first pixel signal s1 and the second pixel signal s2 using information on the second amount of received light RL10 after correction by the correction unit 80. Specifically, when only the first amount of received light RL9 is greater than the saturation level TH, the synthesis unit 70 generates a synthesis signal s10 using the second pixel signal s2 including information on the corrected second amount of received light RL10 without using the first pixel signal s1 including information on the first amount of received light RL9.

When the first amount of received light RL9 and the second amount of received light RL10 are both smaller than the saturation level TH, and when only the second amount of received light RL10 is greater than the saturation level TH, a composite signal s10 is generated in the same manner as in embodiment 1.

The distance calculation unit 90 can calculate the distance to the subject 9 when the first amount of received light RL9 and the second amount of received light RL10 are both smaller than the saturation level TH. For example, the distance calculation unit 90 can calculate the distance to the subject 9 using the first amount of received light RL9 and the second amount of received light RL10. If the distance to the subject 9 is L, the light emission pulse width is Tn, and the speed of light is c, then the distance L is expressed by the following (Equation 8).

In this image generating device 1D, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than a predetermined level (saturation level TH), a composite signal s10 is generated without using the pixel signal that is greater than the predetermined level. This makes it possible to easily generate an image by suppressing the effects of specular reflection. Furthermore, with the image generating device 1D, when both the first amount of received light RL9 and the second amount of received light RL10 are smaller than the saturation level TH, the distance L to the subject 9 can be easily measured.

(Embodiment 3)
[Image Generation Device]
An image generating device 1E according to the third embodiment will be described with reference to Figs. 16 to 18. In the third embodiment, an example will be described in which two first light receiving pulses RG11 and RG11a and two first light receiving periods Tr11 and Tr11a are set. Note that the description of the same configuration as in the first and second embodiments may be omitted.

Figure 16 is a block diagram of an image generating device 1E according to embodiment 3. Figure 17 is a time chart showing the operation of image generating device 1E. Figure 17 shows the operation of image generating device 1E in one frame.

As shown in FIG. 16, the image generating device 1E of the third embodiment also includes a first light emitting unit 10, a second light emitting unit 20, a light receiving unit 30, a drive control unit 50, a saturation level determining unit 60, a synthesis unit 70, and a correction unit 80. The image generating device 1E also includes a distance calculation unit 90 that calculates the distance from the image generating device 1E to the subject 9. The image generating device 1E of the third embodiment not only generates an image, but also has a function of measuring distance.

As shown in FIG. 17, the drive control unit 50 generates a first light emission pulse SG11 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG11 output from the drive control unit 50.

The drive control unit 50 also generates a second light emission pulse SG12 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG11. The second light emission unit 20 irradiates the illumination light L2 toward the subject 9 in accordance with the second light emission pulse SG12 output from the drive control unit 50.

The irradiation lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg11 caused by the first light emission pulse SG11 and the second light emission period Tg12 caused by the second light emission pulse SG12 so that they are equal.

The drive control unit 50 generates two first light receiving pulses RG11 and RG11a that control the light receiving unit 30. The reflected light R1 that is irradiated from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 according to the first light receiving pulses RG11 and RG11a output from the drive control unit 50, and outputs pixel information related to the first received light amounts RL11 and RL11a.

The light receiving unit 30 reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the first light receiving pulse RG11 into a first packet p1, and also reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the first light receiving pulse RG11a into a fourth packet (not shown), and outputs a first pixel signal s1 including information on the first light receiving amounts RL11 and RL11a corresponding to the above signal charges.

The drive control unit 50 also generates a second light receiving pulse RG12 that controls the light receiving unit 30 at a timing different from the first light receiving pulses RG11 and RG11a. Reflected light R2 emitted from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulse RG12 output from the drive control unit 50, and outputs pixel information related to the second amount of received light RL12.

The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light receiving period Tr11 corresponding to the first light receiving pulse RG11 and the first light receiving period Tr11a corresponding to the first light receiving pulse RG11a so that they are equal. The drive control unit 50 also controls the first light receiving start time ts11 from the start of light irradiation by the first light emitting unit 10 to the start of light reception by the light receiving unit 30 and the second light receiving start time ts12 from the start of light irradiation by the second light emitting unit 20 to the start of light reception by the light receiving unit 30 so that they are equal. The first light receiving start time ts11a by the first light receiving pulse RG11a is controlled to be the sum of the first light receiving start time ts11 by the first light receiving pulse RG11 and the first light receiving period Tr11. In addition, the drive control unit 50 controls the period obtained by adding up the first light receiving periods Tr11 and Tr11a to be equal to the second light receiving period Tr12 of the light receiving unit 30 corresponding to the irradiation of light by the second light emitting unit 20.

The saturation level determination unit 60 determines whether the first amount of received light RL11, RL11a, and the second amount of received light RL12 exceed a predetermined level. The result of the determination made by the saturation level determination unit 60 is output to the synthesis unit 70.

Figure 18 is a table showing the judgment results of the saturation level judgment unit 60 and the synthesis process in the synthesis unit 70.

The synthesis unit 70 synthesizes the first pixel signal s1 and the second pixel signal s2 based on the judgment result of the saturation level judgment unit 60 to generate a synthetic signal s10.

For example, if saturation does not occur in either the first pixel signal s1 or the second pixel signal s2, the synthesis unit 70 generates the synthesis signal s10 as follows:

As shown in FIG. 18 (1), when the sum of the first received light amounts RL11 and RL11a and the second received light amount RL12 are both smaller than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a first combination coefficient. The first combination coefficient is a coefficient that combines the first pixel signal s1 and the second pixel signal s2 by regarding the sum of the first received light amounts RL11 and RL11a as half and the second received light amount RL12 as half. In other words, when the sum of the first received light amounts RL11 and RL11a and the second received light amount RL12 are both smaller than the saturation level TH, the combiner 70 uses half each of the first pixel signal s1 and the second pixel signal s2 to generate a combined signal s10.

Also, for example, when saturation occurs in at least one of the first pixel signal s1 and the second pixel signal s2, i.e., when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal greater than a predetermined level, the synthesis unit 70 generates the synthesis signal s10 without using the pixel signal greater than the predetermined level.

Specifically, when only the sum of the first amounts of received light RL11 and RL11a is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with the second combination coefficient. The second combination coefficient is a coefficient that combines the first amounts of received light RL11 and RL11a assuming that the sum is 0. In other words, when only the sum of the first amounts of received light RL11 and RL11a is greater than the saturation level TH, the combiner 70 generates the combined signal s10 using the second pixel signal s2 without using the first pixel signal s1 (see (2) in FIG. 18).

When only the second amount of received light RL12 is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a third combination coefficient. The third combination coefficient is a coefficient for combining the signals while regarding the second amount of received light RL12 as 0. In other words, when only the second amount of received light RL12 is greater than the saturation level TH, the combiner 70 generates a combined signal s10 using the first pixel signal s1 without using the second pixel signal s2 (see (3) in FIG. 18).

In this way, in the image generating device 1E, when the irradiation light L1 irradiated by the first light-emitting unit 10 is specularly reflected by the subject 9, the composite signal s10 is generated using information on the second amount of received light RL12 that is not affected by specular reflection, without using information on the first amount of received light RL11, RL11a. Also, in the image generating device 1E, when the irradiation light L2 irradiated by the second light-emitting unit 20 is specularly reflected by the subject 9, the composite signal s10 is generated using information on the first amount of received light RL11, RL11a that is not affected by specular reflection, without using information on the second amount of received light RL12.

If too much light is irradiated onto the subject 9, the sum of the first received light amounts RL11 and RL11a and the second received light amount RL12 may both exceed the saturation level TH. In this case, the synthesis unit 70 may output an abnormal value. By setting an abnormal value, it is possible to notify that an abnormality has occurred in the image.

In this way, the synthesis unit 70 synthesizes the multiple pixel signals output from the pixels 31 based on the determination result of the saturation level determination unit 60, and outputs a synthesis signal s10. The image generating device 1E generates an image of the subject 9 based on the multiple synthesis signals s10 output from the multiple pixels 31.

In addition, in the image generating device 1E of embodiment 3, it is possible to obtain the distance L to the subject 9 except when both of the first light receiving amounts RL11 and RL11a exceed the saturation level TH.

For example, when only the second amount of received light RL12 is greater than the saturation level TH, the distance calculation unit 90 calculates the distance L to the subject 9 using the following formula (9). When only the first amount of received light RL11 is greater than the saturation level TH, the distance calculation unit 90 calculates the distance L to the subject 9 using the following formula (10). When only the first amount of received light RL11a is greater than the saturation level TH, the distance calculation unit 90 calculates the distance L to the subject 9 using the following formula (11).

This image generating device 1E can also easily generate an image by suppressing the effects of specular reflection. Furthermore, this image generating device 1E can easily measure the distance L to the subject 9, except when both of the first light receiving amounts RL11 and RL11a exceed the saturation level TH.

[Modification of the third embodiment]
An image generating device 1F according to a modification of the third embodiment will be described with reference to Fig. 19 and Fig. 20. In the modification of the third embodiment, an example in which two second light receiving pulses RG14 and RG14a and two second light receiving periods Tr14 and Tr14a are set will be described.

Figure 19 is a time chart showing the operation of image generating device 1F. Figure 19 shows the operation of image generating device 1F in one frame.

As shown in FIG. 19, the drive control unit 50 generates a first light emission pulse SG13 that controls the first light emission unit 10. The first light emission unit 10 irradiates the subject 9 with irradiation light L1 in accordance with the first light emission pulse SG13 output from the drive control unit 50.

The drive control unit 50 also generates a second light emission pulse SG14 that controls the second light emission unit 20 at a timing different from that of the first light emission pulse SG13. The second light emission unit 20 irradiates the illumination light L2 toward the subject 9 in accordance with the second light emission pulse SG14 output from the drive control unit 50.

The irradiation lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg13 caused by the first light emission pulse SG13 and the second light emission period Tg14 caused by the second light emission pulse SG14 so that they are equal.

The drive control unit 50 generates two first light receiving pulses RG13 and RG13a that control the light receiving unit 30. The reflected light R1 that is irradiated from the first light emitting unit 10 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R1 according to the first light receiving pulses RG13 and RG13a output from the drive control unit 50, and outputs pixel information related to the first received light amounts RL13 and RL13a.

The light receiving unit 30 reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the first light receiving pulse RG13 into a first packet p1, and also reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the first light receiving pulse RG13a into a fourth packet (not shown), and outputs a first pixel signal s1 including information on the first light receiving amounts RL13 and RL13a corresponding to the above signal charges.

The drive control unit 50 also generates second light receiving pulses RG14 and RG14a that control the light receiving unit 30 at a timing different from that of the first light receiving pulses RG13 and RG13a. Reflected light R2 emitted from the second light emitting unit 20 and reflected by the subject 9 is incident on the light receiving unit 30. The light receiving unit 30 receives the reflected light R2 in accordance with the second light receiving pulses RG14 and RG14a output from the drive control unit 50, and outputs pixel information related to the received second light amounts RL14 and RL14a.

The light receiving unit 30 reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the second light receiving pulse RG14 into a second packet p2, and also reads out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in accordance with the second light receiving pulse RG14a into a fifth packet (not shown), and outputs a second pixel signal s2 including information on the second light receiving amounts RL14 and RL14a corresponding to the above signal charges.

The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light receiving period Tr13 corresponding to the first light receiving pulse RG13, the first light receiving period Tr13a corresponding to the first light receiving pulse RG13a, the second light receiving period Tr14 corresponding to the second light receiving pulse RG14, and the second light receiving period Tr14a corresponding to the second light receiving pulse RG14a so that they are equal. The first light receiving start time ts13a by the first light receiving pulse RG13a is controlled to be the sum of the first light receiving start time ts13 by the first light receiving pulse RG13 and the first light receiving period Tr13. In addition, the drive control unit 50 controls the first light receiving start time ts13 by the first light receiving pulse RG13 and the second light receiving start time ts14 by the second light receiving pulse RG14 to be equal. The second light receiving start time ts14a by the second light receiving pulse RG14a is controlled to be the sum of the second light receiving start time ts14 and the second light receiving period Tr14.

The saturation level determination unit 60 determines whether the first amount of received light RL13, RL13a and the second amount of received light RL14, RL14a exceed a predetermined level. The result of the determination made by the saturation level determination unit 60 is output to the synthesis unit 70.

Figure 20 is a table showing the judgment results of the saturation level judgment unit 60 and the synthesis process in the synthesis unit 70.

The synthesis unit 70 synthesizes the first pixel signal s1 and the second pixel signal s2 based on the judgment result of the saturation level judgment unit 60 to generate a synthetic signal s10.

For example, if saturation does not occur in either the first pixel signal s1 or the second pixel signal s2, the synthesis unit 70 generates the synthesis signal s10 as follows:

As shown in FIG. 20 (1), when the sum of the first received light amounts RL13, RL13a and the sum of the second received light amounts RL14, RL14a are both smaller than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a first combination coefficient. The first combination coefficient is a coefficient that combines the first pixel signal s1 and the second pixel signal s2 by regarding the sum of the first received light amounts RL13, RL13a as half and the sum of the second received light amounts RL14, RL14a as half. In other words, when the sum of the first received light amounts RL13, RL13a and the sum of the second received light amounts RL14, RL14a are both smaller than the saturation level TH, the combiner 70 uses half of both the first pixel signal s1 and the second pixel signal s2 to generate a combined signal s10.

Also, for example, when saturation occurs in at least one of the first pixel signal s1 and the second pixel signal s2, i.e., when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal greater than a predetermined level, the synthesis unit 70 generates the synthesis signal s10 without using the pixel signal greater than the predetermined level.

Specifically, when only the sum of the first amounts of received light RL13 and RL13a is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with the second combination coefficient. The second combination coefficient is a coefficient that combines the first amounts of received light RL13 and RL13a assuming that the sum is 0. In other words, when only the sum of the first amounts of received light RL13 and RL13a is greater than the saturation level TH, the combiner 70 generates a combined signal s10 using the second pixel signal s2 without using the first pixel signal s1 (see (2) in FIG. 20).

When only the sum of the second amounts of received light RL14 and RL14a is greater than the saturation level TH, the combiner 70 combines the first pixel signal s1 and the second pixel signal s2 with a third combination coefficient. The third combination coefficient is a coefficient that combines the second amounts of received light RL14 and RL14a as 0. In other words, when only the second amounts of received light RL14 and RL14a are greater than the saturation level TH, the combiner 70 generates a combined signal s10 using the first pixel signal s1 without using the second pixel signal s2 (see (3) in FIG. 20).

In this way, in the image generating device 1F, when the irradiation light L1 irradiated by the first light-emitting unit 10 is specularly reflected by the subject 9, the composite signal s10 is generated using information on the second received light amounts RL14, RL14a that are not affected by specular reflection, without using information on the first received light amounts RL13, RL13a. Also, in the image generating device 1F, when the irradiation light L2 irradiated by the second light-emitting unit 20 is specularly reflected by the subject 9, the composite signal s10 is generated using information on the first received light amounts RL13, RL13a that are not affected by specular reflection, without using information on the second received light amounts RL14, RL14a.

When an excessive amount of light is irradiated onto the subject 9, the sum of the first received light amounts RL13 and RL13a and the sum of the second received light amounts RL14 and RL14a may both exceed the saturation level TH. In this case, the synthesis unit 70 may output an abnormal value. By setting an abnormal value, it is possible to notify that an abnormality has occurred in the image.

In this way, the synthesis unit 70 synthesizes the multiple pixel signals output from the pixels 31 based on the determination result of the saturation level determination unit 60, and outputs a synthesis signal s10. The image generating device 1F generates an image of the subject 9 based on the multiple synthesis signals s10 output from the multiple pixels 31.

In addition, in the image generating device 1F of the modified example of embodiment 3, it is possible to obtain the distance L to the subject 9.

For example, when only the first amount of received light RL13 is greater than the saturation level TH, the distance calculation unit 90 calculates the distance L to the subject 9 using the following (Equation 12). Note that when only the first amount of received light RL13a is greater than the saturation level TH, the distance L is calculated using the same equation.

When only the second amount of received light RL14 is greater than the saturation level TH, the distance calculation unit 90 calculates the distance L to the subject 9 using the following (Equation 13). Note that when only the second amount of received light RL14a is greater than the saturation level TH, the distance L is calculated using the same equation.

When the sum of the first received light amounts RL13 and RL13a and the sum of the second received light amounts RL14 and RL14a are both smaller than the saturation level TH, the distance calculation unit 90 calculates the distance to the subject 9 using the following (Equation 14).

In this image generating device 1F, the effects of specular reflection can be suppressed and images can be easily generated. In addition, in this image generating device 1F, the effects of specular reflection can be suppressed and the distance L to the subject 9 can be easily measured.

(Embodiment 4)
[Image Generation Device]
An image generating device 1G according to the fourth embodiment will be described with reference to Fig. 21 and Fig. 22. Note that the description of the same configuration as in the first and second embodiments may be omitted.

Figure 21 is a block diagram of an image generating device 1G according to embodiment 4.

The image generating device 1G according to the fourth embodiment includes a first light emitting unit 10, a second light emitting unit 20, a light receiving unit 30, a drive control unit 50, a saturation level determining unit 60, a synthesis unit 70, a correction unit 80, and a distance calculation unit 90.

The drive control unit 50 generates, in the same frame, a first light emission pulse SG15 emitted N times by the first light emission unit 10 and a second light emission pulse SG16 emitted M times by the second light emission unit 20 at different timings. In addition, in the same frame as the above, the drive control unit 50 generates, in different timings, a first light reception pulse RG15 that receives N times the reflected light R1 generated when the irradiation light L1 emitted from the first light emission unit 10 is reflected by the subject 9, and a second light reception pulse RG16 that receives M times the reflected light R2 generated when the irradiation light L2 emitted from the second light emission unit 20 is reflected by the subject 9. N and M are natural numbers equal to or greater than 1, and are controlled so that at least one of N and M is equal to or greater than 2.

The light receiving unit 30 has a plurality of pixels 31 arranged two-dimensionally, receives reflected light R1 in accordance with N first light receiving pulses RG15, and generates the signal charge accumulated in the pixels 31 by the light reception as N first received light amounts RL15. Similarly, the light receiving unit 30 receives reflected light R2 in accordance with M second light receiving pulses RG16, and generates the signal charge accumulated in the pixels 31 by the light reception as M second received light amounts RL16. The N first received light amounts RL15 and the M second received light amounts RL16 are converted into digital data and output.

The saturation level determination unit 60 compares the N first amounts of received light RL15 with the saturation level, and compares the M second amounts of received light RL16 with the saturation level. The saturation level to be compared with the N first amounts of received light RL15 and the saturation level to be compared with the M second amounts of received light RL16 may be set individually.

The correction unit 80 corrects the N first received light amounts RL15 and the M second received light amounts RL16 that are determined to be greater than the saturation level. For example, the correction unit 80 may correct the information on the first received light amounts RL15 or the second received light amounts RL16 using the received light amounts of surrounding pixels that are less than the saturation level. The received light amounts of the surrounding pixels may be the received light amounts of the pixels closest to pixel 31, or may be the average of the received light amounts of multiple pixels located around pixel 31.

The combining unit 70 adds up N first amounts of received light RL15 to obtain the total value of the first amounts of received light RL15. For example, when N is 1, the combining unit 70 uses one first amount of received light RL15 as the total value of the first amounts of received light RL15. The combining unit 70 also adds up M amounts of second amounts of received light RL16 to obtain the total value of the second amounts of received light RL16. For example, when M is 1, the combining unit 70 uses one second amount of received light RL16 as the total value of the second amounts of received light RL16.

When the N first received light amounts RL15 and the M second received light amounts RL16 are all smaller than the saturation level based on the comparison result of the saturation level determination unit 60, the combining unit 70 selects and combines the first combining coefficient. When at least one of the N first received light amounts RL15 is greater than the saturation level, the combining unit 70 selects and combines the second combining coefficient. When at least one of the M second received light amounts RL16 is greater than the saturation level, the combining unit 70 selects and combines the third combining coefficient.

Figure 22 is a time chart showing the operation of image generating device 1G. Figure 22 shows the case where N is 2 and M is 1.

The drive control unit 50 generates first light emission pulses SG15 and SG15a that control the first light emission unit 10. The first light emission unit 10 emits irradiation light L1 toward the subject 9 in accordance with the first light emission pulses SG15 and SG15a. The drive control unit 50 also generates a second light emission pulse SG16 that controls the second light emission unit 20 at a timing different from that of the first light emission pulses SG15 and SG15a. The second light emission unit 20 emits irradiation light L2 toward the subject 9 in accordance with the second light emission pulse SG16.

The irradiation lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg15 of the first light emission pulse SG15, the first light emission period Tg15a of the first light emission pulse SG15a, and the second light emission period Tg16 of the second light emission pulse SG16 so that they are equal.

The light receiving unit 30 receives the reflected light R1 according to the first light receiving pulse RG15 generated by the drive control unit 50, receives the reflected light R1 according to the first light receiving pulse RG15a, and receives the reflected light R2 according to the second light receiving pulse RG16.

The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls so that the first light reception start time ts15 from the start of irradiation to the first light reception pulse RG15 and the second light reception start time ts16 from the start of irradiation to the second light reception pulse RG16 are equal. The drive control unit 50 controls so that the first light reception period Tr15 and the first light reception period Tr15a are equal. The first light reception start time ts15a from the start of irradiation to the first light reception pulse RG15a is controlled to be the sum of the first light reception start time ts15 and the first light reception period Tr15. The second light reception period Tr16 is controlled to be the sum of the first light reception periods Tr15 and Tr15a.

The light receiving unit 30 generates the signal charge accumulated in the pixel 31 by receiving light in response to the first light receiving pulse RG15 as the first amount of received light RL15. The light receiving unit 30 also generates the signal charge accumulated in the pixel 31 by receiving light in response to the first light receiving pulse RG15a as the first amount of received light RL15a. The light receiving unit 30 also generates the signal charge accumulated in the pixel 31 by receiving light in response to the second light receiving pulse RG16 as the second amount of received light RL16.

The saturation level determination unit 60 determines whether the first amount of received light RL15, RL15a, and the second amount of received light RL16 exceed the saturation level.

When the first received light amounts RL15, RL15a, and the second received light amounts RL16 are all smaller than the saturation level based on the comparison result of the saturation level determination unit 60, the combining unit 70 selects and combines the first combining coefficient. When at least one of the first received light amounts RL15, RL15a is greater than the saturation level, the combining unit 70 selects and combines the second combining coefficient. When at least one of the second received light amounts RL16 is greater than the saturation level, the combining unit 70 selects and combines the third combining coefficient.

In this way, the synthesis unit 70 synthesizes the multiple pixel signals output from the pixels 31 based on the determination result of the saturation level determination unit 60, and outputs a synthesis signal s10. The image generating device 1G generates an image of the subject 9 based on the multiple synthesis signals s10 output from the multiple pixels 31.

The distance calculation unit 90 calculates the distance L to the subject 9 using the first amount of received light RL15, RL15a, and the second amount of received light RL16.

For example, if the saturation level determination unit 60 determines that only the first amount of received light RL15, RL15a, or the second amount of received light RL16 is greater than the saturation level, the distance calculation unit 90 calculates the distance L using (Equation 15) or (Equation 17). If the saturation level determination unit 60 determines that only the second amount of received light RL16 is greater than the saturation level, the distance calculation unit 90 calculates the distance L using (Equation 16). In other cases, the distance calculation unit 90 calculates the distance L using (Equation 16) or (Equation 17).

In this image generating device 1G, the effects of specular reflection can be suppressed and an image can be easily generated. In addition, in this image generating device 1G, the effects of specular reflection can be suppressed and the distance L to the subject 9 can be easily measured.

[Modification of the fourth embodiment]
An image generating device 1H according to a modification of the fourth embodiment will be described with reference to FIG.

Figure 23 is a time chart showing the operation of image generating device 1H. Figure 23 shows the case where N is 2 and M is 1.

The drive control unit 50 generates first light emission pulses SG17 and SG17a that control the first light emission unit 10. The first light emission unit 10 irradiates irradiation light L1 toward the subject 9 in accordance with the first light emission pulses SG17 and SG17a. The drive control unit 50 also generates second light emission pulses SG18 and SG18a that control the second light emission unit 20 at a timing different from that of the first light emission pulses SG17 and SG17a. The second light emission unit 20 irradiates irradiation light L2 toward the subject 9 in accordance with the second light emission pulses SG18 and SG18a.

The irradiation lights L1 and L2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light emission period Tg17 of the first light emission pulse SG17, the first light emission period Tg17a of the first light emission pulse SG17a, the second light emission period Tg18 of the second light emission pulse SG18, and the second light emission period Tg18a of the second light emission pulse SG18a so that they are equal.

The light receiving unit 30 receives the reflected light R1 according to the first light receiving pulse RG17 generated by the drive control unit 50, receives the reflected light R1 according to the first light receiving pulse RG17a, receives the reflected light R2 according to the second light receiving pulse RG18, and receives the reflected light R2 according to the second light receiving pulse RG18a.

The reflected light R1 and R2 are pulse-shaped lights having a predetermined time width. The drive control unit 50 controls the first light receiving start time ts17 from the start of irradiation to the first light receiving pulse RG17 and the second light receiving start time ts18 from the start of irradiation to the second light receiving pulse RG18 to be equal. The drive control unit 50 controls the first light receiving period Tr17, the first light receiving period Tr17a, the second light receiving period Tr18, and the second light receiving period Tr18a to be equal. The first light receiving start time ts17a from the start of irradiation to the first light receiving pulse RG17a is controlled to be the sum of the first light receiving start time ts17 and the first light receiving period Tr17. The second light reception start time ts18a from the start of irradiation to the second light reception pulse RG18a is controlled to be the sum of the second light reception start time ts18 and the second light reception period Tr18.

The light receiving unit 30 generates the signal charge accumulated in the pixel 31 by receiving light in response to the first light receiving pulse RG17 as the first amount of received light RL17. The light receiving unit 30 also generates the signal charge accumulated in the pixel 31 by receiving light in response to the first light receiving pulse RG17a as the first amount of received light RL17a. The light receiving unit 30 also generates the signal charge accumulated in the pixel 31 by receiving light in response to the second light receiving pulse RG18 as the second amount of received light RL18. The light receiving unit 30 also generates the signal charge accumulated in the pixel 31 by receiving light in response to the second light receiving pulse RG18a as the second amount of received light RL18a.

The saturation level determination unit 60 determines whether the first amount of received light RL17, RL17a and the second amount of received light RL18, RL18a exceed the saturation level.

When the first received light amounts RL17, RL17a and the second received light amounts RL18, RL18a are all smaller than the saturation level based on the comparison result of the saturation level determination unit 60, the combining unit 70 selects and combines the first combining coefficient. When at least one of the first received light amounts RL17, RL17a is greater than the saturation level, the combining unit 70 selects and combines the second combining coefficient. When at least one of the second received light amounts RL18, RL18a is greater than the saturation level, the combining unit 70 selects and combines the third combining coefficient.

In this way, the synthesis unit 70 synthesizes the multiple pixel signals output from the pixels 31 based on the determination result of the saturation level determination unit 60, and outputs a synthesis signal s10. The image generating device 1H generates an image of the subject 9 based on the multiple synthesis signals s10 output from the multiple pixels 31.

The distance calculation unit 90 calculates the distance L to the subject 9 using the first amount of received light RL17, RL17a and the second amount of received light RL18, RL18a.

When the saturation level determination unit 60 determines that only the first amount of received light RL17 or RL17a is greater than the saturation level, the distance calculation unit 90 calculates the distance L using (Equation 18). When the saturation level determination unit 60 determines that only the second amount of received light RL18 or RL18a is greater than the saturation level, the distance calculation unit 90 calculates the distance L using (Equation 19). In other cases, the distance calculation unit 90 calculates the distance L using any of (Equation 18), (Equation 19), and (Equation 20).

In this image generating device 1H, the effects of specular reflection can be suppressed and an image can be easily generated. In addition, in this image generating device 1H, the effects of specular reflection can be suppressed and the distance L to the subject 9 can be easily measured.

(Embodiment 5)
An image generating device according to a fifth embodiment will be described.

Figure 24 is a time chart showing the operation of an image generating device according to embodiment 5. Figure 24 shows an example in which the light emission period and light reception period of the drive timing shown in embodiment 1 are executed multiple times in the same frame.

As shown in FIG. 24, the image generating device may repeatedly accumulate electric charge by repeatedly receiving reflected light R1 corresponding to the emission of the first light-emitting unit 10 a number of times, and generate the accumulated electric charge as the first amount of received light. Similarly, the image generating device may repeatedly accumulate electric charge by repeatedly receiving reflected light R2 corresponding to the emission of the second light-emitting unit 20 a number of times, and generate the accumulated electric charge as the second amount of received light.

Figure 25 is a diagram showing another example of the operation of the image generating device according to embodiment 5. Figure 25 shows another example in which the light emission period and the light reception period of the drive timing shown in embodiment 1 are executed multiple times in the same frame.

As shown in FIG. 25, the image generating device may repeatedly accumulate electric charge by repeatedly receiving reflected light R1, R2 corresponding to the emission of light from the first light-emitting unit 10 and the emission of light from the second light-emitting unit 20, and generate the accumulated electric charge as the first amount of received light and the second amount of received light.

Figure 26 is a diagram showing another example of the operation of the image generating device according to embodiment 5. Figure 26 shows an example in which the light emission period and the light reception period are executed multiple times in the same frame when N is 2 and M is 2 in the configuration shown in embodiment 4.

As shown in FIG. 26, the image generating device may repeatedly accumulate charges by repeatedly receiving reflected light R1 in response to the emission of the first light-emitting unit 10 a plurality of times, and generate the accumulated charges as the first amount of received light of the other device. The image generating device may also repeatedly accumulate charges by repeatedly receiving reflected light R1 in response to the emission of the first light-emitting unit 10 a plurality of times, and generate the accumulated charges as the first amount of received light of the other device. Similarly, the image generating device may repeatedly accumulate charges by repeatedly receiving reflected light R2 in response to the emission of the second light-emitting unit 20 a plurality of times, and generate the accumulated charges as the second amount of received light of the other device. The image generating device may also repeatedly accumulate charges by repeatedly receiving reflected light R2 in response to the emission of the second light-emitting unit 20 a plurality of times, and generate the accumulated charges as the second amount of received light of the other device.

Figure 27 is a diagram showing another example of the operation of the image generating device according to embodiment 5. Figure 27 shows another example in which the light emission period and the light reception period are executed multiple times in the same frame when N is 2 and M is 2 in the configuration shown in embodiment 4.

As shown in FIG. 27, the image generating device may repeatedly accumulate electric charge by repeatedly receiving reflected light R1, R2 corresponding to the emission of light from the first light-emitting unit 10 and the emission of light from the second light-emitting unit 20, and generate the accumulated electric charge as the first amount of received light and the second amount of received light.

In this way, by repeating light emission and light reception (exposure) multiple times within the same frame, the amount of light emitted each time can be reduced. Also, by distributing the light emission multiple times, the peak power of the first light-emitting unit 10 and the second light-emitting unit 20 can be reduced. Also, by repeating light emission multiple times, the device becomes less susceptible to the effects of noise, and the S/N ratio can be improved.

(summary)
The image generating devices 1 to 1H of the present embodiment include a first light-emitting unit 10 and a second light-emitting unit 20 that emit light, a light-receiving unit 30 that receives, at a plurality of pixels 31 arranged two-dimensionally, reflected light R1, R2 that is generated when the light (illumination light L1, L2) emitted by the first light-emitting unit 10 and the second light-emitting unit 20 is irradiated onto a subject 9, and reflected light, respectively, and a drive control unit 50 that controls the light-emitting period of the first light-emitting unit 10, the light-emitting period of the second light-emitting unit 20, and the light-receiving period of the light-receiving unit 30, and a combination unit 70 that generates a combined signal s10 by combining a first pixel signal s1 output from the pixel 31 upon reception of light by the light-receiving unit 30 corresponding to the emission of the first light-emitting unit 10, and a second pixel signal s2 output from the pixel 31 upon reception of light by the light-receiving unit 30 corresponding to the emission of the second light-emitting unit 20. The first light-emitting unit 10 and the second light-emitting unit 20 are provided at positions where they do not overlap when viewed from the subject 9. The drive control unit 50 causes the first light-emitting unit 10 and the second light-emitting unit 20 to emit light at different timings in the same frame. When at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal greater than a predetermined level, the synthesis unit 70 generates a synthesis signal s10 without using the pixel signal greater than the predetermined level.

In this way, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal greater than a predetermined level, the composite signal s10 is generated without using the pixel signal greater than the predetermined level, thereby suppressing the effects of specular reflection and easily generating an image.

The specified level may also be set in advance based on the saturation level TH at which the signal charge accumulated in pixel 31 becomes saturated.

As a result, when at least one of the first pixel signal s1 and the second pixel signal s2 is a pixel signal that is greater than the saturation level, the composite signal s10 can be generated without using the saturated pixel signal. This makes it possible to easily generate an image by suppressing the effects of specular reflection.

The light receiving unit 30 may also read out the signal charge accumulated in the pixel 31 in response to the emission of the first light emitting unit 10 into a first packet p1, output a first pixel signal s1 including information on the amount of light received according to the signal charge, and read out the signal charge accumulated in the pixel 31 of the light receiving unit 30 in response to the emission of the second light emitting unit 20 into a second packet p2, and output a second pixel signal s2 including information on the amount of light received according to the signal charge.

In this way, by reading out the signal charge accumulated in pixel 31 into multiple packets and outputting the first pixel signal s1 and the second pixel signal s2, it is possible to generate one image using information within the same frame. In other words, instead of generating one image using multiple frames, it is possible to generate one image using information within one frame. This makes it possible to easily generate images by suppressing the effects of specular reflection.

The first light-emitting unit 10 and the second light-emitting unit 20 may also be arranged on both outer sides of the light-receiving unit 30.

In this way, by arranging the first light-emitting unit 10 and the second light-emitting unit 20 on both sides of the light-receiving unit 30 rather than on one side of the light-receiving unit 30, it is possible to prevent both the first pixel signal s1 and the second pixel signal s2 from exceeding a predetermined level. This makes it possible to easily generate an image by suppressing the effects of specular reflection.

The image generating devices 1 to 1H of the present embodiment include a first light-emitting unit 10 that irradiates light (irradiation light L1) toward a subject 9, a second light-emitting unit 20 that irradiates light (irradiation light L2) toward the subject 9, a light-receiving unit 30 that receives reflected light R1, R2 generated when the light irradiated by the first light-emitting unit 10 and the second light-emitting unit 20 is reflected by the subject 9, using a plurality of pixels 31 arranged two-dimensionally, and a light-emitting period of the first light-emitting unit 10, a light-emitting period of the second light-emitting unit 20, and a light-receiving unit 31 that receives the reflected light R1, R2 generated when the light irradiated by the first light-emitting unit 10 and the second light-emitting unit 20 is reflected by the subject 9. The image display device includes a drive control unit 50 that controls the light receiving period of the light receiving unit 30, a saturation level determination unit 60 that determines whether or not the first pixel signal s1 and the second pixel signal s2 output from the pixels 31 of the light receiving unit 30 in response to the irradiation of light by the first light emitter 10 and the second light emitter 20 exceed a saturation level TH, and a combination unit 70 that generates a combined signal s10 by combining the first pixel signal s1 and the second pixel signal s2 based on the determination result of the saturation level determination unit 60. The first light emitter 10 and the second light emitter 20 are arranged apart from each other. The drive control unit 50 generates, at different timings, a first light emission pulse for causing the first light emission unit 10 to emit light and a second light emission pulse for causing the second light emission unit 20 to emit light in the same frame, and generates, at different timings, a first light reception pulse for receiving reflected light R1 generated when light irradiated by the first light emission unit 10 is reflected by the subject 9, and a second light reception pulse for receiving reflected light R2 generated when light irradiated by the second light emission unit 20 is reflected by the subject 9. The light reception unit 30 outputs a first pixel signal s1 including information on a first amount of received light received in accordance with the first light reception pulse, and outputs a second pixel signal s2 including information on a second amount of received light received in accordance with the second light reception pulse. The saturation level determination unit 60 compares the first amount of received light, the second amount of received light, and the saturation level TH. The combination unit 70
(1) when the first amount of received light and the second amount of received light are both smaller than a saturation level TH, the first pixel signal s1 and the second pixel signal s2 are combined by a first combination coefficient;
(2) when only the first amount of received light is greater than the saturation level TH, the first pixel signal s1 and the second pixel signal s2 are combined by a second combination coefficient;
(3) When only the second amount of received light is greater than the saturation level TH, the first pixel signal s1 and the second pixel signal s2 are combined by a third combination coefficient.

In this way, by combining the first pixel signal s1 and the second pixel signal s2 based on the comparison result between the first amount of received light, the second amount of received light, and the saturation level TH, it is possible to easily generate an image while suppressing the effects of specular reflection.

The second synthesis coefficient may be a coefficient that synthesizes the first amount of received light by regarding it as 0, and the third synthesis coefficient may be a coefficient that synthesizes the second amount of received light by regarding it as 0.

With this, when one of the first amount of received light and the second amount of received light is saturated, an image can be generated without using a pixel signal that includes information about the saturated amount of received light. This makes it possible to easily generate an image by suppressing the effects of specular reflection.

The first synthesis coefficient may be a coefficient that synthesizes the first amount of received light by considering it as half, and the second amount of received light by considering it as half.

With this, when both the first amount of received light and the second amount of received light are not saturated, an image can be generated using half each of the pixel signals containing information on both non-saturated amounts of received light. This makes it possible to easily generate an image by suppressing the effects of specular reflection.

The image generating device 1A may further include a correction unit 80 that corrects the information on the first amount of received light or the information on the second amount of received light. The drive control unit 50 controls the first light receiving period of the light receiving unit 30 corresponding to the irradiation of light by the first light emitter 10 and the second light receiving period of the light receiving unit 30 corresponding to the irradiation of light by the second light emitter 20 so as to be different from each other. The correction unit 80 corrects the information on the first amount of received light or the information on the second amount of received light based on the amount of received light caused by the difference between the first light receiving period and the second light receiving period. The synthesis unit 70 may synthesize the first pixel signal s1 and the second pixel signal s2 using the information on the first amount of received light or the information on the second amount of received light corrected by the correction unit 80.

In this way, the correction unit 80 corrects the information on the first amount of received light or the information on the second amount of received light based on the amount of received light resulting from the difference between the first light receiving period and the second light receiving period, so that the first pixel signal s1 or the second pixel signal s2 can be appropriately generated. This makes it possible to appropriately generate an image in which the effects of specular reflection are suppressed.

The image generating device 1B may further include a correction unit 80 that corrects the information on the first amount of received light or the information on the second amount of received light. The drive control unit 50 controls the first light receiving start time from the start of irradiation of the first light emitter 10 to the start of light receiving by the light receiving unit 30 to be different from the second light receiving start time from the start of irradiation of the second light emitter 20 to the start of light receiving by the light receiving unit 30. The correction unit 80 corrects the information on the first amount of received light or the information on the second amount of received light based on the amount of received light caused by the difference between the first light receiving start time and the second light receiving start time. The synthesis unit 70 may synthesize the first pixel signal s1 and the second pixel signal s2 using the information on the first amount of received light or the information on the second amount of received light corrected by the correction unit 80.

In this way, the correction unit 80 corrects the information on the first amount of received light or the information on the second amount of received light based on the amount of received light caused by the difference between the first light-receiving start time and the second light-receiving start time, so that the first pixel signal s1 or the second pixel signal s2 can be appropriately generated. This makes it possible to appropriately generate an image in which the effects of specular reflection are suppressed.

The image generating devices 1A to 1H may further include a distance calculation unit 90 that calculates the distance L to the subject 9, and the distance calculation unit 90 may calculate the distance L using the corrected first amount of received light or the second amount of received light.

In this way, the distance calculation unit 90 calculates the distance L using the corrected first amount of received light or the second amount of received light, thereby making it possible to appropriately measure the distance L to the subject 9.

Moreover, the image generating devices 1A to 1H of the present embodiment include a first light-emitting unit 10 that irradiates light (irradiation light L1) toward the subject 9, a second light-emitting unit 20 that irradiates light (irradiation light L2) toward the subject 9, a light-receiving unit 30 that receives reflected light R1, R2 generated when the light irradiated by the first light-emitting unit 10 and the second light-emitting unit 20 is reflected by the subject 9, using a plurality of pixels 31 arranged two-dimensionally, and a light-receiving unit 31 that receives a light-emitting period of the first light-emitting unit 10, a light-emitting period of the second light-emitting unit 20, and The image display device includes a drive control unit 50 that controls the light receiving period of the first light emitter 10 and the light receiving unit 30, a saturation level determination unit 60 that determines whether or not a first pixel signal s1 and a second pixel signal s2 output from a pixel 31 of the light receiving unit 30 in response to irradiation of light by the first light emitter 10 and the second light emitter 20 exceed a saturation level TH, and a combination unit 70 that generates a combined signal s10 by combining the first pixel signal s1 and the second pixel signal s2 based on a determination result of the saturation level determination unit 60. The first light emitter 10 and the second light emitter 20 are arranged apart from each other. The drive control unit 50 generates, at different timings, one or more first light emission pulses for making the first light emission unit 10 emit light and one or more second light emission pulses for making the second light emission unit 20 emit light in the same frame, and generates, at different timings, one or more first light reception pulses for receiving reflected light R1 generated by the light irradiated by the first light emission unit 10 being reflected by the subject 9, and one or more second light reception pulses for receiving reflected light R2 generated by the light irradiated by the second light emission unit 20 being reflected by the subject 9. The light reception unit 30 outputs a first pixel signal s1 including information on one or more first light reception amounts received according to one or more first light reception pulses, and outputs a second pixel signal s2 including information on one or more second light reception amounts received according to one or more second light reception pulses. The saturation level determination unit 60 compares the information on one or more first light reception amounts, the information on one or more second light reception amounts, and the saturation level TH. The synthesis unit 70
(1) when the first amount of received light, which is equal to or greater than one, and the second amount of received light, which is equal to or greater than one, are all smaller than a saturation level TH, the first pixel signal s1 and the second pixel signal s2 are combined by a first combination coefficient;
(2) when at least one of the first amounts of received light is greater than a saturation level TH, the first pixel signal s1 and the second pixel signal s2 are combined with a second combination coefficient;
(3) When at least one of the second amounts of received light, which are equal to or greater than one, is greater than a saturation level, the first pixel signal s1 and the second pixel signal s2 are combined by a third combination coefficient.

In this way, by combining the first pixel signal s1 and the second pixel signal s2 based on the comparison result of one or more first received light amounts, one or more second received light amounts, and the saturation level TH, it is possible to suppress the effects of specular reflection and easily generate an image.

The image generating devices 1A to 1H may further include a distance calculation unit 90 that calculates the distance to the subject 9, and the distance calculation unit 90 may calculate the distance L using a sum of one or more first amounts of received light and a sum of one or more second amounts of received light.

This makes it possible to easily measure the distance L to the subject 9 while suppressing the effects of specular reflection.

The drive control unit 50 may also execute each of the light emission period and the light reception period multiple times in the same frame.

In this way, by executing each of the light emission period and light reception period multiple times within the same frame, the amount of light emitted each time can be reduced. In addition, by distributing the light emission over multiple times, the peak power of the first light-emitting unit 10 and the second light-emitting unit 20 can be reduced.

Other Embodiments
Although the embodiments have been described above, the present disclosure is not limited to the above-mentioned embodiments. The present disclosure also includes forms obtained by applying various modifications to the embodiments that a person skilled in the art can conceive, and forms realized by arbitrarily combining the components and functions of each embodiment within the scope of the present disclosure.

The image generating devices 1 to 1H are not limited to being installed in a vehicle, but may be installed in a predetermined space 8, such as inside a building. The image generating devices 1 to 1H may be devices included in an imaging device. The image generating devices 1A to 1H may be devices included in a distance measuring device.

This disclosure is useful for improving the performance of devices that detect a person's health state, alertness, concentration state, etc. from images.

1, 1A, 1B, 1C, 1D, 1E, 1F, 1G, 1H Image generating device 8 Space 9 Subject 10 First light emitting unit 10a, 10b Light source 20 Second light emitting unit 20a, 20b Light source 30 Light receiving unit 31 Pixel 40 Substrate 50 Drive control unit 60 Saturation level determination unit 70 Combining unit 80 Correction unit 90 Distance calculation unit L1, L2 Irradiation light p1 First packet p2 Second packet R1, R2 Reflected light RG1, RG3, RG5, RG7, RG9, RG11, RG11a, RG13, RG13a, RG15, RG15a, RG17, RG17a First received light pulse RG2, RG4, RG6, RG8, RG10, RG12, RG14, RG14a, RG16, RG18, RG18a Second received light pulse RL1, RL3, RL5, RL7, RL9, RL11, RL11a, RL13, RL13a, RL15, RL15a, RL17, RL17a First received light amount RL2, RL4, RL6, RL8, RL10, RL12, RL14, RL14a, RL16, RL18, RL18a Second received light amount s1 First pixel signal s2 Second pixel signal s10 Combined signal SG1, SG3, SG5, SG7, SG9, SG11, SG13, SG15, SG15a, SG17, SG17a First light-emitting pulse SG2, SG4, SG6, SG8, SG10, SG12, SG14, SG16, SG18, SG18a Second light-emitting pulse Tg1, Tg3, Tg5, Tg7, Tg9, Tg11, Tg13, Tg15, Tg15a, Tg17, Tg17a First light-emitting period Tg2, Tg4, Tg6, Tg8, Tg10, Tg12, Tg14, Tg16, Tg18, Tg18a Second light-emitting period TH Saturation level ts1, ts3, ts5, ts7, ts9, ts11, ts11a, ts13, ts13a, ts15, ts15a, ts17, ts17a First light-reception start time ts2, ts4, ts6, ts8, ts10, ts12, ts14, ts14a, ts16, ts18, ts18a Second light-reception start time Tr1, Tr3, Tr5, Tr7, Tr9, Tr11, Tr11a, Tr13, Tr13a, Tr15, Tr15a, Tr17, Tr17a First light-reception period Tr2, Tr4, Tr6, Tr8, Tr10, Tr12, Tr14, Tr14a, Tr16, Tr18, Tr18a Second light receiving period

Claims (6)

a first light emitting unit that irradiates light toward a subject;
A second light emitting unit that irradiates light toward the subject;
a light receiving unit that receives reflected light generated by the light irradiated by the first light emitting unit and the light irradiated by the second light emitting unit being reflected by the subject, using a plurality of pixels that are two-dimensionally arranged;
a drive control unit that controls a light emission period of the first light emitter, a light emission period of the second light emitter, and a light reception period of the light receiver;
a saturation level determination unit that determines whether a first pixel signal and a second pixel signal output from the pixel of the light receiving unit in response to irradiation with light from the first light emitting unit and the second light emitting unit exceed a saturation level;
a synthesis unit that synthesizes the first pixel signal and the second pixel signal based on a determination result of the saturation level determination unit to generate a synthesis signal;
Equipped with
The first light emitting unit and the second light emitting unit are disposed apart from each other,
the drive control unit generates, at different timings in the same frame, a first light emission pulse for causing the first light emission unit to emit light and a second light emission pulse for causing the second light emission unit to emit light, and generates, at different timings, a first light reception pulse for receiving the reflected light generated by the light irradiated by the first light emission unit being reflected by the subject, and a second light reception pulse for receiving the reflected light generated by the light irradiated by the second light emission unit being reflected by the subject,
the light receiving unit outputs the first pixel signal including information on a first amount of light received in accordance with the first light receiving pulse, and outputs the second pixel signal including information on a second amount of light received in accordance with the second light receiving pulse;
the saturation level determination unit compares the first amount of received light, the second amount of received light, and the saturation level;
The synthesis unit is
(1) when the first amount of received light and the second amount of received light are both smaller than the saturation level, the first pixel signal and the second pixel signal are combined by a first combination coefficient;
(2) when only the first amount of received light is greater than the saturation level, the first pixel signal and the second pixel signal are combined with a second combination coefficient;
(3) when only the second amount of received light is greater than the saturation level, the first pixel signal and the second pixel signal are combined by a third combination coefficient;
a correction unit that corrects the information on the first amount of received light or the information on the second amount of received light,
the drive control unit controls a first light receiving period of the light receiving unit corresponding to irradiation of light from the first light emitting unit and a second light receiving period of the light receiving unit corresponding to irradiation of light from the second light emitting unit so as to be different from each other;
the correction unit corrects information about the first amount of received light or information about the second amount of received light based on an amount of received light caused by a difference between the first light-receiving period and the second light-receiving period;
the combining section combines the first pixel signal and the second pixel signal using the information on the first amount of received light or the information on the second amount of received light corrected by the correcting section. the second synthesis coefficient is a coefficient for synthesis with the first amount of received light regarded as 0,
The image generating device according to claim 1 , wherein the third blending coefficient is a coefficient for blending the second amount of received light assuming that the second amount of received light is zero. The image generating device according to claim 1 , wherein the first blending coefficient is a coefficient for blending the first received light amount and the second received light amount, the first received light amount being half and the second received light amount being half, respectively. a first light emitting unit that irradiates light toward a subject;
A second light emitting unit that irradiates light toward the subject;
a light receiving unit that receives reflected light generated by the light irradiated by the first light emitting unit and the light irradiated by the second light emitting unit being reflected by the subject, using a plurality of pixels that are two-dimensionally arranged;
a drive control unit that controls a light emission period of the first light emitter, a light emission period of the second light emitter, and a light reception period of the light receiver;
a saturation level determination unit that determines whether a first pixel signal and a second pixel signal output from the pixel of the light receiving unit in response to irradiation with light from the first light emitting unit and the second light emitting unit exceed a saturation level;
a synthesis unit that synthesizes the first pixel signal and the second pixel signal based on a determination result of the saturation level determination unit to generate a synthesis signal;
Equipped with
The first light emitting unit and the second light emitting unit are disposed apart from each other,
the drive control unit generates, at different timings in the same frame, a first light emission pulse for causing the first light emission unit to emit light and a second light emission pulse for causing the second light emission unit to emit light, and generates, at different timings, a first light reception pulse for receiving the reflected light generated by the light irradiated by the first light emission unit being reflected by the subject, and a second light reception pulse for receiving the reflected light generated by the light irradiated by the second light emission unit being reflected by the subject,
the light receiving unit outputs the first pixel signal including information on a first amount of light received in accordance with the first light receiving pulse, and outputs the second pixel signal including information on a second amount of light received in accordance with the second light receiving pulse;
the saturation level determination unit compares the first amount of received light, the second amount of received light, and the saturation level;
The synthesis unit is
(1) when the first amount of received light and the second amount of received light are both smaller than the saturation level, the first pixel signal and the second pixel signal are combined by a first combination coefficient;
(2) when only the first amount of received light is greater than the saturation level, the first pixel signal and the second pixel signal are combined with a second combination coefficient;
(3) when only the second amount of received light is greater than the saturation level, the first pixel signal and the second pixel signal are combined by a third combination coefficient;
a correction unit that corrects the information on the first amount of received light or the information on the second amount of received light,
the drive control unit controls a first light-receiving start time from a start of irradiation of the first light-emitting unit to a start of light-receiving by the light-receiving unit to be different from a second light-receiving start time from a start of irradiation of the second light-emitting unit to a start of light-receiving by the light-receiving unit;
the correction unit corrects information about the first amount of received light or information about the second amount of received light based on an amount of received light caused by a difference between the first light-receiving start time and the second light-receiving start time;
the combining section combines the first pixel signal and the second pixel signal using the information on the first amount of received light or the information on the second amount of received light corrected by the correcting section. Further, a distance calculation unit is provided for calculating a distance to the subject,
The image generating device according to claim 1 , wherein the distance calculation section calculates the distance using the first amount of received light or the second amount of received light after the correction. the drive control unit executes each of the light emission period and the light reception period a plurality of times in the same frame;
The image generating device according to any one of claims 1 to 5.

Images