JP6590517B2 - Distance image generator - Google Patents

Description

  The present invention relates to a distance image generation apparatus.

  2. Description of the Related Art Conventionally, an apparatus that generates a distance image using a distance image sensor such as a CMOS sensor is known. For example, one light source that irradiates space with light whose intensity is modulated at a predetermined light emission frequency, and a plurality of photoelectric conversions that receive reflected light of light emitted from the light source and output a signal corresponding to the received light intensity A distance image generation device including a distance image sensor having an element has been proposed (see, for example, Patent Document 1).

  The distance image generation apparatus of Patent Document 1 utilizes the fact that the speed of light is known, and receives the reflected light of the light irradiated to the measurement object, thereby allowing a time-of-flight measurement method (TOF: Time of flight). ) Is used to measure the distance from the device to the object to be measured based on the time of flight measured.

JP 2010-71832 A

  For example, when the distance image generation apparatus of Patent Document 1 is used in a vehicle periphery monitoring system provided in front of a vehicle where a wide distance measurement range is required, S / In order to increase the N ratio, it is necessary to increase the light output. However, when the light output is increased, in addition to the increase in power consumption and the amount of heat generated, there is a problem that the influence of heat-induced noise and the like increases, and the measurement distance accuracy decreases.

  Further, in the case of a distance image generation device that requires a wide distance measurement range, if a plurality of light sources are provided in the distance image generation device, light is measured between the measurement object and the distance image generation device due to the difference in position of each light source. There is a problem that an error is likely to occur in the flight time and the measurement distance accuracy is deteriorated.

  The present invention suppresses power consumption and heat generation even when a plurality of light sources are used for a light projecting unit, and improves distance measurement accuracy in a wide measurement distance range, particularly distance measurement accuracy in a short distance range. An object is to provide a generation device.

The distance image generation apparatus of the present invention is
A light projecting unit having a plurality of light sources for irradiating the target space with modulated light;
A control unit for generating a drive signal for driving the plurality of light sources;
A light receiving unit that receives light including reflected light that is irradiated from the light projecting unit and reflected by an object in the target space;
A calculation unit that calculates a distance value to the object based on the drive signal and the light received by the light receiving unit;
A distance image generation unit that generates a distance image having a distance value as a pixel value;
The light projecting unit includes a first region in which at least one light source is disposed around the light receiving unit, and a second region extending from an edge on the opposite side of the light receiving unit in the first region,
The control unit includes a drive circuit that drives each light source of the light projecting unit, and the drive circuit generates a drive signal for turning on only the light source of the first region, and generates the first region. And the generation of a drive signal to be in a fully lit state capable of turning on all of the light sources in the second region, can be selectively performed,
The computing unit is
A primary calculation for calculating an approximate primary distance to the object in the fully lit state;
A light source lighting state instruction signal for determining whether to turn on the light projecting unit in the full lighting state or the partial lighting state based on a result of comparing the primary distance with a predetermined threshold, the control unit A lighting state selection process for feedback to be transmitted to
After the primary calculation, it is configured to perform a secondary calculation that calculates a secondary distance to the object as the distance value,
When the primary distance calculated by the primary calculation exceeds the threshold value, the control unit sets the light projecting unit to the full lighting state based on the light source lighting state instruction signal, and the calculation unit performs the second calculation. Perform the next operation,
When the primary distance calculated by the primary calculation is equal to or less than the threshold, the control unit sets the light projecting unit to the partial lighting state based on the light source lighting state instruction signal, and the calculation unit performs the secondary operation. It is characterized by performing an operation .

  The present invention has the following effects by the above configuration. First, at the start of measurement, primary processing is performed to confirm the presence or absence of the measurement object in the target space and to approximate the distance to the measurement object. However, the primary processing is unnecessary when the existence of the measurement object and the approximate distance to the measurement object are known.

  Specifically, first, all of the plurality of light sources of the light projecting unit are turned on, and the distance value to the object is calculated by the calculating unit. When the distance value exceeds a predetermined threshold (in the case of a non-near distance), the control unit generates a drive signal so that all of the plurality of light sources are turned on. When the distance value is equal to or smaller than the predetermined threshold (near In the case of distance), a drive signal is generated so as to turn on some of the plurality of light sources. Based on the drive signal generated by the control unit, the light projecting unit turns on all of the plurality of light sources in the case of non-short distance, and lights up some of the plurality of light sources in the case of short distance.

  According to the present invention, as described above, when the distance to the measurement object is a non-short distance, all of the plurality of light sources are turned on. For example, the distance image generation device is used outdoors where background light increases. Even in this case, a sufficient output of light emitted from the light projecting unit can be secured, and the S / N ratio can be kept good. Accordingly, it is possible to provide a distance image generation apparatus having a wide distance measurement range in which a plurality of light sources are provided in the distance image generation apparatus. On the other hand, when the distance to the measurement object is a short distance, a part of the plurality of light sources is turned on, so that power consumption and heat generation can be suppressed, and the influence of noise and the like induced by heat is reduced. It is possible to suppress a decrease in measurement distance accuracy and to prolong the life of the distance image generation device.

  According to the present invention, when the distance value is equal to or smaller than the predetermined threshold value, the light source in the first region around the light receiving unit is turned on. Therefore, even if the distance to the measurement object is a short distance, S / N A decrease in the ratio can be suppressed.

  Further, in the present invention, the light receiving unit includes an image sensor, and the image sensor receives light including reflected light that is irradiated from the light projecting unit and reflected by an object in the target space, and generates a received light amount. A plurality of photoelectric conversion elements that convert the charge into a corresponding charge, and the calculation unit distributes the charges converted by the photoelectric conversion element in synchronization with the drive signal to the plurality of charge storage units, and stores them in the charge storage unit It is preferable to calculate a distance value to the object based on the generated charge.

  In order to generate a distance value to an object for each pixel of a distance image, the distance image generation device includes an image sensor having a photoelectric conversion element corresponding to each pixel in a light receiving unit.

  For example, when the distance image generating device coexists with the lamps, the light projecting unit may be difficult to be disposed so as to surround the image sensor in the vicinity of the image sensor of the light receiving unit due to restrictions on the design and the heat dissipation structure. In this case, depending on the position of each light source, there is a difference in the distance from each light source to the imaging device through the measurement object, and an error in the time of flight between the measurement object and the distance image generation device is likely to occur. Measurement distance accuracy deteriorates.

According to the present invention, a plurality of charge storage units are connected to one photoelectric conversion element, and light modulated at a high frequency or pulsed light (hereinafter referred to as “modulated light”) is synchronized with the modulation or light emission time. Thus, the S / N ratio can be increased by separating and accumulating in a plurality of charge accumulating sections and reading out and averaging the charges separated and accumulated at regular intervals. Therefore, the measurement distance accuracy can be improved based on the charges separated and accumulated in the plurality of charge accumulation units.
Preferably, in the distance image generating apparatus of the present invention,
In the primary calculation, a primary charge accumulation time for distributing charges accumulated in the charge accumulation unit is set to a time at which an S / N ratio capable of detecting the threshold distance is obtained, and the primary charge accumulation period has elapsed. Later, a distance value to the object is calculated based on the charge accumulated in the charge accumulation unit,
In the secondary calculation, the secondary charge accumulation time for distributing the charges accumulated in the charge accumulation unit is set to a time at which an S / N ratio capable of detecting the maximum distance of the distance image generation device is obtained. After the secondary charge accumulation period, a distance value to the object is calculated based on the charges accumulated in the charge accumulation unit.

The block diagram which shows the structure of the distance image generation apparatus of this embodiment. The figure which shows the relationship between the light projection part and light-receiving part of the distance image generation apparatus of this embodiment. The figure which shows the other relationship of the light projection part of the distance image generation apparatus of this embodiment, and a light-receiving part. The flowchart of the distance measurement process of the distance image generation apparatus of this embodiment.

  As shown in FIG. 1, the distance image generation device 100 according to the embodiment of the present invention includes a light projecting unit 10 having a plurality of light sources 12, a control unit 20 having a drive circuit 22 for driving the plurality of light sources 12, and A light receiving unit 30 that receives light from the measurement target 200, a calculation unit 40 that calculates a distance value to the target 200, and a distance image generation unit 50 that generates a distance image using the distance value as a pixel value. It is an apparatus which produces | generates a distance image using the optical time-of-flight measuring method comprised as follows.

  The distance image generating apparatus 100 includes a CPU, a memory, a storage device, and the like. The control unit 20, the light receiving unit 30, the calculation unit 40, and the distance image generation unit 50 are realized by the CPU loading a program stored in advance in the storage device and executing it.

  As shown in FIG. 2, the distance image generating apparatus 100 according to the present embodiment has a rectangular shape so that the surface of the light projecting unit 10 in the direction of irradiating light is disposed inside, for example, at the center. Is formed.

  The light projecting unit 10 includes a plurality of light sources 12 capable of high-speed modulation such as LEDs that emit near-infrared light. Each light source 12 has modulated light 101 having the same waveform as that of a drive signal generated by a control unit 20 described later (for example, infrared light or visible light modulated at high speed with a sine wave, a rectangular wave, or the like) as an object. It is a light emission source that irradiates the target space. In the present embodiment, the control unit 20 will be described below as generating a pulsed drive signal.

  As shown in FIG. 2, the light projecting unit 10 of the present embodiment includes six light sources 12 around the light receiving unit 30 (including the image sensor 32 inside) disposed inside the light projecting unit 10. The first region 10A (region surrounded by a substantially circular broken line in FIG. 2) and 42 light sources 12 are arranged extending from the edge of the first region 10A opposite to the light receiving unit 30. The region is divided into second regions 10B (regions sandwiched between the broken line and the outer periphery of the light projecting unit 10).

  Note that the shape of the surface of the light projecting unit 10 in the direction of irradiating light is not limited to a rectangular shape, and can be formed in various shapes such as a circular shape, an elliptical shape, a triangular shape, and a trapezoidal shape. For example, when the distance image generating apparatus 100 is required to coexist with the lamps, the lamp design and the heat dissipation structure for the light projecting unit 10 are limited, but can be formed in various shapes according to the limitation. .

  As an example, as shown in FIG. 3, the surface of the light projecting unit 10 ′ in the direction of irradiating light is formed in a long shape. You may arrange in the shape of a line arranged in two rows.

  The light projecting unit 10 ′ is disposed substantially at the center of the surface of the light projecting unit 10 ′ in the direction in which the light receiving unit 30 ′ (including the imaging element 32 ′ is irradiated), and the light receiving unit 30. A first region 10A ′ (a region surrounded by a substantially circular broken line in FIG. 3) in which eight light sources 12 are arranged in the periphery adjacent to “and a first region 10A ′ in a direction away from the light receiving unit 30 ′. Is divided into a second region 10B ′ (region sandwiched between the broken line and the outer periphery of the light projecting unit 10 ′) where 24 light sources 12 are arranged.

  Further, if the light projecting unit 10 can calculate the distance value based on the light emitted from the light source 12 in the first region 10A when the distance to the measurement object is a short distance, the imaging element 32 of the light receiving unit 30 is used. A region other than the surrounding first region 10A adjacent to the region may be divided into a plurality of regions.

  The control unit 20 includes a drive circuit 22 that drives each light source 12 of the light projecting unit 10, and performs operation control of the light projecting unit 10, the light receiving unit 30, and the calculation unit 40.

  The control unit 20 generates and controls a pulse-shaped drive signal for driving the plurality of light sources 12 and inputs a selection signal for determining whether to turn on / off the light sources 12 in the light projecting unit 10 to the drive circuit 22. Each light source 12 driven using a pulsed drive signal irradiates the target space with modulated light 101 having the same waveform as the pulsed drive signal. In addition, the control unit 20 transmits a synchronization signal synchronized with the modulation frequency of the modulated light 101 to the light receiving unit 30 and the calculation unit 40.

  As shown in FIG. 1, the drive circuit 22 is configured so that a drive power source (not shown) can be selectively connected to each light source 12 of the light projecting unit 10 by a selection signal. Accordingly, the control unit 20 can selectively turn on the light sources 12 in the first region 10A and the second region 10B of the light projecting unit 10 by inputting the selection signal to the drive circuit 22. For example, when the distance to the measurement object is a short distance, only the light source 12 in the first area 10A is turned on (partial lighting state), and when the distance to the measurement object is a non-short distance, the first area 10A and the first area 10A. All of the light sources 12 in the two regions 10B can be turned on (fully turned on).

  The light receiving unit 30 includes an imaging element 32 that receives incident light 102 including reflected light, which is reflected from the measurement object 200 in the target space, from the modulated light 101 emitted from the light source 12 and converts the incident light 102 into charges. The imaging device 32 receives incident light 102 and stores a plurality of photoelectric conversion elements 34 that convert the amount of received light into a charge amount for each pixel, and a charge amount obtained by each photoelectric conversion element 34 for each photoelectric conversion element 34. Two charge storage units 36A and 36B. Note that the number of charge storage units for each photoelectric conversion element 34 is not limited to two, and may be two or more.

  The photoelectric conversion element 34 converts the incident light 112 including the reflected light reflected by the measurement object 200 into a charge amount. Then, according to a synchronization signal from the control unit 20, the obtained charge amount is distributed to each of the two charge storage units 36A and 36B provided in association with each photoelectric conversion element 34. In the present embodiment, each of the modulation periods is divided into two equal charge storage parts 36A and 36B for each period. The charge storage units 36A and 36B may store not only the charge amount itself but also data after AD conversion of the charge amount.

  The calculation unit 40 performs a calculation to be described later based on the charges distributed to the charge storage units 36A and 36B, and calculates the phase difference between the modulated light 101 and the incident light 102 using the synchronization signal from the control unit 20. Then, a distance value to the measurement object is calculated for each pixel based on the phase difference.

  In addition, the calculating part 40 of this embodiment performs the primary calculation which measures the approximate primary distance to a measurement object first, shortening the charge accumulation period by charge distribution in the all lighting state by the light projection part 10. . Based on the result of comparing the primary distance measured by the primary calculation and a preset constant value (threshold distance), the calculation unit 40 determines whether the lighting is performed in the full lighting state or the partial lighting state. The light source lighting state instruction signal to be determined is fed back to the control unit 20.

  Then, according to the light source lighting state instruction signal, the control unit 20 sets the lighting state of the light source 12 after the primary calculation to either the full lighting state or the partial lighting state, and then the calculation unit 40 increases the accuracy of the measurement distance. Using the charge accumulation time to perform charge distribution for improvement, calculate the secondary distance with improved accuracy to the measurement object, and perform secondary calculation to generate distance information consisting of the secondary distance calculated for each pixel Do.

  The distance image generation unit 50 generates a distance image having the calculated distance value as a pixel value.

  Next, distance measurement processing of the distance image generation device 100 of the present embodiment will be described with reference to FIG.

[Primary processing]
First, in order to perform a primary calculation for measuring a rough primary distance to the measuring object 200, the light projecting unit 10 is in a fully lit state in which all the light sources 12 in the first region 10A and the second region 10B are lit. The modulated space 101 is irradiated on the object space including the measurement object 200 (S10). Specifically, the control unit 20 generates a pulse-shaped drive signal for driving the light source 12 and inputs it to the drive circuit 22 together with a selection signal so that the lighting state of the light source 12 becomes the full lighting state, and enters the target space. The modulated light 101 is irradiated.

  Next, the incident light 102 including the reflected light reflected by the measuring object 200 is received by the imaging device 32 of the light receiving unit 30 in the fully lit state. Then, the electric charges are converted into electric charges by the photoelectric conversion elements 34 of the respective pixels of the image pickup element 32, and the converted electric charges are input to the electric charge storage units 36A and 36B based on the synchronization signal input from the control unit 20 to the light receiving unit 30. Sort (S11).

  In this embodiment, in order to shorten the charge accumulation period in which charges are distributed in all lighting states, the primary charge accumulation time (time for repeating charge distribution) for distributing charges to the charge accumulation units 36A and 36B is a predetermined short time. The distance (threshold distance) is set to a short time that can guarantee an S / N ratio that can be detected.

  Next, the calculation unit 40 performs a primary calculation for measuring an approximate primary distance to the measurement object 200. (S12). Specifically, after the primary charge accumulation period has elapsed, the charge amounts Q1 and Q2 of the charges distributed and accumulated in the charge accumulation units 36A and 36B are output from the light receiving unit 30 to the calculation unit 40, respectively. Then, the calculation unit 40 generates distance information including the distance values to the measurement object 200 calculated for each pixel using the values of the charge amounts Q1 and Q2 and the synchronization signal input from the control unit 20. The shortest distance to the measurement object is determined as the primary distance mL.

  The distance value is calculated by multiplying the ratio of Q1 or Q2 with respect to the total charge amount “Q1 + Q2” by the period of the pulse-like drive signal (synchronization signal), Measured by multiplying by 2 ".

  In addition, the primary calculation process (S12) of the distance image generation device 100 of the present embodiment is a process of determining whether or not the measurement object 200 at a short distance exists in the target space. Therefore, since the primary distance determined in the primary calculation process (S12) is an approximate distance including an error in the short distance range, the primary calculation process (S12) shortens the primary charge accumulation time to measure space. Even if the S / N ratio in the entire region is low, it is only necessary to guarantee the desired accuracy for the measurement distance in the range up to the threshold distance L.

[Lighting state selection processing]
Next, in order to select the full lighting state or the partial lighting state as the lighting state of the light projecting unit 10 depending on whether the distance to the measurement object 200 is a short distance, for example, 1 [m], It is determined whether or not the primary distance mL exceeds a predetermined threshold distance L (S13).

  When the primary distance mL exceeds the predetermined threshold distance L (S13 / Yes), the light source is formed so as to form a fully lit state in which all the light sources 12 in the first area 10A and the second area 10B of the light projecting unit 10 are lit. A lighting state instruction signal is fed back to the control unit 20. And the control part 20 to which the light source lighting state instruction | indication signal which shows a full lighting state is input into the drive circuit 22 so that it may become a full lighting state, and it measures from the light projection part 10 of a full lighting state. 200 is irradiated with modulated light (S14).

  On the other hand, when the primary distance mL is equal to or smaller than the predetermined threshold distance L (S13 / No), the light source lighting state instruction is formed so as to form a partial lighting state in which all the light sources 12 of only the first region 10A of the light projecting unit 10 are lit. The signal is fed back to the control unit 20. And the control part 20 to which the light source lighting state instruction | indication signal which shows a partial lighting state was input inputs a selection signal into the drive circuit 22 so that it may be in a partial lighting state, and it measures from the light projection part 10 of a partial lighting state. 200 is irradiated with modulated light (S15).

[Secondary calculation processing and distance image generation processing]
In order to calculate a secondary distance with improved accuracy to the measurement object and perform secondary calculation to generate distance information composed of the secondary distance calculated for each pixel, the imaging element 32 of the light receiving unit 30 turns on the lighting state. In the lighting state selected by the selection process (S13 to S15), the incident light 102 including the reflected light of the light 101 irradiated from the light projecting unit 10 and reflected by the measurement object 200 is received.

  Then, in the secondary charge accumulation period in which the S / N ratio capable of detecting the maximum distance of the distance image generating apparatus 100 of the present embodiment can be guaranteed, the photoelectric conversion element 34 of each pixel of the image sensor 32 converts the charge into a charge. Based on the synchronization signal input from the unit 20 to the light receiving unit 30, the converted charges are distributed to the charge storage units 36A and 36B (S16).

  Next, the calculation unit 40 measures an accurate secondary distance to the measurement target 200 for each pixel, and performs a secondary calculation for generating distance information including the secondary distance calculated for each pixel. Pixel distance information is acquired (S17). The calculation process for generating the distance information in S17 is the same as the calculation process for generating the distance information in S12 except that the secondary charge accumulation time is used instead of the primary charge accumulation time, and thus detailed description thereof is omitted.

  Then, the distance image generation unit 50 generates a distance image based on the distance information of all the pixels generated by the calculation unit 40 (S18), and the distance image generation device 100 determines the secondary distance (distance value) to the measurement object. ) Is generated as a pixel value.

  When the modulated light is irradiated to the measurement space in the partially lit state (S15), the distance from the distance image generation device 100 to the measurement object 200 is short in the secondary calculation process and the distance image generation process (S16 to S18). Since it is a distance, deterioration of the S / N ratio due to the distance to the measurement object can be suppressed.

  Therefore, according to the distance measurement process shown in FIG. 4, since the S / N ratio can be maintained well even outdoors, the power consumption and the heat generation amount can be suppressed even when the size of the light projecting unit 10 is increased. The distance measurement accuracy in a wide measurement distance range, particularly the distance measurement accuracy in a short distance range can be improved.

  Note that the distance measurement process of the present embodiment shown in FIG. 4 is an example of the process, and is not limited to this. A person skilled in the art can perform various processes without departing from the gist of the present invention. For example, when the approximate primary distance to the measurement object is known in advance at the start of measurement, it is also possible to perform the lighting state selection process using the primary distance without performing the primary calculation process. .

DESCRIPTION OF SYMBOLS 100 ... Distance image generation apparatus, 200 ... Measurement object (object), 10, 10 '... Light projection part, 12 ... Light source, 20 ... Control part, 22 ... Drive circuit, 30, 30' ... Light-receiving part, 32, 43 '... imaging device, 34 ... photoelectric conversion element, 36A ... charge accumulating unit 1, 36B ... charge accumulating unit 2, 40 ... calculating unit, 50 ... distance image generating unit.

Claims (3)

A light projecting unit having a plurality of light sources for irradiating the target space with modulated light;
A control unit for generating a drive signal for driving the plurality of light sources;
A light receiving unit that receives light including reflected light that is irradiated from the light projecting unit and reflected by an object in the target space;
A calculation unit that calculates a distance value to the object based on the drive signal and the light received by the light receiving unit;
A distance image generation unit that generates a distance image having a distance value as a pixel value;
The light projecting unit includes a first region in which at least one light source is disposed around the light receiving unit, and a second region extending from an edge on the opposite side of the light receiving unit in the first region,
The control unit includes a drive circuit that drives each light source of the light projecting unit, and the drive circuit generates a drive signal for turning on only the light source of the first region, and generates the first region. And the generation of a drive signal to be in a fully lit state capable of turning on all of the light sources in the second region, can be selectively performed,
The computing unit is
A primary calculation for calculating an approximate primary distance to the object in the fully lit state;
A light source lighting state instruction signal for determining whether to turn on the light projecting unit in the full lighting state or the partial lighting state based on a result of comparing the primary distance with a predetermined threshold, the control unit A lighting state selection process for feedback to be transmitted to
After the primary calculation, it is configured to perform a secondary calculation that calculates a secondary distance to the object as the distance value,
When the primary distance calculated by the primary calculation exceeds the threshold value, the control unit sets the light projecting unit to the full lighting state based on the light source lighting state instruction signal, and the calculation unit performs the second calculation. Perform the next operation,
When the primary distance calculated by the primary calculation is equal to or less than the threshold, the control unit sets the light projecting unit to the partial lighting state based on the light source lighting state instruction signal, and the calculation unit performs the secondary operation. A distance image generating apparatus characterized by performing an operation . The distance image generating device according to claim 1 ,
The light receiving unit includes an image sensor,
The imaging element has a plurality of photoelectric conversion elements that receive light including reflected light that is irradiated from the light projecting unit and reflected by an object in the target space, and converts the light into charges according to the received light quantity,
The calculation unit distributes charges converted by the photoelectric conversion elements in synchronization with the drive signal to a plurality of charge storage units, and calculates a distance value to the object based on the charges stored in the charge storage units A distance image generation apparatus characterized by: The distance image generating device according to claim 2,
  In the primary calculation, a primary charge accumulation time for distributing charges accumulated in the charge accumulation unit is set to a time at which an S / N ratio capable of detecting the threshold distance is obtained, and the primary charge accumulation period has elapsed. Later, a distance value to the object is calculated based on the charge accumulated in the charge accumulation unit,
  In the secondary calculation, the secondary charge accumulation time for distributing the charges accumulated in the charge accumulation unit is set to a time at which an S / N ratio capable of detecting the maximum distance of the distance image generation device is obtained. A distance image generating apparatus that calculates a distance value to the object on the basis of charges accumulated in the charge accumulation unit after the secondary charge accumulation period has elapsed.