JP7605673B2 - Stereo camera device - Google Patents

Description

The present invention relates to a stereo camera device.

In recent years, in the field of automobiles as well as industrial machinery such as forklifts and hydraulic excavators, vehicles have been equipped with various safety devices to prevent accidents caused by carelessness on the part of the driver. As one safety measure, the installation of external recognition sensors that detect surrounding objects is being promoted. External recognition sensors include stereo cameras and millimeter wave radar. By using such safety devices, when objects or workers around the vehicle are detected, the driver is alerted and the vehicle's driving is controlled according to the detected surrounding conditions, thereby avoiding accidents associated with vehicle collisions.

Stereo cameras calculate distance using parallax information from images obtained by two cameras. As a result, their detection distance is shorter than that of monocular cameras, but they can detect objects at close ranges with high accuracy. Distance measurement using stereo cameras is, in principle, premised on the optical axes of the two cameras being parallel. When mounting a stereo camera on a vehicle, it is difficult to maintain a strict parallel arrangement due to changes in lens position over time caused by temperature or vibration, mechanical tolerances of the housing, etc.

A stereo camera has been proposed that includes a camera stay that is arranged side by side in a position where the optical axes of a pair of monocular cameras are aligned and connected together, an image correction means that corrects the images captured by each monocular camera, and a distance calculation means that calculates the parallax from the corrected images corrected by the image correction means and calculates the distance to the subject (see, for example, Patent Document 1). A method has been proposed in which a calibration target is placed in front of the stereo camera and a laser range finder is placed behind it, and the parameters of the stereo camera are calibrated using the distance L1 from the laser range finder to the reference surface of the camera stay and the distance L2 from the laser range finder to the calibration target (see, for example, Patent Document 2). A technology has been proposed that estimates the positional deviation of the lens from the state of image distortion captured by the stereo camera, and corrects the distortion by electronically transforming the image using image correction processing to reproduce the original image (see, for example, Patent Document 3).

The wider the angle of a camera, the more complex the image distortion that occurs and the greater the amount of correction required. It is difficult to achieve sufficient distortion correction using only geometric correction using polynomial approximation, so a mechanical structure that can stably maintain the position between the cameras is required. In environments subject to temperature and weather changes, such as construction sites, a weather-resistant structure that includes waterproofing and dustproofing is desirable.

One aspect of the present invention aims to provide a stereo camera device with improved weather resistance.

In one embodiment, the stereo camera device has multiple cameras, an electronic circuit that calculates the distance to the object based on image data captured by the multiple cameras, and a housing case that houses the multiple cameras and the electronic circuit, the housing case being formed of a first case member and a second case member, the multiple cameras being disposed in the first case member, and the electronic circuit being disposed in the second case member.

This allows for a stereo camera device with improved weather resistance.

1 is an external view of a stereo camera device according to an embodiment; FIG. 2 is a hardware configuration diagram of the stereo camera device of FIG. 1. FIG. 2B is a functional block diagram of the integrated circuit of FIG. 2A. 1A and 1B are diagrams illustrating the principle of distance measurement by a stereo camera device. 11 is a flowchart of calibration of the stereo camera device. FIG. 13 is a diagram showing the positional relationship of devices during calibration. FIG. FIG. 2 is an exploded perspective view of the stereo camera device. FIG. 2 is a vertical sectional view of the stereo camera device. FIG. 2 is a horizontal cross-sectional view of the stereo camera device.

Fig. 1 is an external view of a stereo camera device 100 according to an embodiment, and Fig. 2A is a hardware configuration diagram of the stereo camera device 100. As shown in Figs. 1 and 2A, the stereo camera device 100 has cameras 10A and 10B, a distance measurement sensor 20, an electronic circuit 50, and a housing case 101 that houses these optical devices and the electronic circuit 50. The housing case 101 is an example of a case. In this example, two cameras 10A and 10B are used, but distance measurement may be performed using three or more cameras.

1, the arrangement direction of the cameras 10A and 10B is the X direction, the optical axis direction is the Z direction, and the direction perpendicular to the X direction and the Z direction is the Y direction. As described later, the cameras 10A and 10B and the distance measuring sensor 20 are held in a predetermined positional relationship inside the housing case 101 by the holding member 30 (see Figs. 7, 8A, and 8B). The housing case 101 functions as a protective member for these optical devices and the electronic circuit 50. By arranging the cameras 10A and 10B and the distance measuring sensor 20 inside the housing case 101, robustness against collisions, dustproofness against dust and dirt, waterproofness against rain, etc. are ensured. The housing case 101 does not need to be completely sealed as long as it can protect the optical devices, electronic circuits, etc. inside. In one embodiment, the cameras 10A and 10B and the distance measuring sensor 20 are integrally held in a certain positional relationship by the holding member 30 inside the housing case 101, and positional accuracy is maintained.

The housing case 101 is, for example, a metal or alloy case formed by die casting. In the example of FIG. 1, the housing case 101 is made by aluminum die casting. The housing case 101 is formed of an upper case 15 and a lower case 40 that can be separated at an angle to the bottom surface, making it easy to maintain, repair, and replace the parts housed inside the housing case 101.

The housing case 101 has camera openings 16A and 16B that enable imaging by cameras 10A and 10B, and a distance measurement opening 17 for distance measurement by distance measurement sensor 20. At the very front of the housing case 101, a cover glass 102 is provided as a light-transmitting member. The cover glass 102 covers the camera openings 16A and 16B and the distance measurement opening 17, maintaining light transmittance while keeping the housing case 101 airtight. The cover glass 102 is preferably a single sheet of glass, and prevents optical errors caused by differences in thickness or warping between the camera openings 16A, 16B, and the distance measurement opening 17.

The electronic circuit 50 performs image processing, distance calculation processing, etc. based on the image data (captured image) of the measurement object captured by the cameras 10A and 10B, and measures the distance to the measurement object.

On the other hand, the distance measurement sensor 20 measures the distance to the object by receiving the reflected light of light irradiated on the object in order to calibrate the image distance measurement. The distance measurement sensor 20 is, for example, a ToF sensor that measures the distance to the object based on the time of flight (TOF) from when light (electromagnetic waves) is irradiated on the object to when the reflected light from the object is received.

Figure 2B is a functional block diagram of the electronic circuit 50. The electronic circuit 50 may be a microprocessor configured with a large-scale integrated circuit (LSIC) or may be a logic device such as a field programmable gate array (FPGA). The electronic circuit 50 functions as a control unit 50A, and has as its functions an image processing unit 51, a parallax calculation unit 52, a calibration calculation unit 53, a distance calculation unit 54, and a data input unit 55.

The data input unit 55 receives the captured images output from each of the cameras 10A and 10B and the distance information output from the distance measurement sensor 20. The captured images are input from the data input unit 55 to the image processing unit 51. The distance information is input from the data input unit 55 to the calibration calculation unit 53.

The image processing unit 51 applies appropriate image processing to the captured image to generate a corrected image, and supplies the corrected image to the parallax calculation unit 52. The parallax calculation unit 52 calculates the parallax p between the cameras 10A and 10B from the corrected image by a pattern matching method or the like. The parallax p is input to the distance calculation unit 54 and the calibration calculation unit 53.

The calibration calculation unit 53 calculates the distance Z from the optical center of the cameras 10A and 100B to the measurement object based on the distance information L to the measurement object obtained by the distance measurement sensor 20 and the known positional relationship between the distance measurement sensor 20 and the cameras 10A and 10B. The calibration calculation unit 53 uses the calculated distance Z and the parallax p input from the parallax calculation unit 52 to determine the B*f value and the parallax offset Δp, and supplies the B*f value and Δp to the distance calculation unit 54. Here, the B*f value is the product of the distance (baseline length) B between the actual optical axes of the cameras 10A and 10B and the actual focal length f, as described below.

The distance calculation unit calculates the distance Z to the measurement object using the parallax p input from the parallax calculation unit 52, and the parallax offset Δp and B*f value input from the calibration calculation unit 53, as follows:
Z=B×f/(p+Δp) (1)
and calculate.

Calibration of the stereo camera device 100 may be performed during product inspection of the stereo camera device 100 before it is shipped from the factory, or may be performed using a measurement target when the stereo camera device 100 is used after being installed on a moving object or the like.

<Principle of distance measurement>
3 is a diagram explaining the principle of distance measurement by the stereo camera device 100. The cameras 10A and 10B are installed at a distance B apart. The distance B is the distance between the optical axes of the cameras 10A and 10B, and is also called the baseline. The focal length of the cameras 10A and 10B is f, their optical centers are O _A and O _B , and their imaging planes are s _A and s _B . What is to be found is the distance Z from the optical centers of the cameras 10A and 10B to the measurement target A.

An image of a measurement target A, which is located at a distance Z away from the optical center OA of camera 10A in the optical axis direction (Z direction), is formed at P _A , which is the intersection point between a straight line A-O _A and the imaging plane s _A. Meanwhile, in camera 10B, the same measurement target A is formed at a position P _B on the imaging plane s _B. Here, the intersection point between a straight line PL that passes through the optical center OB of camera 10B and is parallel to the straight line A-O _A , and the imaging plane s _B is denoted as P _A '. The distance between _{P A} ' and P _B is the parallax p between cameras 10A and 10B.

Point P _A ' represents the position of image point P _A relative to the optical axis of camera 10 A. Parallax p is the combined distance between P _A ' and P _B , and corresponds to the amount of positional deviation on the images of the same measurement target A captured by the two cameras 10A and 10B.

Since triangle A-O _A -O _B and triangle O _B -P _A' -P _B are similar,
Z=B×f/p (2)
If the distance (baseline length) B between the optical axes of the cameras 10A and 10B and the focal point f are known, the distance Z can be obtained from the parallax p. However, in reality, the base line length B, focal length f, and parallax p change slightly due to fluctuations over time of the lens optical axis caused by temperature changes, vibrations, and the like. Therefore, Z is obtained from equation (1) using the parallax offset Δp as described above.
Z=B×f/(p+Δp) (1)
By measuring the distance Z to the object A to be measured from equation (1) at multiple positions, the actual values of B and f can be found using simultaneous equations.

<Proofreading process>
Fig. 4 is a flow chart of the calibration, and Fig. 5 is a diagram showing the positional relationship of the devices during the calibration. In the inspection before shipment from the factory, the calibration may be performed using a calibration target T. In the actual distance measurement, the calibration may be performed using a measurement target as described above. In Fig. 5, the distance measurement sensor 20 is drawn behind (on the -Z side) of the cameras 10A and 10B to explain the calibration process, but the distance measurement sensor 20 may be held integrally with the cameras 10A and 10B by a holding member 30.

First, as shown in FIG. 5, a calibration target T is placed in front of the stereo camera device 100 in the measurement direction (Z direction) (S1). Next, the stereo camera device 100 is placed at a predetermined position so that the light beam emitted from the distance measurement sensor 20 irradiates the feature point T1 of the calibration target T (S2).

The distance measurement sensor 20 has a light source that irradiates the calibration target T with a light beam and a photodetector that detects the reflected light from the calibration target T. The light source is, for example, a laser diode, and the photodetector is, for example, a photodiode. The optical axes of the light source and the photodetector are aligned in the same direction. The photodetector detects that the optical axes of the emitted light and the reflected light are aligned in the same direction. The distance measurement sensor 20 measures the time ToF from when the light beam is emitted from the light source to when the reflected light is received, and measures the distance L1 to the calibration target T based on the measured time (S3). The distance calculation unit 54 of the control unit 50A calculates the distance Z between the optical center O of the camera and the calibration target T from the measured distance L1 measured by the distance measurement sensor 20 and the known value ΔZ (S4). The known value ΔZ is the distance between the origin of the distance measurement sensor 20 and the optical center O of the cameras 10A and 10B, but the optical center O of the cameras 10A and 10B attached to the same holding member 30 and the origin of the distance measurement sensor 20 are in approximately the same plane, so ΔZ can be considered to be zero. When attached to the holding member 30, if there is a difference in the Z direction between the optical center O of the cameras 10A and 10B and the distance measurement limit of the distance measurement sensor 20, the distance Z is calculated using ΔZ.

On the other hand, the cameras 10A and 10B capture images of the feature points T1 of the calibration target T (S5). The captured images of the feature points T1 are processed by the image processing unit 51 to generate corrected image data (corrected image) (S6). The parallax calculation unit 52 calculates the parallax p with respect to the calibration target T from the generated corrected image (S7). The position of the calibration target T in the Z direction is changed and steps S1 to S7 are repeated two or more times (YES in S8).

The calibration calculation unit 53 uses two or more pairs of distance Z and parallax p to calculate the actual B*f value and parallax offset Δp. This B*f value and parallax offset Δp are the data for calibration.

When the stereo camera device 100 is attached to a vehicle and used at a work site, the stereo camera device 100 detects obstacles and workers in front, on the sides, and behind. If there is a risk of collision based on the relative speed, relative position, etc. between the obstacle or worker and the vehicle, an alert may be issued or automatic braking may be activated. If the distance measurement accuracy is low, the automatic brake may be activated frequently, which may interfere with work, so it is desirable to activate the automatic brake only when there is a high risk of contact with a person. To achieve this, a resolution is required that can distinguish between workers and materials, etc. In addition, it is necessary to maintain distance measurement performance even in outdoor environments with large temperature differences and the presence of dust and raindrops.

In the embodiment, the mechanical structure described below holds the cameras 10A and 10B and the distance measurement sensor 20 with a holding member 30 separate from the housing case 101, thereby maintaining the positional accuracy between the cameras 10A and 10B and the distance measurement sensor 20 as high as possible. In addition, even if the positional relationship of the optical devices changes over time due to temperature differences, vibrations, etc., the measurement accuracy of the stereo camera device 100 is compensated for by the calibration function described above.

<Device Configuration>
FIG. 6 is an exploded perspective view of the camera 10. The cameras 10A and 10B have the same configuration, and therefore will be described as one camera 10. The camera includes a lens 21 and an image sensor 23 mounted on a printed circuit board 25. The lens 21 has a plurality of lens elements inside a lens cell 211. The image sensor 23 is, for example, a CMOS sensor. The lens 21 and the printed circuit board 25 on which the image sensor 23 is mounted are incorporated into an aluminum die-cast base member 22 to constitute the monocular camera 10. The base member 22 is formed with a cylindrical screw hole 24 for receiving the lens 21. The screw hole 24 is formed coaxially with the optical axis of the lens 21 inside a cylinder 223 protruding in the Z direction of the base member 22. The printed circuit board 25 is fixed to the mounting surfaces 221 and 222 of the base member 22 so that the image sensor 23 is coaxial with the optical axis of the lens 21.

The mounting surface 221 of the base member 22 surrounds the periphery of the image sensor 23, and mounting surfaces 222 are provided at multiple locations on the outside of the mounting surface 221. As described below, the mounting surface 222 is used to temporarily fix the vicinity of the outer edge of the printed circuit board 25 at multiple locations, and the printed circuit board 25 is finally fixed at the mounting surface 221. By providing the mounting surface 222 at a position symmetrical with respect to the central axis of the image sensor 23, it is possible to prevent uneven tensile stress from occurring in the printed circuit board 25 due to the hardening and shrinkage of the adhesive during temporary fixation, and to suppress misalignment.

Figure 7 is an exploded perspective view of the stereo camera device 100, Figure 8A is a vertical cross-sectional view along the YZ plane of Figure 1, and Figure 8B is a horizontal cross-sectional view along the XZ plane of Figure 1. As shown in Figures 7 and 8B, the cameras 10A and 10B and the distance measurement sensor 20 are attached to the holding member 30. As described with reference to Figure 6, the lens 21A of the camera 10A is fitted into the screw hole 24A of the base member 22A, and the optical axis of the lens 21A is aligned with the optical axis of the image sensor 23A mounted on the printed circuit board 25A. The cylinder 223 of the base member 22A is inserted into the fitting hole 13A of the holding member 30 and positioned.

Similarly, the lens 21B of the camera 10B is fitted into the screw hole 24B of the base member 22B, and the optical axis of the lens 21B is aligned with the optical axis of the image sensor 23B mounted on the printed circuit board 25B. The cylinder 223 (see FIG. 6) of the base member 22B is inserted into the fitting hole 13B of the holding member 30 and positioned.

The cameras 10A and 10B are attached to the holding member 30 by the fitting holes 13A and 13B so that their optical axes are parallel. In the cameras 10A and 10B, the printed circuit boards 25A and 25B on which the image sensors 23A and 23B are mounted are aligned with the optical axes of the lenses 21A and 21B, respectively, and the tilt angle around the optical axis as the axis of rotation is preset. Therefore, even after the cameras 10A and 10B are attached to the holding member 30 with their optical axes parallel to each other, the optical arrangement positioned in each camera 10 is maintained.

The distance measurement sensor 20 is mounted on a printed circuit board 31. The distance measurement sensor 20 is positioned approximately halfway between the cameras 10A and 10B so that its light beam entrance/exit window is exposed within the opening 14 of the holding member 30. The distance measurement sensor 20 is attached to the holding member 30 with the optical axes of the light source and the photodetector aligned in the same direction. As a result, the cameras 10A and 10B and the distance measurement sensor 20 are held by the holding member 30 with their respective optical axes appropriately adjusted.

As shown in Figures 8A and 8B, lenses 21A and 21B are each formed by combining multiple lens elements. Lenses 21A and 21B are screwed into screw holes 24A and 24B formed in base members 22A and 22B, and are fixed so that the imaging surfaces of image sensors 23A and 23B are approximately flush with each other.

Lenses 21A and 21B are, for example, retrofocus type lenses combining multiple lens elements, and each lens element is housed in a lens cell 211 with its axis aligned. Lens cell 211 is formed with a fitting portion 62 that positions the optical axis, a screwing portion 63 that screws lens 21 in the optical axis direction with a predetermined axial force, and a locking portion 64. Generally, a wide-angle lens such as a retrofocus lens has a short focal length, and as shown in the figure, the principal point position 65 of the optical system is arranged biased toward the image sensor 23. Even with such a retrofocus type arrangement, it is desirable that the principal point position 65 does not change inside the screwing hole 24 due to positional deviation or change in attitude of each lens element.

In this embodiment, the fitting portion 62 is provided at the end of the lens cell 211 closest to the image sensor 23, thereby fixing the lens 21 at least near the principal point position 65 of the optical system and maintaining the parallelism of each lens element with respect to the optical axis.

The screwing portion 63 that screws into the screw hole 24 is provided closer to the subject (farther from the image sensor 23) than the fitting portion 62, and the locking portion 64 is provided even closer to the subject. When the lens 21 screws into the screw hole 24, the locking portion 64 abuts against the base member 22 with a predetermined axial force, and the lens 21 can be positioned relative to the image sensor 23. Even if there is expansion and contraction due to temperature changes and creep due to aging, an axial force that can withstand vibration and impact is maintained.

Lenses 21A and 21B are constrained by fitting portions 62A and 62B, respectively, so that when the temperature rises, for example, they move away from image sensors 23A and 23B due to thermal expansion. Meanwhile, principal point position 65 moves closer to image sensor 23 relative to screw-in portion 63. This reduces the fluctuation in the distance between principal point position 65 and image sensor 23, and suppresses changes in imaging characteristics due to temperature changes.

The printed circuit boards 25A and 25B on which the image sensors 23A and 23B are mounted are aligned with the lenses 21A and 21B fixed to the base members 22A and 22B, respectively, so that the imaging characteristics are appropriate. In the alignment adjustment, alignment is adjusted for six elements: the position in the optical axis (Z) direction, the coordinates in the XY plane perpendicular to the optical axis, the tilt component α around the x axis, the tilt component β around the y axis, and the tilt component γ around the z axis. After that, while maintaining the aligned position and posture, the ultraviolet (UV) curing adhesive filled in the gap between the mounting surface 222 (see Figure 6) formed on the base members 22A and 22B and the printed circuit boards 25A and 25B is irradiated with UV light to cure it.

By hardening the adhesive, the aligned printed circuit boards 25A and 25B are temporarily fixed in place, and the position and orientation of the imaging surfaces of the image sensors 23A and 23B relative to the lenses 21A and 21B are maintained.

On the other hand, a thermosetting adhesive is applied in advance to the mounting surfaces 221 (see FIG. 6) of the base members 22A and 22B, and the cameras 10A and 10B after alignment adjustment are placed in a heating furnace to harden the thermosetting adhesive. This ensures that the adhesive strength between the base members 22A and 22B and the printed circuit boards 25A and 25B is maintained even if the cameras are left in a high-temperature environment simulating the hot sun.

In the embodiment, the UV-curing adhesive is an adhesive mainly composed of acrylic resin or urethane resin, which is relatively resistant to impact and tension and highly flexible. The thermosetting adhesive is an adhesive mainly composed of epoxy resin, which is highly heat-resistant, hard and tough. Two types of mounting surfaces 221 and 222 are provided on the base member 22 to separate the application positions, and the highly flexible UV-curing adhesive is placed on the outside, thereby preventing peeling due to shear stress caused by the difference in thermal expansion coefficient between the base member 22 and the printed circuit board 25.

It is advisable to select a thermosetting adhesive that has a higher intermolecular cohesive force, i.e., Young's modulus (elastic modulus), than or is equal to that of a UV-curing adhesive. The UV-curing adhesive and the thermosetting adhesive may have the same base agent. For example, an epoxy resin may be used as the base agent, and by selecting the curing accelerator to be added, either a UV-curing type or a thermosetting type may be used depending on the design requirements of the process.

Generally, UV-curing adhesives cure with short UV exposure, so they are effective in ensuring takt time in the assembly process, but their heat resistance may be insufficient. Thermosetting adhesives have high heat resistance, but take a long time (one hour or more) to completely cure, making them difficult to incorporate into the alignment adjustment process.

Therefore, in this embodiment, once alignment adjustment is complete, temporary fixing is first performed with a UV-curing adhesive. After that, the process is designed so that multiple adjusted semi-finished products are put into a heating furnace together and a batch process is performed to cure the thermosetting adhesive. This ensures the final adhesive strength even if the mounting surface 222 of the base member 22 is a location that is difficult for the energy of UV irradiation light to reach.

As shown in FIG. 6, the mounting surface 221 of the base member 22 surrounds the periphery of the image sensor 23, and the printed circuit board 25 and the mounting surface 221 are adhesively sealed with a thermosetting adhesive to prevent dust and water droplets from entering the inside of the camera 10.

In the embodiment, an example is shown in which a UV-curable adhesive and a thermosetting adhesive are used together to shorten the takt time, but the present invention is not limited to this method. For example, a method can be used in which hot air is applied to the thermosetting adhesive for a short period of time to temporarily cure it to the extent that the alignment adjustment does not shift, and then the adhesive is placed together in a heating furnace for complete curing. The latter method also has little effect on the takt time, so it is not necessary to use a UV-curable adhesive.

In order to ensure the mechanical strength and reliability of the bond, any type of adhesive can be combined, not limited to UV-curing or heat-curing types, as long as a process for pre-curing can be added for a short period of time prior to full curing.

After adjusting the alignment of the cameras 10A and 10B individually and allowing the thermosetting adhesive to completely harden, they are inspected with a single eye and attached to the holding member 30. The cylinders 223 of the base members 22A and 22B, which are formed coaxially with the optical axis of the lenses 21A and 21B, are inserted into the fitting holes 13A and 13B of the holding member 30, and the outer surfaces of the base members 22A and 22B are abutted against the locking portions 29 (see Figure 7) in the rotational direction, thereby determining the position and orientation around the optical axis.

In the XY plane perpendicular to the optical axis of the lenses 21A and 21B, the front surfaces of the base members 22A and 22B are abutted against the abutment surface 30a of the holding member 30, and the cameras 10A and 10B are fixed to the holding member 30 by screw fastening. This abutment surface 30a functions as a positioning means common to the cameras 10A and 10B. During the alignment adjustment described above, the position of the printed circuit board 25 relative to the base member 22 can also be adjusted based on the abutment surface 30a.

In the embodiment, the cameras 10A and 10B are individually assembled to the base members 22A and 22B in advance and then fixed to the holding member 30. However, it is also possible to provide the screw holes 24 and the mounting surfaces 221 and 222 in the holding member 30 without using the base members 22A and 22B, and to directly assemble the lenses 21A and 21B and the printed circuit boards 25A and 25B to the holding member 30.

The front of the upper case 15 of the housing case 101 is formed with an opening 16 for the camera and an opening 17 for distance measurement, and the rear of the lower case 40 is formed with an opening 43 for cable wiring 400 (see FIG. 8A) that serves as an external interface. The upper case 15 and the lower case 40 have a joint surface (or dividing surface) 42 that is inclined obliquely with respect to the XZ plane (see FIG. 1) so that functions are shared between the front and rear sides of the housing case 101. The upper case 15 and the lower case 40 are connected flush with each other at the joint surface 42.

The upper case 15 and the lower case 40 each have a triangular prism shape. The upper case 15 constitutes the front and top surfaces of the housing case 101, and the lower case 40 constitutes the back and bottom surfaces of the housing case 101. The side of the upper case 15 is a triangle with its hypotenuse pointing downward, and the side of the lower case 40 is a triangle with its hypotenuse pointing upward. The rectangular housing case 101 is assembled by joining the upper case 15 and the lower case 40 at a joining surface 42 that is inclined obliquely with respect to the top surface of the upper case 15 or the bottom surface of the lower case 40 and a horizontal surface. By making each of the upper case 15 and the lower case 40 into a triangular prism shape obtained by dividing a rectangular parallelepiped case obliquely, the number of divided surfaces can be reduced, and the waterproof effect can be improved. In addition, the housing case 101 is easy to manufacture and assemble.

The upper case 15 and the lower case 40 are connected at the joint surface 42 with a sheet-like silicone rubber seal member 44 sandwiched between them to prevent dust and water droplets from entering the inside of the stereo camera device 100. In the embodiment, in order to suppress heat conduction between the upper case 15 and the lower case 40, a sheet-like seal member 44 is used to prevent the upper case 15 and the lower case 40 from coming into direct contact with each other, but this is not limited to this example. A general sealing method may also be used, such as forming a groove in one of the upper case 15 and the lower case 40 and fitting a packing into it.

The cameras 10A, 10B and the distance measurement sensor 20 are arranged in the upper case 15. As an example, the holding member 30 to which the cameras 10A, 10B and the distance measurement sensor 20 are assembled is screwed to the front inner wall of the upper case 15 in an XY plane perpendicular to the optical axis. The holding member 30 can be attached to and removed from the upper case 15 while maintaining the optical positional relationship between the cameras 10A and 10B and the distance measurement sensor 20. This makes it possible to guarantee distance measurement performance without having to individually readjust the position of each optical device when the cameras 10A, 10B and the distance measurement sensor 20 are housed in the housing case 101.

Even if an external impact is applied to the housing case 101, the provision of a holding member 30 separate from the upper case 15 makes it possible to suppress changes in the relative positional relationship and attitude between the cameras 10A, 10B and the distance measurement sensor 20. The camera openings 16A, 16B and distance measurement opening 17 formed in the upper case 15 may be formed in a diffuse shape (jagged) to prevent internal reflections, or may be painted matte black to avoid strong sunlight and suppress detection failures caused by ambient light.

A cover glass 102 is attached to the front of the upper case 15, covering the camera openings 16A, 16B and the distance measurement opening 17. The outer periphery of the cover glass 102 may be sealed with a sealing material. As described above, the cover glass 18 is made of a single sheet of glass, and even if the cameras 10A, 10B and distance measurement sensor 20 are attached and removed in an integrated state, differences in distortion and shifts in focus position that occur in each camera 10 due to variations in the thickness, warping, etc. of the cover glass 18 are suppressed.

The printed circuit board 41 on which the electronic circuit 50 is mounted is disposed in the lower case 40. As an example, the printed circuit board 41 on which the electronic circuit 50 is mounted is disposed on the inside bottom surface of the lower case 40. By removing the lower case 40 from the upper case 15, connection to the cable wiring 400, maintenance, etc. can be performed.

When the upper case 15 and the lower case 40 are assembled, the printed circuit boards 25A, 25B, and the printed circuit board 31 held in the upper case 15 and the printed circuit board 41 held in the lower case 40 are wired and connected by a flexible printed circuit (FPC) cable 48. The FPC cable 48 is folded and accommodated inside the housing case 101.

The cable wiring 400, which serves as an external interface, includes a relay connector 45, a waterproof packing 46, and a lock nut 47. The cable wiring 400 is attached by inserting the threaded portion of the relay connector 45 from the outside into an opening 43 formed on the back surface of the lower case 40, sandwiching the waterproof packing 46, and tightening the lock nut 47 from the inside. This configuration of the cable wiring 400 prevents water, oil, dust, etc. from entering the stereo camera device 100.

To improve the detection resolution, it is necessary to increase the number of pixels of the image sensor. As the number of pixels of the image sensor increases, the amount of electronic information processed inevitably increases, and the heat generated by the electronic circuit 50 also increases.

In the embodiment, a heat dissipation path is provided in the lower case 40, and heat generated in the printed circuit board 41 mounting the electronic circuit 50 formed by an LSIC, FPGA, etc. is released from the lower case 40 to the outside. Also, as described above, the printed circuit board 25 mounting the image sensor 23 is joined to the base member 22 with a thermosetting or UV-curing adhesive that is difficult to transmit heat, so that heat generated in the printed circuit board 25 is mainly transferred to the printed circuit board 41 via the FPC cable 48. For example, the ground patterns of the FPC cable 48 and the printed circuit board 41 may be stacked as solid wiring, thereby efficiently transferring heat to the lower case 40.

The electronic circuit 50 mounted on the printed circuit board 41 may be attached to the lower case 40 by heat dissipation grease. Heat dissipation grease is a paste-like resin made by mixing silicone oil with a powder of good thermal conductivity such as alumina. The electronic circuit 50 is attached to the lower case 40 by applying heat dissipation grease to the base surface formed on the lower case 40 and assembling the printed circuit board 41 to the lower case 40.

The area of the base of the lower case 40 to which the thermal grease is applied is determined based on the amount of heat (power consumption) generated by the heat sources such as LSIs and FPGAs, and the thermal resistance of the thermal grease. A sheet-like silicone pad may be used instead of thermal grease. In the embodiment, a paste-like heat transfer material is used that is uncured when the printed circuit board 41 is assembled, and flexibly spreads and settles according to the gap between the printed circuit board 41 and the base surface, preventing excessive pressure from being applied to the printed circuit board 41.

The outer surface of the lower case 40 is formed with heat dissipation fins 401 to increase the surface area. The heat dissipation properties may be further improved by subjecting the heat dissipation fins 401 and the outer surface of the lower case 40 to a surface treatment such as black painting. A thermal circuit is formed so that the heat transferred to the lower case 40 is efficiently radiated to the outside air and does not accumulate due to natural convection.

As described above, the imaging function and the calculation function are shared by consolidating the optical system including the camera in the upper case 15 and arranging the electronic circuit 50 in the lower case 40. By combining the upper case 15 and the lower case 40 with a seal member 44 having a higher thermal resistance than either the upper case 15 or the lower case 40 sandwiched between them, heat generated in the printed circuit board 41 mounting the electronic circuit 50 is dissipated from the lower case 40. The seal member 44 with low thermal resistance acts as a buffer material to suppress heat transfer from the lower case 40 to the upper case 15. This configuration minimizes the effect of heat on the base members 22A and 22B that hold the lenses 21A and 21B, and stably maintains the position and orientation of the lenses 21A and 21B for a long period of time.

The stereo camera device 100 can be mounted directly to a frame constituting the head guard of, for example, a forklift or hydraulic excavator. In this embodiment, the upper case 15 is provided with a mounting portion 49 for the vehicle. When mounting the stereo camera device 100 to the vehicle, the field of view can be optically adjusted to maintain the distance measurement performance of the cameras 10A and 10B.

When the stereo camera device 100 is mounted on a forklift or hydraulic excavator, it is necessary to estimate not only the contact distance between the vehicle and the object, but also the contact distance between the load end and the object. In a system in which the stereo camera device 100 is arranged at the front and rear to enable 360-degree detection corresponding to rotational movement, it is necessary to ensure the continuity of the field of view. Therefore, when mounting the stereo camera device 100 on a vehicle, it is desirable to be able to adjust the object identification function according to the situation at the site. When it becomes necessary to update the object identification function, to malfunction the electronic circuit 50, or to replace parts due to deterioration of the seal member 44 over time, the lower case 40 can be removed while the upper case 15 holding the cameras 10A and 10B is fixed to the vehicle, and maintenance work can be performed. The stereo camera device 100 has a configuration suitable for mounting on a vehicle and can maintain distance measurement performance. The stereo camera device 100 is suitable for mounting not only on vehicles, but also on flying objects such as drones and moving objects such as ships.

The cameras 10A and 10B and the distance measuring sensor 20 are held together by a holding member 30 separate from the housing case 101, so that the positional relationship between the cameras 10A and 10B and the distance measuring sensor 20 is accurately maintained. If optical devices such as the cameras 10A and 10B and the distance measuring sensor 20 are directly attached to the housing case 101, the effects of vibration and heat on the optical system will be large, but the configuration of the embodiment minimizes the effects of the environment on the imaging system and maintains distance measurement accuracy. Since the holding member 30 is fixed to the upper case 15, the positional relationship between the imaging system including the cameras 10A and 10B and the distance measuring sensor 20 and the vehicle is maintained even when the lower case 40 is removed. Note that the distance measuring sensor 20 and the cameras 10A and 10B do not necessarily need to be held together by the holding member 30 as long as they are maintained in a predetermined positional relationship. For example, the distance measurement sensor 20 and the cameras 10A and 10B may be attached to separate holding members, and the holding members may be combined or connected to each other so that the positional relationship between the distance measurement sensor 20 and the cameras 10A and 10B is maintained.

The stereo camera device 100 suppresses degradation of distance measurement performance even in outdoor environments. When the stereo camera device 100 is attached to a moving object such as a vehicle or drone, it calculates distance from parallax regardless of the moving speed or direction of the moving object, allowing for accurate object detection and distance estimation in close-range areas. In addition, by using a retrofocus lens, it is possible to distinguish between workers and materials in a wide-angle near field of view with a viewing angle of 120° or more.

10A, 10B Camera 13 Fitting hole (positioning means)
15 Upper case (first case member)
16A, 16B Camera opening 17 Distance measurement opening 20 Distance measurement sensor 22 Base member 25, 25A, 25B, 31, 41 Printed circuit board 30 Holding member 30a Contact surface 40 Lower case (second case member)
42 Joint surface 43 Opening (for external interface)
44 Sealing member 49 Mounting portion 50 Electronic circuit 50A Control portion 51 Image processing portion 52 Parallax calculation portion 53 Calibration calculation portion 54 Distance calculation portion 55 Data input portion 100 Stereo camera device 101 Housing case (case)
102 Cover glass 400 Cable wiring (external interface)
401 Heat dissipation fin

JP 2012-189324 A JP 2012-167944 A Japanese Patent Application Publication No. 11-325890

Claims (9)

Multiple cameras and
an electronic circuit for calculating a distance to an object based on image data captured by the plurality of cameras;
a housing case that houses the cameras and the electronic circuit;
having
the housing case is formed of a first case member and a second case member, the cameras are disposed in the first case member, and the electronic circuit is disposed in the second case member ,
the first case member and the second case member are separated obliquely with respect to a bottom surface of the housing case ;
Stereo camera device. Multiple cameras and
an electronic circuit for calculating a distance to an object based on image data captured by the plurality of cameras;
a housing case that houses the cameras and the electronic circuit;
having
the housing case is formed of a first case member and a second case member, the cameras are disposed in the first case member, and the electronic circuit is disposed in the second case member,
the first case member and the second case member are connected to each other at a joint surface that is inclined with respect to a bottom surface of the housing case, and communicate with each other internally.
Stereo camera device . Multiple cameras and
an electronic circuit for calculating a distance to an object based on image data captured by the plurality of cameras;
a housing case that houses the cameras and the electronic circuit;
having
the housing case is formed of a first case member and a second case member, the cameras are disposed in the first case member, and the electronic circuit is disposed in the second case member,
The first case member and the second case member are in the shape of a triangular prism.
Stereo camera device . The first case member constitutes a front surface and a top surface of the housing case, and the second case member constitutes a back surface and a bottom surface of the housing case.
The stereo camera device according to claim 1 . the first case member has a plurality of openings exposing lens surfaces of the plurality of cameras, and the second case member has an opening receiving an external interface connected to the electronic circuit;
The stereo camera device according to claim 1 . The first case member and the second case member are joined with a seal member having a higher thermal resistance than either the first case member or the second case member.
The stereo camera device according to claim 1 . The second case member has a heat dissipation portion.
The stereo camera device according to claim 1 . The first case member has an attachment portion for attachment to a movable body.
The stereo camera device according to claim 1 . The second case member is detachable from the first case member in a state in which the first case member is attached to the movable body.
The stereo camera device according to claim 8 .

Images