JP7705789B2 - Object detection system for work machines - Google Patents

Description

The present invention relates to an object detection system for a work machine.

For work machines such as hydraulic excavators, a detection system technology is known that restricts vehicle body movement when it detects a worker or object near the work machine.

For example, Patent Document 1 states, "An object detection system that detects objects present around a construction machine that has an upper rotating body mounted on a lower running body via a rotating mechanism has an object detection unit that detects objects based on the output of a scanning distance measurement device attached to the upper rotating body, and the light emitted by the scanning distance measurement device passes through the gap between the lower running body and the upper rotating body. (Summary excerpt)."

JP 2015-229836 A

Detection systems use relatively expensive detection devices such as infrared sensors and laser scanners. In large work machines such as hydraulic excavators, multiple such detection devices are installed to minimize blind spots. Therefore, installing multiple devices to cover blind spots is very costly.

In addition, these devices are often mounted on the top of the vehicle to prevent damage or dirt during operation, which means blind spots are likely to occur near the work machine or directly below the vehicle. Therefore, it is desirable to be able to compensate for blind spots cheaply and efficiently.

In particular, if the vertical detection range is limited to the plane between the upper rotating body and the lower running body, it is possible to exclude from the detection target the lower running body, which rotates relative to the upper rotating body, but there is an issue that it is not possible to detect workers or other obstacles that are located lower than the underside of the upper rotating body.

The present invention was made in consideration of the above-mentioned circumstances, and aims to provide an object detection system for a work machine that can detect objects in blind spots more inexpensively.

In order to solve the above problems, the present invention provides an object detection system for a work machine that detects objects present around a work machine that has an upper rotating body mounted on a lower traveling body via a rotating mechanism, comprising: an imaging device that is provided on the upper part of the upper rotating body and includes the ground around the work machine as an imaging area, and captures the imaging area to generate an image; a light that emits light toward the ground included in the imaging area of the imaging device; and a controller that is connected to the imaging device and detects objects around the work machine based on the captured image acquired from the imaging device, and the controller determines whether the shadow of the object present on the optical path from the light to the ground included in the imaging area is captured in the captured image based on image feature amounts of the captured image, and detects the object based on the determination result.

The present invention provides an object detection system for a work machine that can detect objects in blind spots more inexpensively. Problems, configurations, and advantages other than those described above will become clear from the description of the embodiments below.

FIG. 2 is a top view showing the configuration of a hydraulic excavator. FIG. FIG. 2 is a block diagram showing the hardware configuration of a hydraulic excavator. FIG. 4 is a block diagram showing the functional configuration of a controller. FIG. 2 is an explanatory diagram of a detection range in the object detection system of the work machine according to the first embodiment. 4 is a flowchart showing a processing flow of the object detection system according to the first embodiment. 13 is an explanatory diagram (arranged in the left-right direction) of a detection range in an object detection system for a work machine according to a second embodiment. FIG. 13 is an explanatory diagram (arranged in the vertical direction) of a detection range in an object detection system for a work machine according to a second embodiment. FIG. FIG. 13 is an explanatory diagram showing the processing content of the third embodiment. 13 is a flowchart showing a processing flow of an object detection system according to a third embodiment.

The following describes in detail an embodiment of the present invention with reference to the drawings. In all drawings used to explain the embodiment, components having the same function are given the same or related reference numerals, and repeated explanations are omitted.

The external appearance of the hydraulic excavator 10 will be described with reference to Figures 1 and 2. Figure 1 is a top view showing the configuration of the hydraulic excavator 10. Figure 2 is a left side view of the hydraulic excavator 10.

Figure 1 shows a hydraulic excavator 10 equipped with an object detection system 1. The hydraulic excavator 10 includes a left crawler 11L, a right crawler 11R (corresponding to a lower running body), an upper rotating body 12 supported on the lower running body so as to be freely rotatable via a rotating mechanism, a cab 13 attached to the upper rotating body 12, and a front work machine attached to the upper rotating body 12.

The front work machine includes a boom 14 supported on the upper rotating body 12 so as to be able to swing up and down, an arm 15 supported on the boom 14 so as to be able to swing up and down (see Figure 2), and a bucket 16 supported on the arm 15 so as to be able to swing.

In addition, actuators 106 (see FIG. 3) such as hydraulic motors and hydraulic cylinders are provided to perform forward and backward movements using the lower traveling body, rotation of the upper rotating body 12, and swinging movements of the boom 14, arm 15, and bucket 16. The hydraulic motor is provided for the swinging movement, and the hydraulic cylinder is provided for the swinging movement of the boom 14, etc. Note that although a hydraulic system is assumed here, this is not restrictive and an electric system such as an electric motor or linear actuator may also be used.

A left rear camera 20LB (corresponding to the imaging device of the present invention) is disposed on the upper left rear side of the upper rotating body 12. The imaging area of the left rear camera 20LB is indicated by the symbol 21LB.

On the upper rotating body 12, below the position where the left rear camera 20LB is attached, a left rear light 25LB (see FIG. 2) is provided that emits light toward the ground 22LB included in the imaging area 21LB. The left rear light 25LB emits light with wavelengths ranging from visible light to near infrared light.

A central rear camera 20CB (corresponding to the imaging device of the present invention) is disposed at the center rear of the upper part of the upper rotating body 12, with the ground 22CB (see FIG. 5) at the center rear of the hydraulic excavator 10 included in its angle of view. The imaging area of the central rear camera 20CB is indicated by the symbol 21CB.

On the upper rotating body 12, below the position where the central rear camera 20CB is attached, a central rear light 25CB (see FIG. 5) is provided that emits light toward the ground 22CB included in the imaging area 21CB.

A right rear camera 20RB (corresponding to the imaging device of the present invention) is disposed on the right rear of the upper part of the upper rotating body 12 so that the ground to the right rear of the hydraulic excavator 10 is included in the field of view. The imaging area of the right rear camera 20RB is indicated by the symbol 21RB.

On the upper rotating body 12, below the position where the right rear camera 20RB is attached, a right rear light 25RB (see FIG. 5(d)) is provided that emits light toward the ground included in the imaging area 21RB.

A right camera 20R is disposed on the right side of the upper part of the upper rotating body 12, with the field of view including the ground to the right of the hydraulic excavator 10. The imaging area of the right camera 20R is indicated by the symbol 21R.

On the upper rotating body 12, below the position where the right camera 20R is attached, a right light 25R (see FIG. 5(d)) is provided that emits light toward the ground included in the imaging area 21R.

The left rear camera 20LB, the center rear camera 20CB, the right rear camera 20RB, and the right camera 20R may have the same configuration, and are named differently depending on their placement. Therefore, when there is no need to specifically explain where they are placed, they may simply be referred to as cameras. The same applies to the imaging area, the ground, and the light.

The cab 13 is equipped with operating levers 18 including a right lever and a left lever as shown in FIG. 2. The right lever is operated forward and backward to lower and raise the boom 14, left and right to lower and raise the bucket 16, the left lever is operated forward and backward to push out and retract the arm 15, and left and right to input instructions to rotate the upper rotating body 12 left or right.

The cab 13 is also equipped with a controller 110 that houses a processor 111 that controls the hydraulic excavator 10, and a notification device 60 that notifies the result of object detection. The notification device 60 is, for example, a display, a buzzer, or a warning light.

Figure 3 is a block diagram showing the hardware configuration of the hydraulic excavator 10.

The controller 110 includes a processor 111 consisting of a CPU (Central Processing Unit) etc., a RAM (Random Access Memory) 112, a ROM (Read Only Memory) 113, storage 114, an input I/F 115, and an output I/F 116, which are connected to each other via a bus 117.

Storage 114 can be any type of device that can store data non-volatilely, such as a hard disk drive (HDD) or a solid state drive (SSD).

The input I/F 115 is connected to the operation lever 18, the left rear camera 20LB, the center rear camera 20CB, the right rear camera 20RB, and the right camera 20R.

The left rear light 25LB, the center rear light 25CB, the right rear light 25RB, the right light 25R, and the notification device 60 are connected to the output I/F 116.

The hydraulic excavator 10 also includes a pilot pump 102, a main pump 103, an electromagnetic proportional valve 104, a control valve 105, an actuator 106, an engine 107 serving as a power source, and an ECU (Engine Control Unit) 108. For ease of explanation, it has been described as having one each of the electromagnetic proportional valve 104, the control valve 105, and the actuator 106, but in reality, multiple systems may be provided.

The operation signal from the potentiometer, which is converted into an electrical signal corresponding to the amount of operation of the operating lever 18 and output, drives the electromagnetic proportional valve 104 via the controller 110, and the pilot pressure output from this electromagnetic proportional valve 104 drives the control valve 105 to operate the actuator 106. The pilot pump 102 delivers pressurized oil to the electromagnetic proportional valve 104, and the main pump 103 delivers pressurized oil to the control valve 105.

The output I/F 116 is connected to the solenoid proportional valve 104, and outputs a close signal to the solenoid proportional valve 104 when the result of object detection is to stop the swing, forward, or reverse motion of the hydraulic excavator 10. The output I/F 116 is also connected to the ECU 108. An engine speed control signal (including a signal to reduce the speed or a stop signal) is output from the controller 110 to the engine 107 via the ECU 108, resulting in a reduction in the discharge rate of the main pump 103 or a stoppage. When swing, forward, or reverse motion is performed, a signal to increase the speed is output as the engine speed control signal.

Figure 4 is a block diagram showing the functional configuration of the controller 110.

The processor 111 executes the functions of each of the following units: the operation signal determination unit 111a, the light control unit 111b, the camera control unit 111c, the object detection unit 111d, and the operation control unit 111e. Each of these units is realized by the processor 111 loading an object detection program into the RAM 112 and executing it.

Storage 114 stores object determination threshold data used to determine the size of the shadow.

First Embodiment
The first embodiment relates to an object detection system in which one camera and one light are combined. Fig. 5 is an explanatory diagram of the detection range in the object detection system 1 for a work machine according to the first embodiment. Fig. 5 explains the positional relationship between the imaging area 21CB of the central rear camera 20CB and the central rear light 25CB as an example.

As shown in FIG. 5(a), the central rear camera 20CB has a blind spot 29CB below the upper rotating body 12 and between the left crawler 11L and the right crawler 11R. For ease of illustration, FIG. 5(a) illustrates the blind spot 29CB as seen from the right side with the right crawler 11R not shown. The same is true for FIGS. 5(b) and (c).

The worker 91CB2 is included in the imaging area 21CB and is therefore captured in the image captured by the central rear camera 20CB. Therefore, the worker 91CB2 is detected by the object detection system 1.

However, as shown in FIG. 5(b), if the worker 91CB1 is in a low posture within the blind spot area 29CB, the worker 91CB1 is not included in the imaging area 21CB and therefore is not captured in the image captured by the central rear camera 20CB.

The central rear light 25CB is therefore attached to the bottom of the upper rotating body 12 in a position and orientation where the shadow 92CB1 of the worker 91CB1 is projected onto the ground 22CB. In other words, the central rear light 25CB is attached below the upper rotating body 12 so that it shines light at an angle where the shadow of an object present in the blind spot area 29CB between the outside of the angle of view of the central rear camera 20CB and the hydraulic excavator 10, or more preferably in the space below the upper rotating body 12, reaches the ground 22CB behind the hydraulic excavator 10.

As shown in FIG. 5(a), when there is nothing in the blind spot area 29CB, there is nothing blocking the light path 26CB from the central rear light 25CB to the ground 22CB, so no shadows of objects are cast on the ground 22CB behind the hydraulic excavator 10, and an image with minimal changes in image features is obtained.

On the other hand, as shown in FIG. 5(b), when an operator 91CB1 is in the blind spot area 29CB, the operator 91CB1 blocks the optical path 26CB, and a shadow 92CB1 of the operator 91CB1 is projected onto the ground 22CB behind the hydraulic excavator 10. Therefore, a captured image with a larger change in image feature amount is obtained compared to FIG. 5(a).

Furthermore, FIG. 5(c) shows a state in which a worker 91CB3 is hiding under the body of the upper revolving body 12. In this case, at the mounting position of the central rear light 25CB shown in FIG. 5(b), no worker is present on the light path 26CB, so no shadow is cast. Therefore, the mounting position of the central rear light 25CB is disposed at the lower end of the upper revolving body 12 and forward of the rearmost part of the upper revolving body 12. The mounting angle of the central rear light 25CB is such that the light emitted by the central rear light 25CB reaches the ground 22CB behind the hydraulic excavator 10. As a result, the shadow 92CB3 of the worker 91CB3 present between the central rear light 25CB and the ground 22CB behind the hydraulic excavator 10 is projected onto the ground 22CB behind the hydraulic excavator 10, and a captured image with a larger change in image feature amount is obtained compared to FIG. 5(a).

In this way, when a shadow appears in the captured image of the ground 22CB behind the hydraulic excavator 10, image features such as luminance, saturation, and brightness of the captured image change between the area where the shadow is captured and the surrounding area. Whether a shadow is captured is determined based on this change in image features, and an object that exists outside the capture area 21CB of the central rear camera 20CB is detected based on the determination result.

Figure 5 (d) shows the mounting positions of the left rear light 25LB, the center rear light 25CB, the right rear light 25RB, and the right light 25R.

The left rear light 25LB, the center rear light 25CB, the right rear light 25RB, and the right light 25R are each disposed at the bottom of the upper rotating body 12. They are attached in such a direction that the shadow 92CB1 of the worker 91CB1 hiding under the body of the hydraulic excavator 10 and the shadow 92R of the worker 91R positioned close to the right side of the hydraulic excavator 10 can be captured by the center rear camera 20CB and the right camera 20R.

In FIG. 5, the positional relationship between the central rear camera 20CB and the central rear light 25CB is mainly described as an example, but the same applies to the right rear camera 20RB, the left rear camera 20LB, and the right camera 20R.

Figure 6 is a flowchart showing the processing flow of the object detection system 1 according to the first embodiment.

When the engine 107 of the hydraulic excavator 10 is started, the main power supply of the controller 110 is turned on, and the operation signal determination unit 111a waits to receive an operation signal from the operation lever 18. If no operation signal is received (S01: No), the unit continues to wait.

The operation signal determination unit 111a determines the operation indicated by the acquired operation signal, and when it determines that the acquired operation signal is a right turn, a left turn, forward movement, or reverse movement (S01: Yes), it outputs the operation content indicated by the operation signal to the light control unit 111b.

The light control unit 111b determines which lights to turn on and in what order according to the operation content (S02).

For example, if a right turn is about to be made, the right light 25R, right rear light 25RB, center rear light 25CB, and left rear light 25LB will be lit in this order; if a left turn is about to be made, the order will be reversed; if moving forward, the right rear light 25RB and right light 25R will be lit from back to front; and if moving backward, the right light 25R and right rear light 25RB will be lit from front to back, to give a warning of the operation of the hydraulic excavator 10.

The light control unit 111b outputs a lighting signal to the light that is to be turned on first according to the lighting order, and the light turns on (S03).

The light control unit 111b outputs information indicating the lit lights to the camera control unit 111c, and the camera control unit 111c captures an image using the camera corresponding to the lit light (S04), and outputs the captured image to the controller 110. After capturing the image, the light is turned off.

If all of the lights determined in S02 have not been turned on (S05: No), the light control unit 111b outputs a turn-on signal for the next light (S06). Then, the process returns to S04.

When the light control unit 111b turns on all the lights determined in S02 (S05: Yes), the object detection unit 111d performs subject detection in the captured image and image analysis based on image features (S07).

Specifically, the object detection unit 111d acquires images captured by each camera, performs subject detection on the captured images, performs object recognition processing on the detected subjects, and determines whether an object such as a person, reflective material, or a car has been captured within the detection range of the captured image. If it determines that an object has been captured (S08: Yes), it outputs notification information indicating that an object has been detected to the notification device 60. Furthermore, the object detection unit 111d outputs information indicating that an object has been detected to the operation control unit 111e, and the operation control unit 111e outputs a close signal to the solenoid proportional valve 104 (S09).

If the object detection unit 111d determines that an object has not been detected (S08: No), it determines whether a shadow has been captured based on the image feature of the captured image (S10). As an example of image analysis, the brightness of the captured image is relatively bright when there is no shadow, and the brightness of the captured image is reduced when a shadow is present, so the shadow is captured using this image feature to estimate the presence of an object in the blind spot.

When the object detection unit 111d determines that a shadow has been captured (S10: Yes), it compares the size of the shadow with an object determination threshold for determining whether or not operation needs to be stopped (S11). When it determines that the size of the shadow is equal to or greater than the object determination threshold (S11: Yes), it outputs notification information indicating that an object has been detected to the notification device 60. Furthermore, the object detection unit 111d outputs information indicating that an object has been detected to the operation control unit 111e, and the operation control unit 111e outputs a close signal to the solenoid proportional valve 104 (S09).

The "object determination threshold" is a threshold determined based on the size of the shadow of an object of the size to be detected. For example, if a pylon is not detected as an object, but an object larger than a pylon, such as a worker working in a crouched position, is detected, the length of the shadow of the worker working in a crouched position is measured in advance and stored in storage 114 as the object determination threshold. Then, in step S11, the length of the shadow reflected in the captured image is compared with the object determination threshold, and if the length of the shadow reflected in the captured image is equal to or greater than the object determination threshold, it is determined that the object to be detected is present.

If the object detection unit 111d determines that no shadow is detected (S10: No), or that the size of the shadow is less than the object determination threshold (S11: No), the operation control unit 111e outputs an open signal to the electromagnetic proportional valve 104, and the hydraulic excavator 10 starts operating in accordance with the operation signal (S12).

If turning, moving forward, or reversing is to be continued (S13: Yes), the process returns to S03 and the lights start to light up in the next lighting cycle. If turning, moving forward, or reversing is to be stopped (S13: No), the process ends.

According to this embodiment, light is shone from the blind spot of each camera toward the ground within the imaging area of the camera, so if there is an object on the light path from the light to the ground within the imaging area, the presence of the object in the blind spot can be estimated by capturing the shadow reflected in the imaging area.

Furthermore, based on the size of the shadow, the size of the object can be estimated, and objects that you do not want to detect, such as pylons, can be excluded, while still detecting workers hidden inside the body of a hydraulic excavator, making it possible to detect objects in blind spots while preventing false positives.

Furthermore, the imaging area includes the ground, and subject detection is based on this to determine the presence or absence of an object within the camera's detection range, preventing false detection of unevenness in the ground while also making it possible to include objects near the lower vehicle body, which were blind spots in conventional examples, within the detection range.

Plus, lights and cameras are inexpensive compared to laser scanners or infrared devices, so if they are damaged they can be repaired cheaply.

In the first embodiment, the device may be configured to notify the detection of an object and stop operation (S09) when a shadow is captured in the captured image, without estimating the size of the object from the size of the shadow. In that case, step S11 in FIG. 6 may be omitted.

Second Embodiment
In the second embodiment, a plurality of lights are arranged for one camera to detect the size of a shadow more accurately.

Figure 7 is an explanatory diagram (arranged in the left-right direction) of the detection range in the object detection system for a work machine according to the second embodiment. In Figure 7, an example is described in which three lights, central rear lights 25CB1, 25CB2, and 25CB3, are arranged along the left-right direction of the upper rotating body 12 for the central rear camera 20CB.

As shown in FIG. 7(a), the central rear lights 25CB1, 25CB2, 25CB3 are arranged in a line at the rear of the upper rotating body 12 from left to right of the upper rotating body 12. Each of the central rear lights 25CB1, 25CB2, 25CB3 is attached to the hydraulic excavator 10 at a position and angle that irradiates light from a different angle onto the ground 22CB included in the imaging area 21CB of one central rear camera 20CB.

The worker 91CB1 is working in a low position at the center rear of the hydraulic excavator 10 and is located in the blind spot 29CB (see FIG. 5) of the center rear camera 20CB.

Figure 7 (a) shows a state in which the central rear lights 25CB1, 25CB2, and 25CB3 are all turned off. In this case, the shadow 92CB1 of the worker 91CB1 is not projected onto the ground.

Here, if only the central rear light 25CB1, which is the leftmost of the three central rear lights 25CB1, 25CB2, and 25CB3, is turned on, the worker 91CB1 blocks the light path 26CB1 of the central rear light 25CB1, and a shadow 92CB1 is cast, as shown in FIG. 7(b).

Next, of the three central rear lights 25CB1, 25CB2, and 25CB3, when only the centrally located central rear light 25CB2 is turned on, the worker 91CB1 blocks the light path 26CB2 of the central rear light 25CB2, so a shadow 92CB1 is projected at the position shown in FIG. 7(c).

Furthermore, if only the central rear light 25CB3, which is the rightmost of the three central rear lights 25CB1, 25CB2, and 25CB3, is turned on, the worker 91CB1 blocks the light path 26CB3 of the central rear light 25CB3, and a shadow 92CB1 is projected at the position shown in FIG. 7(d).

In this way, three central rear lights 25CB1, 25CB2, 25CB3 are arranged to illuminate the imaging area 21CB of the central rear camera 20CB from different angles, and by repeatedly switching the central rear lights 25CB1, 25CB2, 25CB3 to turn on, capture an image, turn off, turn on, capture an image, turn off, the size and position of the shadow of the same obstacle changes, and the size and position of the object can be estimated based on this.

For example, as in the first embodiment, if a pylon is not detected as an object, but an object larger than a pylon, such as a worker working in a crouched position, is detected, the shadow of the worker working in a crouched position is captured in advance by switching between the central rear lights 25CB1, 25CB2, and 25CB3, and the size of the worker's shadow is stored in storage 114 as the object determination threshold corresponding to each light. Then, in step S11, the length of the shadow reflected in the captured image captured with each light turned on is compared with the object determination threshold corresponding to that light, and if the length of the shadow reflected in any one of the captured images is equal to or greater than the object determination threshold, it is determined that an object to be detected is present.

In FIG. 7, the positional relationship between the central rear camera 20CB and the central rear lights 25CB1, 25CB2, and 25CB3 is described as an example, but the same applies to the left rear camera 20LB, the right rear camera 20RB, and the right camera 20R.

Figure 8 is an explanatory diagram (arranged in the vertical direction) of the detection range in the object detection system of the work machine according to the second embodiment. In Figure 8, an example is described in which two lights, central rear lights 25CB4 and 25CB5, are arranged vertically relative to the central rear camera 20CB. Note that for convenience of illustration, in Figure 8, the worker 91CB1 is shown as seen from the right side with the right crawler 11R not shown.

When only the lower center rear light 25CB4 is turned on, the worker 91CB1 blocks the light path 26CB4, so a shadow 92CB1 with length L_25CB4 is projected at the position shown in Figure 8(a).

When only the upper center rear light 25CB5 is turned on, the worker 91CB1 blocks the light path 26CB5, so a shadow 92CB1 with length L_25CB5 is projected at the position shown in FIG. 8(b).

In FIG. 8, the positional relationship between the central rear camera 20CB and the central rear lights 25CB4 and 25CB5 is explained as an example, but the same applies to the left rear camera 20LB, the right rear camera 20RB, and the right camera 20R.

7 and 8, the blinking order of the multiple central rear lights 25CB1, 25CB2, 25CB3, 25CB4, and 25CB5 may be changed to be used as a warning light for the turning, forward, or backward movement of the hydraulic excavator 10. For example, in the example of FIG. 7, when the control lever 18 receives an input operation for an instruction to perform a turning operation, the processor 111 receives an operation signal of the control lever 18, and based on this, a lighting signal may be output so that the turning direction, for example, in the case of a right turn, the central rear lights 25CB3, 25CB2, and 25CB1 are repeatedly blinked in this order.

The processing flow of the object detection system in the second embodiment is similar to the processing flow of the first embodiment shown in Figure 6, so we will use the flowchart in Figure 6 and only explain the differences from the first embodiment.

In step S02, the lighting order is determined for each of the multiple lights provided for one camera.

In addition, in determining the size of the shadow in step S11, image analysis is performed on each captured image taken with one camera while a light that emits light from different angles is turned on. Then, if it is determined that the size of the shadow in one or more captured images is equal to or greater than the object determination threshold (S11: Yes), a notification that an object has been detected is sent and operation is stopped (S09).

In addition, in step S11, the shadow of the same object is captured from multiple directions, so the size of the object itself may be estimated from the size of the shadow to determine whether or not it is a target object for detection. For example, in the example of FIG. 8, the height of the object may be calculated based on the shadow lengths L_25CB4, L_25CB5 and the mounting angles of the central rear lights 25CB4, 25CB5 to estimate the size of the object.

According to the second embodiment, multiple lights can be used to illuminate the imaging area of a single camera, allowing detection of objects hidden in the camera's blind spot, enabling object detection over a wider range.

In addition, even if a certain light accidentally casts a short shadow due to the relative position or orientation of the light and the worker 91CB1, other lights will cast a relatively long shadow of the worker 91CB1, improving the accuracy of object detection. It is also possible to estimate the size of an object from the length of the shadow cast in different captured images.

In addition, because shadows are created with the lights and the camera captures and judges the difference in brightness, there are few false positives even if the lights are slightly dirty. Also, by using multiple lights and changing the order in which they are turned on to create shadows, even if one light is dirty, the others can compensate, reducing false positives.

Third Embodiment
The third embodiment is an embodiment in which a dimmable light is used as a light for illuminating a blind spot area. The "dimmable light" here refers to a light whose light quantity and wavelength can be changed. Fig. 9 is an explanatory diagram showing the processing contents of the third embodiment. In Fig. 9, the central rear light 25CB is taken as an example for explanation, but the same applies to the lights provided corresponding to the left rear camera 20LB, the right rear camera 20RB, and the right camera 20R.

Figure 9(a) shows a situation where the ambient light is relatively strong during the day. The controller 110 outputs a dimming signal to increase the light intensity of the central rear light 25CB or change the wavelength to emit light of a different color from the ambient light so that the contrast ratio between the shadow of the worker 91CB1 and the image of the ground captured under the ambient light is good even in the case of strong ambient light in the captured image.

On the other hand, at night there is little or no ambient light. Therefore, as shown in FIG. 9(b), the light output of the central rear light 25CB is reduced to reduce power consumption when it is turned on.

In the evening, the ambient light changes from orange to red. Therefore, as shown in FIG. 9(c), the wavelength of the light of the central rear light 25CB is changed to emit a color that is easily distinguished from the ambient light, for example, a complementary color between orange and red. This prevents the light of the central rear light 25CB from being mixed in with the ambient light, making it easier to distinguish shadows.

Figure 10 is a flowchart showing the processing flow of the object detection system according to the third embodiment. Steps that are the same as those in the flowchart according to the first embodiment shown in Figure 6 are given the same reference numerals, and duplicate explanations will be omitted.

In the third embodiment, a dimming process is added following step S04. Specifically, following step S04, a dimming sequence start determination step is executed (S31). When starting the dimming sequence (S31: Yes), the object detection unit 111d analyzes the amount and wavelength of ambient light based on image features of the captured image, such as the distribution (histogram) of luminance, brightness, and saturation (S32), and outputs the results to the light control unit 111b. The light control unit 111b adjusts the light according to the analysis result. Then, the process proceeds to step S05.

In this embodiment, the ambient light was analyzed based on the captured image, but if the hydraulic excavator 10 is equipped with an illuminance sensor, the light control unit 111b may obtain illuminance data from the illuminance sensor and determine the light intensity based on the amount of ambient light.

Alternatively, if the controller 110 is equipped with an RTC (Real Time Clock), the light control unit 111b may obtain time data from the RTC and, depending on the time, increase the amount of light during the day, switch to a complementary color from orange to red in the evening, and switch to a power saving mode at night.

When dimming is performed based on an illuminance sensor or time, the determination to start the dimming sequence (S31) does not have to be made after step S04, but may be made at any time before S03.

The dimming sequence may be configured to run once and then run again for several hours, for example once every six hours.

According to the third embodiment, the light level is adjusted according to the surrounding light conditions to capture an image of a shadow, thereby improving the accuracy of detecting objects hiding in blind spots while suppressing the effects of surrounding light.

The above embodiment is not intended to limit the present invention, and various modifications are also included in the present invention. For example, the mounting positions and number of cameras and lights are not limited to those described above.

In addition, instead of arranging multiple lights for one camera, a single light with a variable illumination angle may be provided.

1: Object detection system 10: Hydraulic excavator 11L: Left crawler 11R: Right crawler 12: Upper rotating body 13: Cab 14: Boom 15: Arm 16: Bucket 18: Operation lever 20CB: Center rear camera 20LB: Left rear camera 20R: Right camera 20RB: Right rear camera 21CB: Imaging area 21LB: Imaging area 21R: Imaging area 21RB: Imaging area 22CB: Ground 22LB: Ground 25CB: Center rear light 25CB1: Center rear light 25CB2: Center rear light 25CB3: Center rear light 25CB4: Center rear light 25CB5: Center rear light 25LB: Left rear light 25R: Right light 25RB: Right rear light 26CB: Light path 26CB1 : Light path 26CB2 : Light path 26CB3 : Light path 26CB4 : Light path 26CB5 : Light path 29CB : Blind spot area 60 : Notification device 91CB1 : Worker 91CB3 : Worker 91R : Worker 92CB1 : Shadow 92CB3 : Shadow 92R : Shadow 102 : Pilot pump 103 : Main pump 104 : Solenoid proportional valve 105 : Control valve 106 : Actuator 107 : Engine 108 : ECU
110: Controller 111: Processor 111a: Operation signal determination unit 111b: Light control unit 111c: Camera control unit 111d: Object detection unit 111e: Operation control unit 112: RAM
113: ROM
114: Storage 115: Input I/F
116: Output I/F
117: Bus

Claims (8)

An object detection system for a work machine that detects an object present around the work machine having an upper rotating body mounted on a lower traveling body via a rotating mechanism,
an imaging device that is provided on the upper rotating body, includes the ground around the work machine as an imaging area, and captures the imaging area to generate an image;
a light that emits light toward the ground included in an imaging area of the imaging device;
a controller that detects objects around the work machine based on the captured image acquired from the imaging device,
the controller determines whether a shadow of the object existing on an optical path from the light to the ground included in the imaging area is captured in the captured image based on an image feature amount of the captured image, and detects the object based on the determination result.
An object detection system for a work machine comprising: 2. The object detection system for a work machine according to claim 1,
Further provided with an operation lever for inputting either a rotation operation of the upper rotating body or a forward operation or a reverse operation of the work machine,
The controller:
when it is detected that an operation signal indicating an instruction to execute any one of the turning operation, the forward operation, or the backward operation has been input from the operation lever, a lighting signal is transmitted to the light.
An object detection system for a work machine comprising: 3. The object detection system for a work machine according to claim 2,
a notification device for notifying a result of the determination made by the controller;
Further comprising:
the lights are a plurality of lights that irradiate light from different angles onto a ground included in one of the imaging areas, and when the controller detects the object from the captured image based on a light path from any one of the plurality of lights, the controller sequentially blinks the plurality of lights one by one to acquire the captured image generated each time each light is turned on, estimates the size of the object based on the size of the shadow reflected in the plurality of captured images, and outputs the estimation result to the notification device;
An object detection system for a work machine comprising: 4. An object detection system for a work machine according to claim 3,
the controller determines, based on the operation signal from the operation lever, whether the execution instruction indicated by the operation signal instructs the turning operation, the forward movement, or the reverse movement;
outputting a lighting signal for lighting the plurality of lights in an order along the direction in which the work machine is operating in accordance with the content of the execution instruction of the operation signal;
An object detection system for a work machine comprising: 4. An object detection system for a work machine according to claim 3,
The plurality of lights are arranged in the upper rotating body in the vertical direction or the horizontal direction.
An object detection system for a work machine comprising: 4. An object detection system for a work machine according to claim 3,
the controller compares the estimated size of the object with an object determination threshold serving as a criterion for determining whether or not a notification is required, and performs control to invalidate operation of the operating lever when the size of the object is equal to or larger than the object determination threshold.
An object detection system for a work machine comprising: 2. An object detection system for a work machine according to claim 1,
the imaging device includes a rear imaging device attached above and rearward of the upper rotating body at an angle including the ground rearward of the work machine as the imaging area,
The light includes a rear light attached to a lower end of the upper rotating body and forward of a rear end of the upper rotating body,
The mounting angle of the rear light is an angle at which the light emitted by the rear light reaches the ground behind the rear light,
A shadow of an object to be detected that exists between the rear light and the ground behind the vehicle is projected onto the ground behind the vehicle, and the shadow is captured in the captured image.
An object detection system for a work machine comprising: 2. An object detection system for a work machine according to claim 1,
The light is a dimmable light capable of changing the amount and wavelength of light,
The controller analyzes the amount and wavelength of ambient light based on image features of the captured image, and outputs a dimming signal to the dimming light to change at least one of the amount or wavelength of the light of the dimming light when a contrast ratio between the shadow of the object and a ground image captured under the ambient light is less than a contrast threshold for determining whether dimming is necessary.
An object detection system for a work machine comprising:

Images