JP7619089B2 - Rear turning interference monitoring device - Google Patents

Description

This relates to rear rotation interference monitoring, which monitors whether the rear of the upper rotating body of a mobile crane interferes with other objects when rotating.

Driving assistance technology that warns of interference with other objects when reversing or parking a vehicle has been put to practical use. For example, Patent Document 1 describes technology that enables parking to be performed according to a desired parking method. The information processing device of Patent Document 1 calculates the time required for automatic parking to be completed for each mode corresponding to the parking method of automatic parking that automatically parks a vehicle, and controls the display of the required time according to the mode, thereby enabling desired parking to be performed according to the time required for automatic parking to be completed. The technology of Patent Document 1 is touted as being applicable, for example, to an automatic parking system incorporated as part of an automatic driving system equipped in a vehicle capable of automatic driving.

JP 2017-140890 A

When applied to mobile cranes, driving assistance technology can be used not only to check for interference when moving and setting up the crane, but also to monitor rear interference caused by the rotation of the upper rotating body during crane operation. However, driving assistance technology does not know what type of object is interfering with the vehicle. When applying vehicle driving technology to mobile cranes, the operator needs to determine whether the interfering object can be removed before setting up the crane.

The purpose of this invention is to detect when the rear of the upper rotating body interferes with a nearby object due to rotation when setting up a mobile crane, and to display whether the interfering object can be moved.

A rear turning interference monitoring device according to an aspect of the present invention comprises:
an image acquisition unit that acquires an image including a range swept by a rear portion of an upper rotating body due to rotation of the upper rotating body of the mobile crane;
an interference detection unit that detects an interference object that may interfere with the rear portion due to the rotation of the upper rotating body;
a portability determination unit that determines whether or not the interfering object can be moved from the image when the interference detection unit detects the interfering object;
a display unit that displays the presence or absence of the interfering object and whether or not the interfering object can be moved;
Equipped with.

Preferably, the portability determination unit includes a trained model of a neural network that classifies the area of the interfering object in the image into a case where the interfering object can be moved and a case where the interfering object cannot be moved.

Preferably, the portability determination unit includes a trained model of a neural network that classifies the type of the interfering object based on the area of the interfering object in the image.

Preferably, the interference detection unit detects the interfering object from information on the position and size of objects around the mobile crane obtained from at least one of an image recognition device, a radar, a laser scanner, and an ultrasonic detector.

Preferably, when the portability determination unit determines that the interfering object cannot be moved, a movement information acquisition unit acquires information on whether the mobile crane has been moved before starting crane work by the mobile crane;
A storage unit that stores the determination result of the portability determination unit and information on whether the mobile crane was moved before starting crane work in association with each other;
Equipped with.

The rear swing interference monitoring device of the present invention makes it possible to check whether an object that the rear of the upper swing body interferes with can be moved when deciding the installation position of the mobile crane. If there is an interfering object that cannot be moved, the installation position of the mobile crane can be changed before the mobile crane is placed in the working position.

FIG. 1 is an external view showing an example of a mobile crane equipped with a rear swing interference monitoring device according to an embodiment of the present invention. FIG. 2 is a functional block diagram of a rear turning interference monitoring device according to the first embodiment of the present invention; Block diagram of a portability determination unit according to the first embodiment. FIG. 1 is a diagram showing an example of a neural network according to a first embodiment; Another example of a neural network classifying types of interference. A flowchart showing an example of an operation of monitoring rear turning interference according to the first embodiment. A functional block diagram of a rear turning interference monitoring device according to a second embodiment. A flowchart showing an example of an operation of monitoring rear turning interference according to the second embodiment.

The following describes in detail an embodiment of the present invention with reference to the drawings. In the drawings, the same or equivalent parts are denoted by the same reference numerals.

Embodiment 1.
FIG. 1 is an external view showing an example of a mobile crane equipped with a rear swing interference monitoring device according to an embodiment of the present invention. The crane 100 shown in FIG. 1 is a rough terrain crane. The lower traveling body 20 of the crane 100 has an engine housed in an engine room 23 arranged at the rear of a lower frame 24. The lower traveling body 20 has steerable wheels 27 and wheels 28 that can be driven by the engine, and can travel on the ground. The lower traveling body 20 has outriggers 25, 26 at the front and rear of the lower frame 24, respectively. The outriggers 25, 26 can be extended to the left and right of the lower traveling body 20 and extended downward to touch the ground, and the lower frame 24 can be supported by the outriggers 25, 26.

The upper rotating body 30 is mounted on the center of the lower frame 24 so as to be freely rotatable via a swivel bearing 29. The upper rotating body 30 includes a swivel frame 31, a telescopic boom 32, a hoisting cylinder 33, a hoisting device 34, a hook 35, and a wire rope 36.

The base end 38 of the telescopic boom 32 is supported on the rotating frame 31 so that it can be raised and lowered around a fulcrum pin 39. A hoisting cylinder 33 interposed between the rotating frame 31 and the telescopic boom 32 drives the telescopic boom 32 to raise and lower. Figure 1 shows the outriggers 25, 26 folded in and the telescopic boom 32 retracted and lowered. The crane 100 has a cab 22, which is used for both traveling and crane operation, located on the rotating frame 31 to the right of the telescopic boom 32.

The crane 100 is equipped with a hook 35 capable of suspending a load. Figure 1 shows the hook 35 housed in the lower frame 24. When the telescopic boom 32 is raised and extended, the hook 35 is removed from the lower frame 24, and the load is suspended by the wire rope 36 from the top sheave attached to the tip of the telescopic boom 32, the hook 35 can suspend a load. When suspending a load, the top sheave is positioned above the hook 35.

A hoisting device 34 that winds up the wire rope 36 is provided behind the revolving frame 31. The wire rope 36 that suspends the hook 35 is looped around the top sheave, then looped around a slide sheave located behind the base end 38, and wound up on the hoisting device 34.

To keep the crane 100 stable during operation, a counterweight is attached to the rear 37 of the rotating frame 31. When the rotating frame 31 rotates around the central axis of the rotating bearing 29, the rear 37 of the rotating frame 31, including the counterweight, swings out of the range of the lower running body 20. If there is an object within the rotating range of the rotating frame 31 next to the lower running body 20, the rear 37 will interfere with the object when the upper rotating body 30 rotates. The crane 100 is equipped with a rear rotation interference monitoring device that monitors whether the rear 37 interferes with other objects when rotating.

Figure 2 is a functional block diagram of the rear swing interference monitoring device according to the first embodiment of the present invention. The rear swing interference monitoring device 10 (hereinafter sometimes referred to as the monitoring device 10) is connected to an object detection sensor 18 that detects objects around the crane 100, and a camera 19 that captures images of the surroundings of the upper swing body 30. The object detection sensor 18 and the camera 19 are disposed, for example, one on each side of the upper swing body 30. The object detection sensor 18 detects objects in the space on the right and left sides of the upper swing body 30. The camera 19 captures images of the space on the right and left sides of the upper swing body 30, including the range swept by the rear part 37 as the upper swing body 30 swings.

The monitoring device 10 includes an image acquisition unit 11, an interference detection unit 12, a recognition range setting unit 13, a portability determination unit 14, and a display unit 15. The image acquisition unit 11 acquires an image from a camera 19 that includes an area swept by the rear part 37 as the upper rotating body 30 rotates. The interference detection unit 12 acquires information indicating the position and size of objects around the crane 100 from an object detection sensor 18.

The object detection sensor 18 is, for example, an image recognition device, a radar, a laser scanner, or an ultrasonic detector. The interference detection unit 12 detects, as an interfering object, an object within the range swept by the rear part 37 due to the rotation of the upper rotating body 30, based on information indicating the position and size of the objects around the crane 100 relative to the crane 100. If the object detection sensor 18 is an image recognition device, for example, the camera 19 may be a stereo camera, and surrounding objects may be detected from the images of the camera 19. In that case, the object detection sensor 18 can be included in the monitoring device 10.

When the interference detection unit 12 detects an interfering object, the recognition range setting unit 13 sets the recognition range as a range that includes the interfering object in the image acquired by the image acquisition unit 11. The position and imaging range of the camera 19 are set so as to capture an image of the space that includes the range swept by the rear section 37, so the image of the camera 19 includes an image of the interfering object. The recognition range setting unit 13 sets the recognition range so as to include the image of the interfering object.

If there is an interfering object, the portability determination unit 14 determines whether the interfering object can be moved from the recognition range of the image acquired by the image acquisition unit 11. If the interfering object is, for example, a ladder or stepladder, a temporary fence, a vehicle, or a temporary toilet, the portability determination unit 14 determines that the object can be moved. If the interfering object is a structure fixed to the ground, for example, a fixed fence, a fence or wall, an embedded pillar, or a standing tree, the portability determination unit 14 determines that the object cannot be moved.

The display unit 15 displays the presence or absence of an interfering object detected by the interference detection unit 12, and the result of the determination by the portability determination unit 14 as to whether the interfering object can be moved. To indicate the presence of an interfering object, for example, a buzzer may sound or a warning sound may be emitted from a speaker. A lamp may also be turned on or a warning icon or image may be displayed on a screen monitored by the operator. The warning may be a combination of an audio display and a visual display. The result of the determination as to whether the interfering object can be moved may be displayed, for example, as text or an icon together with an image of the interfering object on a screen monitored by the operator. The display unit 15 may distinguish whether the interfering object can be moved by the type, color, flashing or brightness of the text or icon, or a combination of these.

Figure 3 is a block diagram of the portability determination unit according to the first embodiment. In the first embodiment, the portability determination unit 14 includes a trained model 41 of a neural network and a mobility possibility calculation unit 42. The trained model 41 includes a neural network that classifies an interfering object into two states, for example, when the object can be moved and when the object cannot be moved, based on the recognition range of the image.

Figure 4 is a diagram showing an example of a neural network according to the first embodiment. The neural network 43 is composed of an input layer, an intermediate layer, and an output layer, each of which includes nodes composed of artificial neurons, and edges that connect the nodes between adjacent layers. The intermediate layer includes one or more n layers. Each pixel value xi in the recognition range of the image is input to each node i (i = 1...k) of the input layer. In the intermediate layer, the output of each previous layer is combined and the result of the operation using an activation function is transmitted to the subsequent layer and finally output to the output layer. In the intermediate layer, the convolution layer and the pooling layer are repeated to classify the image data in the recognition range into cases where the interfering object can move and cases where it cannot move. The output layer has nodes for cases where the interfering object can move and cases where it cannot move, and each node j (j = 1, 2) outputs the probability pj that the image data is classified into state j.

Multiple pieces of image data classified into cases where the interfering object can move and cases where it cannot move are input to the neural network 43, and machine learning is performed by backpropagating the difference between the state in which the image data is classified and the output of the neural network 43 to adjust each parameter. The image data to be machine-learned includes at least image data when the interfering object can move and image data when the interfering object cannot move.

The movement possibility calculation unit 42 in FIG. 3 calculates whether or not the interfering object can be moved based on the probability that the interfering object can and cannot be moved, which is the output of the neural network 43. For example, if the difference between the probability that the interfering object can be moved minus the probability that it cannot be moved is greater than a set threshold, it calculates that the interfering object can be moved, and if the difference is equal to or less than the threshold, it calculates that the interfering object cannot be moved. The portability determination unit 14 outputs the calculation result of the movement possibility calculation unit 42 to the display unit 15 as the determination result of whether or not the interfering object can be moved.

The rear swing interference monitoring device 10 includes an I/O interface, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU is, for example, a microprocessor, and is a central processing unit that executes various processes and calculations. The monitoring device 10 reads out a control program stored in the ROM, and while using the RAM as a work memory, causes the CPU to execute the control program, thereby performing the operations of the image acquisition unit 11, the interference detection unit 12, the recognition range setting unit 13, the portability determination unit 14, and the display unit 15. The monitoring device 10 may be incorporated into a crane control device (not shown). The monitoring device 10 may also be realized as a function of the crane control device.

Figure 5 is a diagram showing another example of a neural network that classifies the type of interference. Neural network 44 in Figure 5 shows a different example of trained model 41 in Figure 3. Neural network 44 in Figure 5 has a similar structure to neural network 43 in Figure 4. Neural network 44 that classifies the type of interference has an output layer composed of nodes p1, p2, p3, ... pM, the number of which is the number of types to be classified.

The neural network 44 classifies the type of interference object from the image data contained within the recognition range into, for example, a ladder or stepladder, a temporary fence, a vehicle, a temporary toilet, a fixed fence, a wall or fence, an embedded pillar, and a standing tree.

Multiple image data classified into types of interference objects are input to the neural network 44, and machine learning is performed by backpropagating the difference between the types of interference objects for which the image data is classified and the output of the neural network 44 and adjusting each parameter.

When the trained model 41 is the neural network 44 in FIG. 5, the movement possibility calculation unit 42 in FIG. 3 calculates whether or not the interfering object can move from the probability of each type of interfering object output from the neural network 44. The movement possibility calculation unit 42 calculates whether or not the interfering object can move, for example, according to whether or not the node with the highest probability output from the neural network 44, i.e., the most likely type of interfering object, can move. Alternatively, for example, the types of output nodes of the neural network 44 are divided into types that can move and types that cannot move, and the larger total value of the probability of each type is used to calculate whether or not the interfering object can move.

Neural network 44 in FIG. 5 may be a part of neural network 43 in FIG. 4. That is, the neural network 43 in FIG. 4 may be configured by connecting a layer following the output layer of neural network 44 in FIG. 5 that classifies and outputs cases in which an interfering object can be moved and cases in which it cannot be moved based on the type of interfering object.

Figure 6 is a flowchart showing an example of the operation of rear swing interference monitoring according to the first embodiment. The rear swing interference monitoring is initiated by the operation of the operator of the crane 100. For example, the rear swing interference monitoring is initiated when a command is issued to extend the outriggers 25, 26. Alternatively, the rear swing interference monitoring is initiated when the swing lock of the upper swing body is released, or when the boom is released from the fixed lock in the stowed position.

At narrow work sites, there may be situations where materials are placed there and it is not possible to pass through the site in the traveling position. In such cases, the boom may be raised and the upper rotating body may be rotated without extending the outriggers 25, 26 to move around the site. In such cases, rear rotation interference monitoring is also required. Also, in crawler cranes that do not have outriggers, rear rotation interference monitoring is activated when the rotation lock of the upper rotating body is released or when the lock to fix the boom in the stowed position is released.

The image acquisition unit 11 of the rear rotation interference monitoring device 10 acquires from the camera 19 an image including the range swept by the rear section 37 due to the rotation of the upper rotating body 30 (step S10). The interference detection unit 12 acquires information indicating the position and size of objects around the crane 100 from the object detection sensor 18, and detects objects within the range swept by the rear section 37 due to the rotation of the upper rotating body 30 as interfering objects (step S11). If there is no interfering object that the rear section 37 will interfere with due to the rotation of the upper rotating body 30 (step S12; N), the interference monitoring operation ends.

If an interfering object is present (step S12; Y), the display unit 15 displays a warning that an interfering object is present (step S13). Next, the recognition range setting unit 13 sets a recognition range as a range that includes the interfering object in the image acquired by the image acquisition unit 11, and the portability determination unit 14 determines whether the interfering object can be moved based on the image data of the set recognition range (step S14).

If it is determined that the interfering object can be moved (step S15; Y), the display unit 15 displays that the interfering object can be moved (step S16). If it is determined that the interfering object cannot be moved (step S15; N), the display unit 15 displays that the interfering object cannot be moved (step S17). After displaying whether or not the interfering object can be moved, the interference monitoring operation ends.

According to the rear rotation interference monitoring device 10 of the first embodiment, an object in the range swept by the rear part 37 due to the rotation of the upper rotating body 30 is detected as an interfering object, and further, whether or not the interfering object can be moved is determined and displayed. Therefore, when deciding the installation position of the crane 100, it is possible to confirm whether or not the object interfering with the rear part 37 of the upper rotating body 30 can be moved.

The monitoring device 10 of embodiment 1 can be applied to mobile cranes such as truck cranes, all-terrain cranes, crawler cranes, and railway cranes in addition to rough terrain cranes.

Embodiment 2.
7 is a functional block diagram of a rear turning interference monitoring device according to embodiment 2. The rear turning interference monitoring device 10 of embodiment 2 further includes a movement information acquisition unit 16 and a storage unit 17. The other configurations are the same as those of embodiment 1.

The movement information acquisition unit 16 acquires information on whether the crane 100 has been moved before the crane 100 starts its crane operation after the portability determination unit 14 has determined whether the interfering object can be moved. For example, the monitoring device 10 can obtain information on the movement of the crane 100 and the state of the crane operation from a crane control device (not shown).

The movement of the crane 100 may be recognized by a change in the image acquired by the image acquisition unit 11 from the camera 19. The start of crane operation can be recognized, for example, by the extension of the outriggers 25, 26, the raising or lowering of the telescopic boom 32, the rotation of the upper rotating body 30, or the operation of the hoisting device 34.

The memory unit 17 stores the determination result of the portability determination unit 14 and information on whether the crane 100 was moved before starting crane work, in association with each other. It is possible to record the reason why the installation location of the crane 100 had to be changed, and to collect data that is useful for planning crane work.

The information stored in the memory unit 17 can be one of four types: (1) the interfering object can be moved; the crane does not move, (2) the interfering object can be moved; the crane moves, (3) the interfering object cannot be moved; the crane does not move, and (4) the interfering object cannot be moved; the crane moves. The memory unit 17 may further have the operator input whether or not the judgment result of the portability judgment unit 14 differs from the actual situation, and store the input information in association with the judgment result. These data and image data can be fed back as learning data for the portability judgment unit 14.

Figure 8 is a flowchart showing an example of the operation of rear swing interference monitoring according to embodiment 2. Steps S10 to S17 in the flowchart of Figure 8 are the same as steps S10 to S17 in Figure 6. If it is determined that the interfering object cannot be moved (step S15; N), the display unit 15 displays that the interfering object cannot be moved (step S17), and the movement information acquisition unit 16 acquires information on whether or not the crane 100 has been moved before starting crane work (step S18).

After displaying that the interfering object can be moved (step S16) and after acquiring information on whether the crane 100 has been moved (step S18), the memory unit 17 stores the determination result on whether the interfering object can be moved and the information on whether the crane 100 has been moved before starting the crane operation in association with each other (step S19), and ends the interference monitoring operation.

In FIG. 8, when it is determined that the interfering object cannot be moved (step S15; N, step S17), information as to whether the crane 100 has been moved is acquired (step S18). However, even when it is determined that the interfering object can be moved (step S15; Y, step S16), information as to whether the crane 100 has been moved may be acquired before starting the crane operation.

The rear swing interference monitoring device 10 of embodiment 2 can record the reason why the installation location of the crane 100 had to be changed, and can collect data that is useful for planning crane operations.

LIST OF SYMBOLS 10 Rear swing interference monitoring device 11 Image acquisition unit 12 Interference detection unit 13 Recognition range setting unit 14 Portability determination unit 15 Display unit 16 Movement information acquisition unit 17 Memory unit 18 Object detection sensor 19 Camera 20 Lower running body 22 Cab 23 Engine room 24 Lower frame 25, 26 Outriggers 27, 28 Wheels 29 Swing bearing 30 Upper swing body 31 Swing frame 32 Telescopic boom 33 Drawer cylinder 34 Hoisting device 35 Hook 36 Wire rope 37 Rear 38 Base end 39 Fulcrum pin 41 Learned model 42 Mobility possibility calculation unit 43, 44 Neural network 100 Crane

Claims (5)

an image acquisition unit that acquires an image including a range swept by a rear portion of an upper rotating body due to rotation of the upper rotating body of the mobile crane;
an interference detection unit that detects an interference object that may interfere with the rear portion due to the rotation of the upper rotating body;
a portability determination unit that determines whether or not the interfering object can be moved from the image when the interference detection unit detects the interfering object;
a display unit that displays the presence or absence of the interfering object and whether or not the interfering object can be moved;
A rear turning interference monitoring device comprising: The rear turning interference monitoring device according to claim 1, wherein the portability determination unit includes a trained model of a neural network that classifies the area of the interfering object in the image into a case where the interfering object can be moved and a case where the interfering object cannot be moved. The rear turning interference monitoring device according to claim 1 or 2, wherein the portable determination unit includes a trained model of a neural network that classifies the type of the interfering object from the area of the interfering object in the image. The rear swing interference monitoring device according to any one of claims 1 to 3, wherein the interference detection unit detects the interfering object from information on the position and size of objects around the mobile crane obtained from at least one of an image recognition device, a radar, a laser scanner, and an ultrasonic detector. a movement information acquisition unit that acquires information on whether the mobile crane has been moved before starting crane work when the portability determination unit determines that the interfering object cannot be moved;
A storage unit that stores the determination result of the portability determination unit and information on whether the mobile crane was moved before starting crane work in association with each other;
The rear turning interference monitoring device according to any one of claims 1 to 4, comprising:

Images