JP6988713B2 - Remote control system for industrial vehicles - Google Patents

Description

The present invention relates to a remote control system for industrial vehicles.

The fork positioning support device for a forklift disclosed in Patent Document 1 accepts the designation of a target point for moving the fork tip in the image displayed on the display, calculates the distance from the reference point of the fork tip to the target point, and calculates the distance. The target point is tracked in the updated stereo image and the calculated distance is output to the display.

Japanese Unexamined Patent Publication No. 2013-86959

By the way, even if the amount of misalignment itself is displayed when the industrial vehicle is remotely controlled, the operator does not know what kind of operation should actually be performed to eliminate the misalignment.
An object of the present invention is to provide a remote control system for an industrial vehicle that can easily reach a target position.

According to the first aspect of the present invention, there is an industrial vehicle having a cargo handling device on the chassis and a vehicle communication unit, and an operation device communication unit that performs wireless communication with the vehicle communication unit. A remote control system for an industrial vehicle equipped with a remote control device used for remote control of cargo handling by a cargo handling device, which is mounted on the industrial vehicle and has a plurality of cameras that image the surroundings of the industrial vehicle. A display unit provided in the remote control device for displaying images captured by the plurality of cameras, and a target position with respect to the current position of the industrial vehicle by image recognition from the images captured by the plurality of cameras. The gist is to include a calculation means for calculating the amount of misalignment up to, and a display control means for displaying the amount of operation to reach the target position on the display unit according to the amount of misalignment calculated by the calculation means.

According to the first aspect of the present invention, the calculation means calculates the amount of misalignment from the images captured by a plurality of cameras to the target position with respect to the current position of the industrial vehicle by image recognition. The display control means displays on the display unit the amount of operation to reach the target position according to the amount of misalignment calculated by the calculation means. Therefore, it is possible to easily reach the target position.

As described in claim 2, in the remote control system for industrial vehicles according to claim 1, the display control means may also display the operation procedure on the display unit.
As described in claim 3, in the remote control system for industrial vehicles according to claim 1 or 2, the display control means may also display the operation timing on the display unit.

According to the present invention, it is possible to easily reach the target position.

A block diagram showing the electrical configuration of a remote control system for industrial vehicles. Schematic side view of a reach forklift. Schematic plan view of a reach forklift. The figure for demonstrating image switching in a display part. Schematic plan to illustrate the movement of the reach forklift pallet to the front position in the workplace. A flowchart for explaining the action. The figure for demonstrating the display contents in a display part. The figure for demonstrating the display contents in the display part of another example. The figure for demonstrating the display contents in the display part of another example.

Hereinafter, an embodiment embodying the present invention will be described with reference to the drawings.
As shown in FIG. 1, the remote control system 10 for an industrial vehicle includes a reach type forklift 20 as an industrial vehicle and a remote control device 50 used for remotely controlling the traveling and cargo handling of the reach type forklift 20. ing. The reach forklift 20 is located in the workplace. Then, the reach type forklift 20 in the workplace can be remotely controlled from the operation room by using the remote control device 50.
As shown in FIG. 2, a cargo handling device 22 is provided on the machine base 21 of the reach type forklift 20. The cargo handling device 22 includes a pair of reach legs 23 extending forward. Each reach leg 23 is provided with a front wheel 24. That is, a pair of left and right front wheels 24 are provided on the front side of the machine base 21. Further, a rear wheel 25 is provided on the rear side of the machine base 21. In the present embodiment, the rear wheels 25 are the steering wheels and the driving wheels.

The cargo handling device 22 includes a two-stage mast 26. The mast 26 includes an outer mast 27 and an inner mast 28. The cargo handling device 22 includes a lift cylinder 29 connected to the inner mast 28. The cargo handling device 22 includes a reach cylinder 30 connected to the mast 26. The inner mast 28 moves up and down by supplying and discharging hydraulic oil to and from the lift cylinder 29. The mast 26 moves along the reach leg 23 by supplying and discharging hydraulic oil to the reach cylinder 30.

The cargo handling device 22 includes a pair of forks 31 and a lift bracket 32 for fixing the forks 31 to the mast 26. The fork 31 and the lift bracket 32 move up and down as the inner mast 28 moves up and down. The reach type forklift 20 includes a traveling motor 33 that drives the rear wheels 25.

As shown in FIG. 1, the reach type forklift 20 images the surroundings of the radio unit 40 as a vehicle communication unit, the control unit 41, the video signal processing unit 42, the camera switching unit 43, and the reach type forklift 20. It has cameras 71-78. The cameras 71 to 78 mounted on the reach type forklift 20 include traveling cameras 71, 72, 73, 74 and cargo handling cameras 75, 76, 77, 78.

The plurality of cameras 71 to 78 are assigned to each work (for each traveling work and cargo handling work) on the reach type forklift 20 side. The camera is switched by the camera switching unit 43. The video signal processing unit 42 compresses and converts the image captured by the camera. The control unit 41 controls the switching instruction of the cameras 71 to 78. The wireless unit 40 wirelessly transmits an image and a switching instruction signal.

The remote control device 50 includes a wireless unit 51 as an operation device communication unit, a control unit 52, an operation unit 53, a display unit (monitor) 54, a video signal processing unit 55, and a work detection unit 56. ..
The wireless unit 51 wirelessly transmits images, camera switching instructions, and the like. The video signal processing unit 55 restores and converts the image. The display unit 54 displays the image on the display. The operation unit 53 is for performing an operation. The work detection unit 56 can know which work operation is performed by a touch sensor or the like. The control unit 52 controls the switching instruction of the cameras 71 to 78.

Wireless communication can be performed between the wireless unit 40 of the reach type forklift 20 and the wireless unit 51 of the remote control device 50. As described above, the reach type forklift 20 has a wireless unit 40, and the remote control device 50 has a wireless unit 51 that performs wireless communication with the wireless unit 40.

In the remote control device 50, the operation unit 53 is for the operator to remotely control the reach type forklift 20, and the operation contents of the reach type forklift 20 by the operator (lift, reach, tilt operation command values, and tilt operation command values). , Speed, acceleration, operation command value of steering angle, etc.) are sent to the control unit 52 via the work detection unit 56. The control unit 52 wirelessly transmits vehicle control signals such as lift, reach, and tilt operation command values, speed, acceleration, and steering angle operation command values to the radio unit 40 of the reach type forklift 20 via the radio unit 51. Send.

In the reach type forklift 20, the control unit 41 receives instructions from the remote control device 50 side to drive a traveling system actuator (traveling motor 33, steering motor, etc. not shown) and a cargo handling system actuator (lift cylinder 29, reach cylinder 30, tilt cylinder not shown). Etc.) can be driven.

The wireless unit 40 wirelessly transmits vehicle information such as the vehicle speed of the reach type forklift 20 and abnormality information (obstacle detection information and the like) to the control unit 52 of the remote control device 50.
In FIG. 1, the operation unit 53 of the remote control device 50 can remotely control the traveling of the reach type forklift 20 and the cargo handling by the cargo handling device 22 via the radio units 51 and 40 and the control unit 41. ..

Then, in the remote control device 50, when the operator performs a desired operation using the operation unit 53, the operation content is sent to the reach type forklift 20 side by the control unit 52 via the radio unit 51. In the reach type forklift 20, the wireless unit 40 receives the operation content from the remote control device 50, and the control unit 41 drives the actuator unit to execute a desired operation.

The traveling system cameras 71, 72, 73, 74 are provided as shown in FIGS. 2 and 3. The traveling cameras 71, 72, 73, 74 are attached to the machine base 21 of the reach type forklift 20. The traveling cameras 71, 72, 73, 74 take an image of the surroundings of the machine base 21. Specifically, the traveling system camera 71 captures an image of the front in the traveling direction. The traveling camera 72 takes an image of the rear. The traveling system camera 73 captures an image on the left side. The traveling camera 74 captures the right side.

The cargo handling cameras 75, 76, 77, 78 are provided as shown in FIGS. 2 and 3. The cargo handling cameras 75, 76, 77, 78 are attached to the cargo handling device 22 of the reach type forklift 20. The cargo handling cameras 75, 76, 77, 78 take an image of the cargo handling target area in the cargo handling device 22. Specifically, the cargo handling system camera 75 is attached to the upper part on the left side of the lift bracket 32, and images the front of the cargo handling device 22 from the left side. The cargo handling system camera 76 is attached to the upper part on the right side of the lift bracket 32, and images the front of the cargo handling device 22 from the right side. The cargo handling camera 77 is attached to the upper part of the inner mast 28 and images the front lower part. The cargo handling camera 78 is attached to the fork 31 on the left side and images the front of the fork 31.

On the other hand, as shown in FIG. 1, in the reach type forklift 20, the images captured by the cameras 71 to 78 are sent to the remote control device 50 side via the video signal processing unit 42 and the radio unit 40. In the remote control device 50, the camera image from the reach type forklift 20 is received by the wireless unit 51 and displayed by the control unit 52 on the display unit 54 via the video signal processing unit 55.

Using the operation unit 53, it is possible to instruct switching from the image captured by the traveling camera to the image captured by the cargo handling camera, and from the image captured by the cargo handling camera to the traveling camera. You can instruct to switch to the image captured by. This camera selection signal is sent to the reach type forklift 20 via the radio unit 51. The control unit 41 of the reach type forklift 20 is a camera group (traveling cameras 71, 72, 73, 74, or cargo handling) instructed by the camera selection signal based on the camera selection signal wirelessly received via the radio unit 40. The video of the system cameras 75,76,77,78) is wirelessly transmitted to the control unit 52 of the remote control device 50 via the video signal processing unit 42.

As shown in FIG. 4, the display unit 54 provided in the remote control device 50 is an image captured by the traveling camera 71, 72, 73, 74 and the cargo handling camera 75, 76, 77, 78. It is possible to display the captured image while switching. Then, the operator operates while looking at the camera image (screen) displayed on the display unit 54.

Next, the action will be described.
It is assumed that the situation shown in FIG. 5 is now present.
In FIG. 5, the reach type forklift 20 is located at a position away from the pallet 60 in the workplace. The pallet 60 has left and right holes (pallet holes) 60a into which the left and right forks 31 are inserted. To transport the load of the pallet 60 by the reach type forklift 20 (to load), move the reach type forklift 20 from the current position Pc of the reach type forklift 20 to the target position Pt so that the reach type forklift 20 faces the pallet 60. There is a need. After that, the fork 31 of the reach type forklift 20 is inserted into the pallet hole 60a.

Specifically, in FIG. 5, the tip of the fork 31 on the right side of the reach type forklift 20 is set as the current position Pc, and the position separated by the distance d1 in front of the pallet hole 60a on the right side of the pallet 60 is set as the target position Pt. There is. Further, the front surface of the pallet 60 is in the X direction. The direction orthogonal to the X direction of the front surface of the pallet 60 is defined as the Y direction.

At the current position Pc of the reach type forklift 20, the angle formed by the traveling direction L1 of the reach type forklift 20 and the front surface (X direction) L2 of the pallet 60 is θ. In FIG. 5, θ is, for example, about 45 degrees. A position where the reach type forklift 20 is positioned in front of the pallet 60 (facing) and the tip of the fork 31 on the right side of the reach type forklift 20 is separated from the front side of the pallet hole 60a on the right side of the pallet 60 by a predetermined distance d1. To arrange the forklift 20 at (target position Pt), the reach type forklift 20 is driven straight (advanced) by a predetermined distance d2 so as to approach the pallet 60 to travel to the relay point Pr, and θ = 90 ° at the relay point Pr. It is necessary to turn left like this.

By the process of FIG. 6, the operation amount for eliminating the misalignment amount is displayed so that θ = 0 ° and the distance X1 = 0 in the X direction of FIG.
In FIG. 6, the control unit 52 detects the positional relationship between the vehicle (reach type forklift 20) and the target object (pallet 60) from the images of the cameras 71 to 78 in step S100. The control unit 52 detects the state of the vehicle in step S101. The control unit 52 calculates the operation amount for reaching the target position Pt in step S102. That is, the operation amount for reaching the target position (the position where the fork faces the pallet hole) Pt is calculated.

The control unit 52 calculates the optimum operation amount according to the input method in step S103. That is, the control unit 52 calculates the amount of misalignment from the images captured by the plurality of cameras 71 to 78 to the target position Pt with respect to the current position Pc of the reach type forklift 20 by image recognition, and calculates the optimum operation amount. do.

The control unit 52 notifies the operator in step S104. That is, the control unit 52 causes the display unit 54 to display the operation amount to reach the target position Pt according to the calculated position shift amount. Further, the control unit 52 also causes the display unit 54 to display the operation procedure.

Specifically, it will be described.
As shown in FIG. 7, the touch panel type remote controller 100 that serves as both a display unit and an operation unit has a slider 110 for an accelerator and an indicator 120 for displaying a steering angle on the screen. The slider 110 is for indicating the amount of forward / backward accelerator. In FIG. 7, the slider 110 can instruct the amount of advance by the amount of shifting upward, and in FIG. 7, the amount of retreat can be instructed by the amount of shifting downward. In FIG. 7, the right side of the center position 120a is the area for displaying the right steering angle, and the left side of the center position 120a in FIG. 7 is the area for displaying the left steering angle. In the touch panel type remote controller 100, the steering angle can be instructed by turning the remote controller itself like a steering wheel, and the steering angle according to the turning angle is displayed on the indicator 120.

On the screen of the touch panel type remote controller 100, the required operation amount is displayed by the triangular marks 111 and 121. The triangular mark 111 specifies a required traveling amount (a predetermined amount of advance in FIG. 7) at the operating position of the slider 110. The triangular mark 121 specifies a required steering angle (a predetermined amount of left steering in FIG. 7) on the indicator 120.

Further, in FIG. 7, the operation procedure 1 and the operation procedure 2 are displayed. The operation procedure 1 is attached to the triangular mark 111. The operation procedure 2 is attached to the triangular mark 121. As a result, an instruction is given to perform a predetermined amount of left turn operation after a predetermined amount of forward traveling.

From the current position Pc in FIG. 5 to the relay point Pr, the triangular mark 111 (operation procedure 1) and the triangular mark 121 (operation procedure 2) are displayed. Further, from the relay point Pr in FIG. 5 to the target position Pt, only the triangular mark 121 (operation procedure 2), which is the steering angle for setting the angle θ = 90 ° in FIG. 5, is displayed.

The detailed operation procedure is as follows.
As the first step, the slider 110 is moved upward to move the machine base 21 forward.
As the second step, when the advance is advanced by a predetermined amount, the display of the triangle mark 111 (operation procedure 1) disappears, so the finger is released from the slider 110 (accelerator off). That is, the timing at which the display of the triangular mark 111 (operation procedure 1) disappears is the steering start timing.

As the third step, the remote controller 100 is turned counterclockwise. As a result, the center position 120a of the indicator 120 moves to the left.
As the fourth step, when the center position 120a of the indicator 120 reaches the position of the triangular mark 121 (operation procedure 2), the remote controller 100 is returned to the neutral position.

As a fifth step, move forward and insert the fork into the pallet hole.
In this way, the control unit 52 calculates the amount of misalignment from the images captured by the plurality of cameras 71 to 78 to the target position Pt with respect to the current position Pc of the reach type forklift 20 by image recognition, and the calculated position. The display unit 54 displays the operation amount to reach the target position Pt according to the deviation amount. Therefore, it is possible to easily reach the target position Pt. It should be noted that the steering may be automatically performed when the relay point Pr is reached from the current position Pc.

explain in detail.
Conventionally, when lifting a pallet using a forklift, the pallet hole and the fork must face each other. If the fork position and the pallet hole position do not match, it is necessary to reposition them by performing a turning operation or the like. In addition, I want to be able to align the fork with a simple operation.

In the present embodiment, the angle θ between the object (pallet) and the own vehicle (fork) and the amount of misalignment (distance X1) in FIG. 5 are detected by image recognition. Then, the control amount of the vehicle required for the vehicle to reach the point (Pt) facing the pallet hole from the current position (current position Pc) of the reach type forklift 20 (θ = 90 ° in FIG. 5, X1). = 0), the operation order is calculated.

By doing so, the operator can easily align the fork and the pallet. Specifically, if the operation order is not notified, the operator must operate the driving and steering at his / her own discretion, so it may take extra operation and time for an operator with low proficiency to adjust the position. be. Specifically, it does not take much time to drive first and then steer later. This is because, in order to make sure that the vehicle reaches the target position (target point) Pt, the distance between the target position (target point) Pt and the vehicle (reach type forklift 20), the optimum operation order according to the vehicle speed, and the steering angle. It is necessary to perform the operation with (operation amount) and operation timing. If this is not possible, a misalignment will occur and re-operation will be required, resulting in an increase in operation time.

In the present embodiment, the positional relationship between the fork and the pallet is calculated from the camera image, and the operation amount for the fork, which is the target position Pt, to reach the position facing the pallet hole is calculated. Further, the operation procedure is calculated, and the required operation amount and operation procedure are guided to the operator.

That is, the operator is notified of the operation procedure for carrying out the necessary vehicle control. Specifically, the forward operation amount corresponding to the straight running amount (equivalent to d2) and the target value of the steering amount required to set θ to 90 ° are displayed. In FIG. 7, the operation order, operation timing, and operation amount are notified.

This makes it possible to simplify the work of aligning the forks. As a result, the burden on the worker can be reduced. Furthermore, improvement in work efficiency can be expected.
According to the above embodiment, the following effects can be obtained.

(1) As a configuration of the remote control system 10 for an industrial vehicle, a reach-type forklift 20 as an industrial vehicle having a cargo handling device 22 on the machine base 21 and a wireless unit 40 as a vehicle communication unit, and wireless communication with the wireless unit 40. It has a radio unit 51 as an operation device communication unit for performing the operation, and includes a remote operation device 50 used for remotely controlling the traveling of the reach type forklift 20 and the cargo handling by the cargo handling device 22. A plurality of cameras 71 to 78 mounted on the reach type forklift 20 to image the surroundings of the reach type forklift 20, and a plurality of cameras provided on the remote control device 50 for displaying images captured by the plurality of cameras 71 to 78. It includes a display unit 54 and a control unit 52. The control unit 52 as a calculation means calculates the amount of misalignment from the images captured by the plurality of cameras 71 to 78 to the target position (working position) Pt with respect to the current position Pc of the reach type forklift 20 by image recognition. The control unit 52 as the display control means causes the display unit 54 to display the operation amount to reach the target position Pt according to the calculated position shift amount.

That is, since the amount of misalignment (deviation) from the current position to the target point can be known by image recognition, the amount of operation for eliminating the amount of misalignment is displayed. Therefore, it is possible to easily reach the target position Pt.

(2) Since the control unit 52 as the display control means also displays the operation procedure on the display unit 54, it is possible to more easily reach the target position Pt.
The embodiment is not limited to the above, and may be embodied as follows, for example.

○ Instead of FIG. 7, it may be displayed as shown in FIG. In FIG. 8, on the display unit 54, a push button Bp1 indicating forward movement and a push button Bp2 indicating backward movement are displayed at the central portion of the screen 54a. Further, on the screen 54a of the display unit 54, the push button Bp3 indicating left turn is displayed and the push button Bp4 indicating right turn is displayed. A display instructing the operation of the number of taps (specifically, 2) is displayed for the push button Bp1 indicating the advance of the reach type forklift. Further, a display instructing the operation of the number of taps (specifically, 1) is displayed for the push button Bp3 indicating the left turn. In FIG. 8, the operation procedure 1 and the operation procedure 2 are displayed. The operation procedure 1 is attached to a push button Bp1 indicating the advance of the reach type forklift. The operation procedure 2 is attached to a push button Bp3 indicating a left turn.

That is, as an operation procedure, the reach type forklift is operated forward in the traveling direction, and after that, a guide display indicating that the forklift is turned left at a predetermined angle is displayed.
In this way, the operation order and the number of inputs may be displayed. Specifically, the button to be input is guided (such as inputting the push button Bp1 for forward movement twice and then inputting the push button Bp3 for turning left once). In FIG. 8, only the operation order and the operation amount (number of times) are notified, and the operation timing is automatically determined on the vehicle side.

○ Instead of FIG. 7, it may be displayed as shown in FIG. In FIG. 9, on a part of the screen 54a of the display unit 54, the straight running X seconds and the left turn steering Y degree are displayed as character information. Further, in FIG. 9, the operation procedure 1 and the operation procedure 2 are displayed. The operation procedure 1 is given to the straight running X seconds. The operation procedure 2 is applied to the left turn steering Y degree. The operator operates an operating member (accelerator, steering wheel, lever, etc.) (accelerator, steering wheel, lever, etc.) (not shown) according to the displayed character information and the operating procedure.

That is, as an operation procedure, the reach type forklift is operated forward in the traveling direction, and after that, a guide display indicating that the forklift is turned left at a predetermined angle is displayed.
The timing at which the operation procedure 2 is displayed is the timing at which the steering is started, and the control unit 52 as the display control means may also display the operation timing on the display unit 54. That is, instead of setting the timing at which the operation procedure 2 is displayed as the timing at which the steering is started, the timing at which the steering is started after a few seconds may be displayed in characters. In this case, the target position Pt can be reached more easily.

○ Further means for automatically controlling the operation timing may be provided. In this case, the target position Pt can be reached more easily. That is, the operation timing may be further provided with means for automatically controlling the operation timing without depending on the input of the operator.

○ The process performed by the control unit 52 in FIG. 6 may be executed by the control unit 41.
○ In FIG. 5, the target position Pt is set as the facing position before the fork is inserted into the pallet hole. Alternatively, the target position Pt may be the position where the fork is inserted into the pallet hole.

○ The number of in-vehicle cameras does not matter. Also, the orientation of the camera does not matter.
○ In addition to the in-vehicle camera, a camera may be installed on the equipment side (infrastructure side), and the surrounding area may be imaged by the infrastructure camera to use the camera image. For example, the system configuration may be such that the infrastructure cameras C1 and C2 are arranged as shown by virtual lines in FIG.

○ The industrial vehicle was a reach type forklift, but it is not limited to this, and it may be an industrial vehicle other than the reach type forklift.

10 ... Remote control system for industrial vehicles, 20 ... Reach type forklift, 21 ... Machine stand, 22 ... Cargo handling device, 40 ... Radio unit, 50 ... Remote control device, 51 ... Radio unit, 52 ... Control unit, 54 ... Display unit , 71, 72, 73, 74, 75, 76, 77, 78 ... Camera, Pc ... Current position, Pt ... Target position.

Claims (2)

An industrial vehicle equipped with a cargo handling device on the chassis and a vehicle communication unit,
An operation device for wireless communication with the vehicle communication unit A remote operation system for an industrial vehicle having a communication unit and a remote operation device used for remotely controlling the traveling of the industrial vehicle and the cargo handling by the cargo handling device. And,
A plurality of cameras mounted on the industrial vehicle and taking an image of the surroundings of the industrial vehicle,
A display unit provided in the remote control device for displaying images captured by the plurality of cameras, and a display unit.
A calculation means for calculating the amount of misalignment from the images captured by the plurality of cameras to the target position with respect to the current position of the industrial vehicle by image recognition.
A display control means for displaying an operation amount reaching a target position on the display unit according to a position shift amount calculated by the calculation means is provided .
The display control means is a remote control system for industrial vehicles, characterized in that the operation timing is also displayed on the display unit. The remote control system for an industrial vehicle according to claim 1, wherein the display control means also displays an operation procedure on the display unit.

Images