JP7103077B2 - Remote control system for forklifts - Google Patents

Description

The present invention relates to a remote control system for a forklift.

The unmanned forklift described in Patent Document 1 includes an obstacle sensor and a control device. As the obstacle sensor, a plurality of ultrasonic sensors and a plurality of photoelectric sensors are used. Reflectors are installed where forklifts are used. The control device switches the operation timing between the ultrasonic sensor and the photoelectric sensor according to the reception of the reflected light from the reflector by the photoelectric sensor. Obstacle detection ranges differ between ultrasonic sensors and photoelectric sensors. It can be said that the control device changes the detection range of the obstacle sensor according to the light received from the reflector.

Japanese Unexamined Patent Publication No. 4-213705

By the way, an obstacle sensor may be provided on a forklift that is remotely controlled by a remote control device. In this case, when the forklift approaches the pallet and the pallet enters the detection range of the obstacle sensor, the pallet is erroneously detected as an obstacle. Similar to Patent Document 1, it is conceivable to provide equipment such as a reflector at the place where the pallet is placed to shorten the detection range of the obstacle sensor when approaching the pallet. However, in this case, additional equipment is required. Also, unlike unmanned forklifts that travel on a predetermined transport route, in places where remote-controlled forklifts are used, pallet placement locations are often not predetermined, and the location of equipment is determined. It may not be possible.

An object of the present invention is to provide a remote control system for a forklift that can suppress false detection of a pallet as an obstacle without adding equipment.

The remote operation system for a forklift that solves the above problems includes a forklift including a vehicle communication unit that performs wireless communication and a remote operation device that includes a remote communication unit that performs wireless communication, and is a remote transmission transmitted from the remote communication unit. A remote operation system for a forklift in which the forklift is remotely operated by an instruction signal, the forklift takes an image of an obstacle sensor that measures a distance to an obstacle existing in front of the forklift and an image of the front of the forklift. The remote operation system for a forklift including a camera detects an image processing unit that processes image data obtained by the camera and a distance from the forklift to the pallet from the information obtained by the image processing unit. A detection range setting unit is provided, and a detection range setting unit capable of setting the detection range of the obstacle sensor to a first range or a second range in which the range with respect to the front of the forklift is wider than the first range. When the distance from the forklift to the pallet detected by the detection unit is equal to or less than a predetermined value, the detection range setting unit sets the detection range of the obstacle sensor to the first range.

According to this, when the distance from the forklift to the pallet is equal to or less than a predetermined value, the detection range of the obstacle sensor is set to the first range by the detection range setting unit. The first range is narrower than the second range with respect to the front of the forklift. Therefore, the pallet is suppressed from entering the detection range, and the recognition as an obstacle is suppressed. As a result, it is possible to prevent the pallet from being erroneously detected as an obstacle.

Regarding the remote control system for a forklift, the camera is a monocular camera, and the detection unit may detect the distance from the forklift to the pallet from the size of the pallet in the image data.

As the forklift moves, the size of the pallet in the image data changes. The closer the forklift is to the pallet, the larger the pallet in the image data. Therefore, this can be used to detect the distance between the forklift and the pallet. By calculating the distance to the pallet, the load on the detection unit can be reduced as compared with the case where it is detected that the distance between the forklift and the pallet is equal to or less than a predetermined value.

Regarding the remote operation system for a forklift, the camera is a stereo camera, and the detection unit may detect the distance from the forklift to the pallet based on the parallax obtained from the image data.

According to this, the detection unit can detect the distance from the forklift to the pallet based on the parallax. Therefore, it is possible to detect with high accuracy that the distance from the forklift to the pallet is equal to or less than a predetermined value.

Regarding the remote control system for a forklift, the remote control device may include the image processing unit and the detection unit.
According to this, it is possible to detect that the distance from the forklift to the pallet is equal to or less than a predetermined value by the remote control device.

Regarding the remote control system for a forklift, the forklift may include the image processing unit and the detection unit.
According to this, the forklift detects that the distance from the forklift to the pallet is equal to or less than a predetermined value. It is possible to suppress the delay in changing the detection range as compared with the case where the remote control device detects that the distance from the forklift to the pallet is equal to or less than a predetermined value.

According to the present invention, it is possible to suppress false detection of the pallet as an obstacle without adding equipment.

Schematic of a remote control system for a forklift. Top view of a forklift. The block diagram which shows the forklift and the remote control device. A flowchart showing a process for setting a detection range of an obstacle sensor. The figure which shows the relationship between the distance from a forklift to a pallet and a pallet in image data. The figure which shows the relationship between the distance from a forklift to a pallet and a pallet in image data.

(First Embodiment)
Hereinafter, the first embodiment of the remote control system for a forklift will be described.
As shown in FIGS. 1 and 2, the forklift remote control system 10 includes a forklift 20 and a remote control device 50 used to remotely control the forklift 20. The remote control device 50 transmits a remote instruction signal. The forklift 20 that has received the remote instruction signal operates in response to a command included in the remote instruction signal. In the following description, the remote control system 10 for a forklift will be referred to as a remote control system 10.

The forklift 20 of this embodiment is a reach type. The forklift 20 includes a vehicle body 21, wheels 22 provided on the vehicle body 21, a reach leg 23 extending toward the front of the vehicle body 21, and a cargo handling device 24 provided in front of the vehicle body 21. The cargo handling device 24 includes a mast 25 that stands up against the reach leg 23, and a fork 26 that is attached to the mast 25. The forklift 20 loads the pallet P on the fork 26 and conveys the load loaded on the pallet P. In the present embodiment, the pallet P will be described by exemplifying a flat pallet having a rectangular plate shape and having a hole H into which the fork 26 is inserted.

As shown in FIG. 3, the forklift 20 includes a drive mechanism 27 that causes the forklift 20 to perform a traveling operation, a cargo handling mechanism 28 that causes the forklift 20 to perform a cargo handling operation, a receiving unit 29, a transmitting unit 30, and a speed sensor 31. , An obstacle sensor 32, and a main controller 33.

The drive mechanism 27 includes a drive source for rotating the wheels 22, a steering mechanism for steering the forklift 20, and the like. The cargo handling mechanism 28 includes a cargo handling pump that supplies hydraulic oil to the hydraulic cylinder that operates the mast 25, a drive source that drives the cargo handling pump, a valve that controls the flow of hydraulic oil, and the like. The cargo handling mechanism 28 can cause the cargo handling device 24 to perform a reach operation for moving the mast 25 in the front-rear direction along the reach leg 23, a tilt operation for tilting the mast 25, and a lift operation for raising and lowering the mast 25. The cargo handling operation is an operation including any one of a reach operation, a tilt operation, and a lift operation.

The receiving unit 29 receives the remote instruction signal wirelessly transmitted from the remote control device 50. The remote instruction signal is a signal including a command for the forklift 20. The commands include various commands related to the operation of the forklift 20, such as a command related to a traveling operation and a command related to a cargo handling operation. When the receiving unit 29 receives the remote instruction signal, the receiving unit 29 outputs the remote instruction signal to the main controller 33.

The transmission unit 30 wirelessly transmits a vehicle status signal including information about the forklift 20 to the remote control device 50. The information about the forklift 20 is, for example, the speed of the forklift 20. The vehicle status signal is generated by the main controller 33.

The speed sensor 31 detects the speed of the forklift 20. The detection result of the speed sensor 31 is output to the main controller 33.
As the obstacle sensor 32, for example, a distance meter such as a laser range finder using a laser or an ultrasonic sensor using an ultrasonic wave is used. In this embodiment, a case where a laser range finder is used as the obstacle sensor 32 will be described.

As shown in FIGS. 2 and 3, the obstacle sensor 32 is a range finder that measures the distance by irradiating the periphery with a laser and receiving the reflected light reflected from the portion hit by the laser. The obstacle sensor 32 of the present embodiment is a two-dimensional laser range finder that irradiates a laser while changing the irradiation angle in the horizontal direction. Assuming that the portion hit by the laser is the irradiation point, the obstacle sensor 32 measures the distance to the irradiation point in association with the irradiation angle. In other words, the obstacle sensor 32 can measure the relative coordinates of the forklift 20 and the irradiation point. In the following description, a range within the laser irradiation angle θ and within the laser irradiation distance D will be described as the detectable range A. The detectable range A can be said to be the maximum range in which the obstacle sensor 32 can cause the main controller 33 to detect an obstacle.

The obstacle sensor 32 is attached to the forklift 20 so that the distance to an obstacle existing in front of the forklift 20 can be measured. That is, the obstacle sensor 32 is attached so that the detectable range A of the obstacle sensor 32 includes the front of the forklift 20. Further, the obstacle sensor 32 is attached at a position where the laser is not easily blocked by the forklift 20. In this embodiment, the obstacle sensor 32 is attached to the tip of the reach leg 23.

The main controller 33 includes a CPU 34 and a storage unit 35 including a RAM, a ROM, and the like. Various programs for controlling the forklift 20 are stored in the storage unit 35. The main controller 33 may include dedicated hardware that executes at least a part of the various processes, for example, an integrated circuit for a specific application: ASIC. The main controller 33 may be configured as one or more processors operating according to a computer program, one or more dedicated hardware circuits such as an ASIC, or a circuit including a combination thereof. The processor includes a CPU and memories such as RAM and ROM. The memory stores a program code or a command configured to cause the CPU to execute the process. Memory, or computer-readable medium, includes anything that can be accessed by a general purpose or dedicated computer.

The main controller 33 controls the drive mechanism 27 and the cargo handling mechanism 28 in response to a command included in the remote instruction signal acquired from the receiving unit 29. As a result, the forklift 20 can operate in response to the remote instruction signal.

The main controller 33 detects an obstacle existing in front of the forklift 20 from the detection result of the obstacle sensor 32. When an obstacle exists in the detectable range A, the distance detected at the boundary between the position where the laser hits the obstacle and the position where the laser does not hit the obstacle changes. The main controller 33 can detect an obstacle by utilizing this change in distance.

When the main controller 33 detects an obstacle, it imposes a speed limit on the forklift 20. The main controller 33 controls so that the speed of the forklift 20 does not exceed the speed limit value set by the speed limit. That is, even if the command regarding the speed from the remote control device 50 exceeds the speed limit value, the main controller 33 controls the drive mechanism 27 so as not to exceed the speed limit value. The speed limit value is a preset speed, and is set according to the time required to avoid obstacles and the like. The speed limit value includes the stop of the forklift 20 = speed 0 [km / h]. The "speed limit" referred to here is a speed limit when the forklift 20 is advanced, and the speed limit may not be imposed when the forklift 20 is retracted.

The forklift 20 includes a plurality of cameras 41 to 46, a vehicle image processing unit 47, and an image transmitting unit 48. The plurality of cameras 41 to 46 capture the surrounding environment of the forklift 20. The plurality of cameras 41 to 46 are arranged so that their viewpoints are different from each other. Therefore, the viewpoints of the images captured by the plurality of cameras 41 to 46 are different from each other. In other words, it can be said that each image has a different imaging position or imaging angle. In this embodiment, each of the cameras 41 to 46 is a monocular camera.

Of the plurality of cameras 41 to 46, four cameras 41 to 44 are attached to the upper part of the vehicle body 21. The first camera 41 is arranged so as to face the front of the vehicle body 21. The second camera 42 and the third camera 43 are arranged so as to face different directions on both sides in the vehicle width direction. The fourth camera 44 is arranged so as to face the rear of the vehicle body 21. The fifth camera 45 is arranged below the four cameras 41 to 44. The fifth camera 45 is arranged above the mast 25. The fifth camera 45 is arranged so as to face the front of the vehicle body 21 and diagonally downward of the mast. The sixth camera 46 is arranged below the four cameras 41 to 44. The sixth camera 46 is attached to, for example, a fork 26. The sixth camera 46 is arranged so as to face the front of the vehicle body 21. The first camera 41, the fifth camera 45, and the sixth camera 46 are cameras that image the front of the forklift 20.

The vehicle image processing unit 47 of the present embodiment performs image processing on the image data obtained by imaging the cameras 41 to 46, and outputs an image signal including the image-processed image data to the image transmitting unit 48. .. The image data of this embodiment is RGB format data. The image transmission unit 48 wirelessly transmits the image signal transmitted from the vehicle image processing unit 47 to the remote control device 50. The image signal is transmitted periodically.

As described above, the receiving unit 29, the transmitting unit 30, and the image transmitting unit 48 of the forklift 20 are provided for wireless communication with the remote control device 50, and function as a vehicle communication unit that performs wireless communication. ing.

Next, the remote control device 50 will be described.
The remote control device 50 is provided, for example, in a control room where the forklift 20 cannot be visually recognized, specifically, a place different from the work place where the forklift 20 works.

As shown in FIG. 1, the remote control device 50 includes a travel operation unit 51 and a cargo handling operation unit 52. The traveling operation unit 51 and the cargo handling operation unit 52 are, for example, tiltable lever-type operation units. The traveling operation unit 51 is used when the forklift 20 is made to travel by the remote control device 50. The cargo handling operation unit 52 is used when the forklift 20 is made to perform a cargo handling operation by the remote control device 50. The number of cargo handling operation units 52 may be the same as the number of cargo handling operations, depending on the type of cargo handling operation. Further, the cargo handling operation unit 52 may be a single number, and the type of cargo handling operation corresponding to the cargo handling operation unit 52 may be switched by a switch or the like.

As shown in FIG. 3, the remote control device 50 includes a transmission unit 53, a reception unit 54, and a remote controller 55. The remote controller 55 includes a CPU 56 and a storage unit 57 including a RAM, a ROM, and the like. Various programs for controlling the remote control device 50 are stored in the storage unit 57. The remote controller 55 may include dedicated hardware that executes at least a part of the various processes, for example, an integrated circuit for a specific application: ASIC. The remote controller 55 may be configured as one or more processors operating according to a computer program, one or more dedicated hardware circuits such as an ASIC, or a circuit including a combination thereof. The processor includes a CPU and memories such as RAM and ROM. The memory stores a program code or a command configured to cause the CPU to execute the process. Memory, or computer-readable medium, includes anything that can be accessed by a general purpose or dedicated computer.

The remote controller 55 generates commands related to the traveling operation according to the operating amount of the traveling operation unit 51 and the operating direction. The remote controller 55 generates a command regarding the cargo handling operation according to the operation amount of the cargo handling operation unit 52 and the operation direction. The remote controller 55 generates a remote instruction signal including a command related to the traveling operation and a command related to the cargo handling operation.

The transmission unit 53 wirelessly transmits the remote instruction signal generated by the remote controller 55. As a result, the forklift 20 is remotely controlled according to the operation of the traveling operation unit 51 and the cargo handling operation unit 52.

The receiving unit 54 receives the vehicle status signal transmitted from the transmitting unit 30 of the forklift 20. When the receiving unit 54 receives the vehicle status signal, the receiving unit 54 sends the vehicle status signal to the remote controller 55. The remote controller 55 grasps the state of the forklift 20 from the information included in the vehicle state signal.

The remote control device 50 includes an image receiving unit 61, a remote image processing unit 62, and a monitor 63. The image receiving unit 61 receives the image signal transmitted from the forklift 20. The image receiving unit 61 outputs the received image signal to the remote image processing unit 62.

The remote image processing unit 62 performs image processing on the image data included in the image signal. The image processing includes a process for displaying the image data on the monitor 63 and a process for detecting the palette P from the image data.

The monitor 63 is arranged at a position where the operator who operates the traveling operation unit 51 and the cargo handling operation unit 52 can visually recognize the monitor 63. Images captured by the cameras 41 to 46 by the remote image processing unit 62 are displayed on the monitor 63. The operator grasps the surrounding environment of the forklift 20 from the image displayed on the monitor 63, and remotely controls the forklift 20.

As described above, the receiving unit 54, the transmitting unit 53, and the image receiving unit 61 of the remote control device 50 are provided for wireless communication with the forklift 20, and function as a remote communication unit that performs wireless communication. ing.

The remote control system 10 performs a process of setting the detection range of the obstacle sensor 32 so that the pallet P is not erroneously detected as an obstacle by the obstacle sensor 32. Hereinafter, the process of setting the detection range of the obstacle sensor 32 will be described.

As shown in FIG. 4, in step S1, the remote controller 55 derives the detection range of the obstacle sensor 32 according to the speed of the forklift 20. The speed of the forklift 20 can be grasped from the speed information included in the vehicle status signal. The detection range is a range in which the main controller 33 detects an obstacle in front of the vehicle body 21. The remote controller 55 widens the detection range as the speed of the forklift 20 is faster, and narrows the detection range as the speed of the forklift 20 is slower. In the present embodiment, a threshold value is set for the speed of the forklift 20, and when the speed is less than the threshold value, the first range A1 is derived, and when the speed is equal to or more than the threshold value, the second range is derived. The second range A2 is a range in which the forklift 20 with respect to the front is wider than the first range A1. In the present embodiment, the second range A2 coincides with the detectable range A. The first range A1 is a range in which the range with respect to the front of the forklift 20 is narrower than the second range A2, and includes the range 0. That is, the first range A1 also includes an aspect in which obstacles are not detected.

Next, in step S2, the remote image processing unit 62 determines whether or not the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value and the forklift 20 is moving forward. Whether or not the forklift 20 is moving forward can be grasped by, for example, acquiring information on the operation direction of the traveling operation unit 51 from the remote controller 55. The "predetermined value" is set based on the detectable range A of the obstacle sensor 32, and is slightly longer than the distance that the pallet P enters the detectable range A when the forklift 20 approaches the pallet P. Is. The distance from the forklift 20 to the pallet P may be the distance from the obstacle sensor 32 to the pallet P, or the distance from the vehicle body 21 or the cargo handling device 24 of the forklift 20 to the pallet P. .. That is, as long as it is a member constituting the forklift 20, the distance to the pallet P may be set based on any member.

Whether or not the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value can be determined by comparing the size of the pallet P in the image data with a preset threshold value. In the present embodiment, it is determined whether or not the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value from the side surface of the pallet P in the image data captured by the sixth camera 46.

5 and 6 schematically show the palette P in the image data I. The remote image processing unit 62 detects the palette P from the image data I. In other words, it is determined whether or not the palette P exists in the image data I. The detection of the palette P in the image data I is performed by using the luminance information of the image data I. When an object is imaged, the brightness changes greatly at the boundary between the periphery of the object and the space. Therefore, the contour of the object can be detected from the change in the brightness value of RGB. The remote image processing unit 62 compares the matching image PM of the predetermined palette P with the contour of the object in the image data I. When comparing the contour of the object with the matching image PM, the scale of the matching image PM is changed to compare with a plurality of matching image PMs having different scales. Then, among the objects in the image data I, the objects in which both the difference between the vertical dimension (vertical pixel) of the matching image PM and the horizontal dimension (horizontal pixel) of the matching image PM are within a predetermined range are selected. Judged as pallet P. The scale is changed while maintaining the aspect ratio of the matching image PM. Thereby, the pallet P can be detected regardless of the distance from the forklift 20 to the pallet P, in other words, regardless of the size of the pallet P in the image data I.

Next, the remote image processing unit 62 determines whether or not the size of the palette P in the image data I is equal to or larger than the threshold value. In the present embodiment, the size of the pallet P is determined by using both the height direction dimension and the width direction dimension of the side surface of the pallet P. That is, individual threshold values are set for the dimension in the height direction of the pallet P and the dimension in the width direction of the pallet P, and individual determinations are made in the height direction and the width direction. Hereinafter, the threshold value with respect to the dimension in the height direction of the pallet P will be described as the height threshold value, and the threshold value with respect to the dimension in the width direction of the pallet P will be described as the width direction.

As shown in FIGS. 5 and 6, when the forklift 20 approaches the pallet P, the pallet P in the image data I becomes larger. That is, the vertical pixels and horizontal pixels of the palette P become larger. When the height dimension of the pallet P exceeds the height threshold value and the width direction dimension of the pallet P exceeds the width threshold value, the remote image processing unit 62 sets the distance between the forklift 20 and the pallet P to a predetermined value. Judge as follows. It can be said that the remote image processing unit 62 detects the distance between the forklift 20 and the pallet P by using the size of the pallet P in the image data I. The height threshold value and the width threshold value are set in consideration of wear of the pallet P due to long-term use. For example, it is assumed that the pallet P that is not worn is imaged from a position where the distance from the forklift 20 to the pallet P becomes a predetermined value. In this case, a value slightly smaller than the height dimension of the palette P in the image data I is defined as the height threshold, and a value slightly smaller than the width dimension of the palette P in the image data I is defined as the width threshold. do.

The process of step S2 described above can be performed regardless of whether the type of pallet P used in the workplace where the forklift 20 is used is singular or plural. When the type of the pallet P is singular, a single matching image PM, a single height threshold value, and a single width threshold value are set, and these may be compared with the pallet P. When there are a plurality of types of palettes P, a matching image PM, a height threshold value, and a width threshold value corresponding to the plurality of types of palettes P are set. In this case, when the palette P is detected from the image data I, matching is performed using a plurality of types of matching image PMs. Among the plurality of matching image PMs, the remote image processing unit 62 can recognize the type of the palette P in the image data I from the matching image PM that matches the palette P in the image data I. The remote image processing unit 62 determines that the distance between the forklift 20 and the pallet P is equal to or less than a predetermined value by using the height threshold value and the width threshold value corresponding to the recognized type of the pallet P. In the present embodiment, the remote image processing unit 62 functions as an image processing unit and a detection unit.

As shown in FIG. 4, when the determination result in step S2 is negative, the remote control system 10 performs the process in step S3. On the other hand, when the determination result in step S2 is affirmative, that is, when the forklift 20 is moving forward and the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value, the remote control system 10 performs the process of step S4. conduct.

In step S3, the remote image processing unit 62 outputs to the remote controller 55 that the determination result in step S2 is negative. The remote controller 55 generates a remote instruction signal including a command for designating the detection range derived in step S1, and transmits the remote instruction signal from the transmission unit 53. When the main controller 33 receives a command for designating the detection range via the receiving unit 29, the main controller 33 sets the detection range specified by the command. As a result, when the distance between the forklift 20 and the pallet P is longer than a predetermined value, or when the forklift 20 is retracting, the detection range according to the speed is set. That is, the first range A1 and the second range A2 are switched according to the speed of the forklift 20.

Switching between the first range A1 and the second range A2 may be performed by changing the distance information used for detecting an obstacle. When the first range A1 is set as the detection range, the main controller 33 detects an obstacle by using the irradiation point within the first range A1 among the irradiation points irradiated with the laser. As a result, obstacles existing in the first range A1 are detected. That is, the irradiation points outside the first range A1 are not used for detecting obstacles. When the second range A2 is set as the detection range, the main controller 33 detects an obstacle by using the irradiation point within the second range A2 among the irradiation points irradiated with the laser.

Further, switching between the first range A1 and the second range A2 may be performed by using two sensors as the obstacle sensor 32. For example, a sensor whose detectable range A is the first range A1 and a sensor whose detectable range A is the second range A2 are used. When the first range A1 is set as the detection range, the main controller 33 detects an obstacle using a sensor whose detectable range A is the first range A1. When the second range is set as the detection range, the main controller 33 detects an obstacle using a sensor whose detectable range A is the second range A2.

In step S4, the remote image processing unit 62 outputs to the remote controller 55 that the determination result in step S2 is affirmative. The remote controller 55 generates a remote instruction signal including a designation for designating the first range A1 regardless of the detection range derived in step S1, and transmits the remote instruction signal from the transmission unit 53. When the main controller 33 receives a command for designating the first range A1 via the receiving unit 29, the main controller 33 sets the first range A1 as the detection range. As a result, when the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value, the first range A1 is set as the detection range regardless of the speed of the forklift 20. As described above, the first range A1 and the second range A2 can be switched by the remote instruction signal generated by the remote controller 55, and it can be said that the remote controller 55 functions as a detection range setting unit.

The remote control device 50 can change the forklift 20 to be a communication target (connection target). Therefore, by changing the communication target, the remote control device 50 can remotely control a plurality of forklifts 20 in the workplace.

Next, the operation of the first embodiment will be described.
When the distance from the forklift 20 to the pallet P is longer than a predetermined value, the detection range of the obstacle sensor 32 corresponds to the speed of the forklift 20. The faster the speed of the forklift 20, the longer the braking distance. Therefore, when the speed of the forklift 20 is equal to or greater than the threshold value, the second range A2, which is wider than the first range A1, is set. As a result, obstacles are appropriately detected according to the speed.

Here, when the forklift 20 approaches the pallet P when the speed of the forklift 20 is equal to or higher than the threshold value, if the detection range is maintained in the second range A2, the pallet P enters the detection range and the pallet P fails. It can be falsely detected as an object.

On the other hand, in the present embodiment, when the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value, the detection range becomes the first range A1 regardless of the speed. Therefore, it is difficult for the pallet P to enter the detection range, and it is possible to prevent the pallet P from being erroneously detected as an obstacle.

It is also conceivable to detect obstacles using cameras 41 to 46 without providing the obstacle sensor 32. However, when a monocular camera is used as the cameras 41 to 46, even if the remote image processing unit 62 can recognize the existence of an obstacle, it is difficult for the remote image processing unit 62 to recognize the distance to the obstacle. Therefore, it is difficult to determine the timing for imposing a speed limit on the forklift 20. More specifically, if the speed limit is imposed when the obstacles are imaged by the cameras 41 to 46, the speed limit may be imposed even though the distance to the obstacle is sufficiently large. .. On the other hand, by using the obstacle sensor 32 that detects an obstacle by the distance, it is possible to impose a speed limit at an appropriate timing.

The effect of the first embodiment will be described.
(1-1) When the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value, the detection range of the obstacle sensor 32 is set to the first range A1. As a result, it is possible to prevent the pallet P from being erroneously detected as an obstacle. Since the detection range of the obstacle sensor 32 can be set based on the image data I, it is possible to prevent the pallet P from being erroneously detected as an obstacle without adding equipment to the workplace where the forklift 20 is used. .. Since it is possible to suppress false detection of the pallet P as an obstacle even in a place without equipment, it can be said that false detection can be suppressed without depending on the environment of the work place where the forklift 20 is used.

(1-2) Monocular cameras are used as the cameras 41 to 46. The closer the forklift 20 is to the pallet P, the larger the pallet P in the image data I is. Therefore, the distance between the forklift 20 and the pallet P can be detected by using this. By calculating the distance from the forklift 20 to the pallet, the load on the remote image processing unit 62 can be reduced as compared with the case where it is detected that the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value.

(1-3) The remote image processing unit 62 of the remote control device 50 is used as an image processing unit that detects that the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value. When the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value by image processing, higher processing capacity is required as compared with the case where the image processing is not performed. By using the remote image processing unit 62 of the remote control device 50 as the image processing unit, the manufacturing cost can be reduced as compared with the case where the vehicle image processing unit 47 mounted on each forklift 20 is used as the image processing unit.

(Second Embodiment)
Hereinafter, a second embodiment of the remote control system for a forklift will be described. In the second embodiment, stereo cameras are used as the cameras 41 to 46. Further, the processing procedure for detecting that the distance from the pallet P to the forklift 20 is equal to or less than a predetermined value is different. Since other configurations are the same as those in the first embodiment, the description thereof will be omitted.

The stereo camera can obtain a plurality of image data I. In the present embodiment, it is assumed that two image data I can be obtained by imaging with a stereo camera. One of the two image data I will be referred to as the first image data, and the other will be referred to as the second image data. Of the cameras 41 to 46, only the camera used for detecting that the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value may be used as a stereo camera. Further, all the cameras 41 to 46 may be used as stereo cameras.

When a stereo camera is used as the cameras 41 to 46, the image transmission unit 48 transmits the first image data and the image signal including the second image data. When stereo cameras are used as the cameras 41 to 46, the following processing is performed in the determination in step S2 in the first embodiment.

First, the remote image processing unit 62 detects the palette P from the first image data and the second image data, as in the first embodiment. Next, the remote image processing unit 62 calculates the parallax from the first image data and the second image data. The parallax is obtained by comparing the first image data with the second image data and calculating the difference in image position between the first image data and the second image data for the same object reflected in each image data. The difference in image position is the difference in the number of pixels [px] between the object in the first image data and the object in the second image data. The remote image processing unit 62 uses the palette P as an object, and calculates the parallax of the palette P, specifically, the parallax of each pixel constituting the contour of the palette P.

The remote image processing unit 62 calculates the distance from the forklift 20 to the pallet P based on the parallax. When the calculated distance is equal to or less than the predetermined value, the remote image processing unit 62 determines that the distance from the forklift 20 to the pallet P is equal to or less than the predetermined value. Subsequent processing is the same as that of the first embodiment. Therefore, the stereo camera can be used to detect that the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value, thereby setting the detection range of the obstacle sensor 32.

When a stereo camera is used as the cameras 41 to 46, the remote image processing unit 62 can detect an obstacle and measure the distance to the obstacle. Therefore, it is conceivable to detect obstacles by using a stereo camera without providing the obstacle sensor 32. However, when measuring the distance to an obstacle with a stereo camera, it is necessary to calculate the parallax. In the case of a plain wall or a wall in which the same pattern is drawn evenly, it is difficult to calculate the parallax, and it is difficult to detect these walls with a stereo camera. Therefore, when the forklift 20 is traveling toward the wall due to an erroneous operation by the operator, the remote image processing unit 62 cannot recognize the wall as an obstacle, and there is a possibility that the speed limit is not imposed. On the other hand, when the obstacle sensor 32 is used to detect an obstacle, even an obstacle such as a wall that is difficult to be recognized by the stereo camera can be detected.

The effect of the second embodiment will be described.
(2-1) Stereo cameras are used as cameras 41 to 46, and the distance to the palette P is detected based on the parallax. Therefore, it is possible to detect with high accuracy that the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value.

Each embodiment can be modified and implemented as follows. Each embodiment and the following modifications can be implemented in combination with each other within a technically consistent range.
○ In each embodiment, the vehicle image processing unit 47 of the forklift 20 may be used as the image processing unit and the detection unit. In this case, the processing of steps S1 to S4 of the first embodiment, that is, the processing of setting the detection range of the obstacle sensor 32 is performed on the forklift 20 side.

The process of step S1 is performed by the main controller 33. The main controller 33 grasps the speed of the forklift 20 from the detection result of the speed sensor 31. Then, the detection range according to the speed is derived by the same process as the process in the first embodiment.

The process of step S2 is performed by the vehicle image processing unit 47. The vehicle image processing unit 47 determines whether or not the forklift 20 is moving forward by acquiring the detection result of the speed sensor 31 from the main controller 33. Then, the vehicle image processing unit 47 detects that the distance between the forklift 20 and the pallet P is equal to or less than a predetermined value by the same processing as the processing in each embodiment.

The processing of step S3 is performed by the vehicle image processing unit 47. The vehicle image processing unit 47 outputs to the main controller 33 that the determination result in step S2 is negative. The main controller 33 sets the detection range according to the speed. The process of step S4 is performed by the vehicle image processing unit 47. In step S4, the vehicle image processing unit 47 outputs to the main controller 33 that the determination result in step S2 is affirmative. The main controller 33 sets the first range A1 as the detection range. In this case, the main controller 33 functions as a detection range setting unit.

As described above, when the vehicle image processing unit 47 is used as the image processing unit and the detection unit, the detection range is set without the processing by the remote control device 50. Therefore, when setting the detection range, there is no delay due to transmitting the image data I to the remote control device 50, and the detection range can be quickly changed due to a change in speed or a change in the positional relationship with the palette P. Can be done.

○ In each embodiment, the detection of the palette P from the image data I in step S2 may be performed without using the matching image PM. For example, the color of the palette P used in the workplace where the forklift 20 is used may be set in advance, and the palette P may be detected by matching the color in the image data I with the preset color. ..

○ In each embodiment, the method of detecting that the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value may be changed. For example, when the forklift 20 is used in a workplace where the height on which the pallet P is placed is constant, the distance from the forklift 20 to the pallet P can be adjusted by adjusting the mounting positions and mounting angles of the cameras 41 to 46. The palette P is set to enter the imaging range at the timing when becomes equal to or less than a predetermined value. Then, when the remote image processing unit 62 detects the pallet P in the image data I, it determines that the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value.

○ In each embodiment, the number of obstacle sensors 32 for detecting obstacles in front of the forklift 20 may be appropriately changed. For example, obstacle sensors 32 may be provided on both reach legs 23.

-In each embodiment, an obstacle sensor 32 for detecting an obstacle in the vehicle width direction of the forklift 20 or behind the forklift 20 may be provided. With respect to these obstacle sensors 32, it is not necessary to change the detection range when the distance from the forklift 20 to the pallet P becomes equal to or less than a predetermined value.

○ In each embodiment, the mounting position of the obstacle sensor 32 is any position as long as the distance to the obstacle existing in front of the forklift 20 such as the tip of the fork 26, the mast 25, and the upper part of the vehicle body 21 can be measured. It may be in any position.

○ In each embodiment, the detection range of the obstacle sensor 32 may be changed in three or more stages. For example, when the detection range is switched to three stages, the detection range when the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value is set as the first range A1. Then, the detection range when the speed of the forklift 20 is equal to or higher than the threshold value is set to the second range A2 which is wider than the first range A1, and the detection range when the speed of the forklift 20 is less than the threshold value is the first range A1 and the second range A2. It is the third range which is the range between and.

When the detection range is switched to three or more stages, the first range A1 does not have to be the narrowest detection range. For example, when the detection range can be switched in three or more stages, the second widest detection range may be set as the first range A1 and the widest detection range may be set as the second range A2. That is, the first range A1 does not have to be the widest range, and the second range A2 is an arbitrary range wider than the first range A1.

○ In each embodiment, the change of the detection range according to the speed may be changed so as to be proportional to the speed.
○ In each embodiment, the detection range may not be changed according to the speed. That is, the detection range of the obstacle sensor 32 may be changed only by determining whether or not the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value. When the detection range is not changed according to the speed, the detection range is set to the first range A1 when the distance from the forklift 20 to the pallet P is equal to or less than the predetermined value, and the detection range is detected when the distance from the forklift 20 to the pallet P is longer than the predetermined value. The range is the second range A2.

Even if the detection range is not changed according to the speed, the obstacle sensor 32 can be appropriately changed according to whether the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value. Detection is possible. If the detection range is not changed depending on whether the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value and the detection range is constant, in order to prevent the pallet P from being erroneously detected as an obstacle. , It is necessary to keep the detection range short at all times. Then, even if the distance from the forklift 20 to the pallet P is sufficiently long, only obstacles within a short detection range can be detected.

On the other hand, by setting the detection range to the first range A1 only when the distance from the forklift 20 to the pallet P is equal to or less than the predetermined value, if the distance to the pallet P is longer than the predetermined value, it is larger than the first range A1. Obstacles can be detected in the wide second range A2.

○ In each embodiment, when one of the dimension in the height direction of the pallet P and the dimension in the width direction of the pallet P exceeds the threshold value, the distance from the forklift 20 to the pallet P becomes equal to or less than a predetermined value. You may judge. Further, in each embodiment, one of the height threshold value and the width threshold value may be set.

-In each embodiment, as the obstacle sensor 32, a laser range finder, an ultrasonic sensor, a sensor that measures a distance by radio waves, or the like may be used in combination.
○ In each embodiment, when an obstacle is detected, the monitor 63 may indicate that the obstacle is near, instead of imposing a speed limit on the forklift 20. Further, both the speed limit may be imposed and the display on the monitor 63 may be performed.

○ In each embodiment, the format of the image data I may be, for example, the HSV format. It may also be in grayscale format.
○ In each embodiment, a flat pallet is given as an example of the pallet P, but other types of pallets such as a post pallet may be used. By changing the matching image PM and each threshold value according to the type of the palette P, it is possible to suppress false detection of an obstacle regardless of the type of palette P.

○ In each embodiment, the remote image processing unit 62 determines whether or not the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value from the image data obtained by the imaging of the first camera 41 and the fifth camera 45. You may.

○ In each embodiment, the forklift 20 may be a counter type.
○ In the first embodiment, the remote image processing unit 62 determines that the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value by using the area of the side surface of the pallet P as the size of the pallet P. You may. In this case, the remote image processing unit 62 compares the area of the matching image PM with the area of the pallet P in the image data I, and when the difference in area is within a predetermined range, the distance from the forklift 20 to the pallet P is a predetermined value. Judge that it became the following. The area of the palette P in the image data I can be obtained by, for example, setting the color of the palette P to be used in advance and calculating the area of the color in the image data I.

○ In the first embodiment, the determination that the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value may be made without using the threshold value. For example, by using the matching image PM, it may be determined that the distance from the forklift 20 to the pallet P is equal to or less than a predetermined value. More specifically, as the matching image PM, an image of the pallet P captured at a position where the distance from the forklift 20 to the pallet P becomes a predetermined value is used. The remote image processing unit 62 performs matching processing without changing the scale of the matching image PM. When the dimensional difference between the palette P of the image data I and the matching image PM is within a predetermined range, the remote image processing unit 62 determines that the distance from the forklift 20 to the palette P is equal to or less than a predetermined value.

-In each embodiment, the image processing unit and the detection unit may be provided as separate devices.

A1 ... 1st range, A2 ... 2nd range, P ... pallet, 10 ... remote control system for forklift, 20 ... forklift, 29 ... receiver, 30 ... transmitter, 32 ... obstacle sensor, 33 ... main controller, 41 ... 1st camera, 45 ... 5th camera, 46 ... 6th camera, 47 ... vehicle image processing unit, 48 ... image transmission unit, 50 ... remote control device, 53 ... transmission unit, 54 ... receiver unit, 55 ... remote controller , 61 ... Image receiving unit, 62 ... Remote image processing unit.

Claims (5)

For a forklift that includes a forklift including a vehicle communication unit that performs wireless communication and a remote control device that includes a remote communication unit that performs wireless communication, and the forklift is remotely controlled by a remote instruction signal transmitted from the remote communication unit. It ’s a remote control system.
The forklift
An obstacle sensor that measures the distance to an obstacle in front of the forklift,
A camera that captures the front of the forklift
A speed sensor for detecting the speed of the forklift is provided.
The remote control system for forklifts is
An image processing unit that processes image data obtained by the camera, and an image processing unit.
A detection unit that detects the distance from the forklift to the pallet from the information obtained by the image processing unit, and
A detection range setting unit capable of setting the detection range of the obstacle sensor to a first range or a second range in which the range with respect to the front of the forklift is wider than the first range is provided.
The detection range setting unit
The faster the speed of the forklift, the wider the detection range of the obstacle sensor is set with respect to the front of the forklift.
For forklifts that set the detection range of the obstacle sensor to the first range regardless of the speed of the forklift when the distance from the forklift to the pallet detected by the detection unit is equal to or less than a predetermined value. Remote control system. The camera is a monocular camera and
The remote control system for a forklift according to claim 1, wherein the detection unit detects a distance from the forklift to the pallet from the size of the pallet in the image data. The camera is a stereo camera
The remote control system for a forklift according to claim 1, wherein the detection unit detects the distance from the forklift to the pallet based on the parallax obtained from the image data. The remote control system for a forklift according to any one of claims 1 to 3, wherein the remote control device includes the image processing unit and the detection unit. The remote control system for a forklift according to any one of claims 1 to 3, wherein the forklift includes the image processing unit and the detection unit.

Images