JP7768028B2 - Cargo Handling System - Google Patents

Description

This disclosure relates to a cargo handling system.

The cargo handling vehicle disclosed in Patent Document 1 is equipped with a cargo handling device. The cargo handling vehicle performs cargo handling operations. The cargo handling operations include loading and unloading. In loading, cargo is loaded onto the transport vehicle. In unloading, the cargo loaded onto the transport vehicle is unloaded by the cargo handling vehicle.

Japanese Patent Application Laid-Open No. 2021-160860

The direction in which the loading device operates may differ depending on whether the loading vehicle is located on the left or right side of the transport vehicle.

A loading and unloading system that solves the above problem includes an external sensor that detects the position of an object using coordinates in a three-dimensional coordinate system, a loading and unloading vehicle equipped with a loading device, and a control device. The control device determines whether the loading and unloading vehicle is located to the left or right of the transport vehicle from point cloud data, which is a collection of points representing the position of the object, and determines the loading and unloading operation direction, which varies depending on whether the loading and unloading vehicle is located to the left or right of the transport vehicle, based on whether the loading and unloading vehicle is located to the left or right of the transport vehicle.

The control device determines whether the cargo handling vehicle is located to the left or right of the transport vehicle from the point cloud data. The direction of the cargo handling operation differs depending on whether the cargo handling vehicle is located to the left or right of the transport vehicle. By determining whether the cargo handling vehicle is located to the left or right of the transport vehicle, the control device can determine the direction of the cargo handling operation.

In the above-mentioned cargo handling system, the control device may create a side view of the transport vehicle from the point cloud data, extract the driver's seat from the side view using image recognition, and determine whether the cargo handling vehicle is located on the left or right side of the transport vehicle based on the position of the driver's seat in the side view.

In the above-mentioned cargo handling system, the control device may estimate the self-position of the cargo handling vehicle and determine whether the cargo handling vehicle is located to the left or right of the transport vehicle based on the positional relationship between the self-position and the stopping position of the transport vehicle.

According to the present invention, the control device can determine the direction of loading and unloading operations.

1 is a schematic diagram of an area in which a loading vehicle operates; FIG. 2 is a perspective view of a transport vehicle and a loading vehicle. FIG. 2 is a perspective view of a side shift device. FIG. 1 is a schematic configuration diagram of a cargo handling vehicle. 10 is a flowchart showing a cargo handling operation direction determination control. FIG. 1 is a schematic diagram of a point cloud map. FIG. FIG. 1 is a schematic diagram of a work area. 10 is a flowchart showing a modified example of cargo handling operation direction determination control. 10 is a diagram showing the positional relationship between a transport vehicle and a cargo handling vehicle when the area of the stopping position is large. FIG.

An embodiment of the cargo handling system will be described below.
As shown in FIG. 1 , a stopping position PS1 is set in the area A1. A plurality of stopping positions PS1 are set. The stopping positions PS1 are arranged at intervals from each other. A transport vehicle 10 stops at the stopping position PS1. The area of the stopping position PS1 is larger than the area of the transport vehicle 10 when viewed from above when stopped at the stopping position PS1. No structures such as pillars are present at the stopping position PS1. The area A1 is, for example, the entire area or part of a location such as a factory, port, airport, commercial facility, or public facility. A cargo handling vehicle 20 is operated in the area A1. The cargo handling vehicle 20 performs loading and unloading. Loading is the operation of loading a load C1 placed on a pallet PA1 onto the transport vehicle 10. Unloading is the operation of removing the load C1 loaded on the transport vehicle 10 from the transport vehicle 10. In the following description, the front, back, left, right, top and bottom of the transport vehicle 10 refer to the front, back, left, right, top and bottom when the transport vehicle 10 is used as the reference. The left, right, top and bottom of the cargo handling vehicle 20 refer to the left, right, top and bottom when the cargo handling vehicle 20 is used as the reference.

As shown in FIG. 2, the transport vehicle 10 is a wing truck. The transport vehicle 10 may be any type of truck, such as a flatbed truck. The transport vehicle 10 includes a driver's seat 11, a front panel 12, a rear door 13, a cargo bed 14, wing side panels 16, and a tailgate 17.

The driver's seat 11 is a position where the driver of the transport vehicle 10 sits. The front panel 12 is located further rearward of the transport vehicle 10 than the driver's seat 11. The front panel 12 is located adjacent to the driver's seat 11. The rear door 13 is located further rearward of the transport vehicle 10 than the front panel 12. The front panel 12 and the rear door 13 are located at a distance from each other in the fore-and-aft direction of the transport vehicle 10. The loading platform 14 extends in the fore-and-aft direction of the transport vehicle 10 between the front panel 12 and the rear door 13. The loading platform 14 has a loading surface 15. The loading surface 15 is the upper surface of the loading platform 14. Loads C1 placed on pallets PA1 are loaded on the loading surface 15. The wing side panels 16 are located between the front panel 12 and the rear door 13. The wing side panels 16 are rotatable in the vertical direction of the transport vehicle 10 around the center position of the transport vehicle 10 in the vehicle width direction. One wing side panel 16 is provided on each side of the transport vehicle 10 in the vehicle width direction. The tailgate 17 is provided to extend in the front-to-rear direction of the transport vehicle 10. The tailgate 17 is provided along the edge of the loading platform 14, which is the edge extending in the front-to-rear direction of the transport vehicle 10. One tailgate 17 is provided on each side of the transport vehicle 10 in the vehicle width direction.

<Loading vehicle>
The cargo handling vehicle 20 includes a vehicle body 21, drive wheels 22, steering wheels 23, and a cargo handling device 24. The cargo handling device 24 is provided at the front of the vehicle body 21. The cargo handling device 24 includes a mast 25, a lift cylinder 28, a lift bracket 29, two forks 30, and a side shift device 40.

The mast 25 comprises an outer mast 26 and an inner mast 27. The inner mast 27 is movable up and down relative to the outer mast 26. The lift bracket 29 and fork 30 move up and down together with the inner mast 27. The lift cylinder 28 raises and lowers the inner mast 27. The lift cylinder 28 is a hydraulic cylinder.

The side shift device 40 moves the forks 30 left and right on the cargo handling vehicle 20. The side shift device 40 moves the two forks 30 left and right on the cargo handling vehicle 20 while maintaining the distance between the two forks 30.

As shown in FIG. 3, the side shift device 40 is attached to the lift bracket 29. The lift bracket 29 has two finger bars 31, 32. The two finger bars 31, 32 are spaced apart in the vertical direction of the cargo handling vehicle 20.

The side shift device 40 includes a shifter 41 and a shift cylinder 46 .
The shifter 41 is provided so as to be movable to the left and right of the cargo handling vehicle 20 relative to the two finger bars 31, 32. The shifter 41 includes two shifter bars 42, 43 and two connecting members 44, 45 that connect the two shifter bars 42, 43. The two shifter bars 42, 43 are provided at a distance from each other in the up-down direction of the cargo handling vehicle 20. The two connecting members 44, 45 are provided at a distance from each other in the left-right direction of the cargo handling vehicle 20. The connecting members 44, 45 connect the ends of the shifter bars 42, 43 to each other. The forks 30 are connected to the shifter bars 42, 43.

The shift cylinder 46 is a hydraulic cylinder. The shifter 41 moves left and right on the cargo handling vehicle 20 by supplying and discharging hydraulic oil to the shift cylinder 46. The fork 30 also moves left and right on the cargo handling vehicle 20 together with the shifter 41.

As shown in FIG. 4, the cargo handling vehicle 20 includes an external sensor 51, a control device 52, an auxiliary memory device 55, a vehicle control device 56, a travel actuator 59, a cargo handling actuator 60, and a communication device 61. The cargo handling vehicle 20 is a cargo handling system.

The external sensor 51 detects the position of an object using coordinates in a three-dimensional coordinate system. The external sensor 51 is provided on the top of the cargo handling vehicle 20. For example, the external sensor 51 is provided on the head guard of the cargo handling vehicle 20.

Examples of the external sensor 51 include millimeter-wave radar, a stereo camera, a ToF (Time of Flight) camera, and a LIDAR (Laser Imaging Detection and Ranging). In this embodiment, a LIDAR is used as the external sensor 51. The external sensor 51 irradiates the surrounding area with a laser and derives the distance to the point by receiving the light reflected from the point where the laser hits. The point where the laser hits represents a portion of the surface of the object. The position of the point can be expressed by coordinates in a polar coordinate system. The coordinates of the point in the polar coordinate system are converted to coordinates in a Cartesian coordinate system. The conversion from the polar coordinate system to the Cartesian coordinate system may be performed by the external sensor 51 or by the control device 52. In this embodiment, it is assumed that the conversion from the polar coordinate system to the Cartesian coordinate system is performed by the external sensor 51. The external sensor 51 derives the coordinates of the point in the sensor coordinate system. The sensor coordinate system is a three-axis Cartesian coordinate system with the external sensor 51 as the origin. The external sensor 51 outputs the coordinates of multiple points obtained by irradiating the laser to the control device 52 as point cloud data.

The control device 52 includes a processor 53 and a memory unit 54. The memory unit 54 includes RAM (Random Access Memory) and ROM (Read Only Memory). The memory unit 54 stores program code or instructions configured to cause the processor 53 to execute processes. The memory unit 54, i.e., computer-readable medium, includes any available medium accessible by a general-purpose or special-purpose computer. The control device 52 may be configured with hardware circuits such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control device 52, which is a processing circuit, may include one or more processors that operate according to a computer program, one or more hardware circuits such as an ASIC or FPGA, or a combination thereof.

The auxiliary storage device 55 stores information that can be read by the control device 52. Examples of the auxiliary storage device 55 include a hard disk drive and a solid state drive. The auxiliary storage device 55 stores an environmental map D1. The auxiliary storage device 55 stores an image recognition model D2.

Environmental map D1 is information about the physical structure of area A1, such as the shapes of objects present in area A1 and the size of area A1. In this embodiment, environmental map D1 is data that represents the structure of area A1 using coordinates in a map coordinate system. The map coordinate system is a three-axis Cartesian coordinate system. The map coordinate system is a coordinate system with any one point in area A1 as its origin. In the map coordinate system, the horizontal direction is defined by the X-axis and Y-axis, which are perpendicular to each other. The XY plane defined by the X-axis and Y-axis can be said to represent the horizontal plane. In the map coordinate system, the up-down direction is defined by the Z-axis, which is perpendicular to the X-axis and Y-axis. Coordinates in the map coordinate system will be referred to as map coordinates where appropriate. The map coordinate system is a three-dimensional coordinate system that represents three-dimensional positions.

The control device 52 estimates the self-position of the cargo handling vehicle 20. The control device 52 outputs the self-position of the cargo handling vehicle 20 to the vehicle control device 56. The self-position is the position of the cargo handling vehicle 20 on the environmental map D1. The self-position is a coordinate indicating a point on the cargo handling vehicle 20 in the map coordinate system. The point on the cargo handling vehicle 20 can be any point, but could be, for example, the center position of the cargo handling vehicle 20 in the horizontal direction.

The self-location is estimated by comparing the detection results of the external sensor 51 with the environmental map D1. The control device 52 extracts landmarks from the environmental map D1 that have the same shape as the landmarks obtained from the point cloud data. The control device 52 recognizes the positions of the landmarks from the environmental map D1. The positional relationship between the landmark positions and the loading vehicle 20 can be determined from the detection results of the external sensor 51. Therefore, the control device 52 can estimate the self-location by recognizing the positions of the landmarks. A landmark is an object that has characteristics that can be identified by the external sensor 51. A landmark is a physical structure whose position is unlikely to change. Examples of landmarks include walls and pillars. The self-location may be estimated by combining self-location estimation using the external sensor 51 with dead reckoning using an internal sensor. The self-location may be estimated by combining self-location estimation using the external sensor 51 with self-location estimation using satellite signals transmitted from GNSS (Global Navigation Satellite System) satellites.

The vehicle control device 56 has, for example, the same hardware configuration as the control device 52. The vehicle control device 56 has a processor 57 and a storage unit 58.
The travel actuator 59 is an actuator that causes the cargo handling vehicle 20 to travel. The travel actuator 59 includes, for example, a motor that rotates the drive wheels 22 and a steering mechanism. The vehicle control device 56 controls the travel actuator 59 while determining its own position to cause the cargo handling vehicle 20 to travel.

The cargo handling actuator 60 is an actuator that causes the cargo handling vehicle 20 to handle cargo. The cargo handling actuator 60 includes, for example, a motor that drives a pump that supplies hydraulic oil to the hydraulic equipment, and a control valve that controls the supply of hydraulic oil. The hydraulic equipment includes the lift cylinder 28 and the shift cylinder 46. The vehicle control device 56 controls the cargo handling actuator 60 to raise and lower the forks 30 and to side-shift the forks 30.

The cargo handling vehicle 20 travels automatically by controlling the travel actuator 59 with the vehicle control device 56. The cargo handling vehicle 20 automatically handles cargo by controlling the cargo handling actuator 60 with the vehicle control device 56. The cargo handling vehicle 20 is an autonomous forklift.

The communication device 61 is a communication device capable of communicating using any wireless communication method, such as wireless LAN, Zigbee (registered trademark), LPWA (Low Power Wide Area), or a mobile communication system. The communication device 61 is capable of sending and receiving wireless signals. The communication device 61 demodulates the received wireless signals and outputs the demodulated data to the vehicle control device 56.

The communication device 61 receives commands from the host control device 71. The host control device 71 is installed, for example, in area A1. The host control device 71 transmits operation data input by a person to the communication device 61. The operation data includes commands for the cargo handling vehicle 20. Examples of commands include a command to move the cargo handling vehicle 20 forward, a command to start loading the cargo handling vehicle 20, and a command to start unloading the cargo handling vehicle 20. The vehicle control device 56 controls the cargo handling vehicle 20 in accordance with the commands from the host control device 71.

The control for determining the direction of the loading/unloading operation performed by the control device 52 will now be described. The control for determining the direction of the loading/unloading operation is control for determining the direction of the loading/unloading operation. The direction of the loading/unloading operation is the direction in which the loading device 24 is operated, and differs depending on whether the loading/unloading vehicle 20 is located on the left or right side of the transport vehicle 10. One example of the direction of the loading/unloading operation is the side shift direction of the forks 30. When the loading/unloading vehicle 20 loads cargo, cargoes C1 are loaded sequentially onto the loading platform 14 of the transport vehicle 10 from the front to the rear of the transport vehicle 10. The cargoes C1 are loaded starting from a position closest to the driver's seat 11. At this time, the forks 30 are side-shifted by the side shift device 40 to reduce the gap between the cargoes C1 loaded on the loading platform 14. When loading cargo and the loading vehicle 20 is located on the right side of the transport vehicle 10, the side shift direction is to the right of the loading/unloading vehicle 20. When loading cargo and the cargo handling vehicle 20 is located on the left side of the transport vehicle 10, the side shift direction is to the left of the cargo handling vehicle 20. When the cargo handling vehicle 20 is unloading cargo, it sequentially unloads cargoes C1 loaded on the loading platform 14 of the transport vehicle 10 from the rear to the front of the transport vehicle 10. If the gap between the cargoes C1 is small, there is a risk of adjacent cargoes C1 interfering with each other. Therefore, to increase the gap between the cargoes C1, the forks 30 are side-shifted by the side shift device 40. When unloading cargo and the cargo handling vehicle 20 is located on the right side of the transport vehicle 10, the side shift direction is to the left of the cargo handling vehicle 20. When unloading cargo and the cargo handling vehicle 20 is located on the left side of the transport vehicle 10, the side shift direction is to the right of the cargo handling vehicle 20.

For example, the cargo handling vehicle 20 approaches the transport vehicle 10 based on a command from the upper control device 71. Then, when the distance between the cargo handling vehicle 20 and the transport vehicle 10 becomes less than a predetermined distance, control for determining the direction of the cargo handling operation is initiated. The predetermined distance is, for example, the distance that the transport vehicle 10 can detect using the external sensor 51.

<Load handling direction determination control>
As shown in FIGS. 5 and 6 , in step S1, the control device 52 creates a point cloud map PM1. The point cloud map PM1 is obtained by overlaying point cloud data obtained from the detection results of the external sensor 51. The control device 52 acquires point cloud data from the external sensor 51 multiple times while the loading vehicle 20 is moving. The control device 52 converts the coordinates of each point P1 in the point cloud data from the coordinates of the sensor coordinate system to map coordinates based on its own position. The control device 52 can recognize the origin of the sensor coordinate system in the map coordinate system based on its own position estimated by the control device 52. The control device 52 can recognize the deviation between the coordinate axes of the map coordinate system and the coordinate axes of the sensor coordinate system based on the origin of the sensor coordinate system in the map coordinate system and the deviation between the coordinate axes of the map coordinate system and the coordinate axes of the sensor coordinate system. The control device 52 creates the point cloud map PM1 by overlaying each point P1 converted to map coordinates each time point cloud data is acquired. The point cloud map PM1 is a collection of point cloud data. Compared to the points P1 of the point cloud data, the points P1 of the point cloud map PM1 are denser.

Figure 6 shows the point cloud map PM1 obtained by the processing of step S1. Each point P1 included in the point cloud map PM1 represents the map coordinates of an object. For ease of explanation, the points P1 included in the point cloud map PM1 will be classified into a first point P11, a second point P12, a third point P13, and a fourth point P14. The first point P11 is a point P1 obtained by irradiating the gate 17 with a laser. The second point P12 is a point P1 obtained by irradiating the driver's seat 11 with a laser. The third point P13 is a point P1 obtained by irradiating the pallet PA1 and cargo C1 loaded on the loading surface 15 with a laser. The fourth point P14 is a point P1 that does not fall into any of the first point P11, second point P12, or third point P13.

As shown in Figures 5 and 7, in step S2, the control device 52 creates a side view IM1 from the point cloud map PM1. Because the point cloud map PM1 is a collection of point cloud data, the side view IM1 can be said to have been created from the point cloud data. The side view IM1 is a view in which point P1 is projected in the vehicle width direction of the transport vehicle 10. This makes it possible to make the point cloud map PM1 two-dimensional, so the side view IM1 can be treated as image data. An example of the processing performed by the control device 52 when creating the side view IM1 will be described below.

The control device 52 calculates the normal vector of each point P1. A normal vector is a vector oriented in a direction perpendicular to a plane surrounded by multiple points P1. Methods for deriving a normal vector include a method of determining a curved surface from each point P1 and deriving the normal vector of each point P1 from the curved surface, and a method using the cross product of vectors. For example, when the control device 52 determines the normal vector of one point P1, it determines the cross product of the vectors pointing from this point P1 to each of two points P1 located within a specified range. This cross product is the normal vector.

The control device 52 extracts point P1 where the normal vector is oriented horizontally. For example, the control device 52 determines whether the angle of the normal vector with respect to the XY plane of the map coordinate system is within a predetermined range. If the normal vector is oriented horizontally, the normal vector is parallel to the XY plane of the map coordinate system. Taking into account the inclination of the transport vehicle 10 and measurement errors, a predetermined range is set so that point P1 where the normal vector can be considered to be oriented horizontally can be extracted.

The control device 52 derives a plane equation from point P1, whose normal vector is oriented horizontally. The plane equation can be derived using, for example, a robust estimation method such as RANSAC (Random Sample Consensus) or the least squares method. The plane represented by the plane equation is a surface that extends vertically. In the case of the transport vehicle 10 of this embodiment, the plane can be obtained from the first point P11 defined by the gate 17. Furthermore, when unloading goods, since the transport vehicle 10 is loaded with a pallet PA1 and a load C1, the plane can be obtained from the third point P13 defined by the pallet PA1 and the load C1. The control device 52 creates a side view IM1 by projecting point P1 in a direction perpendicular to the plane represented by the plane equation.

Next, in step S3, the control device 52 extracts the driver's seat 11 from the side view IM1 to determine the position of the driver's seat 11 in the side view IM1. The position of the driver's seat 11 in the side view IM1 can be represented by an image coordinate system. The image coordinate system is a coordinate system in which the horizontal direction of the side view IM1 is the X axis and the vertical direction is the Y axis. The position of the driver's seat 11 is determined using image recognition. In this embodiment, the case where image recognition is performed using the image recognition model D2 is described, but image recognition may also be performed by pattern matching.

Image recognition model D2 is a trained model generated by machine learning. Image recognition model D2 uses an algorithm capable of determining the class of an object on a region-by-region basis. The class is set to "driver's seat." Examples of machine learning algorithms include SSD (Single Shot Multibox Detector), R-CNN (Regional Convolutional Neural Network), fast R-CNN, faster R-CNN, and YOLO (You Only Look Once). Image recognition model D2 is generated by supervised learning or semi-supervised learning using training data. The training data includes image data containing an object corresponding to the class, as well as the object's position and label in the image data. The training data can be generated, for example, by surrounding the object in the image data with a frame and labeling the image data. In this embodiment, the training data is obtained by surrounding the driver's seat 11 in the image data with a frame and labeling the image data as "driver's seat." The image data used as training data may be obtained from the point cloud map PM1, as with the side view IM1, or may be obtained by capturing images using an imaging device.

The image recognition model D2 identifies the area including the driver's seat 11 from the input side view IM1. The area including the driver's seat 11 is represented by a bounding box B1. In the example shown in Figure 7, the driver's seat 11 represented by the second point P12 is surrounded by the bounding box B1.

As shown in Figures 5 and 8, in step S4, the control device 52 determines work areas A11 and A12. Specifically, the control device 52 generates a boundary line BL1 that passes through the transport vehicle 10 in the forward/backward direction of the transport vehicle 10 in the map coordinate system. The control device 52 determines the areas on both sides of the boundary line BL1 as work areas A11 and A12. The work areas A11 and A12 are, for example, areas of a predetermined size. The work areas A11 and A12 include a right work area and a left work area. The right work area is the work areas A11 and A12 on the right of the transport vehicle 10. The left work area is the work areas A11 and A12 on the left of the transport vehicle 10.

When the cargo handling vehicle 20 is located to the right of the transport vehicle 10, the driver's seat 11 is located to the right of the side view IM1. The control device 52 determines that the cargo handling vehicle 20 is located to the right of the transport vehicle 10 when the driver's seat 11 is located to the right of the center position in the X-axis direction of the image coordinate system in the side view IM1. In this case, the control device 52 can determine that the work areas A11 and A12 in which the cargo handling vehicle 20 is located are the right work areas. The control device 52 can determine that the work areas A11 and A12 in which the cargo handling vehicle 20 is not located are the left work areas.

When the cargo handling vehicle 20 is located to the left of the transport vehicle 10, the driver's seat 11 is located on the left side of the side view IM1. The control device 52 determines that the cargo handling vehicle 20 is located to the left of the transport vehicle 10 when the driver's seat 11 is located to the left of the center position in the X-axis direction of the image coordinate system in the side view IM1. In this case, the control device 52 can determine that the work areas A11 and A12 in which the cargo handling vehicle 20 is located are the left work areas. The control device 52 can determine that the work areas A11 and A12 in which the cargo handling vehicle 20 is not located are the right work areas.

Next, in step S5, the control device 52 determines the direction of the loading/unloading operation. An example will be described in which the side shift direction is determined as the direction of the loading/unloading operation. As mentioned above, the direction of the loading/unloading operation differs depending on whether the loading/unloading vehicle 20 is loading or unloading. When the loading/unloading vehicle 20 is loading in the right work area, the control device 52 determines the direction of the loading/unloading operation to be to the right of the loading/unloading vehicle 20. When the loading/unloading vehicle 20 is loading in the left work area, the control device 52 determines the direction of the loading/unloading operation to be to the left of the loading/unloading vehicle 20. When the loading/unloading vehicle 20 is unloading in the right work area, the control device 52 determines the direction of the loading/unloading operation to be to the left of the loading/unloading vehicle 20. When the loading/unloading vehicle 20 is unloading in the left work area, the control device 52 determines the direction of the loading/unloading operation to be to the right of the loading/unloading vehicle 20.

After completing step S5, the control device 52 ends the loading/unloading operation direction determination control. The control device 52 outputs the loading/unloading operation direction determined in the loading/unloading operation direction determination control to the vehicle control device 56. The vehicle control device 56 performs control according to the loading/unloading operation direction.

[Effects of this embodiment]
(1) The control device 52 determines whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10 from the point cloud data. The control device 52 can determine the direction of the cargo handling operation depending on whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10.

(2) The control device 52 extracts the driver's seat 11 from the side view IM1 using image recognition. The position of the driver's seat 11 differs depending on whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10. By extracting the driver's seat 11 from the side view IM1, it is possible to determine whether the cargo handling vehicle 20 is located on the left or right side of the transport vehicle 10.

(3) When the stopping position PS1 of the transport vehicle 10 is determined and the stopping orientation of the transport vehicle 10 is determined, the control device 52 can determine from its own position whether the cargo handling vehicle 20 is located to the left or right of the transport vehicle 10. However, as shown in FIG. 1 , when the cargo handling vehicle 20 is located between two stopping positions PS1, the cargo handling vehicle 20 is located to the right when one of the two transport vehicles 10 is used as the reference, and to the left when the other is used as the reference. Therefore, when the cargo handling vehicle 20 is located between two stopping positions PS1, it is not possible to determine whether the cargo handling vehicle 20 is located to the left or right of the transport vehicle 10. In contrast, when the driver's seat 11 is extracted from the side view IM1 by image recognition, it is possible to determine whether the cargo handling vehicle 20 is located to the left or right of the transport vehicle 10, even if the cargo handling vehicle 20 is located between two stopping positions PS1.

(4) The vehicle control device 56 controls the loading vehicle 20 in accordance with commands from the higher-level control device 71. The commands are included in the operation data entered by a person. If the loading operation direction is not determined by the control device 52, the person must specify the loading operation direction when entering the operation data. At this time, there is a risk that the person may enter the wrong loading operation direction. In contrast, by having the control device 52 determine the loading operation direction, it is possible to prevent a person from entering the wrong loading operation direction. This can prevent the loading vehicle 20 from malfunctioning due to a person entering the wrong loading operation direction. It can also reduce the amount of work required to create operation data.

(5) The external sensor 51 used to estimate the vehicle's own position can be used to determine whether the cargo handling vehicle 20 is located to the left or right of the transport vehicle 10. This reduces the number of parts required compared to using a dedicated sensor to determine whether the cargo handling vehicle 20 is located to the left or right of the transport vehicle 10.

[Example of change]
The embodiment can be modified as follows: The embodiment and the following modifications can be combined with each other to the extent that they are not technically inconsistent.

The loading/unloading operation direction determination control may be a control that determines the loading/unloading operation direction based on the positional relationship between the vehicle's own position and the stopping position PS1.
9, in step S11, the control device 52 estimates its own position. The estimation of the own position is the same process as in the embodiment.

Next, in step S12, the control device 52 determines the direction of the loading/unloading operation based on the positional relationship between its own position and stopping position PS1. When determining the direction of the loading/unloading operation based on the positional relationship between its own position and stopping position PS1, it is necessary to determine the stopping orientation of the transport vehicle 10. This makes it possible to determine whether the own position corresponds to the left or right side of the transport vehicle 10 when the transport vehicle 10 is stopped at stopping position PS1. It can be said that the control device 52 can determine whether the loading/unloading vehicle 20 is located to the left or right side of the transport vehicle 10 from its own position. The control device 52 can then determine the direction of the loading/unloading operation based on whether the loading/unloading vehicle 20 is located to the left or right side of the transport vehicle 10.

When determining the direction of loading/unloading operations based on the positional relationship between the vehicle's own position and stopping position PS1, it is preferable to make the area of stopping position PS1 small. If the area of stopping position PS1 is large, even if the position of the loading/unloading vehicle 20 is the same, the loading/unloading vehicle 20 may be located on the left or right side of the transport vehicle 10 depending on the position of the transport vehicle 10. For example, as shown in FIG. 10, assume that the transport vehicle 10 is stopped at first position A2 of stopping positions PS1, and the loading/unloading vehicle 20 is located to the right of the transport vehicle 10. When the loading/unloading vehicle 20 is in the same position, assume that the transport vehicle 10 is stopped at second position A3 of stopping positions PS1. The first position A2 and the second position A3 are located on either side of the loading/unloading vehicle 20. In this case, the loading/unloading vehicle 20 is located to the left of the transport vehicle 10 stopped at second position A3. It is preferable to set the area of the stopping position PS1 so that the loading vehicle 20 is positioned on either the left or right side of the transport vehicle 10 depending on the position of the transport vehicle 10.

The image recognition model D2 may use an algorithm that can determine the class of an object for each side view IM1. In this case, "right-facing" and "left-facing" may be set as classes. An example of a machine learning algorithm is a convolution neural network (CNN). The training data may be data in which image data showing a transport vehicle 10 facing right is labeled "right-facing" and data in which image data showing a transport vehicle 10 facing left is labeled "left-facing."

By inputting the side view IM1 into the image recognition model D2, the control device 52 determines whether the transport vehicle 10 shown in the side view IM1 is facing right or left. If the cargo vehicle 20 is located to the right of the transport vehicle 10, the transport vehicle 10 will be shown facing right in the side view IM1. If the cargo vehicle 20 is located to the left of the transport vehicle 10, the transport vehicle 10 will be shown facing left in the side view IM1. Therefore, the control device 52 can determine whether the cargo vehicle 20 is located to the right or left of the transport vehicle 10 from the output of the image recognition model D2.

The control device 52 does not have to create the point cloud map PM1. In this case, the control device 52 converts the point cloud data acquired from the external sensor 51 into map coordinates and performs the processing from step S2 onwards. In other words, the control device 52 may determine whether the loading vehicle 20 is located to the left or right of the transport vehicle 10 from a single point cloud data, without creating a point cloud map PM1 that aggregates multiple point cloud data.

The loading and unloading device 24 may be equipped with, for example, a robot arm. In this case, the loading and unloading operation direction may be the direction in which the robot arm operates. For example, the loading and unloading operation direction may be the direction in which the robot arm loads the load C1.

The transport vehicle 10 may be any vehicle capable of transporting goods, such as an AGV (Automatic Guided Vehicle) or an AMR (Autonomous Mobile Robot). In this case, the control device 52 may use the image recognition model D2 to determine whether the transport vehicle 10 is facing right or left, thereby determining whether the cargo handling vehicle 20 is located to the right or left of the transport vehicle 10. The control device 52 may also determine whether the cargo handling vehicle 20 is located to the right or left of the transport vehicle 10 from its own position.

Part of the processing for determining the direction of cargo handling operations may be performed by the vehicle control device 56. In this case, the vehicle control device 56 can also be considered a control device.
The upper control device 71 may perform part of the processing for determining the direction of cargo handling operations. In this case, the upper control device 71 can also be considered a control device. The upper control device 71 may perform all of the processing for determining the direction of cargo handling operations. In this case, the upper control device 71 is the control device. In this way, when the upper control device 71 performs at least part of the processing for determining the direction of cargo handling operations, the cargo handling vehicle 20 and the upper control device 71 form a cargo handling system.

P1...point, 10...transport vehicle, 11...driver's seat, 20...cargo vehicle which is a cargo handling system, 24...cargo handling device, 51...external sensor, 52...control device.

Claims (3)

a cargo handling vehicle equipped with an external sensor that detects the position of an object in coordinates of a three-dimensional coordinate system and a cargo handling device;
A cargo handling system comprising:
The control device
determining whether the cargo handling vehicle is located on the left or right side of the transport vehicle from point cloud data, which is a set of points representing the positions of the object;
A loading and unloading system in which the loading and unloading operation direction, which differs depending on whether the loading and unloading vehicle is located on the left or right side of the transport vehicle, is determined based on whether the loading and unloading vehicle is located on the left or right side of the transport vehicle. The control device
creating a side view of the transport vehicle from the point cloud data;
extracting a driver's seat from the side view by image recognition;
The cargo handling system according to claim 1 , wherein it is determined whether the cargo handling vehicle is located on the left or right side of the transport vehicle from the position of the driver's seat in the side view. The transport vehicle stops at a stopping position with a predetermined stopping direction,
The control device estimates a self-position of the cargo handling vehicle,
2. The cargo handling system according to claim 1, wherein it is determined whether the cargo handling vehicle is located to the left or right of the transport vehicle based on the positional relationship between the vehicle's own position and the stopping position.