HK40037444B - Load handling module for a material handling vehicle - Google Patents

Description

Load handling module for material handling vehicles

Cross-references to related applications

This application is based on and claims priority to U.S. Provisional Patent Application No. 62/830,155, entitled “Load Handling Module for a Material Handling Vehicle,” filed April 5, 2019.

Explanation of federally funded research

not applicable.

Technical Field

This invention relates to the field of material handling vehicles, and more particularly to autonomous control systems for material handling vehicles.

Background Technology

Material handling vehicles can be found in warehouses, factories, and shipyards, and typically anywhere there is a need to move pallets, large packages, or cargo loads from one location to another. Material handling vehicles typically include load-bearing members or forks for lifting packages or pallets. They can be designed to include sensing capabilities that make them automated guided vehicles (AGVs). AGVs can be programmed to store travel routes and include a control system integrated with the vehicle's drive, steering, and braking systems.

Summary of the Invention

Embodiments of this disclosure provide systems and methods including a load handling module for autonomously loading and unloading loads.

In one aspect, this disclosure provides a load handling module for a material handling vehicle. The material handling vehicle may include a vehicle body, a mast, a tower, and at least one fork extending from a load backrest to a tip. The tower may extend upward from the load backrest, while the at least one fork may be selectively movable relative to the vehicle body and configured to support a load. The load handling module may include a first camera fixed to the load backrest and disposed proximately to the top surface of the at least one fork, the first camera being configured to determine the position of an object within the field of view of the first camera. A second camera may be fixed to the tower such that the second camera is positioned above the first camera. Further, the second camera may be configured to determine the position of an object within the field of view of the second camera. The load handling module may further include a fork tip sensor fixed proximately to the tip of the at least one fork, the fork tip sensor being configured to detect the presence of an object within the field of view of the fork sensor extending in front of the tip. Furthermore, a controller may communicate with the first camera, the second camera, and the fork tip sensor, and the controller may be configured to autonomously control the movement of the at least one fork.

In another aspect, this disclosure provides a method for picking up a load using a material handling vehicle. The material handling vehicle may include at least one fork extending from a load backrest, a mast, a tower extending upward from the load backrest, a first camera mounted on the load backrest, a second camera mounted on the tower, and fork tip sensors disposed near the tips of the at least one fork. The method may include moving the material handling vehicle to a picking position, identifying a pallet on which the load is resting using the first camera, and aligning the at least one fork with the pallet. Further, the method may include extending the fork to engage the pallet, and raising and retracting the fork to pick up the load using the fork.

In another aspect, this disclosure provides a method for unloading a load supported by a material handling vehicle. The material handling vehicle may include at least one fork extending from a load backrest, a mast, a tower extending upward from the load backrest, a first camera mounted on the load backrest, a second camera mounted on the tower, and a fork tip sensor disposed near the tip of the at least one fork. The method may include moving the material handling vehicle to an unloading position, determining whether the unloading position is vacant, and extending the forks to move the load into the unloading position. Further, the method may include lowering the forks to place the load in the unloading position, retracting the forks to remove the load, and verifying that the position of the load is within a predefined tolerance of the unloading position.

These and other advantages of the invention will be apparent from the detailed description and accompanying drawings. The following is merely a description of some preferred embodiments of the invention. The full scope of the invention should be assessed with reference to the claims, as these preferred embodiments are not intended to be the only embodiments within the scope of the claims.

Attached Figure Description

The invention will be better understood when taken into consideration the following detailed description, and features, aspects, and advantages in addition to those set forth above will become apparent. Such detailed description is illustrated in the following drawings.

Figure 1 is a left front view of a load handling module on a material handling vehicle according to various aspects of the present disclosure;

Figure 2 is a front view of the load handling module and material handling vehicle of Figure 1 according to various aspects of the present disclosure, having a first camera field of view, a second camera field of view and a fork tip sensor field of view.

Figure 3 is a top plan view of the load handling module and material handling vehicle of Figure 2 according to various aspects of the present disclosure, including the field of view of the first camera, the second camera and the fork tip sensor;

Figure 4 is a front view of the load handling module and material handling vehicle of Figure 3 according to various aspects of the present disclosure, wherein the load is supported by the material handling vehicle;

Figure 5 is a front view of the load handling module, material handling vehicle and load of Figure 3 according to various aspects of the present disclosure, including a first camera field of view and a second camera field of view.

Figure 6 is a left front view of a load handling module with an adjustable tower on a material handling vehicle according to various aspects of the present disclosure;

Figure 7 is a front view of the load handling module and material handling vehicle of Figure 6 according to various aspects of the present disclosure, having a first camera field of view and a second camera field of view.

Figure 8 is a flowchart of a method for picking up a load according to various aspects of the present disclosure;

Figure 9 is a flowchart of a method for unloading a load on the ground according to various aspects of the present invention;

Figure 10 is a flowchart of a method for unloading a load onto a rack according to various aspects of this disclosure; and

Figure 11 is a left front view of a load handling module on a material handling vehicle according to various aspects of the present disclosure.

Detailed Implementation

Before elaborating on any aspect of this disclosure, it should be understood that this disclosure is not limited in its application to the construction details and component arrangements set forth in the following description or shown in the following figures. This disclosure can have other configurations and can be practiced or performed in various ways. Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. In this document, “comprising,” “including,” or “having,” and variations thereof mean including the items listed below and their equivalents, as well as any additional items. Unless otherwise specified or limited, the terms “mounting,” “connection,” “support,” and “coupling,” and variations thereof, are used extensively and cover both direct and indirect mounting, connection, support, and coupling. Moreover, “connection” and “coupling” are not limited to physical or mechanical connections or couplings.

Furthermore, as used herein, unless otherwise specified or limited, directional terms are presented only in relation to the specific non-limiting example and perspective described. For example, references to features or directions such as “horizontal,” “vertical,” “front,” “back,” “left,” and “right” are generally made with reference to specific figures or examples and do not necessarily indicate absolute orientations or directions. However, relative directional terms for a specific non-limiting example are generally applicable to alternative orientations of that non-limiting example. For example, “front” and “back” directions or features (or “right” and “left” directions or features, etc.) can generally be understood to indicate relatively opposite directions or features.

The invention will now be described in more detail with reference to the following embodiments. It should be noted that the following embodiments given herein are for illustrative and descriptive purposes only and are not intended to be exhaustive or limited to the precise forms disclosed.

This document primarily discloses a load handling module configurable for autonomous or semi-autonomous material handling vehicles (MHVs). The load handling module may include at least one camera, sensors, and/or any other object detection sensors to selectively control the material handling vehicle to pick up and/or unload loads at various locations, including, for example, space in a rack or space on the floor. Cameras may be positioned such that when the MHV is carrying a load, the bottom portion of the load is within the field of view (FOV) of one camera, while the top portion of the load is within the FOV of another camera. Cameras may also be positioned such that their respective FOVs extend above, below, and/or around the load side. Additionally, sensors may be placed on or near the tips of the load-bearing forks of the MHV and may be used to detect the presence of objects in front of and/or between the forks. Through the cameras and sensors, the load handling module can acquire information about the position and orientation of the load and the space around the MHV, and then use the collected information to pick up or unload the load.

Referring now to the accompanying drawings, and particularly to Figures 1-5, an embodiment of a load handling module 100 that can be attached to a fork carriage or assembly 70 of an MHV 60 is shown. The MHV 60 may include a mast 72 and a fork assembly 70 coupled to the mast 72. The fork assembly 70 may include two forks 74 (i.e., load support members) laterally separated from each other. Each fork 74 extends from the front side 76 of a load backrest 78 to a corresponding fork tip 80 opposite the load backrest 78. The forks 74 may be positioned adjacent to the bottom side 82 of the fork assembly 70 such that the top surfaces 84 of the forks 74 are generally horizontal and coplanar.

In some embodiments, the fork assembly 70 may be movably coupled to the MHV 60 such that the fork assembly 70 may selectively translate, rotate, or a combination thereof in at least one direction relative to the MHV 60. In the illustrated embodiment, the fork assembly 70 may be vertically movable (e.g., in a direction perpendicular to the floor on which the MHV 60 travels) and pivotally rotated relative to the mast 72 (see FIG. 3). Additionally or alternatively, the fork 74 may be configured to selectively translate, rotate, or a combination thereof in at least one direction relative to the fork assembly 70. In some embodiments, the MHV 60 may include a height sensor configured to measure the height of the fork 74, for example, relative to the floor on which the MHV 60 travels. In some embodiments, the MHV may include a fork assembly with more or fewer load support members, and at least one load support member may be located in a different position than the fork 74 shown in FIGS. 1-5.

The load handling module 100 may include a first camera 104 mounted on a load backrest 78 and a second camera 108 mounted in a tower 112 extending upward from the top side of the load backrest 78, above the first camera 104. Typically, the first camera 104 may be positioned slightly above the top surface 84 of the fork 74. For example, the first camera 104 may be positioned such that its lower surface is substantially coplanar with the top surface 84 of the fork 74. In some embodiments, the first camera 104 may be positioned such that its lower surface is above the top surface 84 of the fork 74 (e.g., at least one centimeter (“cm”)). In some embodiments, the first camera 104 may be positioned such that its lower surface is between 1 and 80 cm above the top surface 84 of the fork 74, an example of which is shown in FIG11. The first camera 104 may be positioned between the forks 74 such that the forks 74 are substantially symmetrical about the first camera 104. Furthermore, the first camera 104 can be oriented to project forward away from the load backrest 78, such that the first field of view (FOV) 116 of the first camera 104 (see, for example, FIG. 2) extends to the front of the fork assembly 70. More specifically, as best seen in FIG. 2, the first FOV 116 may extend toward the fork tip 80 of the fork 74. In some embodiments, the first camera 104 may be positioned such that the first FOV 116 is generally angled toward, away from, or parallel to the fork 74.

Referring back to Figure 1, the second camera 108 can be similarly positioned between the forks 74, such that the forks 74 are substantially symmetrical about the second camera 108. The second field of view (FOV) 120 of the second camera 108 can extend to the front of the fork assembly 70. Again, more specifically, the second FOV 120 of the second camera 108 can extend toward the tip 80 of the forks 74. Furthermore, the second camera 108 can be positioned such that the second FOV 120 is generally angled toward, away from, or parallel to the forks 74.

In some embodiments, the height of the second camera 108 above the first camera 104 may be defined by the average height of the load carried by the MHV 60. For example, the vertical distance (i.e., height H) between the first camera 104 and the second camera 108 may be sized to be a predetermined distance above the average height of the load carried by the MHV 60. In some embodiments, the height H may be at least 50 cm. Further, in some embodiments, the height H may be at least 100 cm. When the load 90 is supported on the fork 74 (see, for example, FIG. 5), the second FOV 120 may extend above, beyond, or otherwise exceed the load 90, such that the second camera 108 can view at least a portion of the load 90 and the area above the load 90. This positioning may be useful, for example, allowing the second camera 108 to check a desired unloading position when the load is lifted above a structure on the rack, such as an unloading position load beam (i.e., a horizontally extending beam on the rack on which the load can be placed).

In some embodiments, at least one of the first camera 104 and the second camera 108 may be a 3-D camera (e.g., a time-of-flight camera) capable of measuring distance. More specifically, the 3-D camera is generally a camera capable of providing an output that allows for the evaluation of the position of coordinates in a three-dimensional spatial volume (i.e., x, y, and z coordinates). The 3-D camera is preferably provided with illumination possibility, such that if the surrounding environment is dark, the camera itself can obtain sufficient illumination to provide the correct output. The 3-D camera typically has a field of view (FOV) that can be defined by a specific range. For example, the 3-D cameras included herein may have an FOV of 20° to 180°. In some embodiments, the 3-D cameras included herein may have an FOV of 35° to 70°. Further, some embodiments may include at least one of the first and second cameras having an FOV with a shape, size, and/or orientation different from at least one of the FOVs shown. For example, at least one of the first and second FOVs may extend from the respective camera with a wider or narrower field of view than the shown field of view. Additionally or alternatively, the FOV of at least one camera may be asymmetrical.

Referring again to FIG1, the load handling module may include at least one fork tip sensor 128 positioned in proximity to the fork tip 80 of at least one fork in the fork 74. For example, the load handling module 100 may include at least one fork tip sensor 128 mounted in or on the fork tip 80. Each fork tip sensor 128 may be configured to detect the presence or absence of an object within a fork tip sensor FOV 132 (see, for example, FIG3), which may extend from the fork tip sensor 128 in a direction away from the load backrest 78 (e.g., to the front of the fork tip 80). Additionally, in some embodiments, a portion of at least one fork tip sensor FOV 132 may be laterally angled inward to extend along the direction of another fork 74, which may aid in detecting the presence of an object between the two forks 74. In some embodiments, a portion of the fork tip sensor FOV 132 may extend laterally between the two forks 74. Additionally or alternatively, each of the fork tip sensors 128 may include a plurality of individual sensors arranged in an array. For example, individual sensors may be arranged laterally along the fork tip 80 from the outer edge to the opposite inner edge, the inner edge being closer to the opposite fork 74 than the outer edge. In some embodiments, at least one of the individual sensors arranged near the inner edge of the fork may be laterally angled inward (i.e., toward the centerline between the forks 74 or the opposite forks 74). For example, a central fork tip sensor may be configured to detect a central longitudinal beam (or any other object) between the two forks 74. In some embodiments, at least one fork tip sensor 128 may be (or may include) at least one of a time-of-flight sensor (e.g., a LiDAR sensor), a camera, and any other type of object detection sensor. It should also be understood that some embodiments may include at least one fork tip sensor having a tip sensor FOV that differs in shape, size, and/or orientation from the illustrated tip sensor FOV 132. For example, at least one tip sensor FOV may extend from a fork tip sensor having a wider or narrower field of view than the illustrated field of view. Additionally or alternatively, at least one tip sensor FOV may be asymmetrical.

Referring back to Figure 1, the load handling module 100 may include one or more load positioning sensors 142 configured to detect whether a load is positioned on the fork 74 and fully received by the fork 74. For illustrative purposes only, for example, as shown in Figure 4, a load is considered fully positioned on the fork 74 when it contacts the load backrest 78. For example, the load positioning sensor 142 may be attached to or disposed on the same vertical plane as the load backrest 78, and may be configured to detect when the load has been fully positioned on the fork 74. The load positioning sensor 142 may be a mechanical sensor or switch, a time-of-flight sensor, or any other object detection or position sensing sensor. Additionally, at least one weight sensor (not shown) may be configured to determine the weight of the load carried by the MHV 60 and may be incorporated into at least one of the fork 74, the load backrest 78, or any other part of the fork assembly 70 and the MHV 60.

However, in some embodiments, at least one of the first camera, second camera, fork tip sensor, and load positioning sensor may be positioned differently from those illustrated in the embodiment. For example, at least one of the first and second cameras may be positioned adjacent to one side of the backrest, above (or inside) the fork, on the MHV, and/or at any other location on the fork assembly or MHV. Additionally or alternatively, the load handling module may include at least one additional sensor, which may be the same as or different from the sensor in the illustrated embodiment. For example, the load handling module may include at least one additional camera, fork tip sensor, and/or any other object detection sensor.

Referring again to Figure 1, the load handling module 100 may further include a controller 150 that communicates with at least one of the first camera 104, the second camera 108, at least one fork tip sensor 128, the load positioning sensor 142, a weight sensor, and any other type of sensor (e.g., a fork height sensor) or input device. The controller 150 may also communicate with a warehouse management system (WMS) and a main navigation unit, which may include at least one additional sensor or positioning device that allows the MHV 60 to be automatically guided. For example, the main navigation unit may include at least one of a camera, a global positioning system (GPS), a rotating laser scanner, a motion sensor, a LiDAR system, a simultaneous localization and mapping (SLAM) system, a point navigation system using radio frequency identification (RFID) or magnets, and a wired navigation system. In some embodiments, the controller may be within the MHV or the main navigation unit, and the load handling module 100 may not have a separate controller.

As described above, in some embodiments, the position of at least one of the first camera 104 and the second camera 108 is adjustable. For example, Figures 6 and 7 illustrate a load handling module 500 according to another embodiment of this disclosure. The load handling module 500 is generally similar in design and function to the load handling module 100 of Figures 1-5, and the same reference numerals identify the same elements unless described herein or clearly visible from the figures. For example, the second camera 508 of the load handling module 500 may be included in (or on) a tower 512 mounted on a load backrest 78. The tower 512 may further include an actuator 560 coupled to a movable portion 564 of the tower 512. The actuator 560 may be an electric actuator, a hydraulic actuator, or any other type of actuation system or device, and may be configured to selectively move the movable portion 564 of the tower 512, including the second camera 508, relative to the fork assembly 70. The actuator 560 may be configured to selectively change the height between the first camera 504 and the second camera 508, for example, to compensate for changes in load height. For example, referring to FIG7, if the second camera 508 cannot detect the top of the load when the load has been verified to be on the fork 74 (e.g., via a load-in-place switch), the actuator 560 may adjust the height of the second camera 508 until the second camera FOV 520 extends beyond the top of the load (i.e., the second camera 508 detects the top of the load). Alternatively or additionally, if a load is detected on the fork and the second camera 508 cannot see above the load (i.e., the load substantially or completely blocks the second camera FOV 520), the height of the second camera 508 may be adjusted until a portion of the second camera FOV 520 is unobstructed. For example, the second camera 508 may be moved until at least 25%, 40%, or 60% of the second camera FOV 502 is unobstructed by the load. Further, in some embodiments, the actuator may be configured to move the second camera 508 within the tower 512. That is, the actuator may be configured to pivot, rotate, or translate the second camera 508 relative to the tower 512 and/or the movable portion 564. Additionally, one or more displacement sensors (e.g., string potentiometer, encoder, position transducer) may be included to measure the height displacement of the second camera 508.

Typically, the load handling module disclosed herein can control the MHV to pick up or unload loads using data transmitted to the controller and/or main navigation unit by at least one of the WMS, connected sensors, and operator. Figures 8-10 illustrate exemplary methods of using a load handling module to pick up or unload loads according to a non-limiting example of this disclosure. Although these methods are described with reference to the load handling module 100 and material handling vehicle 60 in Figures 1-5, the same or similar methods can be used for other material handling vehicles and/or load handling modules.

Figure 8 illustrates a method 200 for picking up a load using an MHV 60 with a load handling module 100 as described above with reference to Figures 1-5. The method may include receiving load coordinates in step 204 and traveling to the load 90. More specifically, information about the load may be provided to the controller 150 from, for example, a WMS. Target load information may include the load's grid location (e.g., X-Y location), whether the load is stored on the floor or on a rack, the height of the load pickup location (e.g., rack height), the load's dimensions, the load's weight, and at least one of any other information about the load, the pickup location, the unloading location, or the MHV 60's operating parameters. The MHV 60 can then autonomously travel to the load's X-Y location. In some embodiments, a height sensor may transmit the height of the fork 74 to the controller 150 to determine whether the load handling process described herein can occur at a height (i.e., above the ground).

In step 208, method 200 may further include checking whether the load is on the ground or in a rack (or on a raised structure), which may be performed by controller 150. For example, this operation may include determining whether the target load is on a rack based on the height coordinates provided in step 204. Additionally or alternatively, data acquired from at least one of the first camera 104, the second camera 108, the fork tip sensor 128, any other object detection system on the MHV 60, and any other information transmitted to controller 150 may be used to determine whether the load is on the ground or in a rack. In some embodiments, controller 150 on the MHV 60 may be configured to perform step 208 of checking whether the load is on the ground or in a rack before the MHV 60 is moved to the load location, while the MHV 60 is being moved to the load location, once the MHV reaches the load location, or any combination thereof.

After controller 150 has determined whether the target load is on the ground or on a rack, method 200 may include locating the pallet supporting the target load in step 212. For example, in some embodiments, first camera 104 may identify the center of the pallet by identifying the center longitudinal beam of the pallet. This operation may include moving fork assembly 70 such that the load pickup position is within the first FOV 116 of first camera 104. Additionally or alternatively, some embodiments may be configured to use at least one of a second camera 108, fork tip sensor 128, and any other object detection system to identify the pallet supporting the target load. If the pallet cannot be located, controller 150 may set an error code or send an error message. However, if the pallet is located, in step 216, load handling module 100 may determine whether the pallet is positioned within a predefined tolerance for fork 74 to engage the load. For example, controller 150 may determine whether the center of fork 74 is substantially aligned with the center of the pallet bag located on the opposite side of the center longitudinal beam. In some embodiments, the predefined tolerance can be the position of the pallet relative to the fork 74 (e.g., X, Y, and Z coordinates; roll, pitch, and yaw).

If the pallet is not within a predetermined tolerance relative to fork 74, the position of fork 74 can be corrected in step 220. In some embodiments, correcting the fork position may include performing an adjustment vector at step 224, which adjusts the position of fork 74 relative to the detected pallet position. The adjustment vector may include at least one of moving MHV 60, moving fork assembly 70 relative to MHV 60, and moving fork 74 on fork assembly 70. After the adjustment vector has been performed, in step 228, load handling module 100 may control first camera 104 to re-identify the center of the pallet by recognizing the center, and then determine at step 232 whether the adjustment vector has moved fork 74 to a position engaging the pallet (e.g., aligned with the pallet bag). Step 232 may be performed using the same or different position detection process as used in step 212. In some embodiments, fork position correction step 220 may be performed multiple times (e.g., incrementally correcting the fork position), and steps 224, 228, and 232 may be repeated. However, some position correction steps may be performed only once without repeating any steps. In some embodiments, if the pallet position is completely outside the maximum adjustment vector range, the controller 150 may be configured to set an error code or send an error message. The load handling module 100 may be configured to use a fork position correction process that may have additional steps, or a process that omits at least one of steps 220, 224, 228 and/or 232.

When the pallet position is determined to be within a predefined tolerance, in step 236, the fork tip sensors 128 can be configured to verify the alignment of the forks 74 with the clearance between the pallet girders. For example, if the fork tip sensor 128 does not detect an object in its corresponding tip sensor FOV 132, the forks 74 are confirmed to be correctly aligned between the pallet girders. In step 240, the controller 150 can then instruct the forks 74 to traverse into the pallet bag formed between the pallet girders. In step 244, while the forks 74 are traversing into the pallet bag, the controller 150 can monitor the progress of the traversing forks 74. For example, the controller 150 can monitor at least one of the load placement sensors 142, the pallet center girders (e.g., using the first camera 104), and the fork tip sensors 128 to check whether the forks 74 have traversed correctly into the pallet bag. Additionally or alternatively, the controller 140 can monitor and/or measure the distance traveled by the forks 74.

In step 248, the load handling module 100 can check whether the travel of the fork 74 is complete. For example, the controller 150 can determine whether the load positioning sensor 142 has detected that the load is fully received on the fork 74. If the load positioning sensor 142 does not detect that the load is fully positioned, an error code can be set. If the load positioning sensor 142 detects that the load is fully received on the fork 74, then in step 252, the controller 150 can instruct the fork assembly 70 to be raised to a predefined lift-off height to engage and lift the load.

Once the fork assembly 70 has been raised to a predefined lift-off height, in step 256, the controller 150 may instruct the fork 74 to travel toward the vehicle body (i.e., return to the retracted position) to remove the load from the pick-up position. While the fork 74 returns to the retracted position, in step 260, the controller 150 may monitor the load positioning sensor 142 to ensure the load remains fully positioned on the fork 74. If the load positioning sensor 142 indicates that the load is not fully positioned while the fork 74 is moving, the controller 150 may set an error code (or send an error message). Once the fork 74 has traveled substantially or fully back to the retracted position, the load pick-up operation is complete. In some embodiments, the MHV 60 may travel to the desired location to unload the load.

Figure 9 illustrates a method 300 for unloading a load onto the ground using an MHV 60 with a load handling module 100 as described above with reference to Figures 1-5. Initially, at step 304, the load unloading coordinates (e.g., the X-Y position of the unloading location) can be wirelessly provided to the controller 150 from, for example, a warehouse management system (WMS). At step 308, once the controller 150 receives the load unloading coordinates, the MHV 60 can travel to the X-Y position of the unloading location and can approximately position the carried load in front of the destination bag (i.e., the forks 74 are height-aligned relative to the unloading location).

At step 312, with the load roughly positioned, controller 150 can access the fork-side object detection sensor (e.g., positioning the object detection sensor such that the field of view covers the space area occupied by the load on fork 74). At step 316, controller 150 can then determine whether an object (or obstruction) is within the field of view of the fork-side object detection sensor. If an object is detected in the field of view of the fork-side object detection sensor, controller 150 can set an error code or send an error message. At step 320, if no object is detected in the field of view of the fork-side object detection sensor, controller 150 can check at least one fork tip sensor 136 to confirm that the space in front of fork 74 is empty (i.e., no object is detected). At step 324, after determining that the space in front of fork 74 is empty, fork 74 can be controlled to travel to the load unloading position. At step 326, when fork 74 reaches the desired unloading position, fork 74 can stop and then lower. In some embodiments, using a weight sensor, the controller 150 can determine whether the weight on the fork 74 decreases as the fork 74 is lowered, in order to determine when the ground engages the load. Once the load is substantially or completely supported by the ground, the downward movement of the fork 74 can stop. Additionally or alternatively, the downward movement of the fork 74 can stop after a predetermined distance, or using any other sensor, process, or standard.

At step 330, once the forks 74 have been lowered, they can move away from the load and retract toward the vehicle body. At step 334, as the forks 74 return to the retracted position, the controller 150 can determine whether a load is being dragged by checking at least one load positioning sensor 142. For example, if the load positioning sensor 142 is off (i.e., the load is still fully positioned on the forks 74), the controller 150 can be configured to set an error code or send an error message. If the load positioning sensor 142 is on (i.e., indicating that the pallet/load is not fully positioned on the forks 74), the forks can continue to travel from the load to a predetermined distance.

At step 338, after the fork 74 has traveled a predetermined distance from the load, the first camera 104 can capture an image of the placed load, and the controller 150 can attempt to locate the load on the pallet/pallet in the image. At step 342, the controller 150 can then determine whether the predetermined distance traveled by the fork 74 matches the distance from the load backrest 78 to the load/pallet measured from one or more images from the first camera. If the distances do not match, the controller 150 can be configured to set an error code or send an error message. At step 346, if the distances do match, or are within predefined tolerances of each other, the controller 150 can determine whether the fork 74 has fully traveled (i.e., retracted). If the fork 74 has not fully traveled, the fork 74 can continue to retract. If the fork 74 has fully traveled, the unloading operation is complete, and the MHV 60 can proceed to perform another task.

Figure 10 illustrates a method 400 for unloading a load onto a rack or raised platform using an MHV 60 with a load handling module 100 as described above with reference to Figures 1-5. Similar to method 300 described above, at step 404, the load unloading coordinates (e.g., the X-Y position and height of the unloading location) can be wirelessly provided to the controller 150 from, for example, a WMS. Therefore, the MHV 60 can autonomously travel to the X-Y position of the unloading location. When (or as) the MHV 60 reaches the X-Y position of the unloading location, at step 408, the controller 150 can determine whether the destination height is at or above the current height of the second camera 108. In some embodiments, a height sensor can transmit the height of the fork 74 to the controller 150, and the height of the second camera 108 relative to the height of the fork 74 can be known. In some embodiments, step 408 can be performed when the MHV 60 travels to the unloading location or before the MHV 60 begins to travel.

If the destination height is not above or above the current height of the second camera 108 (i.e., the second camera 108 is above the destination height), then at step 412, the fork 74 can be adjusted to reach a predetermined distance above the unloading height (e.g., one foot above the unloading height). At step 416, the fork 74 can be further lowered from the predetermined distance above the unloading height to a predetermined offset value above the unloading height. When the fork 714 is lowered to the predetermined offset above the unloading height, the fork tip sensors 128 can indicate whether the position on the rack is vacant and whether it is capable of receiving a load. In some embodiments, at least one of the first camera 104, the second camera 108, and any other object detection system can be used to determine whether the unloading position on the rack is vacant.

If the destination height is equal to or higher than the current height of the second camera 108 (i.e., the second camera 108 is below the destination height), then at step 420, the second camera 108 can be positioned approximately in front of the unloading position on the rack by moving at least one of MHV 60 and fork assembly 70 relative to the rack. An image can then be acquired from the second camera 108, and at step 424, the acquired image can be used to determine rack alignment. For example, the controller 150 can use the second camera 108 to determine lateral alignment (i.e., left or right alignment) relative to the rack structure. Further, in some embodiments, the second camera 108 can be used to search for rack connections between upright beams and load beams. If a rack connection is found, the position of the rack connection relative to the second camera 108 can be calculated, and the position of at least one of the fork 74, fork assembly 702, and second camera 108 can be adjusted based on the position of the load connection.

At step 428, the second camera 108 can be used to identify features on the rack, such as, for example, horizontal load bars, upright beams, or adjacent loads (i.e., loads next to the desired unloading position on the rack). If the rack or its components cannot be located, the controller 150 can set an error code or send an error message. However, if the rack is identified, at step 432, one or more of the identified features can be used to quantify the minimum and maximum volumes of interest. For example, the minimum and/or maximum volumes of interest can be based on load dimensions, lift-off height, and desired buffer intervals between loads and between the load and rack uprights. In some embodiments, load dimensions (e.g., length, width, and height) can be predetermined values known to the controller 150. Additionally or alternatively, load dimensions can be provided to the controller 150 by the WMS and/or any other information source. Further, the controller 150 can compare the defined volumes of interest with the volume of unoccupied space at the drop-off position, which can be measured by the second camera 108.

At step 436, once the minimum and maximum volumes of interest have been quantized and it has been determined that the load can fit within them, the second camera 108 can verify that the search volume is empty (e.g., by analyzing pixels in the image used to indicate objects). If an object or another structure is detected (e.g., by the presence of a predetermined number of pixels indicating the intensity of the object/structure), the controller 150 can set an error code or send an error message. At step 440, if it is determined that the search volume is empty, the fork 74 can be raised to the final unloading height. At step 444, while the fork 74 is raised to the unloading height, the controller 150 can monitor the fork tip sensors 128 to identify the transition from a blocked state to an unblocked state. More specifically, the fork tip sensors 128 can be configured to detect when the load beam is in the tip sensor FOV 132, and when the fork 74 is above the load beam and the tip sensor FOV 132 is clear. The controller 150 can use the shift height value (i.e., the height value of the fork tip sensor 128 when it shifts from blocked to unblocked) to calculate a predefined offset value above the shift height. Once the fork 74 reaches the unblocking height, the fork 74 can be moved to the predefined offset above the shift height value.

At step 448, when fork 74 is positioned at a predefined offset above the transition height or unloading height, fork 74 may travel (i.e., extend) into the rack to place a load. While (or as) fork 74 travels into the rack, controller 150 may monitor at least one fork tip sensor 128 to ensure the fork does not enter an existing pallet. For example, controller 150 may determine if at least one center fork tip sensor (i.e., a center-positioned fork tip sensor specifically designed to detect the center longitudinal beam) is blocked. If the center fork tip sensor is blocked (which could indicate that fork 74 is entering another pallet), controller 150 may set an error code or send an error message. Additionally or alternatively, controller 150 may be configured to determine whether a pallet is being pushed. For example, the distance to the pallet may be measured and compared to the distance fork 74 has moved. In some embodiments, the same or different push detection methods may be implemented when the pallet is picked up.

At step 456, if fork 74 remains unobstructed and has traveled a predetermined distance within the frame, the extension of fork 74 can be stopped, and fork 74 can be lowered a predetermined distance. In some embodiments, the predefined distance may be based on at least one of the load weight (which may have been measured using a load weight sensor or provided to controller 150 by the WMS) and a predefined offset height above the load beam. For example, as fork 74 is lowered, controller 150 may determine whether the weight on fork 74 has decreased, or this may indicate that the load is supported by the frame. Additionally or alternatively, controller 150 may compare a preset lowering distance, which may be based on at least one of a transition height value and a predefined offset value, with the distance fork 74 has been lowered. At step 464, once it is determined that the weight on fork 74 has decreased and the preset lowering distance is met, the downward movement of fork 74 can be stopped, and at step 468, fork 74 may be retracted to a retracted position (i.e., away from the frame and toward the vehicle body). However, if the weight on the fork 74 does not decrease during descent and/or does not meet the preset descent distance, the controller 150 may set an error code or send an error message.

At step 472, when the forks are retracted, controller 150 may monitor at least one load position sensor to determine if the pallet has been dragged. For example, if load position sensor 142 remains closed after a predetermined time (providing a positive indication of pallet dragging), movement of fork 74 may be stopped, and controller 150 may set an error code or send an error message. If load position sensor 142 switches to the open state (e.g., indicating that the load is not in place on fork 74), retraction of fork 74 may continue.

The controller 150 can also be configured to monitor the distance the forks 74 have retracted. For example, the first camera 104 can be used to measure the distance between the load and the first camera 104 and/or the load backrest 78 to determine the distance the forks 74 have retracted. In some embodiments, the controller 150 can compare the distance traveled by the forks 74 (distance measured using another method or sensor) to determine if it matches the distance from the load backrest 78 to the load/pallet measured in the first camera image. Additionally or alternatively, the controller 150 can check at least one fork tip sensor 128 to determine whether the pallet has been unloaded by monitoring a portion of the fork tip sensor FOV between the forks 74 (e.g., monitoring the center fork tip sensor) to determine whether the center longitudinal beam on the pallet is still between the forks 74. At step 480, based on the monitored retraction distance and at least one of the presence/absence of the center longitudinal beam sensed by the center fork tip sensor, the controller 150 can determine whether the forks 74 have been fully retracted. If the fork 74 has not fully retracted, it may continue to retract, while the controller 150 may continue to monitor pallet drag, retraction distance, and the presence/absence of the center longitudinal beam. In some embodiments, at least one of steps 472, 476, and/or 480, as well as any other steps for checking pallet drag and/or measuring retraction distance, may be performed at least once.

At step 488, when fork 74 has fully retracted, controller 150 can perform secondary confirmation of load placement using first camera 104. At least one of first camera 104 and second camera 108 can capture an image of the load, which can be used to locate the pallet and/or load to determine the load position. At step 492, controller 150 can determine whether the measured load position matches the desired load unloading coordinates within a predefined tolerance. If the load position is within the predetermined tolerance, the load operation is complete, and MHV 60 can perform another task. If the load position is not within the predefined tolerance, controller 150 can set an error code or send an error message.

Figure 11 illustrates another load handling module 600 according to an embodiment of the present disclosure. The load handling module 600 may be similar in design and function to the load handling module 100 of Figures 1-5, having similar elements identified using similar reference numerals, unless described herein or clearly visible from the figures. For example, the first camera 704 is positioned such that its lower surface is positioned above the top surface 84 of the fork 74. In some embodiments, the first camera 704 may be at least 10, 15, or 20 cm above the top surface 84 of the fork 74.

Although various spatial and directional terms such as top, bottom, lower, middle, lateral, horizontal, vertical, front, etc., may be used to describe examples of this disclosure, it is understood that such terms are used only relative to the orientation shown in the accompanying drawings. Orientation may be reversed, rotated, or otherwise changed such that upper becomes lower, and vice versa, horizontal becomes vertical, etc.

The prior description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the invention is not intended to be limited to the embodiments described herein, but should be accorded the broadest scope consistent with the principles and novel features disclosed herein.

Finally, it is clearly expected that any process or step described herein can be combined, omitted, or reordered. In other embodiments, instructions may reside on a computer-readable medium, wherein these instructions are executed by a processor to implement one or more processes or steps described herein. Thus, it is expressly contemplated that any process or step described herein can be implemented as hardware, firmware, software including program instructions that execute on a computer, and any combination thereof. Therefore, this specification is given by way of example only and does not constitute a limitation on the scope of the invention.

Claims (19)

1. A load handling module for a material handling vehicle, the material handling vehicle including a vehicle body, a mast, a tower, and at least one fork extending from a load backrest to a tip, the tower extending upward from the load backrest, and the at least one fork being selectively movable relative to the vehicle body and configured to support a load, the load handling module comprising: A first camera is fixed to the load backrest and positioned near the top surface of at least one fork, and the first camera is configured to determine the position of an object in the field of view of the first camera. A second camera is fixed to the tower such that the second camera is positioned above the first camera, and the second camera is configured to determine the position of an object in the field of view of the second camera. A fork tip sensor, fixed to the at least one fork in proximal to the tip, configured to detect the presence of an object within the fork tip sensor's field of view extending in front of the tip; and A controller, which communicates with the first camera, the second camera, and the fork tip sensor, is configured to autonomously control the movement of the at least one fork; The tower includes a tower actuator configured to selectively move the second camera relative to the first camera. 2. The load handling module of claim 1, wherein when the load is supported on the at least one fork, the first camera field of view is configured to cover at least a portion of the bottom of the load. 3. The load handling module of claim 1, wherein when the load is supported on the at least one fork, the second camera field of view is configured to cover at least a portion of the top of the load. 4. The load handling module of claim 1, wherein the at least one fork comprises two forks extending from the load backrest to a corresponding tip, and wherein the first camera and the second camera are positioned between the two forks. 5. The load handling module of claim 4, wherein the fork tip sensor is a first fork tip sensor that is fixed to the tip of the first fork in the vicinity of the tip of the first fork of the two forks; The load handling module further includes a second fork tip sensor whose tip proximates to the second fork are fixed to the second fork; The first fork and the second fork are configured to detect the presence of an object between the first fork and the second fork. 6. The load handling module of claim 1, further comprising a load positioning sensor configured to detect whether the load is fully positioned on the at least one fork. 7. The load handling module of claim 1, further comprising a weight sensor configured to determine the weight of the load supported by the at least one fork. 8. A method for picking up a load using a material handling vehicle, the material handling vehicle comprising at least one fork extending from a load backrest, a mast, a tower extending upward from the load backrest, a first camera mounted on the load backrest, a second camera mounted on the tower, and a fork tip sensor disposed near the tip of the at least one fork, the method comprising: Move the material handling vehicle to the pickup location; Identify the pallets on which loads are placed; The at least one fork is aligned with the pallet by moving the at least one fork to align with the pallet bag of the pallet; Determine whether the at least one fork is aligned with the pallet bag; The forks extend relative to the vehicle body of the material handling vehicle to engage the pallet; and Raise and retract the fork to pick up the load using the fork. Identifying the pallet includes using at least one of the first camera and the second camera to identify the center longitudinal beam of the pallet, and determining whether the at least one fork is aligned with the pallet bag includes using at least one of the first camera and the second camera to determine whether the pallet is positioned within a predefined tolerance from the at least one fork, wherein the second camera is movable relative to the first camera. 9. The method of claim 8, further comprising correcting the position of the fork relative to the pallet bag by detecting the pallet position, and executing an adjustment vector that adjusts the position of the fork relative to the detected pallet position. 10. The method of claim 8, further comprising using the fork tip sensor to verify that the at least one fork is aligned with the pallet bag. 11. The method of claim 8, wherein extending the fork to engage the pallet further comprises determining, using a load positioning sensor, that the at least one fork is fully engaged with the load. 12. A method for unloading a load supported by a material handling vehicle, the material handling vehicle comprising at least one fork extending from a load backrest, a mast, a tower extending upward from the load backrest, a first camera mounted on the load backrest, a second camera mounted on the tower, and a fork tip sensor disposed near the tip of the at least one fork, the method comprising: Move the material handling vehicle to the unloading position wirelessly provided to the controller; Whether the unloading position is vacant is determined by searching for the presence of an object at the unloading position using at least one of the second camera and the fork tip sensor; The forks extend relative to the vehicle body of the material handling vehicle to move the load to the unloading position; Lower the fork to place the load in the unloaded position; Retract the fork to disengage from the load; and The location of the load is verified to be within a predefined tolerance of the unloading position by measuring the distance between the load and the first camera and the second camera by using at least one of the first camera and the second camera, and the second camera can move relative to the first camera. 13. The method of claim 12, wherein determining whether the unloading location is free further comprises analyzing pixels in the image used to indicate the object. 14. The method of claim 12, wherein determining whether the unloading position is free further comprises identifying at least one of the horizontal load bars, vertical beams, and adjacent loads of the rack structure. 15. The method of claim 14, wherein determining whether the unloading location is free further comprises quantizing at least one of the minimum volume of interest or the maximum volume of interest. 16. The method of claim 12, further comprising raising the fork to a dismount height, wherein the fork tip sensor quantifies a predefined offset height associated with a horizontal load bar of the dismount frame structure, and wherein the fork is raised by the predefined offset height by a distance. 17. The method of claim 16, wherein lowering the fork to place the load in the unloading position comprises lowering the fork by a distance at least the predefined offset height. 18. The method of claim 12, wherein verifying the location of the load comprises capturing an image of the placed load with the first camera and measuring the distance between the load and the first camera. 19. The method of claim 12, wherein the material handling vehicle further includes a weight sensor configured to determine the weight of the load supported by the at least one fork, and wherein lowering the fork to place the load in the unloading position further includes using the weight sensor to determine whether the weight of the load supported by the at least one fork has decreased.