JP7698982B2 - ANGLE DETECTION METHOD AND ANGLE DETECTION SYSTEM - Google Patents

Description

The present invention relates to an angle detection method and an angle detection system for detecting the angle of a substantially rectangular parallelepiped object placed on a horizontal surface.

For example, Patent Document 1 describes a technology that acquires a point cloud that indicates the three-dimensional shape of an object and determines the position and orientation of the object based on the point cloud.

JP 2011-174891 A

However, conventional technology has the drawback that noise based on the surface shape of the object affects the accuracy of detecting the angle of the object.

The present invention has been made in consideration of the above problems, and provides an angle detection method and angle detection system that detects the angle of an object placed on a horizontal surface with high accuracy.

In order to achieve the above object, one aspect of the present invention is an angle detection method that uses a detector that acquires the distance to an object placed on a horizontal surface as multiple three-dimensional point data, and detects the angle of a detection surface portion of a substantially rectangular parallelepiped object, including the surface facing the detector, relative to a predetermined vertical plane, and includes a point cloud acquisition process that acquires point data of multiple locations on the detection surface portion as a point cloud, a trimming process that extracts a point cloud within a predetermined height range from the point cloud acquired by the point cloud acquisition process, and an angle detection process that detects the angle of the detection surface portion based on the point cloud extracted by the trimming process.

To achieve the above object, another aspect of the present invention is an angle detection system that includes a detector that acquires a distance to an object as multiple three-dimensional point data, a rack for storing multiple roughly rectangular parallelepiped objects on a horizontal surface, and an angle detection device that detects the angle of a detection surface portion including the surface of the object facing the detector with respect to a predetermined vertical plane, and the angle detection device includes a point cloud acquisition unit that acquires point data of multiple locations on the detection surface portion as a point cloud, a trimming unit that extracts a point cloud within a predetermined height range from the point cloud acquired by the point cloud acquisition unit, and an angle detection unit that detects the angle of the detection surface portion based on the point cloud extracted by the trimming unit.

The present invention makes it possible to detect the angle of an object with high accuracy by suppressing the effects of noise caused by the shape of the object.

FIG. 2 is a perspective view showing an angle detection system. FIG. 2 is a perspective view showing the vicinity of a transfer device of the angle detection system. FIG. 2 is a block diagram showing a functional configuration of the angle detection system. FIG. 13 is a diagram showing the result of trimming. 13 is a flowchart showing a process flow of an angle detection unit. 11A to 11C are diagrams illustrating the processing state at each stage of the process of the angle detection unit. FIG. 13 is a diagram showing another example of a projected image. FIG. 13 is a diagram showing a fitting state of an L-shaped template.

Below, an embodiment of an angle detection method and an angle detection system according to the present invention will be described with reference to the drawings. Note that the following embodiment is an example to explain the present invention, and is not intended to limit the present invention. For example, the shapes, structures, materials, components, relative positional relationships, connection states, numerical values, mathematical expressions, the contents of each step in the method, and the order of each step shown in the following embodiment are examples, and may include contents not described below. In addition, geometric expressions such as parallel and orthogonal may be used, but these expressions do not indicate mathematical strictness, and include errors, deviations, etc. that are substantially acceptable. In addition, expressions such as simultaneous and identical also include a substantially acceptable range.

The drawings are schematic diagrams in which emphasis, omissions, or proportions have been appropriately adjusted in order to explain the present invention, and may differ from the actual shapes, positional relationships, and proportions.

In addition, multiple inventions may be collectively described below as a single embodiment. Some of the content described below is also described as an optional component of the present invention.

Figure 1 is a perspective view showing an angle detection system. Figure 2 is a perspective view showing the vicinity of a transfer device of the angle detection system. The angle detection system 100 is a system that detects an angle indicating the posture of an item 200 arranged on a rack 110 in a horizontal plane, and includes a detector 140 and an angle detection device 150 (not shown in Figures 1 and 2). In this embodiment, the angle detection system 100 is a device that can automatically transport the delivered item 200, automatically transfer the item 200 to a storage position for the item 200, and also transport the item 200 that has been automatically transferred from the storage position, and is incorporated in a so-called automated warehouse that includes a transport device 120 and a transfer device 130.

The object 200 to be transferred by the angle detection system 100 is not particularly limited as long as it is approximately rectangular. Approximately rectangular includes shapes formed by six flat rectangular faces, and is described as including shapes with rib-like or flange-like protrusions, handle-like depressions or holes, etc. Specific examples of the object 200 include objects with flat surfaces that do not have uneven surfaces, such as cardboard boxes, paper boxes, and wooden boxes, and objects with uneven surfaces, such as containers, trays, and foldable containers.

The rack 110 is a facility for storing a plurality of roughly rectangular parallelepiped objects 200 arranged so that the detection surface 201 of the object 200 facing the area in which the conveying device 120 moves is aligned along a predetermined arrangement direction (X-axis direction in the figure). In the present embodiment, the rack 110 includes a shelf 111 on which the object 200 is placed on a horizontal surface, and a support 112 that supports the shelf 111. The shelf 111 is flat, and the position in which the object 200 is stored is not particularly limited. The rack 110 may store only the object 200 of the same shape, or may store a mixture of objects 200 of multiple shapes. Note that the rack 110 is shown on one side as viewed in the direction of movement of the conveying device 120 in the figure, but the rack 110 may be arranged on both sides.

The conveying device 120 is a device that holds and conveys the item 200, and is not particularly limited as long as the detector 140 can be attached to it. Examples of the conveying device 120 include an unmanned guided vehicle that holds the item 200 and travels autonomously on a floor surface, and a tracked cart that holds the item 200 and travels along a predetermined track such as a rail. In the case of this embodiment, the conveying device 120 is a so-called stacker crane that includes a rail 121, a cart 122 that travels on the rail 121, a mast 123 that is attached upright to the cart 122 and moves together with the cart 122, and a lifting platform 124 that can hold the item 200 and moves up and down along the mast 123.

The transfer device 130 is a device that transfers the item 200 between the rack 110 and the lifting platform 124 of the transport device 120, and transfers the item 200 by moving it in the depth direction (Y-axis direction in the figure) perpendicular to the arrangement direction of the items 200 (X-axis direction in the figure) in a horizontal plane. The type of transfer device 130 is not particularly limited, and examples include a device that transfers the item 200 by clamping both opposing sides of the item 200, a device that hooks a claw on the rear side or front side of the item 200 and transfers the item 200 while sliding it, and a device that scoops up the item 200 and transfers it.

In this embodiment, the transfer device 130 is attached to the lifting platform 124 of the transport device 120, and is capable of transferring the item 200 between the rack 110 and the lifting platform 124. When the racks 110 are disposed on both sides of the transport device 120, the transfer device 130 is configured to be able to transfer the item 200 to either of the racks 110 on either side.

The detector 140 is a sensor that acquires the distance between the detector 140 and multiple points on the detection surface portion 201 of the article 200, including the surface facing the detector 140, as multiple three-dimensional point data. The type of the detector 140 is not particularly limited, but examples include three-dimensional distance measuring sensors such as a LiDAR (Laser Imaging Detection and Ranging) sensor and a TOF (Time of Flight) camera.

The location where the detector 140 is attached is not particularly limited as long as it is a part of the conveying device 120, but it is preferable to attach it near the location where the item 200 is transferred, because the relative positional relationship between the transfer position and the item 200 can be accurately detected. In the present embodiment, the detector 140 is attached to the lifting platform 124 of the conveying device 120. The number of detectors 140 provided in the angle detection system 100 is not particularly limited, but in the present embodiment, two detectors 140 are provided lined up in the arrangement direction of the items 200. The distance between the two detectors 140 is set to include a range of distances that can acquire point data of a predetermined density or more and can detect at once the expected maximum width of the item 200 stored in the rack 110 in the arrangement direction and the area covering the gap between the adjacent items 200 set on both sides. In the present embodiment, the detector 140 is attached to each side of the lifting platform 124 in the width direction (X-axis direction in the figure). In other words, the detectors 140 are disposed on both sides of the area through which the items 200 transferred by the transfer device 130 pass. This makes it possible to accurately detect the positional relationship between the area through which the items 200 are transferred and the items 200 stored in the racks 110 nearby.

Figure 3 is a block diagram showing the functional configuration of the angle detection system. The angle detection device 150 is a device that detects the angle of the detection surface portion 201, which includes the surface of the article 200 facing the detector 140, with respect to a predetermined vertical plane, and includes a point cloud acquisition unit 151, a trimming unit 171, and an angle detection unit 172 as processing units that are realized by causing a processor to execute a program.

The point cloud acquisition unit 151 acquires point data of multiple locations on the detection surface 201 of the item 200 from the detector 140 as a point cloud. The data structure of the point data is not particularly limited, but includes, for example, three-dimensional data indicating a relative positional relationship with the transfer device 130. In the case of this embodiment, the data includes data of an orthogonal coordinate system including data on the arrangement direction (X-axis direction in the figure) of the items 200 on the shelf 111 of the rack 110, the vertical direction (Z-axis direction in the figure) perpendicular to the arrangement direction on the detection surface 201, and the depth direction (Y-axis direction in the figure) perpendicular to both the arrangement direction and the vertical direction.

In this embodiment, the angle detection system 100 includes multiple detectors 140 arranged in the array direction, and the point cloud acquisition unit 151 acquires a point cloud from each of the detectors 140. The point cloud acquisition unit 151 may statistically process point data included in overlapping areas of the point clouds acquired from the multiple detectors 140, and combine the point clouds of the multiple detectors 140 into a single image. This makes it possible to process point data related to the detection surface portions 201 of multiple items 200 in the array direction as a single image. It is also possible to use images corresponding to the point clouds of the multiple detectors 140 to obtain processing results and combine the results, rather than combining the point clouds of the multiple detectors 140 into a single image.

Figure 4 is a diagram showing the result of trimming. The trimming unit 171 executes a trimming process to extract a point cloud within a predetermined height range based on the plane on which the item 200 is placed from the point cloud acquired by the point cloud acquisition unit 151. Specifically, the trimming unit 171 extracts point data included within a predetermined height range from the point cloud acquired from the point cloud acquisition unit 151. The predetermined height range is not particularly limited, but can be, for example, a strip-shaped area having a predetermined height that includes the center position of the minimum height (vertical length) of the item 200 stored in the rack 110. Specifically, point data having a value in the vertical direction (Z-axis direction in the figure) less than the first threshold value is excluded, and point data having a value in the vertical direction (Z-axis direction in the figure) greater than the second threshold value (>first threshold value) is excluded. In other words, point data in the range of the first threshold value or more and the second threshold value or less is extracted. Because the trimming process detects the no-item section using data from a specified height range, it is possible to suppress noise caused by reflections from the shelves 111 of the rack 110 and noise caused by ribs, flanges, holes, etc. provided on the surface of the item 200, and provide accurate point data regarding the angle of the detection surface 201. In addition, it is possible to reduce the amount of point cloud data and expedite the processing of the next step.

The angle detection unit 172 detects the angle of the detection surface 201 based on the point group extracted by the trimming unit 171. Figure 5 is a flowchart showing the flow of processing by the angle detection unit. Figure 6 is a diagram showing the processing state at each stage of the process by the angle detection unit.

The angle detection unit 172 projects the point cloud extracted in the trimming step (S101) by the trimming unit 171 in the vertical direction to generate a two-dimensional projected image shown in FIG. 6(a) (S102, projection step). Specifically, the two-dimensional projected image is generated by excluding data in the vertical direction (Z-axis direction in the figure) from each point data.

Next, the angle detection unit 172 performs morphology processing on the projected image, and changes the data so that the points that existed as rough points are filled in and the part corresponding to the detection of the item 200 as shown in FIG. 6(b) becomes a single mass (the white part in the image in FIG. 6) (S103, morphology step). In this embodiment, the angle detection unit 172 performs a closing process that repeats expansion and contraction as the morphology processing.

Next, the angle detection unit 172 removes noise from the point data that has been subjected to the morphology processing (S104, noise removal step). In the case of this embodiment, the angle detection unit 172 performs a labeling process to distinguish between groups of point data as shown by the white arrows 0-9 in FIG. 6(c). Then, a group of point data that is equal to or shorter than a predetermined length threshold is removed as being noise. This results in an image as shown in FIG. 6(d).

Next, as shown by the vertical lines at the ends of the arrows in FIG. 6(d), the angle detection unit 172 removes each end of the labeled block of point data (S105, cutting process).

Next, the angle detection unit 172 performs straight line fitting on each block of point data with the ends cut off, as shown in FIG. 6(e) (S106, fitting process). As a result, it is possible to exclude sequences of point data that change significantly and perform straight line fitting on a stable block of point data, making it possible to fit a straight line that corresponds to the angle (tilt) of the detection surface portion 201 of the actual item 200.

Finally, as shown in FIG. 6(f), the angles (θ1, θ2, θ3, θ4) of the straight lines determined by the fitting process with respect to the predetermined vertical plane are detected (S107, angle detection process). The predetermined vertical plane is not particularly limited, but may be, for example, a plane parallel to the frontage of the portion of the rack 110 where the items 200 are stored. Also, a plane parallel to the imaging plane of the detector 140 may be the predetermined vertical plane.

According to the angle detection system 100 of the above embodiment, a point cloud within a predetermined height range from the horizontal plane on which the article 200 is placed is extracted based on the point cloud detected by the detector 140, and the angle of the detection surface 201 relative to a predetermined vertical plane is detected based on the extracted point cloud, thereby suppressing noise caused by the uneven shape present on the detection surface 201. Therefore, the angle of the surface of the article can be detected with high accuracy.

In addition, by performing straight line fitting on a projected image in which the three-dimensional point data obtained from the detector 140 is converted into two-dimensional data projected in the vertical direction, it is possible to accurately detect the angle indicating the posture of the item 200 in the horizontal plane.

In addition, by performing morphological processing, noise processing, etc. on the projected image, the range in the projected image where the cluster of point data corresponding to the item 200 exists can be properly visualized, making it possible to perform straight line fitting with high accuracy.

The present invention is not limited to the above-described embodiment. For example, the components described in this specification may be combined in any way, or some of the components may be removed to create another embodiment of the present invention. The present invention also includes modifications that are made to the above-described embodiment by those skilled in the art without departing from the spirit of the present invention, i.e., the meaning of the words in the claims.

For example, in the angle detection process, a straight line was fitted to the block of point data, but an L-shaped template may be fitted to detect the angle of the detection surface. Specifically, in the trimming process (S101) of the above embodiment, a trimming process is performed to leave the point cloud in the depth direction, and after processing the point cloud up to the noise removal process (S104), in the fitting process (S106), as shown in FIG. 7, a corner of the block of point data (part surrounded by a circle in FIG. 7) that appears in the projected image is matched with a corner of the L-shaped template, and as shown in FIG. 8, the L-shaped template (black line in FIG. 8) is rotated relatively around the matched corner at a predetermined angle (for example, 1°) to fit each of the block of point data (white in FIG. 8), and the angle of the L-shaped template with the highest matching score may be detected as the angle of the detection surface 201.

Although the example shows a case where multiple detectors 140 are provided, a single detector 140 may be used. The angle of view of one detector 140 may be such that it can simultaneously detect an area that is the width of the central item 200 among three items 200 lined up consecutively on the shelf 111 of the rack 110, plus the width of the item-free sections on both sides.

In addition, although the above example shows a conveying device 120 that can freely convey the item 200 in a vertical plane, the conveying device 120 may be a tracked or trackless conveying cart that can convey the item 200 in a horizontal plane.

Although the rack 110 is exemplified as being capable of storing items 200 two-dimensionally in the horizontal and vertical directions, the rack 110 may be capable of storing items 200 one-dimensionally along the direction in which the transport cart travels.

The present invention can be applied to automated warehouses, logistics centers, factory facilities, and robots that stack boxes and containers, where items are transported by a transport device and transferred between racks and the transport device.

Reference Signs List 100 Angle detection system 110 Rack 111 Shelf 112 Support 120 Conveyor device 121 Rail 122 Cart 123 Mast 124 Lifting platform 130 Transfer device 140 Detector 151 Point cloud acquisition unit 170 Angle detection device 171 Trimming unit 172 Angle detection unit 200 Article 201 Detection surface unit

Claims (5)

1. An angle detection method for detecting an angle of a detection surface portion of a substantially rectangular parallelepiped object, the detection surface portion including a surface facing the detector, with respect to a predetermined vertical plane, using a detector that acquires a distance to an object placed on a horizontal plane as three-dimensional point data, the method comprising:
a point cloud acquisition step of acquiring point data of a plurality of points on the detection surface portion as a point cloud;
a trimming step of extracting a point cloud within a predetermined height range from the point cloud acquired by the point cloud acquisition step;
an angle detection step of projecting the point cloud extracted by the trimming step in a vertical direction to generate a projection image, and detecting the inclination of a straight line obtained by fitting to point data appearing in the projection image as the angle of the detection surface ;
The angle detection method includes: In the angle detection step,
removing noise from the projection image and then performing straight line fitting;
The angle detection method according to claim 1 . In the angle detection step,
removing the ends of the cluster of point data appearing in the projection image and then performing straight line fitting;
The angle detection method according to claim 1 or 2 . In the angle detection step,
the fitting is performed using an L-shaped template, and an inclination of a straight line portion of the fitted L-shaped template is detected as an angle of the detection surface portion;
The angle detection method according to claim 1 . A detector that acquires a distance to an article as three-dimensional multiple point data;
A rack for storing a plurality of substantially rectangular parallelepiped-shaped articles on a horizontal surface;
an angle detection device that detects an angle of a detection surface portion of the article, the detection surface portion including a surface facing the detector, with respect to a predetermined vertical plane;
The angle detection device is
a point cloud acquisition unit that acquires point data of a plurality of points on the detection surface as a point cloud;
a trimming unit that extracts a point cloud within a predetermined height range from the point cloud acquired by the point cloud acquisition unit;
an angle detection unit that projects the point cloud extracted by the trimming step in a vertical direction to generate a projection image, and detects the inclination of a straight line obtained by fitting to point data appearing in the projection image as the angle of the detection surface ;
An angle detection system comprising:

Images