JP3789218B2 - Automatic operation construction machine and operation method thereof - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an automatic driving construction machine, and more particularly, to an automatic driving construction machine that performs automatic loading work on work machines such as dump trucks and crashers.
[0002]
[Prior art]
Normally, loading work on a dump truck or crusher using a construction machine such as a hydraulic excavator is performed at a crushed stone site by excavating debris from a certain location and releasing it to a certain location such as a dump truck vessel or crusher hopper. It is a monotonous repetitive task of doing. Such a monotonous repetitive operation is painful for the operator, and there is an increasing demand for automatic operation without human intervention.
[0003]
Conventionally, as one of the methods for automating such loading work, the excavation position and the earthing position are taught, and the swivel work is mainly repeatedly performed between these positions without running the hydraulic excavator. It was.
[0004]
Japanese Patent Publication No. Sho 54-7121 discloses a method of automation using a hydraulic excavator. This prior art stores operation state quantities such as an angle and a position obtained by a teaching operation, and performs a reproduction operation. Sometimes the hydraulic excavator is regenerated using the stored value described above.
[0005]
[Problems to be solved by the invention]
However, the above-described conventional method is a self-contained system of a type that does not take into consideration a change in the positional relationship between a hydraulic excavator that performs automatic operation and a work machine such as a dump truck during a regeneration operation.
[0006]
Therefore, when loading debris mined by a hydraulic excavator on a dump truck, etc., it is released at the same position taught each time, so the debris is loaded only at the same location high and proper loading is not possible, Even if the dump truck is not always arranged at the same position, and even if the dump truck is arranged at an appropriate position, if the excavator itself is displaced while repeating the regeneration operation, There was a problem that the dump truck could not be dumped at the proper place to be dumped.
[0007]
In view of the above problems, an object of the present invention is to provide an automatic driving construction machine provided with an imaging means, and capable of calculating an appropriate earthing position to a working machine such as a dump truck from the imaging data. To provide construction machinery.
[0008]
[Means for Solving the Problems]
The present invention employs the following means in order to solve the above problems.
[0009]
In the automatic operation construction machine that repeatedly operates the revolving operation from the excavation to the earthing to be taught, the automatic operation construction machine includes an imaging unit that images the work machine that receives the earthing , and the imaged Based on the imaging data, the coordinates of the feature point of the work machine in the camera coordinate system, which is a coordinate system based on the imaging means, are calculated , and the coordinate of the feature point is automatically controlled by the automatic operation construction machine. and a Hodo position data calculating means for calculating the data of the appropriate Hodo position to the working machine is converted into a control coordinate system of the coordinate is serving as a reference coordinate system for, during playback operation, the calculated Hodo position based on the Hodo position data, characterized in that it is taught.
[0010]
The automatic operation construction machine according to claim 1, wherein the release position data calculation means includes a plurality of release positions for dividing an appropriate release position for the work machine into a plurality of regions based on the captured image data. calculating a soil location data, the reproducing operation, a plurality of Hodo position based on a plurality of Hodo position data the calculated, characterized in that it is sequentially taught.
[0011]
Further, in an operation method of an automatic operation construction machine that repeatedly performs a one-round operation from excavation to earth release taught by regenerating operation , a work machine that receives the earth release at a predetermined work position of the automatic operation construction machine is provided. a step of imaging, and calculates the coordinates of the feature points of the working machine in the camera coordinate system which is an imaging means coordinate system based on the by Hodo position data calculating means on the basis of the imaging data obtained by the imaging, the The coordinates of the feature points are converted into the coordinates of the control coordinate system , which is a reference coordinate system for automatically controlling the automatic operation construction machine, and the appropriate earthing position for the work machine is divided into a plurality of areas. The method includes a step of calculating a plurality of earthing position data, and a step of sequentially discharging to a plurality of earthing positions based on the calculated plurality of earthing position data .
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to FIGS.
[0013]
FIG. 1 is a diagram showing a working mode of a hydraulic excavator (hereinafter referred to as an automatic driving excavator) that performs automatic driving according to the present embodiment.
[0014]
In the figure, 1 is an automatic driving excavator that excavates stored debris 3 to be described later and discharges it to a dump truck 2 to be described later, and 2 is provided with a vessel 14 on which debris is loaded and is discharged from the automatic operation shovel 1. A dump truck 3 loads and transports debris, 3 is debris stored in a debris reservoir, and 15 is a remote control device that is remotely located from the automatic excavator 1 and remotely operates the automatic excavator 1.
[0015]
The automatic driving excavator 1 includes a traveling body 4, a revolving body 5 provided on the traveling body 4 so as to be able to turn, a boom 6 provided on the revolving body 5 so as to be able to be lifted and lowered, and a turnable to the tip of the boom 6. The arm 7 provided, the bucket 8 rotatably provided at the tip of the arm 7, the angle sensor 9 for detecting the elevation angle between the swing body 5 and the boom 6, and the rotation angle between the boom 6 and the arm 7 The angle sensor 10 for detecting the angle, the angle sensor 11 for detecting the rotation angle between the arm 7 and the bucket 8, the cab 12 provided on the revolving body 5, the roof 12 such as the cab 12, and the dump truck 2 And a video camera 13 provided for imaging the loading location.
[0016]
The installation position of the video camera 13 is preferably a position where the vessel 14 that is the target for earthing can be seen at all times.
[0017]
FIG. 2 is a block diagram showing a control mechanism at the time of regeneration operation of the automatic operation construction machine according to the present embodiment. In this figure, the same reference numerals as those shown in FIG. 1 are the same as those shown in FIG. Indicates.
[0018]
In the figure, reference numeral 17 denotes an apparatus mounted on the automatic driving excavator 1, 171 is an automatic driving controller mainly composed of a computer, and 172 is driven by a driving signal output from a servo control unit 1717 described later. An electromagnetic control valve 173 for controlling the amount of oil flowing into the actuator or the hydraulic pressure, 173 is a detector for detecting the operation amount of the actuator.
[0019]
The automatic operation controller 171 compares the current position calculation unit 1711 that calculates the operation position of the automatic operation shovel based on the detection signal from the detector 173, the feedback value from the current position calculation unit 1711 and the input value, and Servo control unit 1712 that outputs a drive signal for correcting the deviation to electromagnetic control valve 172, teaching position storage unit 1713 that is taught at the time of teaching or reproducing operation and stores teaching position data, and is operated automatically at the time of teaching. A teaching command storage unit 1720 that stores the teaching command of the excavator 1, a teaching position output processing unit 1714 that outputs teaching position data stored in the teaching position storage unit 1713, and a teaching that is output from the teaching position output processing unit 1714 Servo pre-processing unit 1715 that interpolates between position data, and teaching command from teaching command storage unit 1720 A command interpreter 1716 that outputs the command speed signal for sequentially reading and outputting the teaching position data to the teaching position output processing unit 1714 and outputting the moving speed value between the teaching points, and an image captured by the video camera 13 are output. An image processing unit 1717 that calculates the coordinates of the feature points of the image based on this, and an image capture that stores the position of the automatic driving excavator 1 that is considered appropriate for the video camera 13 to capture the vessel 14. A position storage unit 1718 and an earthing position data calculation unit 1719 that calculates the earthing position from the coordinates of the calculated feature points and outputs the earthing position data to the teaching position storage unit 1713 are configured.
[0020]
As described above, among the teaching position data stored in the teaching position storage unit 1713 such as the earthing position, the excavation position, and the turning direction of the revolving structure 5, the earthing position data is taught by other teaching position data. It is stored in advance during operation, whereas it is stored as release position data every time it is calculated by the release position data calculation unit 1719 during a reproduction operation. Note that processing in the image processing unit 1717 and the earthing position data calculation unit 1719 will be described in detail with reference to FIG.
[0021]
The remote operation device 15 includes a teaching operation unit 151 that stores teaching position data in the teaching position storage unit 1713 during teaching and stores teaching commands in the teaching command storage unit 1720 during playback operation, and a command interpreter unit during playback operation. And a playback operation unit 152 that activates 1716.
[0022]
Next, the operation of the automatic operation construction machine according to the present embodiment will be described with reference to FIGS.
[0023]
First, as shown in FIG. 1, after excavating the stored debris 3 by the bucket 8 of the automatic operation shovel 1, the bucket 8 is moved to the position of the vessel 14 of the dump truck 2, and a predetermined amount of the vessel 14 is determined. A case will be described in which the recirculation operation is performed until the soil is discharged at the position and the bucket 8 is returned to the storage location of the debris 3 again.
[0024]
When the playback operation unit 152 is operated to start playback, a playback start signal is transmitted from the remote operation device 15 to the automatic driving excavator 1 by wireless communication means (not shown) and input to the command interpreter unit 1716. When the command interpreter 1716 receives the reproduction start signal, the command interpreter 1716 sequentially reads out the commands stored in the teaching command storage 1720, and causes the teaching position output processing unit 1714 to output teaching position data from the teaching position storage 1713. The teaching position data is transferred to the servo preprocessing unit 1715, and interpolation calculation is performed so that each front joint of the automatic driving shovel 1 operates at a target speed given from the command interpreter unit 1716. Outputs the angle value. The servo control unit 1712 performs feedback control based on the current position calculated by the current position calculation unit 1711 and outputs a drive signal to the electromagnetic control valve 172 . Thereby, the operation taught in advance is reproduced.
[0025]
Next, a processing procedure in the image processing unit 1717 and the earthing position data calculation unit 1719 shown in FIG. 2 will be described using the flowchart shown in FIG.
[0026]
First, processing in the image processing unit 1717 will be described. In step 101, the automatic driving excavator 1 is set to the image capture position by comparing the current position data input from the current position calculation unit 1711 with the image capture position data stored in the image capture position storage unit 1718. It is determined whether or not the image capture position has been reached, and when the image capture position is reached, an image is captured from the video camera 13 in step 102. Next, in step 103, the captured image signal is denoised by a filter, and in step 104, an edge is extracted using an edge extraction operator. Next, a straight line is extracted from the edge extracted in step 105 using a technique such as Hough transform. Next, at step 106, end points and intersections of line segments in the image are detected from the straight line extracted at step 105 and the edge extracted at step 104. Next, in step 107, the shape of the object is recognized from the object registered in advance, here, the model of the vessel 14 of the dump truck 2 and the line segment information obtained in step 106. If the shape can be recognized, the position information of the feature points on the vessel 14 is obtained in step 108.
[0027]
Here, FIG. 4 shows an example of the shape of the vessel 14 recognized by the image processing unit 1717 and the positions of the feature points a, b, c, and d. Note that the image processing from step 102 to step 108 described above can be realized in whole or in part by hardware, and conversely, in whole or in part, can also be realized by software. .
[0028]
Next, processing in the earthing position data calculation unit 1719 will be described. First, in step 109, performs the coordinate calculation of feature points in the camera coordinate system from the coordinates of the feature points a~d obtained in step 108.
[0029]
In this calculation, as shown in FIG. 5, generally, at least the distance between the three feature points of the object such as the vessel 14 is known, and these points in the screen imaged by the video camera 13 are known. If the positions a to d are known, the position of each point in the space can be obtained by a known calculation method described later. Here, Xo, Yo, Zo shown in 18 is a target coordinate system set for a target such as the vessel 14, and this coordinate system may take anywhere in the space, but here, among the feature points a to d One point a is set as the origin of the target coordinate system. Xc, Yc, Zc shown in 19 is a camera coordinate system with the lens center of the camera 13 as the origin. Xw, Yw, and Zw shown in 20 are control coordinate systems that serve as a reference for automatically controlling the automatic driving excavator 1. As a known calculation method, Japanese Patent Application Laid-Open No. Sho 62-54108, IEICE Transactions D-1 Vol. J72-D-INo. 9pp-1441-1447 is known.
[0030]
Next, in step 110 of FIG. 3, coordinate conversion is performed in order to convert the coordinates of the feature points obtained in the camera coordinate system calculated in step 109 into the coordinates of the control coordinate system.
[0031]
This coordinate transformation can be calculated when the video camera 13 is installed in the automatic driving excavator 1.
[0032]
Next, in step 111, from the coordinates of the feature points converted into the control coordinate system in step 110, the unloading position data to be unloaded on the vessel 14 by the automatic driving shovel 1 is calculated by the method described later.
[0033]
Here, a method (area division method 1) for calculating the earthing position data from the calculated coordinates of the feature points a to d will be described with reference to FIG.
[0034]
In the figure, an area divided by a diagonal line ac and a diagonal line bd of a quadrangle abcd is a loading area. Next, a triangle amd, a triangle bma, a triangle cmb, and a triangle dmc when the intersection of the diagonal line ac and the diagonal line bd is m are defined as a region f11, a region f12, a region f13, and a region f14, respectively. Next, the center position of each area | region f11-f14 is calculated, and let the calculated | required center position be an earth release position.
[0035]
FIG. 7 shows another example of the method (area division method 2) for calculating the earthing position data from the calculated coordinates of the feature points a to d.
[0036]
In the figure, the midpoint m1 of the side ad, the midpoint m2 of the side ab, the midpoint m3 of the side bc, and the midpoint m4 of the side cd are detected from the feature points a to d. Next, let m5 be the intersection of the straight line m1m3 and the straight line m2m4. Here, the square am2m5m1 is the area f21, the square bm3m5m2 is the area f22, the square cm4m5m3 is the area f23, and the square dm1m5m4 is the area f24. Next, in the region f21, a position away from the straight line m1m3 by the distance d1 and a distance d2 from the straight line m2m4 is set as the earthing position. Similarly, in the region f22, the position d3 away from the straight line m1m3 and the distance d4 from the straight line m2m4 is the earthing position, in the region f23 the distance d5 from the straight line m1m3, the position d6 from the straight line m2m4 is the earthing position, and in the region f24 A position that is a distance d7 from the straight line m1m3 and a distance d8 from the straight line m2m4 is defined as a discharge position.
[0037]
Next, in step 112 of FIG. 3, the plurality of earthing position data obtained by each of the above methods are sequentially input into the region f11 (f21), the region f12 (f22), the region f13 (f23), and the region f14 (f24). In order to release the earth, it is sequentially stored in the teaching position storage unit 1713 for each round of the reproduction operation.
[0038]
In each of the above methods, four regions are set, but how many regions can be set can be arbitrarily set according to various conditions.
[0039]
【The invention's effect】
As described above, according to one aspect of the present invention, an image of a work machine such as a dump truck or a crusher discharged from an automatic operation construction machine is picked up, and the discharging position data calculation unit is based on the picked-up imaging data. calculating the coordinates of the feature points of the working machine in the camera coordinate system. Thus to calculate the Hodo position data to the working machine by converting the coordinates of the feature point in the control system, appropriate to the work machine It can be dumped at appropriate locations to enable proper loading.
Further, according to another invention of the present invention, an image of a work machine such as a dump truck or a crusher released from an automatic operation construction machine is imaged, and the earthing position data is extracted based on the imaged imaging data. Since the calculation means calculates from a plurality of earthing position data divided into a plurality of areas by the calculation means, during the reproduction operation, the earthing is sequentially performed at a plurality of earthing positions based on the calculated plurality of earthing position data. Therefore, the work machine can be loaded without any deviation, and can be loaded evenly in each area, so that the work efficiency can be improved.
[Brief description of the drawings]
FIG. 1 is a diagram showing a working mode of an automatic driving excavator according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a control mechanism during a regeneration operation of the automatic driving shovel according to the present embodiment.
FIG. 3 is a flowchart showing a processing procedure in the image processing unit 1717 and the earthing position data calculating unit 1719 shown in FIG. 2;
4 is a diagram showing the shape of the vessel 14 recognized by the image processing unit 1717 shown in FIG. 2 and the positions of the feature points a, b, c, and d.
FIG. 5 shows the relationship among a target coordinate system set for a target such as a vessel 14, a camera coordinate system with the lens center of the camera 13 as an origin, and a control coordinate system serving as a reference for automatically controlling the automatic driving shovel 1. FIG.
FIG. 6 is a diagram for explaining a method for calculating earthing position data from coordinates of calculated feature points.
FIG. 7 is a diagram for explaining another method for calculating earthing position data from coordinates of calculated feature points.

Claims (3)

In the automatic operation construction machine that repeatedly operates by regenerating operation, the one-round operation from excavation to earthmoving taught
The automatic operation construction machine is
Imaging means for imaging the work machine receiving the earth;
Calculates the coordinates of the feature points of the working machine in the camera coordinate system is a coordinate system based on said image pickup means based on the imaging data obtained by the imaging,
The coordinates of the feature points are converted into the coordinates of a control coordinate system , which is a reference coordinate system for automatically controlling the automatic operation construction machine, and data for an appropriate earthing position on the work machine is calculated. Soil position data calculating means,
An automatic operation construction machine characterized in that the earthing position based on the calculated earthing position data is taught during the regenerating operation. The automatic operation construction machine according to claim 1,
The earthing position data calculating means calculates a plurality of earthing position data for dividing an appropriate earthing position to the work machine into a plurality of areas based on the captured image data,
Playback operation, a plurality of Hodo position based on a plurality of Hodo position data the calculated, automatic operation of construction machine characterized in that it is sequentially taught. In the operation method of the automatic operation construction machine in which the one-round operation from excavation to earthmoving taught is repeatedly operated by the regeneration operation,
Imaging a work machine that receives the earthing at a predetermined work position of the automatic operation construction machine;
It calculates the coordinates of the feature points of the working machine in the camera coordinate system which is an imaging means coordinate system based on the by Hodo position data calculating means on the basis of the imaging data obtained by the imaging,
The coordinates of the feature points are converted into the coordinates of the control coordinate system , which is the reference coordinate system for automatically controlling the autonomous driving construction machine, and the appropriate earthing position to the work machine is divided into a plurality of areas. Calculating a plurality of earthmoving position data ,
A step of sequentially discharging to a plurality of discharging positions based on the plurality of calculated discharging position data ;
A method for operating an automatic operation construction machine, comprising:

Images