JPH0790493B2 - Control method for robot with hand vision - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a control method for a robot with a hand tip equipped with an imaging device at the hand of the robot.

[Conventional technology]

Conventionally, in a robot with a visual tip, when measuring the position of a work target by a visual device attached to the tip of the robot and correcting the reference position, as described in JP-A-60-249583, a TV camera or the like is used. The robot temporarily stops at the position where the recognition target enters the visual field of the hand visual device, and after the visual device captures the image and the image processing device recognizes the position of the work target, the robot receives the position correction value, and then the mechanical hand, etc. The target position of the work device was corrected to perform the positioning operation on the work target.

[Problems to be solved by the invention]

In the above-mentioned conventional technology, since the imaging device is attached to the hand of the robot, the robot moves so that the work target is within the field of view of the imaging device, and after the robot stops, the work target is imaged and Since the position of the work target is recognized and the position of the robot is corrected based on the recognition result, the robot cannot be stopped or cannot work on the work target from the time of visual recognition imaging to the end of the recognition process. However, there is a problem that the work time is increased due to the attachment of the imaging device to the hand of the robot.

It is an object of the present invention that a visual device attached to a hand of a robot captures an image of a work target without stopping while the robot is moving, and before the work device attached to the hand of the robot reaches the vicinity of the work target. Based on the recognition result, the trajectory of the robot can be corrected while the robot is moving, so that the increase in the work time due to the visual correction is eliminated.

[Means for solving problems]

In the robot with a hand tip having a means for picking up the hand of the robot, a means for recognizing the picked up image, and a means for correcting the trajectory of the robot based on the recognition result, an image of the hand end of the moving robot is obtained. When a work target is imaged from the device, a device for adjusting the exposure time of the image pickup device according to the moving speed of the robot's hand is added so that the shape and position of the work target can be recognized from the captured image. The position of the robot at the time of imaging while the robot is moving
In order to accurately detect the posture, a device that communicates time timing between the robot control device and the image processing device,
By adding a device that reads the position and orientation of the robot at a certain time from the joint angle of the robot at high speed, the position of the imaging device at the hand of the moving robot at the time of imaging can be accurately measured. This is achieved by eliminating the need to stop the imaging device of a part and measure the position / orientation of the work target.

[Action]

According to the present invention, the robot control is provided by providing a device for reading the position / orientation of the robot at a certain time from the joint angle of the robot at high speed, and a device for communicating time timings between the robot control device and the image processing device. It is possible to calculate the position / orientation of the imaging device attached to the robot hand when the device issues an imaging command to the imaging device via the image processing device or when the imaging device actually images the work target. Like Further, by providing a device for adjusting the exposure time of the image pickup device, even if the image of the work target is picked up by the image pickup device at the hand of the moving robot, the contour of the work target in the image is not blurred, and the image is accurately displayed in the image. It is possible to recognize the position and orientation of the work target of. With these, it is possible to measure the position / orientation of the work target by using the imaging device of the hand of the moving robot. Therefore, the work device of the hand part of the robot should be stopped for the work target. Instead, the position can be corrected accurately.

〔Example〕

 Hereinafter, examples of the present invention will be described in detail.

FIG. 1 is a configuration block diagram showing an example of an apparatus according to the present invention. In the figure, 1 is a robot having working devices such as an imaging device, a lighting device, and a hand, which will be described later, in a hand portion,
Reference numeral 2 denotes an image pickup apparatus that picks up an image of a recognition target by using an illuminating apparatus to be described later. Reference numeral 3 denotes an illuminating apparatus that is used simultaneously with the image pickup apparatus 2 to adjust an exposure time or an exposure light amount during image pickup. Is a work device that comes into contact with the recognition target to perform a specific work, and here is a hand capable of gripping the target. Reference numeral 11 denotes an image pickup command signal from a robot control device described later, which is received by a synchronization signal high-speed communication circuit described later to control the image pickup device 2 and the lighting device 3 to input an image of a recognition target and input an image. Is an image processing device for recognizing the position and orientation of the recognition target, and 12 is a robot control device that comprehensively controls the robot body 1, hand 4, and image processing device 11; The robot controller sends an imaging command to 11 image processing devices at high speed,
When the second image pickup device starts image pickup, from the 11th image processing device,
Synchronous signal high-speed communication for transmitting an image pickup start signal to 12 robot control units at high speed to accurately synchronize the current position of the robot in the 12 robot control units with the time when the image pickup units 2 picked up images. Reference numeral 14 is a circuit, and reference numeral 14 is a recognition data transmission circuit for transmitting the recognition data recognized by the image processing device 11 to the robot controller 12 by a serial interface.

The operation of the apparatus configured as described above will be described with reference to the flow charts of the processing shown in FIGS. 2, 3 and 4 and 5.
In FIG. 2, the parts corresponding to the respective parts of the apparatus of FIG. 1 are shown with the same reference numerals, the letters with the subscript P indicate the specific points on the work space, and 21 is the robot 4 22 is a movement path through the center point of the gripping mechanism of the hand (hereinafter, abbreviated as “center of the hand”), 22 is a work starting plane including a point P _{S at} which the center of the hand starts moving, and 23 is Target points P _E and 4 for moving the center of the hand to perform work
Is a work plane including a point P _C that represents a work target position where the hand actually performs work. First, according to the processing flow of FIG. 4, the center of the hand starts to rise from the movement start point P _S , and smoothly moves from the ascending movement to the horizontal movement in accordance with the movement path of the robot 21. As soon as the image pickup device 2 moves ahead of the center of the hand 4 and the center of the hand 4 passes the point P _V that issues the image pickup command to the image processing device 11, the robot controller 12 immediately. _Stores the position of P _V , transmits an imaging start command signal to 11 image processing devices via the 13 sync signal high-speed communication circuit, and 11 image processing devices receive the imaging start command signal, When the second image pickup device actually starts to pick up an image, the counter counts (the count value is time T), and when the second image pickup device actually starts picking up an image, From the image capture start notification output unit in the image processing device An imaging start signal is transmitted to the imaging start reception unit in the 12 robot control devices via the high-speed communication circuit, and the 12 robot control devices confirm the imaging start by receiving the imaging start signal and issue the imaging command. 10 ms from point P _V
Memorize the center position P _M of the hand after ec. On the other hand, the second image pickup device adjusts the exposure time of the TV camera by using the lighting device of 3 and so on, in the vicinity of the target point P _E of the movement of the center of the hand on the work plane of 23 so that the image does not shake Actual work target point
The field of view including P _C is captured, and the captured image immediately calculates the position of P _C in the captured image in the image coordinate system in the image processing apparatus 11 and after the calculation is completed, the recognition data transmission circuit 14 is set. Via the position of the work target point P _{C in} the image coordinate system and the above counter value T to the 12 robot controller, and within the 12 robot controller, the received counter value T and P _V point P _M
The center position P _T of the true hand at the time when the image is captured is calculated using the storage position of the point. Next, when the center position of the hand is P _T , the position of the work target point P _{C in} the robot coordinate system is calculated by coordinate conversion between the image coordinate system in the second imaging device and the robot coordinate system. , So that the center of the hand of 4 reaches the work point P _C at the point P _D (called the route correction point) before reaching the target point P _E of the movement of the center of the hand on the movement path of 21. The robot motion path of 21 is corrected, and the center of the hand of 4 is finally reached to the work target point P _C.

In the above case, a method of calculating the true center position P _T of the hand at the time when the image is captured will be described with reference to FIG. FIG. 3 is a detailed view of a part of the robot motion path of FIG. 2, and shows the imaging command issuing point P _V and the imaging command issuing point.
The position P _M of the center of the hand after 10 msec is obtained by reading each joint angle faster than the encoder attached to each joint of one robot, and the spatial position of P _V point and P _M point is obtained. From the time difference of 10 msec between the two points, the movement vector M _V of the position of the center of the hand of the robot is calculated from the imaging command issuing point P _V to the point P _M. On the other hand, in the 11th image processing apparatus, in the case of a general TV camera, the image can be captured only at the 16.7 msec cycle, and therefore, there is a variation of 0 to 16.7 msec from the reception of the image capturing instruction to the actual image capturing. The counter in the 12 image processing devices measures the delay time from when the imaging command is received until the actual imaging is started,
Up to 16.7 msec until the actual image is taken after receiving the imaging command
During this period, it is assumed that the robot is moving at a uniform speed and direction, and the counter value obtained by measuring the above-mentioned movement vector M _V
The center position P _T of the hand at the time of capturing an image is calculated by performing an internal division or an external division based on Tmsec.

When the processing flow of FIG. 5 is followed, the difference from the processing flow of FIG. 4 is that the center position P _T of the hand at the time of imaging is calculated using the movement vector M _V as described above. At the moment when the image pickup is actually started in 11 image processing devices, the image pickup start notification is transmitted via the synchronization signal high-speed communication circuit in 13 and the image pickup start notification is received in 12 robot control devices. Only the processing part for directly calculating the center position P _T of the hand at the time of imaging from the value of each joint angle of one robot at the moment is the same as the other processing. As described above, by some means, the center position of the robot hand at the time of imaging
It suffices if there is a means for calculating P _T at high speed while the robot is moving.

Next, the relationship between the counter value T in the image pickup apparatus 2 of 3, the illumination apparatus 3 of 3 and the image processing apparatus 11 of 11 will be described with reference to two examples shown in the time charts of FIGS. 6 and 7. FIG. 6 shows a television camera using a solid-state image sensor or an image pickup tube for the image pickup device 2 and a strobe lighting device for the lighting device 3.
For example, FIG. 7 is a time chart in the case of being configured with a flash of a xenon lamp. FIG. 7 shows a television camera using a solid-state image sensor or an image pickup tube in the image pickup device 2 and a mechanical shutter, an electronic shutter, and the like. 7 is a time chart in the case of being configured with an object (hereinafter referred to as a camera with a shutter) and an illuminating device of 3 that constantly emits a light bulb, a fluorescent light, or the like. First, referring to FIG.
When an image pickup start command signal is input into the image processing device of
At the first trailing edge of the vertical synchronizing signal in the television camera of the image capturing device of 2, the strobe lighting device of 3 is flashed, and after the flash, one field of the first video output signal becomes a valid video signal, and the counter value is Indicates the time from the start command signal input to the strobe flash. It should be noted that the condition for strobe flashing at the first trailing edge of the vertical synchronization signal is not limited to the type of the image pickup device of 2, and any camera having any image pickup structure can obtain an effective image. This is an example for flashing the strobe during the vertical blanking period, and "if the time from the image pickup start command signal input to the strobe lighting device flash such that a valid video signal is obtained can be counted, the above flash timing is It is not limited.
Next, in FIG. 7, when the image pickup start command signal is input into the image processing device 11 described above, the vertical synchronization signal in the camera with shutter of the image pickup device 2 matches the image pickup start command signal,
The shutter opens for a specified period of time after the trailing edge of the vertical sync signal, and one field of the video output signal immediately after that is a valid video signal. The counter value is the time from the input of the image capture start command signal to the opening of the shutter. Indicates.
Note that the above is an example of a camera with a shutter, and if “the time when the shutter is open is known” in “the time from the input of the imaging start command signal to the opening of the shutter can be counted”, the shutter open The timing is not limited. Further, the present invention is not limited to the embodiment of the combination of the image pickup device and the illumination device described in FIGS. 6 and 7, and the image pickup means attached to the hand of the moving robot does not shake the work target by any method. Any image pickup device may be used as long as it can adjust the exposure time according to the moving speed of the robot so that an image can be picked up, and has a means for detecting the time of exposure.

According to the present embodiment, in the robot with a hand tip, which has an image pickup device capable of instantaneously picking up an image on the hand part of the robot, and which has a means for synchronizing the image pickup time and the position of the robot at high speed,
While the robot is moving, the work target is imaged in synchronization with the robot position, the captured image is recognized, and the work position of the robot can be corrected with respect to the work target based on the recognition result. There is an effect that it is possible to move without stopping a series of movement paths reaching an object, and it is not necessary to stop the robot for visual recognition unlike the conventional robot with a hand-vision.

〔The invention's effect〕

As described above, according to the present invention, by providing the image pickup means capable of adjusting the exposure time and picking up the image and the means for detecting the instantaneous robot position at which the image is picked up, in the robot with a fingertip vision having the visual sense in the hand portion of the robot. While the robot is moving, the position of the work target can be visually recognized and the working position of the robot can be corrected, so that the robot does not need to stop for visual recognition, and the robot has no visual recognition function at hand. There is an effect that the work can be performed in the same time and the increase of the work time can be suppressed.

In addition, as in the conventional robot with a hand tip, it is possible to stop the robot, visually recognize it, and correct the position by making the moving speed of the robot at the time of image capturing to zero. It is possible to selectively use visual recognition during or during stoppage.

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of the present invention, and FIG.
FIG. 3 is a partial detailed view of the robot operation path of the second part, FIG. 4 and FIG.
FIG. 6 is a diagram showing a flow of processing in the robot and image processing in the operation explanatory diagram of FIG. 6, FIG. 6 is a time chart diagram showing a relationship between an imaging command and a strobe flash, and FIG. 7 is an imaging command and opening of a shutter of a camera with a shutter. It is a time chart figure showing a relation. 1 ... Robot, 2 ... Imaging device 3 ... Illumination device, 4 ... Hand 11 ... Image processing device, 12 ... Robot control device 13 ... Synchronous signal high-speed communication circuit 21 ... Robot movement path

Claims (4)

[Claims] 1. A robot body, a robot control device for controlling the robot, a work device attached to a hand portion of the robot, and an image pickup device attached to the work device in the hand portion of the robot. In a robot including an image processing device for recognizing input image data from the imaging device and a communication device between the image processing device and the robot control device, a target work by the robot is performed. A method for controlling the positioning of the working device from another place on a work object placed in the vicinity of a first teaching point is performed by the following procedure. (1) The working device is placed in the other place. Aim the image pickup device so that the image pickup device precedes the work device in the direction of moving to the first teaching point for performing the target work, and (2) The working device reaches the first teaching point from the current position via the second teaching point that captures the first teaching point for performing the target work by the imaging device in the visual field. (3) The work device is operated along the motion path after or during the calculation of the motion path, and the work device determines the second teaching point. Upon passing, the robot controller sends an image capturing start command to the image processing device, and (4) the image processing device receives the image capturing start command,
The delay time from the time when the imaging start command is received to the start time of the next imaging cycle of the imaging apparatus having a predetermined imaging cycle is counted, and the work target is processed by the imaging apparatus at the start time of the next imaging cycle. An object is imaged, the imaged image is recognized, the position of the work object is recognized, the recognition result and the counted delay time are transmitted to the robot control device, and (5) the robot control device , The position of the robot is obtained after a predetermined time has elapsed after sending an image capturing start command to the image processing device, and based on the recognition result and the delay time received from the image processing device, the image capturing device at the time when the image capturing is started. Is calculated by approximating the position of the work object and the error between the position of the work object and the first teaching point, and from the result, the work pair is started from the position immediately before reaching the first teaching point. The correction operation path to the elephant is calculated, and (6) the working device is operated along the operation path calculated in the second procedure, and then the operation apparatus is operated along the correction operation path. A method for controlling a robot with a hand tip, which is characterized by a procedure for positioning a work object. 2. The method for controlling a robot with a hand tip according to claim 1, wherein in the fourth procedure, the image processing apparatus which has received the image capturing start command has a predetermined image capturing cycle. At the start time of the next imaging cycle of the imaging device, the robot control device is notified that the imaging is started, and at the same time, the work object is imaged by the imaging device, and the captured image is recognized to perform the recognition process. The position of the work object is recognized, the recognition result is transmitted to the robot control device, and the fifth step is that the robot control device controls the robot at the time when the imaging start notification is received from the image processing device. The position of the image pickup device is obtained from the position, and then the error between the position of the work target and the first teaching point is obtained based on the recognition result received from the image processing device. The method of the hand vision with a robot, characterized in that the position immediately before reaching the teaching point calculates a correction operation path to the work object, it is. 3. The method for controlling a robot with a hand tip according to claim 1 or 2, wherein the image processing device in the fourth procedure images the work target by the image pickup device. The control of the robot with a hand tip, characterized in that the processing is to flash the stroboscopic illuminating device and perform imaging by the TV camera at the start time of the next imaging cycle of the imaging apparatus having a predetermined imaging cycle. Method. 4. A method for controlling a robot with a hand tip according to claim 1 or 2, wherein the image processing device in the fourth step images the work target by the image pickup device. The control of the robot with a hand-eye vision, characterized in that the processing is to open the shutter of the camera with a shutter for a predetermined time and perform imaging at the start time of the next imaging cycle of the imaging device having a predetermined imaging cycle. Method.