JPH0731533B2 - Three-dimensional coordinate transformation device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial field of application) The present invention includes a visual device such as a camera in a hand of a robot, and the visual device calculates an accurate transport position and posture of a workpiece to be transported. The present invention relates to a three-dimensional coordinate conversion device, and more particularly to a three-dimensional coordinate conversion device capable of automatically correcting the displacement even when the visual device itself is dislocated during operation of a robot. .

(Prior Art) With the recent automation of production processes, for example, in assembly work, a robot automatically grabs a work automatically conveyed by an automated guided vehicle or the like to perform a predetermined work.

In this case, the robot is provided with a visual function such as a camera, and the posture and position of the conveyed work are recognized by this visual function, and the robot always grasps a specific position of the work regardless of the conveyance posture and position of the work. There is. In this way, in order to make the robot grasp a specific position of the work, the coordinate system of the camera in the work is coordinate-transformed to the coordinate system of the robot by the visual control device that controls the visual function, and the signal after this coordinate transformation is performed. This is performed by outputting to a control device that controls the robot.

FIG. 6 shows a schematic configuration diagram of a generally used three-dimensional coordinate conversion device.

As shown in the figure, the camera 1 that captures the characteristic points of the workpiece conveyed on the production line is connected to the image memory 3 via the image input unit 2 that inputs the image from the camera 1. . Therefore, the image input from the camera 1 is stored in the image memory 3 as digital data.

Then, in the image memory 3, a calibration pattern extraction unit 4 for extracting a specific pattern required for coordinate conversion from the digital data stored in the image memory 3 and the digital data stored in the image memory 3 are extracted. A feature point extraction unit 8 for extracting feature points from is connected.
Further, the calibration pattern extraction unit 4 is connected to a coordinate conversion formula creation unit 5 that creates a conversion formula for matching the coordinate system of the camera 1 with the coordinate system of the robot. An absolute coordinate storage unit 6 that stores the coordinates of the robot system in a specific pattern of the work and a conversion formula storage unit 7 that stores the conversion formula created by the coordinate conversion formula creation unit 5 are connected, The coordinate conversion formula creation unit 5 calculates a coordinate conversion formula based on the position of the specific pattern extracted by the calibration pattern extraction unit 4 and the position of the specific pattern of the workpiece in the robot system stored in the absolute coordinate storage unit 6. Is created and this coordinate conversion formula is stored in the conversion formula storage unit 7.

Further, an output unit 10 is connected to the conversion formula storage unit 7 via a coordinate conversion unit 9, and the position of the feature point of the workpiece extracted by the feature point extraction unit 8 in the camera coordinate system is the coordinate conversion unit 9. Is converted into a position in the robot coordinate system based on the conversion formula stored in the conversion formula storage unit 7, and is output from the output unit 10 to a robot controller (not shown). As a result, the robot can grab and move a specific position of the work, or can machine a predetermined position.

(Problems to be Solved by the Invention) However, such a conventional three-dimensional coordinate conversion device is based on the premise that the mounting state of the camera does not deviate during the operation of the robot. Therefore, if the mounting bolts of the camera 1 naturally loosen with the passage of the operating time of the robot and the mounting position shifts, or if the mounting position of the camera 1 shifts during maintenance, the robot work accuracy Will be worse.

Therefore, in order to maintain the work accuracy of the robot, it is necessary to perform maintenance on a regular basis and to recreate the coordinate conversion formula at regular time intervals, assuming such a situation.

For this reason, there are various problems such that maintenance requires time to maintain the work accuracy, the work is complicated, and the reliability of the work accuracy of the robot during the maintenance cycle is low.

The present invention has been made in view of such a conventional problem, and a third-order system is provided in which a mounting position deviation of the camera 1 is appropriately recognized during the work of the robot and the deviation is automatically corrected. An object is to provide an original coordinate transformation device.

(Means for Solving Problems) In order to achieve the above object, the present invention provides a robot hand with a visual device for calculating an accurate transport position and posture of a workpiece to be transported. A three-dimensional coordinate conversion device for visualizing a reference pattern arranged at a position other than a feature point of a workpiece conveyed on a production line and a work position of the robot, and an image captured by the visual device. An image memory for storing as a digital data, a feature point of a work necessary for coordinate conversion from the image digital data stored in the image memory, and a calibration pattern extraction unit for extracting a specific pattern in the reference pattern. A feature point extracting section for extracting feature points of the workpiece from the image digital data stored in the image storage means; A coordinate conversion formula creating unit that creates a conversion formula that matches the coordinate system of the visual device of the specific pattern extracted by the calibration pattern extraction unit with the coordinate system of the robot, the feature point of the workpiece, and the reference pattern. An absolute coordinate storage unit that pre-stores a coordinate system of the robot having a specific pattern, and a coordinate system of the visual device of the feature points of the workpiece created by the coordinate conversion formula creating unit that match the coordinate system of the robot. A first conversion formula storage unit that stores one coordinate conversion formula, and a second coordinate system that matches the coordinate system of the specific pattern in the reference pattern created by the coordinate conversion formula creation unit in the visual device with the coordinate system in the robot. A second conversion formula storage unit that stores a conversion formula, and the reference pattern captured by the visual device. The current coordinate conversion formula creating unit that creates the current coordinate conversion formula of the specific pattern in the above, and the second coordinate conversion formula and the current coordinate conversion formula creating unit stored in the second conversion formula storage unit. The conversion formula stored in the first conversion formula storage unit is corrected by the deviation amount detection unit that detects the difference between the reference patterns from the current coordinate conversion formula and the difference between the reference patterns detected by the deviation amount detection unit. The deviation correction unit, the position in the coordinate system in the visual device of the feature points of the workpiece extracted by the feature point extraction unit, by the conversion formula to which the correction output from the deviation correction unit is added,
A coordinate conversion unit for converting a position in the robot coordinate system,
It is a three-dimensional coordinate conversion device characterized by having.

(Operation) In the present invention configured as described above, the calibrated sean pattern extraction unit extracts a specific pattern of the feature point of the work and the reference pattern from the image captured by the visual device, and the coordinate conversion formula is used. In the creation unit, a coordinate conversion formula from the visual device coordinate system to the robot coordinate system is created, and the first coordinate conversion formula at the feature point of the work is stored in the first conversion formula storage unit, and the specific pattern of the reference pattern is stored. The second coordinate conversion formula in is stored in the second conversion formula storage unit. Then, in addition to the first and second coordinate conversion formulas, a current coordinate conversion formula creating unit creates a current coordinate conversion formula for performing coordinate conversion of a specific pattern of the reference pattern. As a result, the second coordinate conversion formula in the specific pattern of the reference pattern at a certain time is stored in the second conversion formula storage unit, and the current coordinate conversion formula at another certain time is created as the current coordinate conversion formula. Created by the department. Then, the deviation amount detection unit detects the difference between the reference patterns from these two coordinate conversion formulas, and the deviation correction unit corrects the first coordinate conversion formula for performing the coordinate conversion of the feature point of the work according to the detected deviation amount. To do.

By converting the feature points of the workpiece extracted by the feature point extraction unit from the visual device coordinate system to the robot coordinate system by the corrected first coordinate conversion formula, a shift occurs in the visual device during the work of the robot, for example. But, for example, the second
The coordinate conversion formula stored in the conversion formula storage unit is stored in the initial state, and while the work is being performed thereafter, the current coordinate conversion formula is appropriately calculated, and by comparing these,
It is possible to recognize the deviation of the visual device and automatically correct the deviation.

(Example) Below, the Example of this invention is described in detail based on drawing.

FIG. 1 shows a block diagram of a three-dimensional coordinate transformation device according to the present invention.

As shown in FIG. 1, a camera 1 which is a visual device for picking up a feature point and a reference pattern of a work conveyed on a production line receives an image from an image input unit 2 for inputting an image from the camera 1. It is connected to the memory 3. The image input unit 2 converts an image captured by the camera 1 from analog data into digital data, and therefore, the camera 1
The image captured by is stored in the image memory 3 as digital data.

Then, in the image memory 3, a calibration pattern extraction unit 4 that extracts a feature point of a work necessary for coordinate conversion and a specific pattern in the reference pattern from the digital data stored in the image memory 3, and an image. A characteristic point extraction unit 8 for extracting characteristic points of the work from the digital data stored in the memory 3 is connected.

In addition, the calibration pattern extraction unit 4 has coordinates for creating a conversion formula for matching the position of the specific pattern extracted by the calibration pattern extraction unit 4 in the coordinate system of the camera 1 with the position of the robot in the coordinate system. The conversion formula creating unit 5 is connected. That is,
In the present embodiment, in the coordinate conversion formula creation unit 5, the position in the coordinate system of the camera 1 of the feature point of the work when the work extracted by the calibration pattern extraction unit 4 is located in the standard state is the coordinate of the robot. The calibration pattern extraction unit 4 uses the first coordinate conversion formula that matches the position in the system and the reference pattern in the initial state.
A second coordinate conversion formula that matches the position of the specific pattern extracted by the coordinate system of the camera 1 with the coordinate system of the robot is created. Here, the standard state is the position of the work when the feature points of the work are stored as work points for the robot to perform work on the work, and the initial state is necessary for the robot to perform work. When the position at a different working point is stored or changed, or when the camera 1 is operated to perform the first operation after being attached to the hand portion of the robot,
Or when you start the operation from the stopped state,
It means the time when the robot starts operating.

An absolute coordinate storage unit 6 in which the position of the characteristic point of the work in the robot coordinate system in the state where the work is conveyed to the coordinate conversion formula creating unit 5 is stored in advance is stored in the coordinate conversion formula creating unit 5. , A first conversion formula storage unit 7A that stores the first coordinate conversion formula created by the coordinate conversion formula creation unit 5 and a second storage unit that similarly stores the second coordinate conversion formula created by the coordinate conversion formula creation unit 5 The conversion formula storage unit 11 and the current position of the specific pattern extracted from the reference pattern imaged by the camera 1 during the operation of the robot in the coordinate system of the camera 1 to the position in the coordinate system of the robot. And the current coordinate conversion formula creating unit 12 for creating the coordinate conversion formula of FIG.

Therefore, the first conversion formula storage unit 7A stores the first coordinate conversion formula for converting the position of the feature point of the workpiece imaged by the camera 1 into the robot coordinate system, and the second conversion formula storage unit 11 stores the first coordinate conversion formula by the camera 1. The second coordinate conversion formula for converting the position of the specific pattern extracted from the reference pattern in the imaged initial state into the robot coordinate system is stored.

Further, the second conversion formula storage unit 11 and the current coordinate conversion formula generation unit 12
The deviation amount detecting unit 13 is connected to the deviation amount detecting unit 13.
13 is the difference in the reference pattern from the second coordinate conversion formula stored in the second conversion formula storage unit 11 and the current coordinate conversion formula calculated by the current coordinate conversion formula creation unit 12, that is, the camera 1
The amount of deviation, which is the amount of mounting deviation, is detected.

The deviation amount detection unit 13 is connected to a deviation correction unit 14 that corrects the first coordinate conversion formula stored in the first conversion formula storage unit 7A based on the deviation amount detected by the deviation amount detection unit 13. There is.

The deviation correction unit 14 is connected to the coordinate conversion unit 9, and the coordinate conversion unit 9 outputs the position of the feature point of the workpiece extracted by the feature point extraction unit 8 in the camera coordinate system from the deviation correction unit 14. Based on the corrected first coordinate conversion formula, the coordinate is converted to the position in the robot coordinate system, and the corrected coordinate value of the work is output from the output unit 10 to the robot controller (not shown). There is.

Further, FIG. 2 shows a schematic configuration diagram of the periphery of the three-dimensional coordinate conversion device according to the present invention.

The camera 1 has an I / 0 port 2 provided in the visual control device 20 via an A / D converter 23 that converts analog data regarding an image output from the camera 1 into digital data.
Connected to 4. In addition, the I / 0 port 24 is connected to a ROM 25 that stores an operation program of the visual control device 20 and a RAM 26 that stores various kinds of data sent to the visual control device 20, and the visual control device 20 is operated according to an operational flowchart in the ROM 25. A CPU 27 for controlling the robot 15 and an I / 0 port 28 provided in the robot controller 19 for controlling the robot 15 are connected. Further, the I / 0 port 28 is connected to a ROM 29 which stores an operation program of the robot controller 19 and a RAM 30 which stores various data sent thereto, and the robot controller 19 is operated according to an operation flowchart in the ROM 29. A CPU 31 for controlling the overall operation of the robot is connected to a drive circuit 32 for outputting a drive signal to an arm or the like constituting the robot 15 in response to a command from the CPU 31.

The three-dimensional coordinate conversion device configured as described above operates as follows based on the operation flowchart shown in FIG. Hereinafter, this operation flowchart will be described in detail with reference to FIGS. 1 to 4.

In the operation of this apparatus, first, calibration in an initial state is performed in steps 1 to 6, and then, a work operation of the robot is performed in steps 7 to 12. Each step will be described below.

Step 1 As shown in FIG. 4, in the camera 1 mounted on the arm 15A of the robot 15, the work 22 sequentially conveyed on the conveyor 21 is in the standard state as shown in FIG. The image of the portion of the characteristic points in (the characteristic points are usually determined in advance for each type of work), and this image is A / D
It is converted into digital data by the converter 23, and the digital data is converted into the I / 0 port 2 provided in the visual control device 20.
Store in RAM 26 via 4. The RAM 26 functions as the image memory 3 shown in FIG. 1, and the A / D converter 23 functions as the image input unit 2.

Step 2 The CPU 27 extracts predetermined feature points on the work from the digital data stored in step 1, calculates the coordinates of the feature points in the camera system, and the coordinates and the robot system stored in advance in the RAM 26. A first coordinate conversion formula for converting the coordinates in the camera system into the coordinates in the robot system is calculated from the coordinates of the feature points in. In this case, the CPU 27 acts as the calibration pattern extraction unit 4 and the coordinate conversion formula creation unit 5, and the RAM 26 acts as the absolute coordinate storage unit 6.

Step 3 The CPU 27 stores the first coordinate conversion formula calculated in Step 2 in the RAM 26. The RAM 26 in this case functions as the first conversion type storage unit 7A.

Step 4 The arm 15A equipped with the camera 1 is connected to the robot controller 19
In response to a command from the robot 15, the robot 1 moves to the reference pattern display base 16 arranged at the retracted position of the robot 15, and the camera 1 displays the reference pattern drawn on the reference pattern display base 16. 17 is imaged, and the image of each black circle 18, ..., 18 that is a specific pattern of this reference pattern is converted into an A / D converter 2
Convert to digital data by 3 and use this digital data as a calibration pattern for visual control
The data is stored in the RAM 26 via the I / 0 port 24 provided in the 20. The RAM 26 functions as the image memory 3 shown in FIG.

Step 5 The CPU 27 determines the reference pattern 17 stored in Step 4
From the digital data of, each black circle 18, ..., 18 that is the specific pattern is extracted, and the coordinates in the camera system are calculated.
A second coordinate conversion formula for converting the coordinates in the camera system in the initial state into the coordinates in the robot system is calculated from the coordinates of the black circles 18, ..., 18 in the robot coordinate system stored in advance in the RAM 26. In this case, the CPU 27 acts as the calibration pattern extraction unit 4 and the coordinate conversion formula creation unit 5, and the RAM 26 acts as the absolute coordinate storage unit 6.

Step 6 The CPU 27 stores the second coordinate conversion formula calculated in Step 5 in the RAM 26. The RAM 26 in this case functions as the second conversion type storage unit 11.

By the steps so far, the first coordinate conversion formula and the second coordinate conversion formula, which are the conversion formulas from the camera coordinate system to the robot coordinate system of the specific pattern of the workpiece and the reference pattern in the initial state as the calibration, are obtained. can get.

Step 7 The robot 15 takes an image of the reference pattern 17 in the same manner as in Step 4 every time when the predetermined work is performed on the work 22 conveyed on the conveyor 21, and the black circles 18 ,. The image is converted into digital data by the A / D converter 23, and this digital data is used as a calibration pattern for I / 0 provided in the visual control device 20.
Store in RAM 26 via port 24.

Step 8 The CPU 27 causes the black circles 18, ..., 1 extracted from the reference pattern 17 to be extracted.
A current coordinate conversion formula for converting the coordinates in the visual system into the coordinates in the robot system is calculated from the digital data in 8 and the digital data in the black circles 18, ..., 18 in the robot coordinate system stored in advance in the RAM 26.
In this case, the CPU 27 functions as the calibration pattern extraction unit 4, the coordinate conversion formula generation unit 5, and the current coordinate conversion formula generation unit 12, and the RAM 26 functions as the absolute coordinate storage unit 6.

Step 9 The CPU 27 uses the second coordinate conversion formula stored in the RAM 26 and the current coordinate conversion formula calculated in step 8 to calculate the camera 1
The deviation amount, which is the mounting deviation, is calculated.

At this time, the CPU 27 acts as the deviation amount calculation unit 13.

Step 10 Robot arm 15A moves onto conveyor 21 and camera 1
Therefore, the captured image of the work in the transporting posture is converted into digital data by the A / D converter 23, and this digital data is converted into the I / 0 port 24 provided in the visual control device 20.
To RAM 26 via. Then, the CPU 27 extracts the feature points of the work from the digital data of this image. The CPU 27 at this time functions as the feature point extraction unit 8.

Step 11 The CPU 27 calculates the deviation amount in the mounted state of the camera 1 calculated in step 9 and the deviation amount in the transporting posture of the work, and based on these deviation amounts, the stored first coordinate conversion formula Is corrected, and the position of the work in the camera coordinate system or the like is converted into coordinates in the robot system based on the corrected coordinate conversion formula. CPU27 in this case is
It functions as the deviation correction unit 14 and the coordinate conversion unit 9.

Step 12 The CPU 27 uses the converted data calculated in Step 11 as I
Robot controller 1 via / 0 port 24 and I / 0 port 28
It outputs to CPU31 in 9 and CPU31 outputs this conversion data to RAM30.
The operation of the robot 15 thereafter is performed based on this conversion data, and the hand 15B can grasp a predetermined position of the work. In this case, the CPU 27 uses the output unit 1
Acts as 0.

When the work for one work is completed, the process returns to step 7, and steps 7 to 12 are repeated for each work of each work.

In the operation flow chart described above, the mounting deviation of the camera 1 is detected and corrected for each work of the work. However, in a line with a short tact time, the work shift is performed every predetermined number of work works (for example, 10 work). The correction may be performed once for each work.

(Effects of the Invention) As is apparent from the above description, according to the present invention, the coordinate conversion formula creating unit that creates the coordinate conversion formula at the time of calibration in the initial state and the coordinate conversion formula during the working operation are created. By having a coordinate conversion formula creation unit at the same time, it is possible to appropriately recognize the mounting position deviation of the visual device while the robot is working, and this deviation is automatically corrected, which simplifies maintenance. It is possible to improve the reliability of work accuracy of the robot. Further, even when the mounting accuracy of the arm or the like of the robot on which the visual device is mounted is lowered, the same effect as described above can be obtained.

[Brief description of drawings]

FIG. 1 is a block diagram of a three-dimensional coordinate conversion device according to the present invention, FIG. 2 is a schematic configuration diagram around the three-dimensional coordinate conversion device shown in FIG. 1, and FIGS. FIG. 5 is an explanatory reference diagram of the operation flowchart shown in FIG. 5, FIG. 5 is an operation flowchart of the three-dimensional coordinate conversion apparatus shown in FIG. 1, and FIG. 6 is a block diagram of a general three-dimensional coordinate conversion apparatus. 1 ... Camera (visual device), 15 ... Robot 15A ... Arm, 15B ... Hand, 16 ... Calibration pattern display base, 17 ... Calibration pattern (reference pattern), 18 ... Black circle, 21 ... … Conveyor, 22 …… Workpiece.

Claims (1)

[Claims] 1. A three-dimensional coordinate transformation device for providing an accurate visualizing position and posture of a workpiece conveyed by a visual device provided on a robot hand, which is conveyed on a production line. A visual device (1) for picking up a characteristic pattern of a workpiece and a reference pattern arranged at a position other than the work position of the robot, and an image memory (3) for storing the image picked up by the visual device as digital data. A calibration pattern extraction unit (4) for extracting a feature point of a work required for coordinate conversion and a specific pattern in the reference pattern from image digital data stored in the image memory (3); A feature point extracting section (8) for extracting feature points of the work from image digital data stored in the means (3); A coordinate conversion formula creating unit (5) for creating a conversion formula for matching the coordinate system of the visual device (1) of the specific pattern extracted by the image pattern extracting unit (4) with the coordinate system of the robot; An absolute coordinate storage unit (6) that stores in advance a coordinate system of a point and a specific pattern of the reference pattern in the robot; and the visual device for the feature point of the work created by the coordinate conversion formula creating unit (5). A first conversion formula storage unit (7A) that stores a first coordinate conversion formula that matches the coordinate system in (1) with the coordinate system in the robot; and a specific pattern in the reference pattern created by the coordinate conversion formula creation unit 5. A second conversion formula storage unit (1) for storing a second coordinate conversion formula for matching the coordinate system of the pattern in the visual device (1) with the coordinate system of the robot. 1), a current coordinate conversion formula creating unit (12) for creating a current coordinate conversion formula of a specific pattern in the reference pattern captured by the visual device, and the second conversion formula storage unit (11). Said second
A deviation amount detection unit (13) for detecting the difference between the reference patterns from the coordinate conversion formula and the current coordinate conversion formula created by the current coordinate conversion formula creation unit (12), and the deviation amount detection unit (13) A deviation correction unit (14) that corrects the conversion formula stored in the first conversion formula storage unit (7A) based on the difference between the reference patterns, and the feature points of the workpiece extracted by the feature point extraction unit (8). The position in the coordinate system of the visual device (1) of
A three-dimensional coordinate conversion device, comprising: a coordinate conversion unit 9 for converting the position in the robot coordinate system by a conversion formula added with the correction output from the deviation correction unit (14).