JP7562827B2 - Camera position deviation measuring device and method - Google Patents

Description

This specification discloses a camera position deviation measuring device and a position deviation measuring method.

A robot system has been proposed that includes a robot having a robot arm and a robot camera, and an external camera installed separately from the robot, and captures an image of a calibration pattern with a pre-calibrated robot camera and an image of the calibration pattern with the external camera to obtain calibration data for the external camera (see, for example, Patent Document 1). This system captures an image of a calibration pattern with the robot camera and the external camera, respectively, to obtain pattern images, and obtains calibration data for the external camera based on the obtained pattern images and known calibration data for the robot camera.

JP 2019-69493 A

However, although the technology described in the above-mentioned Patent Document 1 describes calibrating an external camera, it makes no mention of calibrating a robot camera.

The primary objective of this disclosure is to properly measure the misalignment of a camera attached to a robot arm.

This disclosure takes the following measures to achieve the above-mentioned primary objective.

The camera position deviation measuring device of the present disclosure is
1. A camera position deviation measuring device for measuring a position deviation of a hand camera in a robot system including a robot having a robot arm and a hand camera attached to the robot arm, and an external camera installed outside the robot, comprising:
a jig member disposed above the external camera and having a first measurement hole extending vertically therethrough;
a first position acquisition unit that captures an image of the jig member using the external camera and acquires a position of the first measurement hole from the captured image of the first measurement hole;
a second position acquisition unit that moves the hand camera to the position acquired by the first position acquisition unit, captures an image of the jig member with the hand camera, and acquires the position of the first measurement hole from the captured image of the first measurement hole;
a positional deviation measuring unit that moves a hand of the robot to the position of the first measurement hole acquired by the second position acquisition unit, captures an image of the hand of the robot with the external camera, and measures a positional deviation of the camera from the captured image;
The gist of the invention is to provide the following:

The camera positional deviation measuring device disclosed herein uses an external camera to measure the positional deviation of a hand camera attached to a robot arm, and includes a jig member that is installed above the external camera and has a first measurement hole that penetrates vertically. Furthermore, the positional deviation measuring device images the jig member with the external camera to obtain the position of the first measurement hole, moves the hand camera to the obtained position, images the jig member with the hand camera, and obtains the position of the first measurement hole. The positional deviation measuring device then moves the robot's hand to the position of the obtained first measurement hole, images the robot's hand with the external camera, and measures the positional deviation of the camera. This allows the positional deviation of the camera attached to the robot's arm to be measured appropriately.

FIG. 2 is an external perspective view of the robot system. FIG. 2 is a block diagram showing an electrical connection relationship between the robot and a control device. FIG. 2 is an explanatory diagram for explaining a world coordinate system. FIG. 2 is an explanatory diagram for explaining a base coordinate system, a camera coordinate system, and a mechanical interface coordinate system. FIG. 11 is an explanatory diagram showing a procedure for measuring the hand camera position. FIG. 2 is an external view of an end effector with a mark. 13 is a flowchart showing a process of measuring the position of a work camera for measuring the position of a hand camera. FIG. 11 is an explanatory diagram showing how a work camera position is measured. FIG. 11 is an explanatory diagram illustrating the amount of deviation of a work camera position. FIG. 11 is an explanatory diagram showing how a hole jig is attached. 13 is a flowchart showing an example of a hole position measuring process for measuring a hand camera position. 11 is an explanatory diagram showing how a work camera captures an image of three holes in a hole jig; FIG. 13 is a flowchart illustrating an example of a hand camera angle measurement process. FIG. 11 is an explanatory diagram showing a state in which an image of a hole jig is captured by a hand camera. FIG. 13 is an explanatory diagram showing the phases of three holes measured by a work camera and a hand camera, respectively. 13 is a flowchart illustrating an example of a hand camera position measurement process. FIG. 11 is an explanatory diagram showing how an end effector with a mark is imaged by a work camera. 11 is an explanatory diagram illustrating the distance from the center of the hand camera to the center hole. FIG. 11 is an explanatory diagram showing the relationship between the mark pin position (average value), the center position of the hole for measuring the hand camera position, and the amount of positional deviation. FIG.

Next, the form for implementing the present disclosure will be explained with reference to the drawings.

Figure 1 is an external perspective view of the robot. Figure 2 is a block diagram showing the electrical connection between the robot and the control device.

Robot system 1 is a work robot system that, for example, picks up a work W (component) supplied by a work supply unit P and mounts it on a substrate S. As shown in Figures 1 and 2, this robot system 1 includes a work table 2, a robot 10, a hand camera 60 attached to a robot arm 20 of the robot 10, a work camera 70 as an external camera, and a control device 90 that controls the robot 10. On the work table 2, a main pole 3a extending vertically and a sub-pole 3b extending vertically so as to be spaced apart from the main pole 3a are installed.

In this embodiment, the robot 10 is a SCARA robot and comprises a base 11 and a robot arm 20.

The base 11 is fixed to the workbench 2 and supports the base end side of the robot arm 20. The robot arm 20 includes a first arm 21, a first arm drive unit 30, a second arm 22, a second arm drive unit 40, a shaft 23, and a shaft drive unit 50. The base end of the first arm 21 is connected to the base 11 via a first joint axis J1, and is configured to be rotatable (horizontally rotated) in a horizontal plane relative to the base 11 by rotation of the first joint axis J1. The base end of the second arm 22 is connected to the tip of the first arm 21 via a second joint axis J2, and is configured to be rotatable (horizontally rotated) in a horizontal plane relative to the first arm 21 by rotation of the second joint axis J2. The shaft 23 is connected to the tip of the second arm 22 via a third joint axis J3, and is configured to be rotatable around the axis of the third joint axis J3 relative to the second arm 22 and to be raised and lowered along the axial direction of the third joint axis J3. In the robot system 1 of this embodiment, a workpiece holding unit 24 that serves as an end effector and picks up and holds the workpiece W is provided at the tip of the shaft 23. Examples of the workpiece holding unit 24 include a suction nozzle that uses negative pressure to suction the workpiece W, a mechanical chuck that grips the workpiece W with a pair of claws, and an electromagnetic chuck that uses an electromagnet to suction the workpiece W.

The first arm driving unit 30 includes a motor 32 and an encoder 34. The rotating shaft of the motor 32 is connected to the first joint shaft J1 via a reduction gear (not shown). The first arm driving unit 30 drives the motor 32 to rotate the first arm 21 around the first joint shaft J1 as a fulcrum by torque transmitted to the first joint shaft J1 via the reduction gear. The encoder 34 is attached to the rotating shaft of the motor 32 and is configured as a rotary encoder that detects the amount of rotational displacement of the motor 32.

The second arm driving unit 40, like the first arm driving unit 30, includes a motor 42 and an encoder 44. The rotating shaft of the motor 42 is connected to the second joint shaft J2 via a reduction gear (not shown). The second arm driving unit 40 drives the motor 42 to rotate the second arm 22 around the second joint shaft J2 as a fulcrum by torque transmitted to the second joint shaft J2 via the reduction gear. The encoder 44 is attached to the rotating shaft of the motor 42 and is configured as a rotary encoder that detects the amount of rotational displacement of the motor 42.

The shaft drive unit 50 includes motors 52a and 52b and encoders 54a and 54b. The rotation shaft of the motor 52a is connected to the shaft 23 via a belt (not shown). The shaft drive unit 50 drives the motor 52a to rotate the shaft 23 around its axis. The rotation shaft of the motor 52b is connected to the shaft 23 via a ball screw mechanism (not shown). The shaft drive unit 50 drives the motor 52b, and the ball screw mechanism converts the rotational motion of the motor 52b into linear motion, thereby raising and lowering the shaft 23. The encoder 54a is configured as a rotary encoder that detects the amount of rotational displacement of the shaft 23. The encoder 54b is configured as a linear encoder that detects the raised and lowered position of the shaft 23.

2, the control device 90 includes a CPU 91, a ROM 92 that stores a processing program, a RAM 93 as a work memory, a storage device 94 such as a HDD or SSD, and an input/output interface (not shown). Position signals from the encoders 34, 44, 54a, and 54b and image signals from the hand camera 60 are input to the control device 90 via the input/output interface. Drive signals for the motors 32, 42, 52a, and 52b are output from the control device 90 via the input/output interface.

The hand camera 60 is attached to the tip of the second arm 22. The hand camera 60 captures an image of the workpiece W in the work supply section P from above and outputs the captured image to the control device 90. The control device 90 recognizes the position of the workpiece W by processing the captured image. Note that since the hand camera 60 is attached to the second arm 22, in this embodiment, the robot arm 20 cannot move the hand camera 60 in the vertical direction.

The work camera 70 is installed between the work supply unit P and the substrate S on the workbench 2. The work camera 70 is housed in a storage box 71 having a rectangular opening 71a at the top. The work camera 70 captures an image of the work W held by the work holding unit 24 of the robot 10 from below, and outputs the captured image to the control device 90. The control device 90 processes the captured image to determine whether the work W is being held properly by the work holding unit 24.

Next, the operation of the robot system 1 thus configured will be described. The operation of the robot system 1 includes a work position recognition operation, a collection operation, a collection confirmation operation, and a mounting operation. The work position recognition operation involves capturing an image of the work W with the hand camera 60 to recognize the position of the work W. The collection operation involves moving the hand (work holding portion 24) of the robot system 1 to the position of the recognized work W to collect the work W. The collection confirmation operation involves moving the work W collected by the hand above the work camera 70 and capturing an image of the work W with the work camera 70 to check whether the collection state of the work W is good or not. The mounting operation involves mounting the collected work W on the board S.

Here, the coordinate system of the robot system 1 will be described. In this embodiment, as shown in Figs. 2 and 3, the robot system 1 has a world coordinate system _Σw , a base coordinate system _Σb , a camera coordinate system _Σc, and a mechanical interface coordinate system _Σm as its coordinate systems. The world coordinate system _Σw has an origin set at the tip of the main pole 3a, an X-axis set in a direction passing through the tip of the main pole 3a and the tip of the sub-pole 3b, a Z-axis set in a vertical direction, and a Y-axis set in a direction perpendicular to the X-axis and Z-axis. The base coordinate system _Σb has an origin set at the bottom surface of the base 11 of the robot 10, and is set so that the X-axis, Y-axis, and Z-axis are aligned with the corresponding axes of the world coordinate system _Σw . The mechanical interface coordinate system _Σm has an origin set at the tip of the robot arm 20, and is set so that the X-axis, Y-axis, and Z-axis are aligned with the corresponding axes of the world coordinate system _Σw . The camera coordinate system _Σc has its origin set at the bottom surface of the second arm 22, which is the center of the third joint axis J3, and is set so that the X-axis, Y-axis, and Z-axis are aligned with the corresponding axes of the world coordinate system _Σw . The world coordinate system _Σw , base coordinate system _Σb , camera coordinate system _Σc , and mechanical interface coordinate system _Σm are mutually converted using a conversion matrix.

In the workpiece position recognition operation, the CPU 91 of the control device 90 first sets target positions Xbtag, Ybtag, and _Zbtag of the hand camera 60 as viewed from the base 11 (base coordinate system _Σb shown in FIG. 1) for capturing an image of the workpiece W supplied from the workpiece _supply unit P. Next, the CPU ₉₁ solves inverse kinematics for the target positions _Xbtag , _Ybtag , and _Zbtag of the hand camera 60 to calculate an angle _θJ1 of the first joint axis J1 and an angle _θJ2 of the second joint axis J2 for matching the position of the hand camera 60 with the target positions _Xbtag , _Ybtag , and _Zbtag .

Next, the CPU 91 calculates estimated positions X best , Y best , and Z best of the hand camera 60 by solving forward kinematics based on the angle _{θ J1} of the first joint axis J1 and the angle θ _J2 of the second joint axis J2, taking into account error factors such as torsion and _deflection of the first arm 21 and deflection of the _second arm _22. That is, assuming that the angles θ _J1 and θ _J2 of the first and second joint axes J1, J2 calculated by inverse kinematics are controlled as angle command values so that the position of the hand camera 60 coincides with the target positions X _btag , Y _btag , Z _btag , the CPU 91 calculates estimated positions X _best , Y _best , and Z _best that are positions deviated from the target positions X _btag , Y btag , Z _btag due to error factors such as torsion and deflection of the first arm 21 and deflection of the second arm 22.

After calculating the estimated positions _Xbest , _Ybest , and _Zbest , the CPU 91 calculates positional deviations _ΔXb , _ΔYb , and _ΔZb by taking the differences between the calculated estimated positions _Xbtag , _Ybest , and _Zbest and the target positions _Xbtag , _Ybtag , and _Zbtag . Next, the CPU 91 offsets the target positions _Xbtag , _Ybtag , and _Zbtag by the positional deviations _ΔXb , _ΔYb , and _ΔZb , and then solves the inverse kinematics to calculate an angle command value _θJ1 * for the first joint axis J1 and an angle command value _θJ2 * for the second joint axis J2 for moving the hand camera 60 to the target positions _Xbtag , _Ybtag , and _Zbtag . Then, the CPU 91 controls the corresponding motors 31, 32 by feedback control so that the angle of the first joint axis J1 detected by the encoder 34 matches the calculated angle command value θ _J1 * and the angle of the second joint axis J2 detected by the encoder 44 matches the calculated angle command value θ _J2 *.

Next, the CPU 91 captures an image of the workpiece W supplied from the workpiece supply unit P with the hand camera 60, and recognizes the position of the workpiece W by performing image processing on the captured image. The CPU 91 then sets target positions X _btag , Y _btag of the hand (workpiece holding unit 24) for picking the workpiece W to the recognized position of the workpiece W, and moves the hand to the target positions X _btag , Y _btag , Z _btag to begin a picking operation in which the workpiece W is picked up. The target positions X _btag , Y _btag are set based on the recognized position of the workpiece W. The target position Z _btag is set based on height information of the workpiece W that has been input in advance.

In the collection operation, the CPU 91 first calculates the angle θ _J1 of the first joint _axis J1, the angle θ J2 of the second joint axis J2, the angle θ _J4 of the shaft 23 (shaft angle), _and the lift position _Zs of the shaft 23 by inverse kinematics, for moving the hand to a set target position (X btag , _{Y btag} , Z btag ). Next, the CPU 91 calculates the estimated positions X best , Y best , and Z best of the hand by forward kinematics based on the calculated θ _J1 , θ _J2 , θ _J4 , and _Zs, taking into account error factors such as the twist and deflection of the first arm 21 and the deflection of the second arm 22. Next, the CPU 91 calculates the positional deviation amounts ΔX _{b ,} ΔY _{b ,} and ΔZ _b by taking the differences between the estimated positions X _best , Y _best , and Z _best of the hand and the target positions X _btag , Y _btag , and _{Z btag} . Next, the CPU 91 offsets the target positions _Xbtag , _Ybtag , _Zbtag by the positional deviations _ΔXb , _ΔYb , _ΔZb, and then solves the inverse kinematics to calculate an angle command value _θJ1 * for the first joint axis J1, an angle command value _θJ2 * for the second joint axis J2, an angle command value _θJ4 * for the shaft 23, and a lift position command value _{Zs *} for the shaft 23 for moving the hand to the target positions _Xbtag , _Ybtag , Zbtag.The CPU 91 then controls the corresponding motors 31, 32, 33a, 33b by feedback control based on each command value.

In this way, the robot system 1 recognizes the position of the workpiece W by capturing an image of the workpiece W with the hand camera 60 attached to the robot arm 20, and moves the hand of the robot arm 20 to the recognized position to pick up the workpiece W. For this reason, unless the positional relationship between the robot system 1 and the hand camera 60 is properly grasped, the position of the workpiece W cannot be accurately recognized by the hand camera 60. In other words, the hand of the robot system 1 cannot properly pick up the workpiece W. Therefore, in this embodiment, the positional deviation amount of the hand camera 60 from the designed position is measured in advance, and the measured positional deviation amount is reflected in the control of the workpiece position recognition operation, thereby enabling the position of the workpiece W to be accurately recognized. It is assumed that the robot system 1 has been calibrated.

Figure 5 is an explanatory diagram showing the hand camera position measurement procedure. In step S10, a marked end effector EE (mark pin) is attached to the tip of the shaft 23. Figure 6 is an external view of the marked end effector EE. The marked end effector EE is a pin-shaped member that extends vertically when attached to the shaft 23. A mark M is provided on the tip of the pin of the marked end effector EE.

Step S20 causes the robot system 1 to execute a work camera position measurement process for measuring the hand camera position, which measures the position of the work camera 70 in the world coordinate system _Σw (work camera position _Xwwc , _Ywwc , _θwwc for measuring the hand camera position). FIG. 7 is a flowchart showing an example of the work camera position measurement process for measuring the hand camera position, which is executed by the CPU 91 of the control device 90. In the work camera position measurement process for measuring the hand camera position, the CPU 91 first accepts an input of a control point Zm (eigenvalue) of the marked end effector EE from the operator (step S100). The control point Zm is the distance between the tip of the shaft 23 and the tip (mark M) of the marked end effector EE. The coordinates of the tip of the marked end effector EE are set at a position Zm away from the origin of the mechanical interface coordinate system _Σm in the Z-axis direction downward. Next, the CPU 91 waits until the operator presses the work camera position measurement button (step S110). When the CPU 91 determines that the work camera position measurement button has been pressed, it moves the marked end effector EE above the work camera 70 (step S120), images the mark M provided on the tip surface of the marked end effector EE with the work camera 70, and measures the position of the work camera 70 (work camera positions X _wwc , Y _wwc , θ 8 is an explanatory diagram showing how to measure the position of the work camera for measuring _the hand camera position. As shown in the figure, the CPU 91 captures the mark M with the work camera 70 while moving the tip of the marked end effector EE in the X-axis direction and the Y-axis direction so as to pass through the design center position of the work camera 70. The CPU 91 acquires the movement trajectories of the mark M in the X-axis direction and the Y-axis direction, respectively, captured in the acquired captured image. Then, as shown in FIG. 9, the CPU 91 calculates the positional deviation amounts Δx and Δy of the work camera 70 by calculating the difference between the image center of the captured image and the intersection point of each of the acquired movement trajectories in the X-axis direction and the Y-axis direction of the captured image. Furthermore, as shown in FIG. 9, the CPU 91 calculates the rotational deviation amount Δθ of the work camera 70 based on the difference in the rotational direction between the X-axis direction of the captured image and the X-axis direction of the movement trajectory. Then, the CPU 91 determines the position offset by the positional deviation amounts Δx and Δy from the design center position of the work camera 70 as the work camera positions X wwc , Y _wwc , Y wwc . The CPU 91 determines the work camera position Xwwc, _Ywwc , _θwwc by measuring the work camera position _Xwwc , _Ywwc , _θwwc in the rotational direction, and registers the measured work camera positions _Xwwc , _Ywwc , _θwwc in the storage device 94 (step S140), and ends this process.

In step S30, a hole jig 80 for measuring the hand camera position is attached to the opening 71a provided at the top of the storage box 71. FIG. 10 is an explanatory diagram showing how the hole jig is attached. As shown in the figure, the hole jig 80 is fixed by a fixing device 82 in a state where it is fitted into the opening 71a of the storage box 71. The hole jig 80 has a center hole 81a located directly above the work camera 70 and penetrating vertically, and three-point holes 81b located around the center hole 81a at intervals in the circumferential direction and penetrating vertically. The number of holes in the three-point holes 81b is not limited to three, and may be two, or may be four or more.

Step S40 causes the robot system 1 to execute a hole position measurement process for measuring the hand camera position, in which the work camera 70 measures the position (center hole position _Xwwv , _Ywwv , _Zwwv ) of the center hole 81a of the hole jig 80 in the world coordinate system _Σw . FIG. 11 is a flowchart showing an example of the hole position measurement process for measuring the hand camera position executed by the CPU 91 of the control device 90. In the hole position measurement process for measuring the hand camera position, the CPU 91 first waits until the hand camera position measurement button is pressed by the operator (step S200). When the CPU 91 determines that the hand camera position measurement button has been pressed, it images the hole jig 80 with the work camera 70 and measures the distance a _m of the three holes 81b of the hole jig 80 from the obtained image (step S210). FIG. 12 is an explanatory diagram showing the state in which the work camera images the three holes of the hole jig. The distance a _m of the three-point holes 81b is measured by measuring the distance between the three-point holes 81b in the captured image. The CPU 91 then calculates the center hole position Z _wwv for measuring the hand camera position using the following formula (1) based on the measured distance a _m of the three-point holes 81b (step S220), and registers the calculated center hole position Z _wwv in the storage device 94 (step S230). In formula (1), a ₀ indicates the designed distance of the three-point holes 81b, WD indicates the working distance of the work camera 70, and Z indicates the imaging height of the work camera 70.

Next, the CPU 91 measures the central hole positions _Xwwv , _Ywwv in the world coordinate system _Σw of the hole jig 80 from the captured image obtained in step S210 (step S240). Then, the CPU 91 registers the measured central hole positions _Xwwv , _Ywwv in the storage device 94 (step S250), and ends this process.

Step S50 causes the robot system 1 to execute a hand camera angle measurement process for measuring the angle (position θ _ci ) of the hand camera 60. Fig. 13 is a flowchart showing an example of the hand camera angle measurement process executed by the CPU 91 of the control device 90. In the hand camera angle measurement process, the CPU 91 first captures an image of the hole jig 80 for measuring the hand camera position with the work camera 70, measures the phase θ _wwv of the three-point hole 81b of the hole jig 80 in the world coordinate system Σ _w (step S300), and registers the measured three-point hole phase θ _wwv in the storage device 94 (step S310). The measurement of the three-point hole phase θ _wwv is performed by determining the angle of the triangle connecting each hole of the three-point hole 81b shown in the obtained captured image with respect to the X-axis. 14, the CPU 91 moves the control point of the hand camera 60 to the work camera positions _Xwwc , _Ywwc for measuring the hand camera position registered in step S20 and the central hole position _Zwwv for measuring the hand camera position registered in step S40 (step S320). Next, the CPU 91 images the hole jig 80 with the hand camera 60, and measures the phase _θwhv of the three-point hole 81b of the hole jig 80 in the world coordinate system _Σw based on the captured image obtained in the same manner as in step S300 (step S330). Then, the CPU 91 calculates the hand camera position _θci based on the three-point hole phase _θwwv measured using the work camera 70 in steps S300 and S310 and the three-point hole phase _θwhv measured using the hand camera 60 in step S330 (step S340), registers the calculated hand camera position _θci in the storage device 94 (step S350), and ends this process. 15 is an explanatory diagram showing the three-point hole phase θ measured by the work camera and the hand camera. As shown in the figure, the hand camera position θ _ci is calculated by calculating the difference θe (=θ wwv -θ _whv ) between the three-point hole phase θ _wwv measured from below by the work camera 70 and the three-point hole _phase θ _whv measured from above by the hand camera 60, and offsetting the designed hand camera position (eigenvalue) in the rotational direction by the difference θe.

In step S60, the robot system 1 executes a hand camera position measurement process for measuring the position of the hand camera 60 (hand camera positions X _ci , Y _ci ). FIG. 16 is a flowchart showing an example of the hand camera position measurement process executed by the control device. In the hand camera position measurement process, the CPU 91 first waits until the hand camera position measurement button is pressed by the operator (step S400). When the CPU 91 determines that the hand camera position measurement button has been pressed, it moves the control point of the hand camera 60 to the work camera positions X _wwc , Y _wwc for measuring the hand camera position registered in step S20 and the central hole position Z _wwv for measuring the hand camera position registered in step S40 (step S410). Next, the CPU 91 images the hole jig 80 with the hand camera 60, and measures the central hole positions X _whv , Y _whv from the obtained captured image (step S420). Next, the CPU 91 calculates the distance L between the image center of the captured image and the measured central hole positions X _whv and Y _whv as shown in FIG. 18 (step S430). The distance L can be calculated by the following formula (2) where the image center is the origin, the central hole position X _whv is a, and the central hole position Y _whv is b. Then, the CPU 91 judges whether the calculated distance L is equal to or less than a predetermined allowable value Lref (step S440). If the CPU 91 judges that the distance L is not equal to or less than the allowable value Lref, the CPU 91 returns to step S410 and repeats the process until the distance L becomes equal to or less than the allowable value Lref, and when the distance L becomes equal to or less than the allowable value Lref, the CPU 91 proceeds to step S450. It should be noted that the CPU 91 may proceed to step S450 regardless of whether the distance L is equal to or less than the allowable value Lref.

When the CPU 91 determines that the distance L is equal to or less than the allowable value Lref, it moves the control point of the marked end effector EE (mark pin) to the work camera positions _Xwwc , _Ywwc for measuring the hand camera position registered in step S20 and the hole position _Zwwv for measuring the hand camera position registered in step S40 (step S450). Next, the CPU 91 rotates the marked end effector EE to angles of 0°, 90°, 180°, and 270°, respectively, and captures an image of the mark M with the work camera 70 at each angle to _measure the mark pin positions (X0, y0), (X90, Y90), (X180, Y180), and (X270, Y270) in the world coordinate system Σw (step S460). Next, the CPU 91 calculates the average values _Xmc , _Ymc of the mark pin positions (X0, Y0), (X90, Y90), (X180, Y180), and (X270, Y270) using the following formula (3). Furthermore, the CPU 91 calculates the difference between the hand camera position measurement center hole positions _Xwwc , _Ywwc measured by the work camera 70 in step S40 and the mark pin positions (average values) _Xmc , _Ymc using the following formula (4) to calculate the positional deviation amounts _Xme , _Yme from the hand camera position measurement hole center positions _Xwwv , _Ywwv of the end effector (step S190). Figure 19 shows the relationship between the mark pin positions (average values) _Xmc , _Ymc , the hand camera position measurement hole center positions _Xwwv , _Ywwv , and the positional deviation amounts _Xme , _Yme .

Then, the CPU 91 converts the coordinate system of the positional deviations _Xme and _Yme from the world coordinate system _Σw to the camera coordinate system _Σc , and calculates new hand camera positions _Xci and _Yci by offsetting the designed hand camera positions _Xmo and _Ymo (eigenvalues) by the converted positional deviations _Xrme and _Yrme (step S470). In order to convert the coordinate system of the positional deviations _Xme and _Yme from the world coordinate system _Σw to the camera coordinate system _Σc , it is sufficient to rotate the positional deviations _Xme and _Yme by the phase difference θrm between the world coordinate system _Σw and the camera coordinate system _Σc . If the phase of the base origin of the robot 10 in the world coordinate system _Σw is _θrw , and the rotation angles of the first joint axis J1 and the second joint axis J2 when the hand camera 60 captures an image of the center hole 81a of the hole jig 81 are _θJ1 and _θJ2 , the phase difference θrm is calculated by the following formula (5). Therefore, the post-conversion positional deviation amounts _Xrme , _Yrme can be calculated using the rotation matrices of the following equations (6) and (7). The new hand camera positions _Xmm , _Ymm can be calculated by the following equation (8) using the eigenvalues of the hand camera positions _Xmo , _Ymo , which become the hand camera positions _Xci , _Yci . After calculating the new hand camera positions _Xci , _Yci in this way, the CPU 91 registers the calculated hand camera positions _Xci , _Yci in the storage device 94 (step S480).

Furthermore, the CPU 91 calculates the hand camera position change amount E by the following formula (9) (step S490), and determines whether or not the hand camera position change amount E is equal to or less than a predetermined allowable value Eref (step S500). If the CPU 91 determines that the hand camera position change amount E is not equal to or less than the allowable value Eref, the process returns to step S400. If the CPU 91 determines that the hand camera position change amount E is equal to or less than the allowable value Eref, the process ends. As a result, the accurate hand camera positions X _ci , Y _ci , and θ _ci are stored in the storage device 94. Therefore, by controlling the position of the hand camera 60 using the hand camera positions X _ci , Y _ci , and θ _ci stored in the storage device 94, the CPU 91 can accurately grasp the position of the workpiece W (object) imaged by the hand camera 60, and therefore it becomes possible to perform work on the workpiece W more accurately.

Step S70 is to remove the marked end effector EE and the hole jig 80 for measuring the hand camera position. This completes the hand camera position measurement procedure.

Here, the correspondence between the main elements of the embodiment and the main elements of the present disclosure described in the claims will be described. That is, in this embodiment, the robot 10 corresponds to the robot, the hand camera 60 corresponds to the hand camera, the work camera 70 corresponds to the external camera, the hole jig 80 corresponds to the jig member, the center hole 81a corresponds to the first measurement hole, the CPU 91 of the control device 90 that executes the processing of steps S240 and S250 of the hole position measurement processing for hand camera position measurement corresponds to the first position acquisition unit, the CPU 91 of the control device 90 that executes the processing of steps S410 and S420 of the hand camera position measurement processing corresponds to the second position acquisition unit, and the CPU 91 of the control device 90 that executes the processing of steps S450 to S480 of the hand camera position measurement processing corresponds to the position deviation measurement unit. Moreover, the CPU 91 of the control device 90 that executes the work camera position measurement process for measuring the hand camera position corresponds to an external camera position acquisition unit, the CPU 91 of the control device 90 that executes the processes of steps S300 and S310 of the hand camera angle measurement process corresponds to a first phase acquisition unit, the CPU 91 of the control device 90 that executes the processes of steps S320 and S330 of the hand camera angle measurement process corresponds to a second phase acquisition unit, and the CPU 91 of the control device 90 that executes the processes of step S340 of the hand camera angle measurement process corresponds to a phase shift measurement unit. Moreover, the CPU 91 of the control device 90 that executes the processes of steps S210 and S220 of the hole position measurement process for measuring the hand camera position corresponds to a height acquisition unit.

It goes without saying that the present disclosure is in no way limited to the above-described embodiments, and may be implemented in various forms as long as they fall within the technical scope of the present disclosure.

For example, in the above-described embodiment, the robot 10 is configured as a horizontal multi-joint robot (SCARA robot), but this is not limited to this and may be configured in any other manner, such as a vertical multi-joint robot.

As described above, the camera positional deviation measuring device disclosed herein is a camera positional deviation measuring device for measuring the positional deviation of the hand camera in a robot system including a robot having a robot arm and a hand camera attached to the robot arm, and an external camera installed outside the robot, and includes: a jig member installed above the external camera and having a first measurement hole that penetrates vertically; a first position acquisition unit that images the jig member with the external camera and acquires the position of the first measurement hole from the obtained captured image of the first measurement hole; a second position acquisition unit that moves the hand camera to the position acquired by the first position acquisition unit, images the jig member with the hand camera, and acquires the position of the first measurement hole from the obtained captured image of the first measurement hole; and a positional deviation measuring unit that moves the robot's hand to the position of the first measurement hole acquired by the second position acquisition unit, images the robot's hand with the external camera, and measures the positional deviation of the camera from the obtained captured image.

The camera positional deviation measuring device disclosed herein includes a jig member that is installed above the external camera and has a first measurement hole that penetrates vertically. Furthermore, the positional deviation measuring device images the jig member with the external camera to obtain the position of the first measurement hole, moves the hand camera to the obtained position and images the jig member with the hand camera to obtain the position of the first measurement hole. The positional deviation measuring device then moves the robot's hand to the obtained position of the first measurement hole and images the robot's hand with the external camera to measure the positional deviation of the camera. This allows the positional deviation of the camera attached to the robot's arm to be properly measured.

In the camera positional deviation measuring device of the present disclosure, a jig member is installed above the external camera and has a plurality of second measurement holes that penetrate vertically in addition to the first measurement holes; an external camera position acquisition unit that moves the hand of the robot above the external camera to capture an image of the hand with the external camera and acquires the position of the external camera from the captured image of the hand; a first phase acquisition unit that images the jig member with the external camera and acquires the phase of the second measurement hole from the captured image of the second measurement hole; a second phase acquisition unit that moves the hand camera to the position of the external camera acquired by the external camera position acquisition unit, captures an image of the jig member with the hand camera, and acquires the phase of the second measurement hole from the captured image of the second measurement hole; and a phase deviation measurement unit that measures the phase deviation of the hand camera based on the phase of the second measurement hole acquired by the first phase acquisition unit and the second phase acquisition unit, respectively. In this way, the jig member and the external camera can be used to appropriately measure the phase deviation in addition to the position deviation of the hand camera. The second measurement holes may be formed at intervals in the circumferential direction around the first measurement hole of the jig member.

The camera positional deviation measuring device of the present disclosure further includes a jig member that is installed above the external camera and has a plurality of second measurement holes that penetrate vertically in addition to the first measurement holes, and a height acquisition unit that images the jig member with the external camera and acquires the heights of the first measurement holes from the acquired image of the second measurement holes, and the second position acquisition unit may move the hand camera to the position acquired by the first position acquisition unit and the height acquired by the height acquisition unit. In this way, even if the installation height of the jig member includes an error, the positional deviation of the hand camera can be measured satisfactorily.

Furthermore, the camera positional deviation measuring device of the present disclosure includes a jig member that is installed above the external camera and has a plurality of second measurement holes that penetrate vertically in addition to the first measurement holes, and a height acquisition unit that images the jig member with the external camera and acquires the height of the first measurement holes from the acquired image of the second measurement holes, and the positional deviation measuring unit may move the hand camera to the position acquired by the second position acquisition unit and the height acquired by the height acquisition unit. In this way, even if the installation height of the jig member includes an error, the positional deviation of the hand camera can be measured satisfactorily.

Although this disclosure is in the form of a camera position deviation measuring device, it may also be in the form of a camera position deviation measuring method.

This disclosure can be used in the robot system manufacturing industry, etc.

1 robot system, 2 work table, 3a main pole, 3b sub pole, 10 robot, 11 base, 20 robot arm, 21 first arm, 22 second arm, 23 shaft, 24 workpiece holder, 30 first arm drive unit, 31 motor, 32 motor, 33a motor, 33b motor, 34 encoder, 40 second arm drive unit, 42 motor, 44 encoder, 50 shaft drive unit, 52a motor, 52b motor, 54a encoder, 54b encoder, 60 hand camera, 70 workpiece camera, 71 storage box, 71a opening, 80 hole jig, 81a center hole, 81b three-point hole, 82 fixture, 90 control device, 91 CPU, 92 ROM, 93 RAM, 94 storage device, EE end effector with mark, J1 First joint axis, J2 second joint axis, J3 third joint axis, M mark, P work supply part, S board.

Claims (6)

1. A camera position deviation measuring device for measuring a position deviation of a hand camera in a robot system including a robot having a robot arm and a hand camera attached to the robot arm, and an external camera installed outside the robot, comprising:
a jig member disposed above the external camera and having a first measurement hole extending vertically therethrough;
a first position acquisition unit that captures an image of the jig member using the external camera and acquires a position of the first measurement hole from the captured image of the first measurement hole;
a second position acquisition unit that moves the hand camera to the position acquired by the first position acquisition unit, captures an image of the jig member with the hand camera, and acquires the position of the first measurement hole from the captured image of the first measurement hole;
a positional deviation measuring unit that moves a hand of the robot to the position of the first measurement hole acquired by the second position acquisition unit, captures an image of the hand of the robot with the external camera, and measures a positional deviation of the camera from the captured image;
A camera position deviation measuring device comprising: 2. The camera position deviation measuring device according to claim 1,
a jig member that is installed above the external camera and has a plurality of second measurement holes that penetrate vertically in addition to the first measurement holes;
an external camera position acquisition unit that moves a hand of the robot above the external camera, captures an image of the hand with the external camera, and acquires a position of the external camera from the captured image of the hand;
a first phase acquisition unit that captures an image of the jig member using the external camera and acquires a phase of the second measurement hole from the captured image of the second measurement hole;
a second phase acquisition unit that moves the hand camera to the position of the external camera acquired by the external camera position acquisition unit, images the jig member with the hand camera, and acquires a phase of the second measurement hole from the obtained captured image of the second measurement hole;
a phase shift measurement unit that measures a phase shift of the hand camera based on the phases of the second measurement hole acquired by the first phase acquisition unit and the second phase acquisition unit, respectively;
A camera position deviation measuring device comprising: 3. The camera position deviation measuring device according to claim 1,
a jig member that is installed above the external camera and has a plurality of second measurement holes that penetrate vertically in addition to the first measurement holes;
a height acquisition unit that images the jig member using the external camera and acquires a height of the first measurement hole from the captured image of the second measurement hole;
Equipped with
the second position acquisition unit moves the hand camera to the position acquired by the first position acquisition unit and to the height acquired by the height acquisition unit;
Camera position deviation measuring device. 4. The camera position deviation measuring device according to claim 1,
a jig member that is installed above the external camera and has a plurality of second measurement holes that penetrate vertically in addition to the first measurement holes;
a height acquisition unit that images the jig member using the external camera and acquires a height of the first measurement hole from the captured image of the second measurement hole;
Equipped with
the positional deviation measurement unit moves the hand camera to the position acquired by the second position acquisition unit and to the height acquired by the height acquisition unit;
Camera position deviation measuring device. 1. A method for measuring a position shift of a camera in a robot system including a robot having a robot arm and a hand camera attached to the robot arm, and an external camera installed outside the robot, the method comprising:
a jig member having a first measurement hole penetrating vertically is placed above the external camera;
acquiring a position of the first measurement hole from an image of the first measurement hole obtained by imaging the jig member with the external camera, moving the hand camera to the acquired position, imaging the jig member with the hand camera, acquiring a position of the first measurement hole from an image of the first measurement hole obtained, moving the tip of the robot to the acquired position of the first measurement hole, imaging the tip of the robot with the external camera, and measuring a positional deviation of the camera from the acquired image.
How to measure camera position deviation. 6. The camera position deviation measuring method according to claim 5,
a jig member having a plurality of second measurement holes that are vertically penetrating in addition to the first measurement holes is installed above the external camera;
moving a hand of the robot above the external camera to image the hand with the external camera and obtain the position of the external camera from the image of the hand; imaging the jig member with the external camera and obtaining the phase of the second measurement hole from the image of the second measurement hole; moving the hand camera to the position of the obtained external camera and imaging the jig member with the hand camera and obtaining the phase of the second measurement hole from the image of the second measurement hole; measuring a phase shift of the hand camera based on the phase of the second measurement hole acquired by the external camera and the hand camera, respectively.
How to measure camera position deviation.

Images