JP7744979B2 - Camera position deviation correction method and robot device - Google Patents

Description

This specification discloses a method for correcting camera position deviation and a robotic device.

A robot system has been proposed that includes a robot with a robot arm and a robot camera, and an external camera installed separately from the robot, and that captures an image of a calibration pattern with a pre-calibrated robot camera and also captures an image of the calibration pattern with the external camera to determine calibration data for the external camera (see, for example, Patent Document 1). This system captures an image of the calibration pattern with both the robot camera and the external camera, acquiring pattern images, and then determines calibration data for the external camera based on the acquired pattern images and known calibration data for the robot camera.

Japanese Patent Application Laid-Open No. 2019-69493

However, while the technology described in Patent Document 1 above describes calibrating an external camera, it makes no mention of calibrating a robot camera.

The main purpose of this disclosure is to appropriately correct the positional misalignment of a camera attached to a robot arm using a simple method.

This disclosure takes the following measures to achieve the above-mentioned main objective.

The camera positional deviation correction method disclosed herein is a method for correcting the positional deviation of a camera relative to an arm in a robot device having an arm and a camera attached to the arm. The method involves controlling the arm and the camera so that an image of a single mark member is captured by the camera, determining the position of the mark member in a base coordinate system based on the captured image, and pre-storing the determined position of the mark member as a reference position. When correction of the positional deviation is requested, the arm and the camera are controlled so that the camera captures an image of the mark member using a first position above the mark member as the imaging position, and pre-storing the position of the mark member in the base coordinate system based on the captured image. a first correction process for determining the position of the mark member in the base coordinate system and correcting the horizontal positional relationship of the camera with respect to the arm so that the determined position of the mark member coincides with the reference position; after performing the first correction process, the arm and the camera are controlled so that the camera captures an image of the mark member, with a second position horizontally offset in a predetermined direction from the first position being used as an imaging position; and a second correction process for determining the position of the mark member in the base coordinate system based on the obtained captured image and correcting the positional relationship of the camera with respect to the arm in the rotational direction so that the determined position of the mark member coincides with the reference position.

The camera misalignment correction method disclosed herein controls the arm and camera to capture an image of a single mark member with the camera, determines the position of the mark member in a base coordinate system based on the captured image, and pre-stores the determined position of the mark member as a reference position. When a request for misalignment correction is received, the method executes a first correction process and a second correction process to correct the misalignment of the camera relative to the arm. The first correction process controls the arm and camera to capture an image of the mark member with the camera using a first position above the mark member as the imaging position. The first correction process determines the position of the mark member in the base coordinate system based on the captured image and corrects the horizontal positional relationship of the camera with the arm so that the position of the mark member coincides with the reference position. The second correction process controls the arm and camera to capture an image of the mark member with the camera using a second position horizontally offset from the first position in a predetermined direction. The second correction process determines the position of the mark member in the base coordinate system based on the captured image and corrects the rotational positional relationship of the camera with the arm so that the position of the mark member coincides with the reference position. As a result, if the position of the mark member is measured using an arm or camera with guaranteed accuracy before the robot device is shipped and stored in advance as a reference position, even if an error occurs in the positional relationship between the arm and camera of the robot device at the shipping destination after the robot device is shipped, the positional relationship between the arm and camera can be corrected on site without the need for a special measuring jig.

The robot device disclosed herein comprises an arm, a camera attached to the arm, one mark member, a storage device that pre-stores the position of the mark member in a base coordinate system as a reference position, and a control device that, when a request is made to correct a positional deviation of the camera relative to the arm, controls the arm and the camera to capture an image of the mark member using the camera at a first position above the mark member as an imaging position, and performs a first correction process to determine the position of the mark member in the base coordinate system based on the captured image and correct the horizontal positional relationship of the camera relative to the arm so that the determined position of the mark member coincides with the reference position; after performing the first correction process, controls the arm and the camera to capture an image of the mark member using the camera at a second position offset horizontally in a predetermined direction from the first position as an imaging position, and performs a second correction process to determine the position of the mark member in the base coordinate system based on the captured image and correct the rotational positional relationship of the camera relative to the arm so that the determined position of the mark member coincides with the reference position.

The robot device disclosed herein comprises an arm, a camera, one marking member, a memory device, and a control device. The position of the marking member in the base coordinate system is pre-stored in the memory device as a reference position. When a request is made to correct the positional deviation of the camera relative to the arm, the control device executes a first correction process and a second correction process similar to the camera positional deviation correction method disclosed herein above. As a result, similar to the camera positional deviation correction method disclosed herein, even if an error occurs in the positional relationship between the arm and camera of the robot device at the shipping destination after shipping, the positional relationship between the arm and camera can be corrected on-site without the need for special measuring tools.

FIG. 1 is an external perspective view of a robot system. FIG. 2 is a block diagram showing the electrical connection relationship between the robot device and a control device. FIG. 2 is an explanatory diagram illustrating a base coordinate system, a camera coordinate system, and a mechanical interface coordinate system. 10 is a flowchart illustrating an example of a reference position registration process. 10 is a flowchart illustrating an example of a hand camera position correction process. FIG. 10 is an explanatory diagram showing the field of view of each hand camera when capturing images of a mark at two locations.

Next, the form for implementing this disclosure will be described with reference to the drawings.

Figure 1 is an external perspective view of the robot system. Figure 2 is a block diagram showing the electrical connection between the robot device and the control device.

The robot system 1 is a work robot system that, for example, picks up workpieces W (components) supplied by a workpiece supply unit P and mounts them on a target object (substrate S). As shown in Figures 1 and 2, the robot system 1 comprises a workbench 2, a robot device 10 having a robot arm 20, a hand camera 60 attached to the robot arm 20, a workpiece camera 70 as an external camera, and a control device 90 that controls the robot device 10. A pole-shaped marking member 3 extending vertically is installed on the workbench 2.

In this embodiment, the robot device 10 is a SCARA robot and comprises a base 11 and a robot arm 20.

The base 11 is fixed to the workbench 2 and supports the base end of the robot arm 20. The robot arm 20 includes a first arm 21, a first arm driver 30, a second arm 22, a second arm driver 40, a shaft 23, and a shaft driver 50. The base end of the first arm 21 is connected to the base 11 via a first joint axis J1 and is configured to be rotatable (horizontally pivotable) within a horizontal plane relative to the base 11 by rotation of the first joint axis J1. The base end of the second arm 22 is connected to the tip end of the first arm 21 via a second joint axis J2 and is configured to be rotatable (horizontally pivotable) within a horizontal plane relative to the first arm 21 by rotation of the second joint axis J2. The shaft 23 is connected to the tip end of the second arm 22 via a third joint axis J3 and is configured to be rotatable around the axis of the third joint axis J3 relative to the second arm 22 and to be movable up and down along the axial direction of the third joint axis J3. In the robot system 1 of this embodiment, a workpiece holding unit 24 that serves as an end effector and picks up and holds the workpiece W is provided at the tip of the shaft 23. Examples of the workpiece holding unit 24 include a suction nozzle that attracts the workpiece W by negative pressure, a mechanical chuck that grips the workpiece W with a pair of claws, and an electromagnetic chuck that attracts the workpiece W by an electromagnet.

The first arm driving unit 30 includes a motor 32 and an encoder 34. The rotation shaft of the motor 32 is connected to the first joint axis J1 via a reducer (not shown). By driving the motor 32, the first arm driving unit 30 rotates the first arm 21 around the first joint axis J1 as a fulcrum using torque transmitted to the first joint axis J1 via the reducer. The encoder 34 is attached to the rotation shaft of the motor 32 and is configured as a rotary encoder that detects the amount of rotational displacement of the motor 32.

Like the first arm driver 30, the second arm driver 40 includes a motor 42 and an encoder 44. The rotation shaft of the motor 42 is connected to the second joint shaft J2 via a reducer (not shown). By driving the motor 42, the second arm driver 40 transmits torque to the second joint shaft J2 via the reducer, causing the second arm 22 to rotate around the second joint shaft J2 as a fulcrum. The encoder 44 is attached to the rotation shaft of the motor 42 and is configured as a rotary encoder that detects the amount of rotational displacement of the motor 42.

The shaft drive unit 50 comprises motors 52a and 52b and encoders 54a and 54b. The rotational axis of motor 52a is connected to shaft 23 via a belt (not shown). By driving motor 52a, the shaft drive unit 50 rotates shaft 23 around its axis. The rotational axis of motor 52b is connected to shaft 23 via a ball screw mechanism (not shown). By driving motor 52b, the shaft drive unit 50 converts the rotational motion of motor 52b into linear motion via the ball screw mechanism, thereby raising and lowering shaft 23. Encoder 54a is configured as a rotary encoder that detects the amount of rotational displacement of shaft 23. Encoder 54b is configured as a linear encoder that detects the raised and lowered position of shaft 23.

2, the control device 90 includes a CPU 91, a ROM 92 that stores processing programs, a RAM 93 as work memory, a storage device 94 such as an HDD or SSD, and an input/output interface (not shown). Position signals from encoders 34, 44, 54a, and 54b, image signals from hand camera 60, and image signals from work camera 70 are input to the control device 90 via the input/output interface. Drive signals for motors 32, 42, 52a, and 52b, drive signals for hand camera 60, and drive signals for work camera 70 are output from the control device 90 via the input/output interface.

The hand camera 60 is attached to the tip of the second arm 22. The hand camera 60 captures an image of the workpiece W in the workpiece supply section P from above and outputs the captured image to the control device 90. The control device 90 processes the captured image to recognize the position of the workpiece W.

The work camera 70 is installed between the work supply unit P and the substrate S on the work table 2. The work camera 70 captures an image of the work W held by the work holding unit 24 of the robot device 10 from below and outputs the captured image to the control device 90. The control device 90 processes the captured image to determine whether the work W is being held properly by the work holding unit 24.

Next, the operation of the robot system 1 configured in this manner will be described. The operation of the robot system 1 includes a work position recognition operation, a collection operation (pick operation), a collection confirmation operation, and an implementation operation (place operation). In the work position recognition operation, the hand camera 60 captures an image of the work W to recognize the position of the work W. In the collection operation, the hand (work holding unit 24) of the robot system 1 is moved to the position of the recognized work W to collect the work W. In the collection confirmation operation, the work W collected by the hand is moved above the work camera 70, and an image of the work W is captured by the work camera 70 to check whether the collection status of the work W is good or bad. In the implementation operation, the collected work W is mounted on the board S.

Here, the coordinate systems of the robot system 1 will be described. In this embodiment, as shown in FIG. 2 , the robot system 1 has a base coordinate system _Σb , a camera coordinate system _Σc , and a mechanical interface coordinate system _Σm as its coordinate systems. The base coordinate system _Σb has its origin set at the bottom surface of the base 11 of the robot device 10, its X-axis set in the direction of the home position of the robot arm 20, its Z-axis set in the vertical direction, and its Y-axis set in a direction perpendicular to the X-axis and Z-axis. The mechanical interface coordinate system _Σm has its origin set at the hand of the robot arm 20, and its X-axis, Y-axis, and Z-axis are set so that their directions coincide with those of the corresponding axes of the base coordinate system _Σb . The camera coordinate system _Σc has its origin set at the bottom surface of the second arm 22, which is the center of the third joint axis J3, and its X-axis, Y-axis, and Z-axis are set so that their directions coincide with those of the corresponding axes of the base coordinate system _Σb . The base coordinate system _Σb , the camera coordinate system _Σc , and the mechanical interface coordinate system _Σm are mutually converted using a conversion matrix.

In the workpiece position recognition operation, the CPU 91 of the control device 90 first sets target positions X btag , Y btag , and Z _btag of the hand camera 60 (hand camera center) as viewed from the base 11 (base coordinate system Σ _b ) for capturing an image of _the workpiece _W supplied from the workpiece supply unit P. Next, the CPU 91 solves inverse kinematics for the target positions X _btag , Y _btag , and Z _btag of the hand camera 60 to calculate the angle θ _J1 of the first joint axis J1 and the angle θ _J2 of the second joint axis J2 for matching the position of the hand camera 60 with the target positions X _btag , Y _btag , and Z _btag . Here, the position of the hand camera 60 in the camera coordinate system Σ _c is registered in advance in the storage device 94 as hand camera positions X _ci , Y _ci , and θ _ci . Because the origin of the camera coordinate system _Σc is set at the center of the shaft 23 connected to the hand of _the robot arm 20, the CPU 91 can convert the position of the hand camera 60 in the base coordinate system _Σb to the position of the hand based on the hand camera positions _Xci , _Yci , and _θci , and can convert the position of the hand in _the base coordinate system _Σb to the position of the hand camera 60 based on _the hand camera positions _Xci , _Yci , and _θci . The CPU 91 can convert the target positions _Xbtag and Ybtag of the hand camera 60 in the base coordinate system _Σb to the target positions of the hand based on the hand camera positions Xci, Yci, _{and θci} , and calculate the angle _θJ1 of the first joint axis J1 and the angle _θJ2 of the second joint axis J2 by solving inverse kinematics based on the converted target positions of the hand.

Next, the CPU 91 calculates an estimated position of the hand by solving forward kinematics based on the angle θ _J1 of the first joint axis J1 and the angle θ _J2 of the second joint axis J2, taking into account error factors such as torsion and deflection of the first arm 21 and deflection of the second arm 22, and then calculates estimated positions X best , Y _best , and Z _best of the hand camera 60 from the calculated estimated position of the hand, based on the hand camera positions X _ci , Y _ci , _and θ _ci . That is, assuming that the angles θ _J1 , θ _J2 of the first and second joint axes _J1 , _J2 calculated by inverse kinematics are controlled as angle command values so that the position of the hand camera 60 coincides with the target positions X _btag , Y btag , Z btag , the CPU 91 calculates estimated positions X best , Y best , Z _best which are positions deviated from the target positions X _btag , Y _btag , Z _btag due to error factors such as twisting and bending of the first arm ₂₁ and bending of the second _arm 22.

After calculating the estimated positions _Xbest , _Ybest , and _Zbest , the CPU 91 calculates positional deviation amounts _ΔXb , ΔYb, and ΔZb by taking the differences between the calculated estimated positions _Xbtag , _Ybtag , and _Zbtag and the target positions _Xbtag , _Ybtag , and _Zbtag . Next, the CPU 91 offsets the target positions _Xbtag , _Ybtag , _{and Zbtag} by the positional deviation amounts _ΔXb , _ΔYb , and _{ΔZb, and} then solves the inverse kinematics to calculate an angle command value _θJ1 * for the first joint axis J1 and an angle command value _θJ2 * for the second joint axis J2 for moving the hand camera 60 to the target positions _Xbtag , _Ybtag , and _Zbtag . Then, the CPU 91 controls the corresponding motors 32, 42 by feedback control so that the angle of the first joint axis J1 detected by the encoder 34 matches the calculated angle command value θ _J1 * and the angle of the second joint axis J2 detected by the encoder 44 matches the calculated angle command value θ _J2 *.

Next, the CPU 91 captures an image of the workpiece W supplied from the workpiece supply unit P using the hand camera 60, and performs image processing on the captured image to recognize the position of the workpiece W in the base coordinate system _Σb . The image processing is performed by calculating the position of the marking member 3 in the camera coordinate system _Σc based on the pixel position of the marking member 3 shown in the captured image and the hand camera positions _Xci , _Yci , and _θci , and then converting the calculated position of the marking member 3 from the camera coordinate system _Σc to the base coordinate system _Σb based on the angle θJ1 of the first joint axis _J1 and the angle _θJ2 of the second joint axis J2. The CPU 91 then sets target positions _Xbtag and _Ybtag of the hand (workpiece holding unit 24) for picking the workpiece W to the position of the recognized workpiece W, and moves the hand to the target positions _Xbtag , _Ybtag , and _Zbtag to begin the picking operation. The target positions _Xbtag and _Ybtag are set based on the position of the recognized workpiece W. The target position Z _btag is set based on the height information of the workpiece W that has been input in advance.

In the collection operation, the CPU 91 first calculates the angle _θ J1 of _the first joint axis J1, the angle θ _J2 of the second joint axis J2, the angle θ _J4 of the shaft 23 (shaft angle), and the elevation position _Zs of the shaft 23 using inverse kinematics, which are required to move the hand to a set target position ( _{X btag} , Y _btag , Z btag ). Next, the CPU 91 calculates the estimated positions X best , Y best , and _{Z best} of the hand using forward kinematics based on the calculated θ _J1 , θ _J2 , θ _J4 , and _Zs , taking into account error factors such as torsion and deflection of the first arm 21 and deflection of the second arm 22. Next, the CPU 91 calculates the positional deviation amounts ΔX _{b , ΔY b ,} and ΔZ _b by taking the differences between the estimated positions X _best , Y _best , and Z _best of the hand and the target positions X _btag , Y _btag , and Z _btag . Next, the CPU 91 offsets the target positions X _btag , Y _btag , Z _btag by the positional deviations ΔX _b , ΔY _b , ΔZ _b and then solves the inverse kinematics to calculate an angle command value θ _J1 * for the first joint axis J1, an angle command value θ _J2 * for the second joint axis J2, an angle command value θ _J4 * for the shaft 23, and an elevation position command value Z _s * for the shaft 23 for moving _the hand to _{the target positions} X btag , _Y btag , Z btag. Then, the CPU 91 controls the corresponding motors 32, 42, 52a, 52b by feedback control based on each command value.

In this way, the robot system 1 captures an image of the workpiece W with the hand camera 60 attached to the robot arm 20, recognizes the position of the workpiece W, and then moves the endoscope of the robot arm 20 to the recognized position to pick up the workpiece W. Therefore, if the positional relationship between the endoscope of the robot arm 20 and the hand camera 60 (hand camera positions X _ci , Y _ci , θ _ci ) is not properly grasped, the hand camera 60 cannot accurately recognize the position of the workpiece W, resulting in a positional deviation when picking up the workpiece W. The robot system 1 of this embodiment is shipped after measuring the positional relationship between the endoscope of the robot arm 20 and the hand camera 60 using a dedicated measuring jig. However, if the robot arm 20 collides with a peripheral device or the like at the shipping destination, an error will occur in the positional relationship between the endoscope of the robot arm 20 and the hand camera 60 due to deformation or misalignment of components constituting the robot arm 20. Because dedicated measuring jigs are expensive, it is not practical to remeasure the positional relationship using a dedicated measuring jig on-site if an error occurs in the positional relationship between the endoscope of the robot arm 20 and the hand camera 60 at the shipping destination.

In the robot system 1 of this embodiment, the workbench 2 is provided with a marking member 3, and before shipping, a dedicated measuring jig is used to measure the positional relationship between the hand of the robot arm 20 and the hand camera 60 (hand camera positions _Xci , _Yci , _θci ). The hand camera 60 then images the marking member 3, measures its position (mark positions _Xb , _Yb ), and stores the measured mark positions _Xb , _Yb as reference positions _Xr , _Yr in the storage device 94. When an error occurs in the positional relationship between the hand of the robot arm 20 and the hand camera 60 at the shipping destination, the robot system 1 images the marking member 3 with the hand camera 60, measures the mark positions _Xb , _Yb , and re-corrects the hand camera positions _Xci , _Yci , _θci so that they are the same as the reference positions _Xr , _Yr that were measured and stored in advance before the error occurred. This makes it possible to re-correct the positional relationship between the hand of the robot arm 20 and the hand camera 60 without using an expensive measuring jig. Hereinafter, first, the reference position registration process for registering the reference position will be described in detail, and then the hand camera position correction process for re-correcting the position of the hand camera 60 will be described in detail.

FIG. 4 is a flowchart showing an example of the reference position registration process. The reference position registration process is executed before shipping of the robot system 1. In the reference position registration process, the CPU 91 of the control device 90 first determines whether the reference position measurement button has been pressed (step S100). If the CPU 91 determines that the reference position measurement button has not been pressed, it simply ends the reference position registration process. On the other hand, if the CPU 91 determines that the reference position measurement button has been pressed, it moves the hand camera 60 to a target position (image capture position) set above the mark member 3, captures an image of the mark member 3 with the hand camera 60, and performs image processing on the captured image to calculate the position of the mark member 3 in the base coordinate system _Σb (mark position _Xb , _Yb ) (step S110). As described above, the image processing is performed by calculating the position of the mark member 3 in the camera coordinate system _Σc based on the pixel position of the mark member 3 shown in the captured image and the hand camera positions _Xci , _Yci , and _θci, and then converting from the camera coordinate system _Σc to the base coordinate system _Σb . After measuring the mark positions _Xb , _Yb , the CPU 91 registers the measured mark positions _Xb , _Yb as reference positions _Xr , _Yr in the storage device 94 (step S120), and ends the reference position registration process.

Figure 5 is a flowchart showing an example of the hand camera position correction process. As described above, the hand camera position correction process is executed when the robot arm 20 collides with a peripheral device or the like, resulting in an error in the positional relationship between the hand of the robot arm 20 and the hand camera 60. Note that if a collision with a peripheral device or the like causes deformation or the like in a component constituting the robot arm 20, a positional deviation may occur in the hand of the robot arm 20. In this case, prior to the hand camera position correction process, the positional deviation of the hand of the robot arm 20 based on the deformation or the like of the component is first corrected. The positional deviation of the hand can be determined, for example, by attaching a marking member to the hand of the robot arm 20, capturing an image of the marking member with the work camera 70, and measuring the position of the marking member. For example, the positional deviation of the hand can be determined by setting a target position for the hand above the work camera 70, moving the hand to the target position, capturing an image of the marking member attached to the hand with the work camera 70, measuring the position of the marking member, and calculating the difference between the measured position of the marking member and the target position.

In the hand camera position correction process, the CPU 91 first determines whether the hand camera position correction button has been pressed (step S200). If the CPU 91 determines that the hand camera position correction button has not been pressed, it terminates the hand camera position correction process. On the other hand, if the CPU 91 determines that the hand camera position correction button has been pressed, it moves the control point of the hand camera 60 above the mark member 3, for example, directly above it (step S210). Next, the CPU 91 captures an image of the mark member 3 with the hand camera 60 and performs image processing on the captured image to measure the mark positions X _b and Y _b in the base coordinate system Σ _b (step S220). As described above, the image processing is performed by calculating the position of the mark member 3 in the camera coordinate system Σ _c based on the pixel positions of the mark member 3 in the captured image and the hand camera positions X _ci , Y _ci , and θ _ci , and then converting the position from the camera coordinate system Σ _c to the base coordinate system Σ _b .

After measuring the mark positions _Xb , _Yb , the CPU 91 calculates the positional deviations Xm, Ym of the mark positions in the _{Xb -} axis direction and the _{Yb- axis} direction in the base coordinate system _Σb by taking the difference between the measured mark positions _Xb , _Yb and the reference positions _Xr , _Yr previously registered in the storage device 94 in the reference position registration process described above (step S230). Then, the CPU 91 calculates the hand camera positions _Xci , _Yci and the camera position change amount E based on the calculated positional deviations _Xm , _Ym (step S240). The hand camera positions _Xci , _Yci are calculated using the following equations (1) to (4). Equations (1) and (2) are conversion equations for converting the positional deviations _Xm , _Ym in the base coordinate system _Σb into positional deviations _Xcie , _Ycie in the camera coordinate system _Σc . In the formula, θ _J1 is the angle of the first joint axis J1, and θ _J2 is the angle of the second joint axis J2. Formulas (3) and (4) correct (offset) the hand camera positions X _cio and Y _cio currently registered in the storage device 94 by the positional deviations X _cie and Y _cie . The camera position change amount E is calculated using the following formula (5). Formula (5) calculates the linear distance between the reference positions X _r and Y _r and the mark positions X _b and Y _b . After calculating the hand camera positions X _ci and Y _ci and the camera position change amount E, the CPU 91 newly registers the calculated hand camera positions X _ci and Y _ci in the storage device 94 (step S250). This allows the horizontal positional relationship between the end effector of the robot arm 20 and the hand camera 60 to be re-corrected. Then, the CPU 91 determines whether the calculated camera position change amount E is equal to or less than the allowable value Eref (step S260).

If the CPU 91 determines that the camera position change amount E is greater than the allowable value Eref, the process returns to step _S210 , and repeats the processes of steps S210 to S250 to measure the position of the mark member 3 with the hand camera 60 using the newly registered hand camera positions X _ci , Y _ci and hand camera position θ _ci , and calculate the hand camera positions _{X ci, Y ci and the camera position change amount E. This is because the hand camera positions X ci ,} Y _ci cannot be determined with high accuracy with just one measurement due to differences in resolution and errors from the true value of the hand camera position θ ci.

When the CPU 91 determines that the camera position change amount E is equal to or less than the allowable value Eref, it moves the control point (field of view) of the hand camera 60 to a position offset by a predetermined amount δ from directly above the mark member 3 in the _Yb- axis direction of the base coordinate system _Σb , as shown in FIG. 6 (step S270). Next, the CPU 91 captures an image of the mark member 3 with the hand camera 60 and processes the captured image to measure the mark positions _Xb and _Yb in _the base coordinate system _Σb (step S280). Next, the CPU 91 calculates the positional _deviation amounts _Xm and _Ym of the mark positions in the base coordinate system _Σb by calculating the difference between the mark positions _Xb and _Yb and the above-mentioned reference positions Xr and Yr (step S290). The CPU 91 then calculates the hand camera position _θci based on the calculated positional deviation amounts _Xm and _Ym (step S300). The hand camera position _θci is calculated using the following equations (6) and (7). Equation (6) is a conversion equation for converting the positional deviation amounts X _m and Y _m into an arctangent angle (positional deviation amount θ _cie ). Equation (7) is used to correct (offset) the hand camera position θ _cio currently registered in the storage device 94 by the positional deviation amount θ _cie .

After calculating the hand camera position θ _ci in this way, the CPU 91 newly registers the calculated hand camera position θ _ci in the storage device 94 (step S310), and ends the hand camera position correction process.

In this way, the CPU 91 images one mark member 3 with the hand camera 60, recognizes the positions _Xb , _Yb of the mark member 3, and corrects the hand camera positions _Xci , _Yci (the horizontal positional relationship of the hand camera 60 with respect to the robot arm 20) so that the recognized positions _Xb , _Yb match the pre-registered reference positions _Xr , _Yr . The CPU 91 then images one mark member 3 with the hand camera 60 from another location, recognizes the positions _Xb , _Yb of the mark member 3, and corrects the hand camera position _θci (the rotational positional relationship of the hand camera 60 with respect to the robot arm 20) so that the recognized positions _Xb , _Yb match the pre-registered reference positions _Xr , _Yr . This makes it possible to re-correct the positional relationship between the endoscope of the robot arm 20 and the hand camera 60 without using an expensive measuring jig.

Here, we will explain the correspondence between the main elements of the embodiment and the main elements of the present disclosure described in the claims. That is, in this embodiment, the robot device 10 corresponds to the robot, the robot arm 20 corresponds to the arm, the hand camera 60 corresponds to the camera, and the mark member 3 corresponds to the mark member. Furthermore, the CPU 91 of the control device 90 that executes the hand camera position correction process corresponds to the control device.

It goes without saying that the present disclosure is in no way limited to the above-described embodiments, and can be implemented in various forms as long as they fall within the technical scope of the present disclosure.

For example, in the above-described embodiment, the robot device 10 is configured as a horizontal articulated robot (SCARA robot), but this is not limited to this and may be any other type of robot device, such as a vertical articulated robot, as long as it has a camera attached to its arm.

As described above, the camera misalignment correction method disclosed herein controls the arm and camera to capture an image of a single mark member with the camera, determines the position of the mark member in a base coordinate system based on the captured image, and pre-stores the determined position of the mark member as a reference position. When a request for misalignment correction is received, the method executes a first correction process and a second correction process to correct the misalignment of the camera relative to the arm. The first correction process controls the arm and camera to capture an image of the mark member with the camera using a first position above the mark member as the imaging position. The first correction process determines the position of the mark member in the base coordinate system based on the captured image and corrects the horizontal positional relationship of the camera with the arm so that the position of the mark member coincides with the reference position. The second correction process controls the arm and camera to capture an image of the mark member with the camera using a second position offset horizontally in a predetermined direction from the first position. The second correction process determines the position of the mark member in the base coordinate system based on the captured image and corrects the rotational positional relationship of the camera with the arm so that the position of the mark member coincides with the reference position. As a result, if the position of the mark member is measured using an arm or camera with guaranteed accuracy before the robot device is shipped and stored in advance as a reference position, even if an error occurs in the positional relationship between the arm and camera of the robot device at the shipping destination after the robot device is shipped, the positional relationship between the arm and camera can be corrected on site without the need for a special measuring jig.

In the camera positional deviation correction method disclosed herein, the first correction process may involve calculating the distance between the determined position of the mark member and the reference position in the orthogonal first and second axis directions of the base coordinate system, and then correcting the horizontal positional relationship between the arm and the camera by coordinate transforming the determined distance into the camera coordinate system. This allows the horizontal positional relationship between the arm and the camera to be corrected by simple calculation.

Furthermore, in the camera positional deviation correction method disclosed herein, the first correction process may be repeated until the amount of positional deviation between the position of the mark member in the base coordinate system and the reference position becomes equal to or less than a predetermined amount. This allows the positional relationship between the arm and the camera to be corrected with high precision.

Furthermore, in the camera positional deviation correction method disclosed herein, the second correction process may involve calculating the distance between the determined position of the mark member and the reference position in the orthogonal first and second axis directions of the base coordinate system, and converting the determined distance into an arctangent angle to correct the positional relationship between the arm and the camera in the rotational direction. This allows the positional relationship between the arm and the camera in the rotational direction to be corrected by simple calculation.

Note that this disclosure is not limited to the form of a camera position misalignment correction method, but may also be in the form of a robotic device.

This disclosure can be used in the robotic device manufacturing industry, etc.

1 Robot system, 2 Work table, 3 Marking member, 10 Robot device, 11 Base, 20 Robot arm, 21 First arm, 22 Second arm, 23 Shaft, 24 Workpiece holder, 30 First arm drive unit, 32 Motor, 34 Encoder, 40 Second arm drive unit, 42 Motor, 44 Encoder, 50 Shaft drive unit, 52a Motor, 52b Motor, 54a Encoder, 54b Encoder, 60 Hand camera, 70 Workpiece camera, 90 Control device, 91 CPU, 92 ROM, 93 RAM, 94 Storage device, J1 First joint axis, J2 Second joint axis, J3 Third joint axis, P Workpiece supply unit, S Board, W Workpiece.

Claims (5)

1. A method for correcting a positional deviation of a camera with respect to an arm in a robot device having an arm and a camera attached to the arm, comprising:
controlling the arm and the camera so as to capture an image of one marking member with the camera, determining the position of a mark affixed to the marking member in a base coordinate system based on the captured image, and pre-storing the determined position of the mark of the marking member in the base coordinate system as a reference position;
When correction of the positional deviation is requested, the arm and the camera are controlled so that the camera captures an image of the mark member using a first position above the mark member as an imaging position, and a first correction process is performed to determine the position of the mark affixed to the mark member in a base coordinate system based on the captured image, and to correct the horizontal positional relationship of the camera with respect to the arm so that the determined position of the mark of the mark member in the base coordinate system coincides with the stored reference position; after performing the first correction process, the arm and the camera are controlled so that the camera captures an image of the mark member using a second position moved horizontally a predetermined distance from the first position in a predetermined direction so that the mark can be captured, and a second correction process is performed to determine the position of the mark affixed to the mark member in the base coordinate system based on the captured image, and to correct the positional relationship of the camera in the rotational direction with respect to the arm so that the determined position of the mark of the mark member in the base coordinate system coincides with the stored reference position.
Camera position correction method. 2. The camera position deviation correction method according to claim 1,
the first correction process is to calculate distances in a first axis direction and a second axis direction orthogonal to each other in the base coordinate system between the calculated position of the mark of the mark member in the base coordinate system and the stored reference position, and to correct the positional relationship between the arm and the camera in the horizontal direction by coordinate conversion of the calculated distances into a camera coordinate system;
Camera position correction method. 3. The camera position deviation correction method according to claim 1, further comprising:
repeating the first correction process until a positional deviation between the position of the mark of the mark member in the base coordinate system and the stored reference position becomes equal to or less than a predetermined amount;
Camera position correction method. 4. The camera position deviation correction method according to claim 1, further comprising:
the second correction process is to calculate distances in a first axis direction and a second axis direction orthogonal to each other in the base coordinate system between the calculated position of the mark of the mark member in the base coordinate system and the stored reference position, and convert the calculated distances into arctangent angles to correct the positional relationship between the arm and the camera in the rotation direction.
Camera position correction method. Arm and
a camera attached to the arm;
one marking member;
a storage device that stores in advance the position of the mark on the mark member in a base coordinate system as a reference position;
a control device that, when a correction of positional deviation of the camera with respect to the arm is requested, controls the arm and the camera so that the camera captures an image of the mark member using a first position above the mark member as an imaging position, determines the position of the mark affixed to the mark member in a base coordinate system based on the captured image, and performs a first correction process to correct a positional relationship of the camera in a horizontal direction with respect to the arm so that the determined position of the mark of the mark member in the base coordinate system coincides with the stored reference position, and after performing the first correction process, controls the arm and the camera so that the camera captures an image of the mark member using a second position moved horizontally a predetermined distance from the first position so that the mark can be captured, and performs a second correction process to determine the position of the mark affixed to the mark member in the base coordinate system based on the captured image, and corrects a positional relationship of the camera in a rotational direction with respect to the arm so that the determined position of the mark of the mark member in the base coordinate system coincides with the stored reference position;
A robotic device comprising: