JP7699516B2 - Component mounting device and component mounting method - Google Patents

Description

The technology disclosed in this specification relates to a component mounting device and a component mounting method.

In the production of electronic devices, component mounting devices are used to mount components onto a substrate. Patent Document 1 discloses a component mounting device that mounts lead components onto a substrate.

JP 2021-093560 A

Lead components are mounted on a board by inserting the leads of the lead components into holes in the board. Depending on the condition of the leads, it may be difficult to insert the leads into the holes in the board, which may result in a decrease in productivity of the component mounting device. For example, if the leads are bent, it may be difficult to insert the leads into the holes in the board, which may result in a situation where mounting of the lead components must be abandoned. To prevent a decrease in productivity of component mounting devices, there is a demand for technology that can recognize the condition of the leads and insert the leads into the holes in the board according to their condition.

This specification discloses a component mounting device. The component mounting device includes a robot manipulator, a robot hand provided at the tip of the robot manipulator and holding the body of a lead component, a projection device that irradiates stripe pattern light onto the lead component while the body is held by the robot hand, an imaging device that images the lead component irradiated with the stripe pattern light from a predetermined viewpoint, a calculation device that performs image processing on the image data of the lead component imaged by the imaging device, and a control device that controls the robot manipulator so that the lead of the lead component is inserted into the hole of the board based on the image processing result of the calculation device. The calculation device has a three-dimensional image generation unit that performs arithmetic processing on the image data of the lead component based on a phase shift method to generate three-dimensional image data, a three-dimensional point cloud conversion unit that converts the three-dimensional image data into three-dimensional point cloud data, a segmentation unit that segments the three-dimensional point cloud data and extracts lead point cloud data indicating the three-dimensional shape of the surface of the lead from the three-dimensional point cloud data, and a lead state calculation unit that performs principal component analysis on the lead point cloud data to calculate the three-dimensional data of the lead.

The component mounting device disclosed in this specification reduces declines in productivity.

FIG. 1 is a perspective view showing a component mounting apparatus according to an embodiment. FIG. 2 is a side view showing the component mounting apparatus according to the embodiment. FIG. 3 is a perspective view showing a robot hand according to the embodiment. FIG. 4 is a side view showing a lead component held by a robot hand according to the embodiment. FIG. 5 is a view of the lead component according to the embodiment as viewed from below. FIG. 6 is a perspective view showing a three-dimensional measuring apparatus according to the embodiment. FIG. 7 is a diagram for explaining the operation of the robot manipulator according to the embodiment. FIG. 8 is a block diagram showing a component mounting apparatus according to an embodiment. FIG. 9 is a flowchart showing an image processing method of the arithmetic device according to the embodiment. FIG. 10 is a diagram illustrating an example of three-dimensional point cloud data according to the embodiment. FIG. 11 is a diagram illustrating an example of integrated point cloud data according to the embodiment. FIG. 12 is a diagram illustrating an example of lead point cloud data according to the embodiment. FIG. 13 is a diagram for explaining a segment point group according to the embodiment. FIG. 14 is a diagram for explaining the operation of inserting the leads of the lead component according to the embodiment into holes in the substrate. FIG. 15 is a diagram for explaining the operation of inserting the leads of the lead component according to the embodiment into holes in the substrate. FIG. 16 is a block diagram showing a computer system according to an embodiment.

Below, embodiments of the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiments. The components of the embodiments described below can be combined as appropriate. Also, some components may not be used.

In the embodiment, a local coordinate system is set in the component mounting device 1, and the positional relationship of each part will be described with reference to the local coordinate system. An XYZ Cartesian coordinate system is set as the local coordinate system. The direction parallel to the X axis in a specified plane is the X axis direction. The direction parallel to the Y axis perpendicular to the X axis in the specified plane is the Y axis direction. The direction parallel to the Z axis perpendicular to each of the X axis and Y axis is the Z axis direction. The direction of rotation or tilt around the X axis is the θX direction. The direction of rotation or tilt around the Y axis is the θY direction. The direction of rotation or tilt around the Z axis is the θZ direction. The specified plane is the XY plane. The Z axis is perpendicular to the specified plane. In the embodiment, the specified plane is parallel to the horizontal plane. The Z axis direction is the vertical direction. The specified plane may be tilted with respect to the horizontal plane.

[Component mounting equipment]
Fig. 1 is a perspective view showing a component mounting apparatus 1 according to an embodiment. Fig. 2 is a side view showing the component mounting apparatus 1 according to an embodiment. As shown in Figs. 1 and 2, the component mounting apparatus 1 includes a base 2, a component supply member 3, a board support member 4, a robot hand 5, a robot manipulator 6, and a three-dimensional measuring device 7.

The base 2 supports the component supply member 3, the board support member 4, the robot manipulator 6, and the three-dimensional measuring device 7.

The component supply member 3 supplies the lead components 100. In an embodiment, the component supply member 3 includes a tray on which the lead components 100 are arranged. A plurality of lead components 100 are arranged on the component supply member 3. The types of the plurality of lead components 100 may be the same or different.

The substrate support member 4 supports the substrate 200 on which the lead components 100 are mounted. The substrate support member 4 supports the substrate 200 so that the top surface of the substrate 200 is parallel to the XY plane.

The robot hand 5 holds the lead component 100. The robot hand 5 is provided at the tip of the robot manipulator 6.

The robot manipulator 6 moves the robot hand 5. The robot manipulator 6 includes a multi-joint robot. In the embodiment, the robot manipulator 6 is a vertical multi-joint robot. The robot manipulator 6 may also be a horizontal multi-joint robot. The robot manipulator 6 has a base member 6A fixed to the base 2, a rotating member 6B supported by the base member 6A, a first arm 6C connected to the rotating member 6B, a second arm 6D connected to the first arm 6C, and a third arm 6E connected to the second arm 6D.

The rotating member 6B is supported by the base member 6A so as to be rotatable around a rotation axis TX. The rotation axis TX is parallel to the Z axis. The first arm 6C is connected to the rotating member 6B so as to be rotatable around a first rotation axis AX1. The first rotation axis AX1 is perpendicular to the Z axis. The second arm 6D is connected to the first arm 6C so as to be rotatable around a second rotation axis AX2. The second rotation axis AX2 is parallel to the first rotation axis AX1. The third arm 6E is connected to the second arm 6D so as to be rotatable around a third rotation axis AX3. The third rotation axis AX3 is parallel to the second rotation axis AX2. The robot hand 5 is attached to the third arm 6E.

The robot manipulator 6 has a rotation actuator that rotates the rotation member 6B, a first rotation actuator that rotates the first arm 6C, a second rotation actuator that rotates the second arm 6D, and a third rotation actuator that rotates the third arm 6E.

The three-dimensional measuring device 7 measures the lead component 100 held by the robot hand 5. The three-dimensional measuring device 7 detects the position of the lead component 100 in the local coordinate system based on the phase shift method.

[Robot Hand]
3 is a perspective view showing a robot hand 5 according to an embodiment. The robot hand 5 has a connecting member 5A attached to a third arm 6E, a rotating member 5B supported by the connecting member 5A, and a pair of moving members 5C supported by the rotating member 5B.

The rotating member 5B is supported by the connecting member 5A so as to be rotatable around a rotation axis RX. The rotation axis RX is perpendicular to the third rotation axis AX3. The pair of moving members 5C move toward and away from each other. A grip portion 5D is provided at the lower end of the moving member 5C. The pair of grip portions 5D move toward and away from each other.

The robot hand 5 has a rotation actuator that rotates the rotating member 5B and a grip actuator that moves the pair of moving members 5C closer to or farther away from each other.

With the lead component 100 placed between the pair of grip portions 5D, the pair of grip portions 5D move closer to each other, thereby holding the lead component 100 in the grip portions 5D. When the pair of grip portions 5D move away from each other, the lead component 100 is released from the grip portions 5D.

A force sensor 8 is disposed on one of the movable members 5C. The force sensor 8 can detect the load applied to the grip portion 5D.

[Lead parts]
Fig. 4 is a side view showing the lead component 100 held by the robot hand 5 according to the embodiment. Fig. 5 is a view of the lead component 100 according to the embodiment as viewed from below.

The lead component 100 has a body 101 and a number of leads 110 protruding from the body 101.

The body 101 includes a housing made of synthetic resin. An element such as a coil is disposed in the internal space of the body 101. The lead 110 is a metal protrusion. The lead 110 is connected to the element disposed in the internal space of the body 101, for example.

The leads 110 protrude downward from the bottom surface of the body 101. When the lead component 100 is mounted on the substrate 200, the bottom surface of the body 101 faces the top surface of the substrate 200.

The robot hand 5 holds the body 101 of the lead component 100. The pair of gripping parts 5D holds the lead component 100 by clamping the body 101.

[3D measurement device]
6 is a perspective view showing a three-dimensional measuring device 7 according to an embodiment of the present invention. As shown in FIG. 6, the three-dimensional measuring device 7 measures the three-dimensional shape of the lead component 100 in a state in which the body 101 is held by the robot hand 5.

The three-dimensional measuring device 7 has a projection device 7A, an imaging device 7B, and a calculation device 7C. In the embodiment, the projection device 7A and the imaging device 7B are each housed in a housing 7D. The projection device 7A and the imaging device 7B are each fixed to the housing 7D. A transparent member 7E is disposed in an opening at the upper end of the housing 7D. An example of the transparent member 7E is a glass plate.

The projection device 7A irradiates stripe pattern light onto the lead component 100 while the body 101 is held by the robot hand 5. The projection device 7A has a light source, a light modulation element that optically modulates the light emitted from the light source to generate stripe pattern light, and an emission optical system that emits the stripe pattern light generated by the light modulation element. Examples of the light modulation element include a digital mirror device (DMD), a transmissive liquid crystal panel, or a reflective liquid crystal panel.

The imaging device 7B images the lead component 100 illuminated with the stripe pattern light from a predetermined viewpoint. The viewpoint of the imaging device 7B refers to the imaging position and imaging angle of the imaging device 7B relative to the lead component 100. The imaging device 7B has an imaging optical system that forms an image of the stripe pattern light reflected by the lead component 100, and an imaging element that acquires image data of the lead component 100 via the imaging optical system. Examples of the imaging element include a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor) or a CCD image sensor (Charge Coupled Device Image Sensor).

The calculation device 7C performs image processing on the imaging data of the lead component 100 captured by the imaging device 7B. The calculation device 7C includes a computer system. The calculation device 7C has a processor such as a CPU (Central Processing Unit), a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory), and an input/output interface including an input/output circuit capable of inputting and outputting signals and data.

The three-dimensional measuring device 7 measures the three-dimensional shape of the lead component 100 held by the robot hand 5 based on the phase shift method.

The projection device 7A irradiates the lead component 100 with stripe pattern light, for example, having a sinusoidal brightness distribution, while shifting the phase. The lead component 100 held by the robot hand 5 is positioned above the transparent member 7E. The stripe pattern light emitted from the projection device 7A is irradiated onto the lead component 100 through the transparent member 7E.

The imaging device 7B images the lead component 100 illuminated with the stripe pattern light. The imaging device 7B images the lead component 100 through the transparent member 7E. The imaging device 7B images the lead component 100 from below the lead component 100. The viewpoint of the imaging device 7B is set below the lead component 100. When the robot manipulator 6 is operated and the position and posture of the lead component 100 held by the robot hand 5 changes, the relative position and relative angle between the lead component 100 and the imaging device 7B change. When the relative position and relative angle between the lead component 100 and the imaging device 7B change, the relative viewpoint of the imaging device 7B with respect to the lead component 100 changes.

The calculation device 7C processes the image data of the lead component 100 captured by the imaging device 7B based on the phase shift method to calculate three-dimensional data of the lead 110. The three-dimensional data of the lead 110 includes the amount of bending of the lead 110 and the coordinates of the tip of the lead 110 in a three-dimensional space defined by a local coordinate system. If the angle of the lead 110 in the design value relative to the body 101 is defined as the ideal angle, and the actual angle of the lead 110 relative to the body 101 is defined as the actual angle, the amount of bending of the lead 110 refers to the difference between the ideal angle and the actual angle.

[Robot manipulator movement]
Fig. 7 is a diagram for explaining the operation of the robot manipulator 6 according to the embodiment. The component mounting apparatus 1 includes a control device 9 that controls the robot manipulator 6. The control device 9 includes a computer system. As shown in Fig. 7, the substrate 200 is provided with holes 210 into which the leads 110 of the lead component 100 are inserted. The control device 9 controls the robot manipulator 6 based on the image processing result of the calculation device 7C of the three-dimensional measurement device 7 so that the leads 110 of the lead component 100 are inserted into the holes 210 of the substrate 200.

[Arithmetic device]
8 is a block diagram showing the component mounting apparatus 1 according to the embodiment. As shown in FIG. 8, the component mounting apparatus 1 includes a calculation device 7C and a control device 9.

The calculation device 7C has a 3D image generation unit 11, a 3D point cloud conversion unit 12, a 3D point cloud integration unit 13, a segmentation unit 14, a candidate point cloud generation unit 15, a lead condition calculation unit 16, and an output unit 17.

The three-dimensional image generating unit 11 acquires imaging data of the lead component 100 from the imaging device 7B. The three-dimensional image generating unit 11 performs computational processing on the imaging data of the lead component 100 based on the phase shift method to generate three-dimensional image data Da of the lead component 100.

In an embodiment, the control device 9 controls the robot manipulator 6 so that the lead component 100 is imaged by the imaging device 7B from each of the multiple viewpoints. The three-dimensional image generating unit 11 generates multiple three-dimensional image data Da of the lead component 100 viewed from each of the multiple viewpoints based on multiple imaging data captured from each of the multiple viewpoints.

The three-dimensional point cloud conversion unit 12 converts the three-dimensional image data Da generated by the three-dimensional image generation unit 11 into three-dimensional point cloud data Db. The three-dimensional point cloud data Db represents the three-dimensional shape of the surface of the lead component 100. The three-dimensional point cloud data Db is a collection of multiple measurement points on the surface of the lead component 100 measured by the three-dimensional measurement device 7. The position of each of the multiple measurement points is defined by an X coordinate, a Y coordinate, and a Z coordinate.

As described above, in the embodiment, the three-dimensional image generating unit 11 generates a plurality of three-dimensional image data Da based on a plurality of pieces of imaging data captured from each of a plurality of viewpoints. The three-dimensional point cloud converting unit 12 generates a plurality of three-dimensional point cloud data Db of the lead component 100 viewed from each of the plurality of viewpoints based on the plurality of pieces of three-dimensional image data Da.

The 3D point cloud integration unit 13 aligns the multiple 3D point cloud data Db generated by the 3D point cloud conversion unit 12 based on a specified algorithm to generate integrated point cloud data Dc.

An example of the specified algorithm is the Iterative Closest Point (ICP) matching algorithm. The three-dimensional point cloud integration unit 13 aligns multiple three-dimensional point cloud data Db in three-dimensional space based on an existing algorithm such as the ICP matching algorithm, and generates integrated point cloud data Dc.

The segmentation unit 14 segments the three-dimensional point cloud data Db to extract lead point cloud data Dd from the three-dimensional point cloud data Db. The lead point cloud data Dd indicates the three-dimensional shape of the surface of the lead 110. The surface of the lead 110 includes the outer peripheral surface and the lower end surface of the lead 110. By segmentation, measurement points corresponding to the lead 110 and measurement points corresponding to the background or noise are separated in the three-dimensional point cloud data Db. That is, by segmentation, measurement points corresponding to the lead 110 are extracted and measurement points corresponding to the background or noise are removed.

Segmentation refers to a process of separating a point group of interest from the point groups that make up the three-dimensional point cloud data Db according to specified rules. In an embodiment, the segmentation unit 14 captures feature points of the three-dimensional point cloud data Db and classifies the measurement points that make up the three-dimensional point cloud data Db into multiple groups. The segmentation unit 14 segments the three-dimensional point cloud data Db based on a specified segmentation method.

Examples of segmentation methods include the Locally Convex Connected Patches (LCCP) method and the Region Growing method. The segmentation unit 14 extracts lead point cloud data Dd from the three-dimensional point cloud data Db based on an existing segmentation method such as the LCCP method or the Region Growing method.

In an embodiment, segmenting the three-dimensional point cloud data Db includes segmenting the integrated point cloud data Dc generated by the three-dimensional point cloud integration unit 13. The segmentation unit 14 segments the integrated point cloud data Dc generated by the three-dimensional point cloud integration unit 13, and extracts lead point cloud data Dd from the integrated point cloud data Dc.

The candidate point cloud generation unit 15 generates candidate point cloud data Df that is a candidate for the lead 110 by connecting multiple segment point clouds De based on the design value of the lead part 100, the viewpoint of the imaging device 7B when the lead part 100 is imaged, and the specified connection conditions. Although the lead point cloud data Dd is extracted from the integrated point cloud data Dc by segmentation, the lead point cloud data Dd may be separated into multiple segment point clouds De. When the lead point cloud data Dd is separated into multiple segment point clouds De by segmentation, it may become unclear which segment point cloud De corresponds to the lead 110 and which segment point cloud De does not correspond to the lead 110. Therefore, the candidate point cloud generation unit 15 extracts multiple segment point clouds De that are predicted to correspond to the lead 110 based on the design values of the lead component 100, the viewpoint of the imaging device 7B when the lead component 100 is imaged, and the specified connection conditions, and connects these multiple segment point clouds De to generate candidate point cloud data Df that are candidates for the lead 110.

The design values of the lead component 100 are known data. The design values of the lead component 100 are stored in advance in the candidate point cloud generating unit 15. Examples of the design values of the lead component 100 include the length of the lead 110, the relative position between the body 101 and the lead 110, the angle (ideal angle) of the lead 110 with respect to the body 101, and the spacing between the multiple leads 110.

The candidate point cloud generation unit 15 can calculate the viewpoint of the imaging device 7B when the lead part 100 is imaged, based on the control command output from the control device 9 to the robot manipulator 6 when the lead part 100 is imaged by the imaging device 7B. If the robot manipulator 6 is provided with a sensor capable of detecting the position and orientation of the robot manipulator 6, the candidate point cloud generation unit 15 can calculate the viewpoint of the imaging device 7B when the lead part 100 is imaged, based on the detection value of the sensor.

The specified connection condition is determined in advance. The specified connection condition is stored in advance in the candidate point group generating unit 15. In the embodiment, the specified connection condition includes a first connection condition and a second connection condition. The first connection condition includes that the relative distance between the two segment point groups De to be connected is equal to or less than a predetermined specified distance. The second connection condition includes that the condition [α<β×γ] is satisfied when the area of the smallest face of the axis-aligned bounding box (AABB: Axis-Aligned Bounding Box) when the segment point group De after connection is surrounded by the axis-aligned bounding box is α, the diameter of the lead 110 is β, and the coefficient is γ. The axis-aligned bounding box refers to a rectangular parallelepiped boundary figure whose sides are parallel to the X-axis, Y-axis, and Z-axis of a three-dimensional space defined in a local coordinate system.

The lead condition calculation unit 16 performs principal component analysis on the lead point cloud data Dd to calculate three-dimensional data of the lead 110. The lead condition calculation unit 16 calculates, as the three-dimensional data of the lead 110, the amount of bending of the lead 110 and the coordinates of the tip of the lead 110 in a three-dimensional space defined by a local coordinate system.

Principal component analysis is an analytical technique in which principal components are determined by maximizing the variance of the first principal component, and then selecting the second and subsequent principal components so as to maximize the variance under conditions that are orthogonal to the previous principal components.

In an embodiment, performing principal component analysis on the lead point cloud data Dd includes performing principal component analysis on the candidate point cloud data Df generated by the candidate point cloud generation unit 15. The lead condition calculation unit 16 performs principal component analysis on the candidate point cloud data Df generated by the candidate point cloud generation unit 15 to calculate three-dimensional data of the lead 110.

In the embodiment, the lead condition calculation unit 16 extracts, as the lead 110, the principal components of the candidate point cloud data Df whose principal axis vectors and coordinates of the tip of the lead 110 are approximate to the principal axis vectors and coordinates of the tip of the lead 110 predicted based on the design values of the lead 110 and the viewpoint of the imaging device 7B when the lead component 100 is imaged.

The output unit 17 outputs the three-dimensional data of the lead 110 calculated by the lead state calculation unit 16 to the control device 9. The image processing results of the calculation device 7C include the three-dimensional data of the lead 110 calculated by the lead state calculation unit 16. The control device 9 controls the robot manipulator 6 based on the three-dimensional data of the lead 110 so that the lead 110 is inserted into the hole 210 of the substrate 200.

[Image processing method]
9 is a flowchart showing an image processing method of the computing device 7C according to the embodiment. In order to mount the lead component 100 on the board 200, the control device 9 controls the robot manipulator 6 so that the robot hand 5 approaches the component supply member 3. The robot hand 5 that has moved to the component supply member 3 holds the body 101 of the lead component 100 arranged on the component supply member 3. After the body 101 of the lead component 100 is held by the robot hand 5, the control device 9 controls the robot manipulator 6 so that the lead component 100 held by the robot hand 5 is measured by the three-dimensional measuring device 7. That is, the control device 9 controls the robot manipulator 6 so that the lead component 100 held by the robot hand 5 is placed above the three-dimensional measuring device 7, as described with reference to FIG. 6.

In the embodiment, the control device 9 controls the robot manipulator 6 so that the lead component 100 is imaged by the imaging device 7B from each of a plurality of viewpoints. That is, the control device 9 controls the robot manipulator 6 so that the position and angle of the lead component 100 held by the robot hand 5 changes above the three-dimensional measuring device 7. The imaging device 7B images the lead component 100 from each of a plurality of different viewpoints.

The three-dimensional image generating unit 11 acquires imaging data of the lead component 100 from the imaging device 7B (step S1).

The three-dimensional image generating unit 11 performs computational processing on the imaging data of the lead component 100 based on the phase shift method to generate three-dimensional image data Da of the lead component 100. The three-dimensional image generating unit 11 generates multiple three-dimensional image data Da of the lead component 100 viewed from each of the multiple viewpoints based on multiple imaging data captured from each of the multiple viewpoints (step S2).

The three-dimensional point cloud conversion unit 12 converts the three-dimensional image data Da into three-dimensional point cloud data Db. The three-dimensional point cloud conversion unit 12 generates a plurality of three-dimensional point cloud data Db of the lead component 100 viewed from each of a plurality of viewpoints based on the plurality of three-dimensional image data Da generated in step S2 (step S3).

FIG. 10 is a diagram showing a schematic example of three-dimensional point cloud data Db according to the embodiment. When the lead component 100 is imaged from multiple viewpoints, the three-dimensional point cloud conversion unit 12 generates multiple three-dimensional point cloud data Db of the lead component 100 seen from each of the multiple viewpoints. For example, when the lead component 100 is imaged from three viewpoints, as shown in FIG. 10, the three-dimensional point cloud conversion unit 12 generates three-dimensional point cloud data Db1 of the lead component 100 seen from the first viewpoint, three-dimensional point cloud data Db2 of the lead component 100 seen from the second viewpoint, and three-dimensional point cloud data Db3 of the lead component 100 seen from the third viewpoint.

The 3D point cloud integration unit 13 aligns the multiple 3D point cloud data Db generated in step S3 based on a specified algorithm to generate integrated point cloud data Dc (step S4).

Figure 11 is a diagram showing a schematic example of integrated point cloud data Dc according to an embodiment. As shown in Figure 10, when three three-dimensional point cloud data Db (Db1, Db2, Db3) are generated, the three-dimensional point cloud integration unit 13 aligns the three three-dimensional point cloud data Db generated in step S3 in three-dimensional space based on an existing algorithm such as an ICP matching algorithm to generate integrated point cloud data Dc.

The segmentation unit 14 segments the integrated point cloud data Dc generated in step S4 and extracts the lead point cloud data Dd from the integrated point cloud data Dc (step S5).

FIG. 12 is a diagram illustrating an example of lead point cloud data Dd according to an embodiment. The segmentation unit 14 segments the integrated point cloud data Dc based on an existing segmentation method such as the LCCP method or the region growing method, and extracts lead point cloud data Dd from the integrated point cloud data Dc. As shown in FIG. 12, the lead point cloud data Dd is extracted by segmentation. Furthermore, measurement points corresponding to the background or noise are removed by segmentation.

Although the lead point cloud data Dd is extracted from the integrated point cloud data Dc by the segmentation in step S5, the lead point cloud data Dd may be separated into multiple segment point clouds De. The candidate point cloud generation unit 15 connects the multiple segment point clouds De based on the design values of the lead component 100, the viewpoint of the imaging device 7B when the lead component 100 is imaged, and the specified connection conditions, to generate candidate point cloud data Df that is a candidate for the lead 110 (step S6).

FIG. 13 is a diagram for explaining the segment point group De according to the embodiment. As shown in FIG. 13, although the lead point group data Dd is extracted from the integrated point group data Dc by segmentation, the lead point group data Dd may be separated into a plurality of segment point groups De. In the example shown in FIG. 13, the segment point group De includes a first segment point group De1, a second segment point group De2, a third segment point group De3, and a fourth segment point group De4. When the lead point group data Dd is separated into a plurality of segment point groups De by segmentation, it may become unclear which segment point group De corresponds to the lead 110 and which segment point group De does not correspond to the lead 110. Therefore, the candidate point cloud generation unit 15 extracts multiple segment point clouds De that are predicted to correspond to the lead 110 based on the design values of the lead component 100, the viewpoint of the imaging device 7B when the lead component 100 is imaged, and the specified connection conditions, and connects these multiple segment point clouds De to generate candidate point cloud data Df that are candidates for the lead 110.

As described above, in the embodiment, the specified connection condition includes a first connection condition and a second connection condition. The first connection condition includes that the relative distance between the two segment point groups De to be connected is equal to or less than a predetermined specified distance. The second connection condition includes that the area of the smallest face when the connected segment point group De is enclosed in an axis-parallel bounding box is α, the diameter of the lead 110 is β, and the coefficient is γ, and the condition [α<β×γ] is satisfied.

The two segment point groups De to be connected are determined based on the design value of the lead component 100 and the viewpoint of the imaging device 7B when the lead component 100 is imaged. The candidate point group generating unit 15 predicts the extension direction of the actual lead 110 based on the design value of the lead component 100 and the viewpoint of the imaging device 7B when the lead component 100 is imaged. The candidate point group generating unit 15 determines multiple segment point groups De to be connected that are arranged along the extension direction of the actual lead 110.

In the example shown in FIG. 13, the first segment point group De1, the second segment point group De2, the third segment point group De3, and the fourth segment point group De4 are arranged so that the predicted actual lead 110 is aligned along the extension direction. Therefore, the first segment point group De1, the second segment point group De2, the third segment point group De3, and the fourth segment point group De4 are determined as connection targets. More specifically, the first segment point group De1 and the second segment point group De2 adjacent to the first segment point group De1 are determined as connection targets. The second segment point group De2 and the third segment point group De3 adjacent to the second segment point group De2 are determined as connection targets. The third segment point group De3 and the fourth segment point group De4 adjacent to the third segment point group De3 are determined as connection targets.

In the example shown in FIG. 13, when focusing on the first connection condition, the first segment point group De1 and the second segment point group De2 are close to each other, and the relative distance between the first segment point group De1 and the second segment point group De2 is equal to or less than a specified distance. The second segment point group De2 and the third segment point group De3 are close to each other, and the relative distance between the second segment point group De2 and the third segment point group De3 is equal to or less than a specified distance. On the other hand, the third segment point group De3 and the fourth segment point group De4 are separated from each other, and the relative distance between the third segment point group De3 and the fourth segment point group De4 exceeds the specified distance. Therefore, in the example shown in FIG. 13, the fourth segment point group De4 is determined not to satisfy the first connection condition.

13, when focusing on the second connection condition, when the first segment point group De1, the second segment point group De2, and the third segment point group De3 are connected, the area α of the bottom surface, which is the smallest surface Aa of the axis-parallel bounding box Ba that encloses the three segment point groups De (De1, De2, De3) after the connection, is less than [β×γ]. On the other hand, when the first segment point group De1, the second segment point group De2, the third segment point group De3, and the fourth segment point group De4 are connected, the area α of the bottom surface, which is the smallest surface Ab of the axis-parallel bounding box Bb that encloses the four segment point groups De (De1, De2, De3, De4) after the connection, is more than [β×γ]. Therefore, in the example shown in FIG. 13, the fourth segment point group De4 is determined not to satisfy the second connection condition either.

In addition, the extension direction of the three segment point groups De (De1, De2, De3) after connection matches the predicted extension direction of the actual lead 110. On the other hand, the extension direction of the four segment point groups De (De1, De2, De3, De4) after connection does not match the predicted extension direction of the actual lead 110.

In this way, in the example shown in FIG. 13, the candidate point group generation unit 15 predicts that the first segment point group De1, the second segment point group De2, and the third segment point group De3 correspond to the lead 110, and predicts that the fourth segment point group De4 does not correspond to the lead 110.

The candidate point cloud generation unit 15 extracts multiple segment point clouds De (De1, De2, De3) that are predicted to correspond to the lead 110, and connects these multiple segment point clouds De (De1, De2, De3) to generate candidate point cloud data Df that is a candidate for the lead 110 (step S6).

In the example shown in FIG. 13, the candidate point cloud data Df is composed of a first segment point cloud De1, a second segment point cloud De2, and a third segment point cloud De3.

The lead condition calculation unit 16 performs principal component analysis on the candidate point cloud data Df generated in step S6 to calculate three-dimensional data of the lead 110 (step S7).

In an embodiment, the lead condition calculation unit 16 extracts, as the lead 110, principal components whose principal axis vectors of the principal components of the candidate point cloud data Df and whose coordinates of the tip of the lead 110 are approximate to the principal axis vector of the lead 110 and the coordinates of the tip of the lead 110 predicted based on the design value of the lead 110 and the viewpoint of the imaging device 7B when the lead component 100 is imaged. As shown in FIG. 13, the predicted principal axis vector of the lead 110 includes the predicted extension direction of the actual lead 110 described above.

The lead condition calculation unit 16 calculates the amount of bending of the lead 110 and the coordinates of the tip of the lead 110 in a three-dimensional space defined by a local coordinate system as three-dimensional data of the lead 110.

The output unit 17 outputs the three-dimensional data of the lead 110 calculated in step S7 to the control device 9 (step S8).

The control device 9 controls the robot manipulator 6 based on the three-dimensional data of the lead 110 so that the lead 110 is inserted into the hole 210 in the substrate 200.

[Insert Lead]
14 and 15 are diagrams for explaining the operation of inserting the leads 110 of the lead component 100 according to the embodiment into the holes 210 of the substrate 200. In the example shown in Fig. 14 and 15, the lead component 100 has two leads 110. The leads 110 include a first lead 111 and a second lead 112. The holes 210 of the substrate 200 include a first hole 211 into which the first lead 111 is inserted and a second hole 212 into which the second lead 112 is inserted.

In the embodiment, the robot manipulator 6 is a multi-joint robot. Therefore, the robot manipulator 6 can tilt the lead component 100 with respect to the upper surface of the substrate 200. The robot manipulator 6 can arbitrarily adjust the angle between the upper surface of the substrate 200 and the lower surface of the body 101 held by the robot hand 5.

As shown in FIG. 14, for example, if the second lead 112 is bent, the control device 9 controls the robot manipulator 6 based on the image processing results of the computing device 7C so that the second lead 112 is inserted into the second hole 212 before the first lead 111 is inserted into the first hole 211. The robot manipulator 6 inserts the second lead 112 into the second hole 212 while tilting the lead component 100.

After the second lead 112 is inserted into the second hole 212, the control device 9 moves the lead component 100 in the -X direction until the tip of the first lead 111 faces the first hole 211 of the substrate 200. This causes the second lead 112 to be corrected. After the second lead 112 has been corrected until it faces the first hole 211 of the substrate 200, as shown in FIG. 15, the control device 9 controls the robot manipulator 6 so that the first lead 111 is inserted into the first hole 211 with the second lead 112 positioned in the second hole 212.

[Computer System]
FIG. 16 is a block diagram showing a computer system 1000 according to an embodiment. Each of the above-mentioned arithmetic device 7C and the control device 9 includes a computer system 1000. The computer system 1000 has a processor 1001 such as a central processing unit (CPU), a main memory 1002 including a non-volatile memory such as a read only memory (ROM) and a volatile memory such as a random access memory (RAM), a storage 1003, and an interface 1004 including an input/output circuit. The functions of the arithmetic device 7C and the control device 9 are stored in the storage 1003 as computer programs. The processor 1001 reads the computer program from the storage 1003, expands it in the main memory 1002, and executes the above-mentioned processing according to the computer program. The computer program may be distributed to the computer system 1000 via a network.

The computer program, according to the above-described embodiment, enables the computer system 1000 to perform the following operations: irradiating stripe pattern light onto the lead component 100 while holding the body 101 of the lead component 100 with the robot hand 5 provided at the tip of the robot manipulator 6; imaging the lead component 100 irradiated with the stripe pattern light from a predetermined viewpoint; image processing the image data of the lead component 100; and controlling the robot manipulator 6 based on the image processing results so that the lead 110 of the lead component 100 is inserted into the hole 210 of the substrate 200. In image processing, the computer program can cause the computer system 1000 to perform the following operations according to the above-described embodiment: perform arithmetic processing of the imaging data of the lead component 100 based on the phase shift method to generate three-dimensional image data Da; convert the three-dimensional image data Da into three-dimensional point cloud data Db; segment the three-dimensional point cloud data Db to extract lead point cloud data Dd indicating the three-dimensional shape of the surface of the lead 110 from the three-dimensional point cloud data Db; and perform principal component analysis of the lead point cloud data Dd to calculate the three-dimensional data of the lead 110.

[effect]
As described above, according to the embodiment, the component mounting apparatus 1 includes the robot manipulator 6, the robot hand 5, and the three-dimensional measuring device 7. The three-dimensional measuring device 7 can recognize the amount of bending of the lead 110 and the coordinates of the tip of the lead 110 in a three-dimensional space defined by a local coordinate system as the state of the lead 110. The robot manipulator 6 and the robot hand 5 can insert the lead 110 into the hole 210 of the board 200 in accordance with the state of the lead 110. Therefore, a decrease in productivity of the component mounting apparatus 1 is suppressed.

The arithmetic unit 7C of the three-dimensional measuring device 7 includes a three-dimensional image generating unit 11 that performs arithmetic processing on the imaging data of the lead component 100 based on the phase shift method to generate three-dimensional image data Da, a three-dimensional point cloud converting unit 12 that converts the three-dimensional image data Da into three-dimensional point cloud data Db, a segmentation unit 14 that segments the three-dimensional point cloud data Db and extracts lead point cloud data Dd indicating the three-dimensional shape of the surface of the lead 110 from the three-dimensional point cloud data Db, and a lead state calculating unit 16 that performs principal component analysis on the lead point cloud data Dd to calculate three-dimensional data of the lead 110. Measurement points corresponding to the background or noise are removed from the three-dimensional point cloud data Db by segmentation by the segmentation unit 14. A principal axis vector representing the state of the lead 110 is calculated by the principal component analysis by the lead state calculating unit 16. Therefore, the three-dimensional measuring device 7 can properly recognize the state of the lead 110.

When the lead point cloud data Dd is separated into multiple segment point clouds De by segmentation, the candidate point cloud generation unit 15 connects the multiple segment point clouds De based on the design values of the lead component 100, the viewpoint of the imaging device 7B when the lead component 100 is imaged, and the specified connection conditions, to generate candidate point cloud data Df that is a candidate for the lead 110. The three-dimensional measurement device 7 can properly recognize the state of the lead 110 based on the candidate point cloud data Df.

The three-dimensional image generating unit 11 generates a plurality of three-dimensional image data Da based on a plurality of imaging data captured from a plurality of viewpoints. The three-dimensional point cloud converting unit 12 generates a plurality of three-dimensional point cloud data Db based on the plurality of three-dimensional image data Da. The three-dimensional point cloud integrating unit 13 aligns the plurality of three-dimensional point cloud data based on a prescribed algorithm to generate integrated point cloud data Dc. As a result, even if the positioning accuracy of the robot manipulator 6 is insufficient when the imaging device 7B images the lead component 100, the generation of the integrated point cloud data Dc absorbs errors in the three-dimensional point cloud data Db caused by the positioning accuracy of the robot manipulator 6.

1...Component mounting device, 2...Base, 3...Component supply member, 4...Substrate support member, 5...Robot hand, 5A...Connecting member, 5B...Rotating member, 5C...Moving member, 5D...Grip portion, 6...Robot manipulator, 6A...Base member, 6B...Pivoting member, 6C...First arm, 6D...Second arm, 6E...Third arm, 7...3D measuring device, 7A...Projection device, 7B...Imaging device, 7C...Calculation device, 7D...Housing, 7E...Transparent member, 8...Force sensor, 9...Control device, 11...3D image generation unit, 12...3D point cloud conversion unit, 13...3D point cloud integration unit, 14...Segmentation unit, 15...Candidate point cloud generation unit, 16...Lead state calculation unit, 17...Output unit, 100...Lead component, 101...Body, 110...Lead, 111...First lead, 112...Second lead, 200...Substrate, 2 10...hole, 211...first hole, 212...second hole, 1000...computer system, 1001...processor, 1002...main memory, 1003...storage, 1004...interface, Aa...minimum surface, Ab...minimum surface, AX1...first rotation axis, AX2...second rotation axis, AX3...third rotation axis, Ba...axis-parallel bounding box, Bb...axis-parallel bounding box, Da...three-dimensional image data , Db...3D point cloud data, Db1...3D point cloud data, Db2...3D point cloud data, Db3...3D point cloud data, Dc...integrated point cloud data, Dd...lead point cloud data, De...segment point cloud, De1...first segment point cloud, De2...second segment point cloud, De3...third segment point cloud, De4...fourth segment point cloud, Df...candidate point cloud data, RX...rotation axis, TX...turning axis.

Claims (5)

A robot manipulator;
a robot hand provided at a tip of the robot manipulator and holding a body of a lead component;
a projection device that irradiates stripe pattern light onto the lead component while the body is held by the robot hand;
an imaging device that images the lead component irradiated with the stripe pattern light from a predetermined viewpoint;
a computing device that processes image data of the lead component captured by the imaging device;
a control device that controls the robot manipulator so that the leads of the lead component are inserted into the holes of the substrate based on the image processing results of the arithmetic device,
The computing device includes:
a three-dimensional image generating unit that performs arithmetic processing on the imaging data of the lead component based on a phase shift method to generate three-dimensional image data;
a three-dimensional point cloud conversion unit that converts the three-dimensional image data into three-dimensional point cloud data;
a segmentation unit that segments the three-dimensional point cloud data and extracts lead point cloud data indicating a three-dimensional shape of a surface of the lead from the three-dimensional point cloud data;
and a lead condition calculation unit that performs principal component analysis on the lead point cloud data to calculate three-dimensional data of the lead.
Component mounting equipment. The segmentation separates the lead point cloud data into a plurality of segment point clouds;
The computing device includes:
a candidate point cloud generating unit that generates candidate point cloud data for the lead by connecting a plurality of the segment point clouds based on a design value of the lead part, a viewpoint of the imaging device when the lead part is imaged, and a specified connection condition;
performing principal component analysis on the lead point cloud data includes performing principal component analysis on the candidate point cloud data;
The component mounting device according to claim 1 . the control device controls the robot manipulator so that the lead part is imaged by the imaging device from each of a plurality of viewpoints;
The three-dimensional image generating unit generates a plurality of three-dimensional image data based on a plurality of pieces of imaging data captured from a plurality of viewpoints,
The three-dimensional point cloud conversion unit generates a plurality of three-dimensional point cloud data based on a plurality of three-dimensional image data;
The computing device includes:
a three-dimensional point cloud integration unit that aligns a plurality of three-dimensional point cloud data based on a specified algorithm to generate integrated point cloud data;
Segmenting the three-dimensional point cloud data includes segmenting the integrated point cloud data.
The component mounting device according to claim 1 or 2. the lead condition calculation unit calculates, as the three-dimensional data of the lead, an amount of bending of the lead and coordinates of a tip of the lead in a three-dimensional space;
The component mounting device according to any one of claims 1 to 3. irradiating a stripe pattern light onto the lead component while holding a body of the lead component with a robot hand provided at the tip of a robot manipulator;
capturing an image of the lead component irradiated with the stripe pattern light from a predetermined viewpoint;
image processing image data of the lead component;
and controlling the robot manipulator based on a result of image processing so that the lead of the lead component is inserted into the hole of the substrate;
The image processing step includes:
processing the imaging data of the lead component based on a phase shift method to generate three-dimensional image data;
converting the three-dimensional image data into three-dimensional point cloud data;
Segmenting the three-dimensional point cloud data to extract lead point cloud data indicating a three-dimensional shape of a surface of the lead from the three-dimensional point cloud data;
and calculating three-dimensional data of the lead by subjecting the lead point cloud data to principal component analysis.
Component mounting method.

Images