JPH07109835B2 - Image recognition method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognition method suitable for detecting the positions of light emitting diodes, transistors and the like.

[0002]

2. Description of the Related Art When manufacturing a semiconductor device, the position of a semiconductor chip is often detected by image recognition. In this type of conventional image recognition, image data obtained by binarizing an analog image signal taken in from a TV camera and preset binarized image data are compared for each address, and predetermined. It was assumed that the chip position could be recognized when the coincidence rate (about 96% even when low) was reached.

Further, in Japanese Patent Laid-Open No. 57-85178, an analog signal obtained by scanning a semiconductor chip is binarized, and the position of the semiconductor chip is recognized by the level change of the binarized signal. A method of doing so is disclosed.

[0004]

By the way, when there is a defect or a special region on the surface of the semiconductor chip or the electrode which is the object to be recognized, there is a possibility that it may be erroneously recognized as the boundary of the pattern of the object to be recognized, Accurate pattern recognition was not always possible.

Therefore, an object of the present invention is to provide a novel image recognition method capable of excellent image recognition even in the above case.

[0006]

The present invention for achieving the above object will be described with reference to the reference numerals of FIG. 5 showing an embodiment. Brightness and darkness of a large number of pixels corresponding to a specific region including a pattern to be recognized. Y perpendicular the image data and to the X-axis showing the
Arranged coordinates on consisting of a shaft, placed in each position on the X axis
Te, and X-axis data x1~x8 consisting sum of a plurality of data that is an array in the Y-axis direction together with the respectively obtained at each position on the Y axis, a plurality of the X-axis direction are sequence <br /> Y-axis data y1 to y8 composed of the sum of data are respectively obtained, and the X-axis data x1 to x1 at each position on the X-axis are obtained.
x8 array adjacent to each other on one end side
X-axis data addition value by adding a fixed number of X-axis data
A1 is obtained, and the predetermined plurality is the X-axis data on the X-axis.
The X-axis data addition value is calculated with the number smaller than
One of the arrays of the X-axis data x1 to x8
Shift the X-axis data one by one from one end to the other
Repeatedly execute the array of X-axis data addition values A1 to A8.
The Y-axis data y1 at each position on the Y-axis
Adjacent to each other on one end side of the sequence y8
Add a predetermined number of Y-axis data and add Y-axis data
The value B1 is obtained, and the predetermined number is the Y-axis data on the Y-axis.
The Y-axis data addition value
One of the arrangements of the Y-axis data y1 to y8
Shift Y-axis data one by one from the end of the to the other end
Array of Y-axis data addition values B1 to B8
And the array of the X-axis data addition values A1 to A8 and
The present invention relates to an image recognition method characterized by recognizing the presence or absence or the position of the recognized pattern based on the array of Y-axis data added values B1 to B8 .

[0007]

According to the present invention, the X-axis data x1 to x8 formed by the sum of the data at the respective positions of the X-axis and the Y-axis.
X-axis data addition values A1 to A8 and Y-axis data addition obtained by adding the X-axis data and the Y-axis data, respectively, instead of recognizing the presence or absence or the position of the pattern to be recognized with the Y-axis data y1 to y8. Since the presence or absence or the position of the pattern to be recognized is recognized based on the values B1 to B8, the accuracy of this recognition is increased. That is, clearly in the recognized pattern from other parts
Even if there are indistinguishable parts, this is the X-axis data.
Ru is averaged by determining a data addition value A1~A8 and Y-axis data addition value B1 to B8. Also, add X-axis data
Value of calculation value A1 to A8 and Y axis data addition value B1 to B8
Has a large difference between the recognized pattern and other parts
Therefore, the detection accuracy of the presence or absence of the recognized pattern or the position is high.
Become.

[0008]

EXAMPLE An LED wire bonding system according to an example of the present invention will be described below with reference to FIGS. First, the state of LED wire bonding will be described with reference to FIG. In FIG. 3, 20 is a dice-shaped LED chip having a substantially square surface shape, 21 is a lead frame, and 22 and 23 are leads in the lead frame 21. The chip 20 is fixed to a dish-shaped recess 24 provided at one end of the lead 22 with a silver paste 25. The bottom periphery 26 of the dish-shaped recess 24 is not covered with the silver paste 25. 27 is an Au wire, one end of which is ball-bonded (A) to the center of the minute circular electrode 28 on the chip surface.
The tip of u is made into a ball shape, and this ball-shaped portion is subjected to thermocompression bonding while applying ultrasonic energy.
Nail head bonding is applied to the end face of No. 3. The present system will be described below by taking wire bonding to the electrode 28 on the surface of the chip as an example.

The system for bonding shown in FIG. 3 is constructed as shown in FIGS. FIG. 2 is a conceptual diagram of a system according to the present invention. In this system, a plurality of (8 in this example) automatic bonders 1a, 1b to 1n are provided, which are relatively low-level image recognition functions with emphasis on high speed, versatility, and low cost. , One image recognition support device 4 having a higher image recognition function than the automatic bonders 1a to 1n is provided, and a plurality of automatic bonders 1a to 1n and one support device 4 are selectively connected. A switching device 5 is provided for this purpose.

FIG. 1 shows the system conceptually shown in FIG. 2 in more detail. Each of the automatic bonders 1a to 1n includes a bonding stage 6 and a lead frame 21 on the stage 6.
A camera 7 for picking up an image of the LED chip 20 mounted on the camera, a bonding tool 8 that is movable in the X, Y, and Z axis directions integrally with the camera 7, an image reading device 9, a computer 10, and controllers of respective parts. (Not shown) and the like are included.

The camera 7 as an image pickup device is a general one which sends out an analog image signal in a television signal format. The image reading device 9 connected to the camera 7 is
An analog image signal including a D converter and a frame memory, which corresponds to the brightness of a subject (chip) obtained from the camera 7, is converted into binary image data by an A / D converter,
It is configured to write this in the frame memory. The frame memory has an address corresponding to the pixel of the camera 7, and the image data is written therein. Note that another memory such as a field memory may be used instead of the frame memory.

The computer 10 is related to the image reading device 9, the camera 7, the bonding tool 8, the bonding stage 6, etc., and the pattern is based on the binary image data written in the frame memory of the image reading device 9. Image calculation is performed by the matching method to determine the bonding position. The computer 10 also includes an automatic bonder 1 that includes movements of the camera 7, the bonding tool 8, and the bonding stage 6 and checks and reports of their operating states.
It also controls and manages a.

In this example, a single lead frame 2 is used.
As many (30) LEDs are mounted on 1,
Bonding positions (coordinates) of a plurality (5) of LEDs are continuously determined and stored in a memory.
After that, the data is read from the memory and a plurality of LEDs are read.
Each of the automatic bonders 1a to 1n is configured to continuously perform wire bonding with respect to.

Second to nth automatic bonders 1
Although the internal structure of b to 1n is not shown, the structure is exactly the same as that of the first automatic bonder 1a. Each of the automatic bonders 1a to 1n in FIG. 1 is an automatic bonder having a low-grade image recognition function.

The support device 4 includes an image capturing device 12 and a computer 13. The support device 4 is provided in a ratio of one to n automatic bonders 1a to 1n, and is selectively connected to the automatic bonders 1a to 1n by a switching device 5.

The switching device 5 includes image signal lines 14a, 14 for transmitting the outputs of the cameras 7 of the respective automatic bonders 1a-1n.
Switches 16a, 16b ... 16 for selectively connecting b ... 14n to the common image signal acquisition line 15
17n connected to each automatic bonder side computer 10 and a signal line 18 connected to the assisting device side computer 13
Switches 19a and 19b for selectively connecting and
..With 19n The automatic bonder side computer 10 and the support device side computer 13 exchange signals with each other.

The image capturing device 12 of the support device 4 captures an analog image signal from the camera 7 though the automatic bonders 1a to 1n are incapable of image recognition through the switching device 5, and A / D the same. The converter is configured to convert it into digital image data of 256 gradations and write it in the frame memory. Compared with the image reading device 9 that performs A / D conversion into two gradations of the automatic bonders 1a to 1n, the image capturing device 12 of the support device 4 performs A / D conversion into 256 gradations, so it is possible to perform detailed identification. It is possible to obtain various image data.

The assisting device side computer 13 is configured to execute a three-step image recognition process as shown in principle in FIGS. 5, 6 and 7. That is, the image data of 256 gradations is read from the frame memory of the image capturing device 12, high-grade image recognition calculation processing is performed based on this data, and the result is transferred to the unrecognizable automatic bonders 1a to 1n by the signal line 18. It is configured to send. In this example, the switches 16a to 16 of the switching device 5 are
n, 19a to 19n are controlled by the support device side computer 1
A computer 13 is connected to the switching device 5 by means of a line 5a in order to carry out at 3. Further, the support device side computer 13 is also configured to check the operation status of the automatic bonders 1a to 1n, totalize and the like.

[0019]

[Operation] The plurality of automatic bonders 1a to 1n automatically detect the center coordinates of the electrode 28 to be wire-bonded to the Au wire 27 based on the image recognition and processing functions of the respective automatic bonders 1a to 1n, and wire bond the wire at that position. I do. When the unrecognizable state does not occur in the automatic bonders 1a to 1n, the support device 4 patrols the automatic Hondas 1a to 1n (checks the operation state and counts). That is, the switching device 5 receives the command from the support device 4 and then connects the signal line 17a to the signal line 17a.
~ 17n are connected cyclically. This operation is performed by switch 1
This is easily achieved by making 9a to 19n a multiplexer configuration. When the automatic bonder side computer 10 and the support device side computer 13 are cyclically connected via the signal line 18, an image recognition state display signal (a signal indicating disabled or possible) obtained by the automatic bonder side computer 10 assists. It is given to the device-side computer 13. Since the switches 19a to 19n can be switched at extremely high speed,
Support device side computer 13 has a plurality of automatic bonders 1a
This means that ~ 1n is always monitored.

In this patrol, when the support device 4 detects that the first automatic bonder 1a is incapable of image recognition, the patrol is interrupted by the operation of the switching device 5 based on the command of the support device 4. The image signal line 14a of the automatic bonder 1a in which the image cannot be recognized is connected to the line 15 via the switch 16a, and the analog image signal obtained from the camera 7 is input to the image capturing device 12 and A / 256 is displayed in 256 gradations. D-converted and written in the frame memory. The computer 13 on the supporting device side performs image recognition based on the image data of 256 gradations in the frame memory by a higher-level algorithm than the computer 10 on the automatic bonder side. As a result, target image recognition can be performed except in extremely exceptional cases, and the center coordinates of the electrode 28 to which the Au wire 27 is wire-bonded are determined.

When the first automatic bonder 1a cannot be recognized, the switch 19a is turned on and the data indicating the bonding position obtained from the computer 13 is transferred to the computer 10 of the automatic bonder 1a. In addition,
Lines 17a-17n and 18 are used in a time division manner for both patrol and position data transmission. The automatic bonder-side computer 10 writes the data determined by the support-apparatus-side computer 13 in the internal memory, and executes the target automatic bonding based on the data.

When the data transfer from the computer 13 on the supporting device side to the computer 10 on the bonder side is completed, the supporting image recognition function of the supporting device 4 is completed, and the patrol operation is started again.

The automatic bonder 1a, which has been disabled, is a support device 4
When the predetermined bonding is completed with the help of the above, the normal operation is restored, and the bonding in the image recognition processing of itself is started.

In exceptional cases, the image recognition may not be possible even by the support device 4. In this case, bonding is not performed there (pass bond), and the process proceeds to bonding to the next chip 20. The pass-bonded LED is excluded as a defective product in the subsequent characteristic inspection process of the product. Of course, when the image recognition of the support device 4 fails, the corresponding automatic bonder 1a ...
It can also be set such that 1n issues a call signal to continue the stop, and the worker 2 performs a coping work.

[0025]

[Image Recognition Operation] Next, image recognition of the automatic bonders 1a to 1n and the support device 4 will be described. First, in the automatic bonders 1a to 1n, the analog image signal taken in by the TV camera 7 using the image pickup tube is converted by the image reading device 9 into binary image data. That is, the predetermined density (brightness)
The above pixels are white (logic 1), and the pixels having a density lower than the predetermined density are black (logic 0) to simplify the image information into two gradations. Then, the binarized image data is written in a frame memory, the computer 10 compares the image data with a preset binarized standard pattern, and a region matching the standard pattern is detected from the binarized image data. To do. In this example, since the circular electrode 28 of the chip is brighter than the surroundings, a standard pattern including a white (logic 1) circle corresponding to the circular electrode 28 is set. When the coordinates of the area matching the standard pattern are detected, the coordinates of the bonding point preset for the standard pattern can be determined, and therefore the bonding is performed at these coordinates. In this example, the center point of the white (logic 1) circle is the bonding point. The number of pixels in the frame memory of the image reading device 9 is 120 × 120 = 14400, which corresponds to one half of the pixels in the central portion of the tube surface (about half of the tube surface) of the camera 7. Of course, the pixel at the central portion of the tube surface of the camera 7 may be extracted in a non-sampling state.

By the above image recognition, even at a low time, 9
Wire bonding is performed on about 6% of the chips 20. In other words, when the number is large, the image recognition of the chip 20 becomes impossible (a state in which the area matching the standard pattern cannot be detected) at a rate of several%. In this case, the operator 2 instantaneously shifts to image recognition by the support device 4. The support device 4 executes three-step image recognition. FIG. 4 shows an example of an image of a region 29 which is the target of pixel calculation processing in the first stage, and an analog image signal captured by a TV camera is projected on a cathode ray tube (not shown). Although the image actually has a slight density difference, in the drawing, the dark portion is shown by hatching and the whitish portion is shown by white. Reference numerals in FIG. 4 correspond to the respective portions in FIG. In this FIG. 4, black stripes 30 and 31 that may make an image impossible are formed on the chip 20. If the stripes 31 are applied to the electrodes 28, the automatic bonders 1a to 1a
In n, the image cannot be recognized. A blackish ring-shaped portion 32 appears around the circular electrode 28. The ring-shaped portion 32 is generated because the electrode 28 is a truncated cone-shaped protrusion and the inclined side surface is dark.

In the first step of image recognition by the support device 4, a whitish quadrangle image having a size in a predetermined range is extracted from the image of FIG. 4 to obtain a rough position of the chip 20. To explain this in detail, the image data is read from the frame memory of the image capturing device 12 in which the analog image signal from the TV camera 7 is written as image data of 256 gradations (256 values), and the computer 13 binarizes it. Convert to image data. Then, pixel calculation processing is performed so as to perform square feature extraction based on the binarized pixel data. The square feature extraction is performed, for example, by ignoring white portions other than the square having a predetermined size and removing unnecessary black portions in the predetermined square. FIG. 5 shows the principle for determining the chip-corresponding square image 33 indicated by the dotted line and the center position thereof in the first stage. A logic "1" is obtained in the white part and a logic "0" is obtained in the black part corresponding to each pixel. As a result, a distribution pattern of logic "1" corresponding to the pattern of the chip 20 is obtained. Since the threshold value for obtaining the data in the first stage is different from the threshold value for reading the data into the frame memory on the automatic bonder side, the electrode 2
8 as well as the entire upper surface of the chip 20 is detected as white. In FIG. 5, the logical "1" is ideally distributed in a square, but the distribution is not necessarily such.
Therefore, the sum of the data arranged in the Y-axis direction at each position on the X-axis is calculated as x1 = 0, x2 = 0, x3 = 4, x
4 = 4, x5 = 4, x6 = 4, x7 = 0, x8 = 0, and these x1, x2 ...
Addition values A1 and A2 of the number of pixels (4 pixels) corresponding to the width of 0
・ ・ ・ Sequentially obtained by moving A8 pixel by pixel to the right. In this example, since four sums are sequentially obtained, A1 = x
1 = 0, A2 = x1 + x2 = 0, A3 = x1 + x2 + x
3 = 4, A4 = x1 + x2 + x3 + x4 = 8, A5 = x
2 + x3 + x4 + x5 = 12, A6 = x3 + x4 + x5
+ X6 = 16, A7 = x4 + x5 + x6 + x7 = 12,
A8 = x5 + x6 + x7 + x8 = 8, the maximum value A6
It can be seen that x6 on the X axis at which is obtained is the right end of the square image 33 corresponding to the chip 20. Similarly, at each position on the Y axis, y1, y2, y3, ... Y8 are obtained, and A1 ~
B1 to B8 are calculated so as to correspond to A8, and the square image 33
Determines that the upper edge of is at the y7 position on the Y-axis. Since the size of the chip 20 (four pixels in both the X-axis and Y-axis directions) is known in advance, the positions of the square image 33 on the X-axis and the Y-axis should be determined as x3 to x6 and y4 to y7, respectively. You can In FIG. 5, the number of pixels is set to 8 × 8 = 64 in order to simplify the drawing, but in reality, for example, 18 × 19 =
There are many pixels such as 342. The frame memory of the image capturing device 12 of the support device 4 is, for example, 128 ×
It has 128 = 16384 pixels. This corresponds to one in which 512 × 512 pixels in the camera 7 are ¼ sampled. The 18 × 19 pixels in the first stage are obtained by sampling the frame memory 1/6. Since data is written in the frame memory with 256 gradations, binarized data is obtained with an appropriate threshold value, and this is used as the data of each pixel in FIG.

After the extraction of the quadrilateral image 33 is completed, the computer 13 judges whether or not the condition (which is set in advance) for the chip 20 to be satisfied is satisfied. Move on to the stage. When not satisfied,
If the threshold value for binarization is changed and the extraction is performed again and the determination condition is still not satisfied, the image is recognized as unrecognizable.

In the second step, the pixel calculation processing target area is narrowed based on the result of the first step, and a more accurate chip 2 is obtained.
Find the position of 0. At this stage, as shown in FIG. 6, the horizontal band 35 and the vertical band 3 are substantially aligned with the center 34 of the rectangular image 33.
The area of 6 is targeted for pixel calculation processing, and the position of each side of the square image 39 corresponding to the chip 20 shown in FIG. 7 is obtained. In this case, the gradation of the image data that is the target of the pixel calculation processing is set to 256 gradations so that a slight temperature difference can be dealt with. That is, the 256 gradation data obtained from the frame memory of the image capturing device 12 is used as it is, and the data of each pixel of the horizontal band 35 and the vertical band 36 is arithmetically processed according to the same principle as in FIG. Accurately find the position of each side of. For convenience of illustration, the horizontal band 35 and the vertical band 36 are each shown as 11 × 3 pixels in FIG. 6, but in reality, the horizontal band 35 is 18 × 5 pixels, and the vertical band 36 is 5 × 14 pixels. 1 / 128x128 pixel frame memory of capture device 12
Obtained by sampling 4 times.

In the third stage, as shown in FIG. 7, a horizontal band 37 consisting of an x-direction array of one pixel and a y-axis of one pixel are aligned with the center of the position of the chip 20, that is, the square image 39, obtained in the second stage. The area with the vertical band 38 formed of the directional arrangement is specified, pixel calculation processing is performed, and the position (center coordinates) of the circular electrode 28 is obtained. At the start of the pixel calculation process of the third stage, the horizontal band 3
The intersection 40 of 7 and the vertical band 38 is located at the center of the square image 39.
Here, the position of the circular electrode 28 is obtained by detecting the dark ring-shaped portion 32 (see FIG. 4) around the circular electrode 28. The gradation of the image data that is the target of the pixel calculation is 256 gradations so that it can correspond to the slight density difference on the surface of the chip 20. For the calculation result, it is determined whether or not the preset condition for the electrode 28 is satisfied, and if it is not satisfied, it is treated as being incapable of image recognition. Note that, in FIG. 7, for convenience of illustration, the horizontal band 37 and the vertical band 38 are each 9 pixels, but in actuality, the horizontal band 37 is, for example, 30 pixels, and the vertical band 38 is, for example, 25 pixels. The number of pixels. The pixels of the horizontal band 37 and the vertical band 38 of FIG. 7 are obtained in a non-sampling state by making a 1: 1 correspondence with the pixels of the frame memory of 128 × 128 pixels.

The image data described so far are all monochrome (black and white). However, the image recognition in the support device 4 may handle color image data. For example, in the case of image data separated into three primary colors of red, green, and blue, even if the image data of each of the three primary colors is converted into binary image data corresponding to the density and handled, a single color multi-valued image is obtained. Equivalent information can be obtained.

The support device 4 is adapted to set individual image recognition conditions (threshold value for binarization, chip size, electrode size, etc.) for each automatic bonder 1a-1n. There is. Even when all the automatic bonders 1a to 1n are used to manufacture LEDs of the same specification using the chips 20 of the same specification, for example, the brightness of the illumination of each automatic bonder 1a to 1n is slightly different, and each automatic bonder 1a to The threshold value is set for each 1n. When a mere chip 20 is used in units of the automatic bonders 1a to 1n due to the need for multi-product parallel production, the chip size and the electrode size are individually set accordingly.

As a result of the three-step image recognition by the support device 4, 90% or more of the chips 20 that could not be image-recognized by the automatic bonders 1a to 1n can be image-recognized, and even if the total is low, about 99.6%. Wire bonding is automatically performed on the chip 20. When the number of chips 20 is 0.4% or less when the image cannot be recognized,
Make a pass bond or call signal as described above. The image recognition and processing time of the support device 4 is
It is an average of 0.7 seconds, which is a value suitable for managing the eight automatic bonders 1a to 1n under the above image recognition rate. The image recognition and processing time of the support device 4 is longer than the image recognition time of the automatic bonders 1a and 1n. This is contrary to the general relationship that higher systems are faster than lower systems. In this way, the image recognition and processing device does not require high speed, so
In the support device 4, an inexpensive general-purpose 16-bit personal computer is operated under a program written in C language and assembler language to perform pixel calculation processing.

[Brief description of drawings]

FIG. 1 is a block diagram showing a bonding system according to an embodiment of the present invention.

FIG. 2 is a conceptual diagram of the system of FIG.

3 is a perspective view showing an LED to be bonded in the system of FIG. 1. FIG.

FIG. 4 is a diagram illustratively showing an image of an object (LED) on a cathode ray tube.

5 is a diagram showing in principle the first stage image recognition in the support device of FIG.

FIG. 6 is a diagram showing in principle the image recognition in the second stage of the support device.

FIG. 7 is a diagram showing in principle the image recognition in the third stage of the support device.

[Explanation of symbols]

 A1 to A8 X-axis added value B1 to B8 Y-axis added value

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication H01L 21/68 E (56) References JP-A-55-121584 (JP, A) JP-A-59 -66785 (JP, A) JP-A-59-51536 (JP, A)

Claims (1)

[Claims] 1. Image data indicating the brightness of a large number of pixels corresponding to a specific region including a pattern to be recognized is arranged on a coordinate composed of an X axis and a Y axis orthogonal thereto, and each position on the X axis.
In, X-axis data consisting of the sum of the plurality of data being an array in the Y-axis direction (X1 to X8) with respectively obtained are array Oite, the X-axis direction at each position on the Y axis Y-axis data (y1 to y that is the sum of a plurality of data
8) respectively, and the X-axis data (x1 to x ) at each position on the X-axis is obtained.
8) Places adjacent to each other on one end side of the array
X-axis data addition value by adding a fixed number of X-axis data
(A1) is obtained, and the predetermined plurality of X-axis data are on the X-axis.
The number of X-axis data added is smaller than the number of data
To obtain the array of the X-axis data (x1 to x8)
One X-axis data from one end to the other
X-axis data addition value (A1-A
8) is made, and the Y axis data (y1 to y) at each position on the Y axis is created.
8) Places adjacent to each other on one end side of the array
Y-axis data addition value by adding a fixed number of Y-axis data
(B1) is determined, and the predetermined plurality of Y-axis data are on the Y-axis.
The number of Y-axis data added is smaller than the number of data
To obtain the array of the Y-axis data (y1 to y8)
One Y-axis data from one end to the other
Shift and repeat to execute Y-axis data addition value (B1-B
8) Make an array and add the X-axis data added value (A1 to A8) to the Y-axis
An image recognition method characterized in that the presence or absence or the position of the recognized pattern is recognized based on an array of data addition values (B1 to B8) .