JP3759983B2 - Image processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention provides an image processing apparatus.In placeIn particular, a device for processing an image captured by an imaging device.In placeIt is related.
[0002]
[Prior art]
This type of image processing apparatus generally includes an imaging apparatus that captures an image processing object, an image data storage unit that stores data of an image captured by the imaging apparatus, and image data stored in the image data storage unit. Image data processing means for processing.
For example, in an image processing device in which the imaging device is configured by a CCD camera, binarized data is obtained based on the respective charge amounts of a large number of charge-coupled elements that constitute the imaging surface, whereby the edge of the image processing object, Size, position, rotation angle, etc. can be obtained. Conventionally, a normalized correlation method and a vectorized correlation method are known as methods for processing image data and recognizing an image processing target.
[0003]
[Problems to be solved by the invention]
An object of the present invention is to provide an apparatus and a method capable of processing image data by a method different from the normalized correlation method and the vectorized correlation method.
[0004]
[Means for solving the problems and functions and effects of the invention]
In order to solve the above problems, the image processing apparatus according to the present invention is configured as follows, and the following operations and effects are obtained.
(1) An imaging device that includes an imaging surface in which a large number of imaging elements are arranged, and that images an image processing object;
Image data storage means for storing data of an image captured by the imaging device;
While sandwiching the edge of the image processing object in between, a pair of two points located at a certain distance larger than the arrangement pitch of the image pickup device of the image pickup device, and at a plurality of positions on the edge Corresponding search template data storage means for storing data of a search template having a plurality of preset point pairs;
When the search template of the search template data storage means is overlaid on the screen on which the image represented by the image data of the image data storage means exists, the optical characteristic values of the pairs of points constituting the plurality of point pairs When the difference state is equal to or greater than the set state, it is assumed that one of the pair of points is located inside the edge of the image processing object and the other is in the matching state located in the background, and the plurality of sets of point pairs And determining means for determining that the image processing target is a search target that matches the search template if a set amount or more is in the matching state.
(Claim 1).
When an image is picked up by the image pickup apparatus in the image processing apparatus of this section, the image pickup surface is decomposed into a large number of pixels, and image data is created for each pixel and stored in the image data storage means. An “image represented by image data” is an image obtained on the basis of image data, and may be an image in which a pixel is used as a unit and an optical characteristic value is obtained for each pixel. It may be an image whose value is obtained at an arbitrary point.
An example of the former is an image that is considered as a set of electrical signals for each imaging element obtained by an imaging apparatus that includes a large number of imaging elements and generates an electrical signal corresponding to the light receiving state of each imaging element. Can be said to be an image physically formed on an imaging surface on which imaging elements are arranged. In this sense, this image is referred to as a physical image, and a screen on which the physical image exists (which matches the imaging surface) is referred to as a physical screen. The physical image data can be stored and reproduced by storing the electrical signal data for each image sensor in the storage means in association with the position of each image sensor.
In addition, image data representing a physical image actually exists. In this sense, a physical image can also be referred to as a real image in contrast to a virtual image described later, and a screen on which a real image exists may be referred to as a real screen. it can.
The image data representing the actual image may be analog data or digital data representing the magnitude of the electric signal for each image sensor, and for example, multi-stage digital data (referred to as gradation data) represented by 256 discrete values. Or binarized data binarized depending on whether or not the electrical signal for each image sensor exceeds a threshold value.
An example of an image in which an optical characteristic value is obtained at an arbitrary point without using a pixel as a unit is that the electrical signal data of each image sensor on the physical screen represents the optical characteristic value at the center point of each image sensor. It is an image that is considered as a set of continuous optical characteristic values that define the curved surface when it is assumed that the curved surface satisfies the optical characteristic values of these many points. The image data of this image can be stored as, for example, data of an expression representing the curved surface, and this image data is also a kind of real image.
On the other hand, when an arbitrary point on the screen is designated instead of obtaining the curved surface data in advance, the optical characteristic value of only that point is obtained by calculation every time each point is designated. Is also possible. In this case, the image does not actually exist, and if it exists, it is only virtualized. Therefore, it is referred to as a virtual image, and a screen on which the virtual image exists is referred to as a virtual screen.
Search template data is data that defines the position of the two points that make up a point pair on the screen. “To superimpose the search template on the screen on which the image represented by the image data exists” means that the screen is a physical screen. At each position specified by the search template data. It is to read the image data of a pixel from the image data storage means and obtain the optical characteristic value. When the screen is a virtual screen, it is assumed that the search template is placed on the virtual screen and specified by the search template data. The optical characteristic value of each point on the virtual screen is calculated from the physical image data. The physical image data in this case needs to be analog data or digital data representing the magnitude of the electric signal for each image sensor itself, or gradation data, and binary data has no meaning.
In any case, there is a difference between the image data corresponding to the image processing object and the image data corresponding to the background. In other words, unless there is a difference from the background, it is not possible to acquire information about the image processing object by image processing.
Therefore, if one point of the point pair is located inside the edge of the image processing object and the other point is in the background, in other words, the image data corresponding to one point of the point pair is the image processing object. If the data corresponding to the other point is data representing the background, the difference state of the optical characteristic values of the two points constituting the point pair should be equal to or greater than the set state. When the number (or rate) of pairs of point pairs whose state is not equal to or higher than the set state is equal to or less than a predetermined set number (or set rate), it is determined that the image processing target is a search target. be able to. Further, by searching for such a pair of point pairs, an image of the image processing object can be searched for on the screen.
“Optical characteristic value” is, for example, luminance, hue, etc., and “above set state” means that the difference in luminance, hue, etc. is greater than or equal to a set value, and the ratio of brightness, hue, etc. That is all.
When an optical characteristic value is obtained by superimposing a search template on a physical screen, the image data may be analog data or gradation data (hereinafter referred to as gradation data) or binarized data. In the case of gradation data or the like, it is optical that the difference between the value of one of the two points of the point pair and the value of the other point is greater than or equal to the set value, or the ratio of both values is greater than or equal to the set ratio. The difference state of the characteristic value is not less than the set state. In the case of binarized data, if one value of two points of the point pair is 0 and the other value is 1, the difference state of the optical characteristic value is equal to or greater than the set state.
When the position of the image processing object is substantially constant, the search template may be superimposed on the screen at a certain position, but when at least one of the position of the image processing object and the rotation angle is indefinite. The determination means repeats the determination while changing at least one of the position and the rotation angle of the search template until the determination result becomes affirmative or a predetermined change limit is reached. When a search template is changed, even if multiple search templates with different positions and rotation angles are prepared in advance, only the standard search templates with standard positions and standard rotation angles are prepared, and search templates with general positions and rotation angles are prepared. May be created by coordinate transformation as necessary. The “change limit” means that, for example, when a search area for searching for an image processing object is set, if the position of the search template is further changed, the search template has reached a position where the search template protrudes from the search area. The position of the search template has been changed a predetermined number of times.
One of the preferred fields of use of the image processing apparatus according to the invention is an electronic circuit assembly line. For example, in an electronic component mounting apparatus, an electronic component to be mounted on a mounting target material such as a printed circuit board or a reference mark provided on the printed circuit board is an image processing target. In a screen printing machine, the reference mark is provided on a screen. Reference marks and the like are image processing objects. Further, the image processing object is not limited to the outline of the entire object, and a part may be an image processing object. For example, in the case of a square chip, the chip itself is an image processing target. However, in an electronic component having leads, solder bumps, etc., the main body having the leads, solder bumps, etc. is used as the image processing target. In some cases, the lead or one solder bump is an object to be image processed. In addition to the leads positioned outside the outline of the main body, objects positioned inside the outline of other members, such as leads positioned inside and solder bumps, may be image processing objects. In the latter case, the search template is created with the outline of the object located inside the outline of the entire object as a search target, and inside the image of the outline of the entire object on the physical screen or the virtual screen. Then, the image processing target object is searched by superimposing the physical image or the virtual image of the object located inside at a position where the formation of the virtual image is scheduled.
As is clear from the above description, according to the invention of this section, it is possible to know the presence or absence of a search target and whether or not the image processing target is a search target using image data and a search template. An image processing apparatus can be obtained that can determine by performing the calculation and comparison of the optical characteristic values of two points constituting a point pair by the number of point pairs, and can perform processing easily and quickly. .
(2) Further, the search template data is generated by changing at least one of a rotation angle and a position of a master search template set with respect to the edge of the image processing object having no positional deviation, and generating the search template data. A search template generation means to be stored in the storage means;
Repeating means for repeating the search template generation by the search template generation means and the determination by the determination means until the determination result of the determination means becomes affirmative;
including (1) An image processing apparatus according to claim 2 (claim 2).
(3) Furthermore, means for generating a re-search template including a plurality of straight lines connecting two points constituting the plurality of point pairs of the search template as a plurality of seek lines,
Means for re-searching the search object by calculating coordinates of intersections of the plurality of seek lines of the generated re-search template and the edge of the image object;
including (1) Term or (2) An image processing apparatus according to claim 3 (claim 3).
If a re-search template is set by defining both ends of the seek line by two points constituting a plurality of point pairs of the search template, the re-search template can be easily set. The search template in this case is a search template that is determined that the image processing object is a search object that matches the search template, and a point pair that is greater than or equal to the set amount is in a matching state, so that the position of the image processing object is shifted to the image processing object. Even if there is a shift in the rotation angle or the like, it is guaranteed that the edge point can be calculated for a seek line larger than the set amount.
(4) Further, based on the search template in which the determination result of the determination unit is affirmative, the edges of the image processing target object are sandwiched, and a certain distance larger than the array pitch of the image pickup elements of the image pickup device is set. A measurement template generating means for generating a measurement template having a plurality of seek lines formed by connecting two points positioned apart from each other in a straight line and corresponding to a plurality of positions on the edge;
Measurement template data storage means for storing measurement template data which is data of the measurement template generated by the measurement template generation means;
The measurement template of the measurement template data storage means is overlaid on the screen on which the image represented by the image data of the image data storage means exists, and the coordinates of the edge points of the image processing object on each of the plurality of seek lines are set. Edge point coordinate calculation means to calculate and
including (1) Term or (3) An image processing apparatus according to any one of claims 1 to 4 (claim 4).
The above “screen on which the image represented by the image data exists” (1) Similarly to the image processing apparatus according to the item invention, it may be a physical screen, a real screen, or a virtual screen.
“Overlay the measurement template on the screen” means, for example, reading image data corresponding to the pixel specified by the measurement template data on the physical screen from the image storage device, or by specifying the seek line on the virtual screen The image data (optical characteristic value) of the point to be calculated is calculated based on the image data of the physical screen.
Since there is a difference in the optical characteristic value between the image processing object and the background, the change gradient of the optical characteristic value is maximized at the edge point position. Therefore, the edge point can be determined by obtaining the change in the optical characteristic value obtained for each pixel on the physical screen and by obtaining the change gradient of the optical characteristic value on the virtual screen.
On the physical screen, the edge point is obtained in units of pixels, but on the virtual screen, the edge point is obtained at an arbitrary point on the seek line, and the resolution of the image processing is increased.
According to the invention of this section, an image on which both searching for a search object and calculation of edge point coordinates is performed. An image processing apparatus is obtained. Therefore, for example, among the image processing objects, the edge point can be calculated only for the object determined to be the search object, and there is an object different from the object for which the edge point is to be calculated for the image processing object. When they are mixed, it is not necessary to perform the calculation for all the image processing objects, and a decrease in the image processing speed can be avoided.
If the measurement template is set by defining both ends of the seek line by two points constituting a plurality of point pairs of the search template, the measurement template can be easily set. The search template in this case is a search template that is determined that the image processing object is a search object that matches the search template, and a point pair that is greater than or equal to the set amount is in a matching state, so that the position of the image processing object is shifted to the image processing object. Even if there is a shift in the rotation angle or the like, it is guaranteed that the edge point can be calculated for a seek line larger than the set amount.
However, it is not essential to define the seek line by the two points that make up the point pair, and may be set separately from the point pair, and the number of seek lines may not be the same as the number of point pairs. .
According to the invention of this section, the edge point of the image processing object can be obtained. Thereby, for example, when the shape of the image processing object is known in advance, the size, position, rotation angle, etc. of the image processing object can be calculated with relatively few seek lines. If the number of seek lines is increased and many edge points are obtained even if the shape of the image processing object is not known in advance, the shape is obtained from the set of edge points, and the size, position, and rotation are obtained. An angle or the like can be obtained.
(5) Further, the image processing object includes a confirmation unit that confirms that the image processing target is a search target based on a plurality of edge point coordinates calculated by the edge point coordinate calculation unit, and the confirmation unit includes A midpoint reference type confirmation unit for performing the confirmation based on a deviation between an edge point of the image processing object on the seek line and a midpoint of the seek line. (Four) An image processing apparatus according to claim 5 (claim 5).
In the image processing apparatus according to the invention of this section, it is confirmed that the image processing object is a search object. For example, when there are multiple types of image processing objects and only the determination by the determination means can only determine that the image processing object may be a search object, and it is not possible to determine whether it is really a search object In addition, the confirmation means confirms that the image object is really a search object based on the calculation result of the edge point by the edge point coordinate calculation means.
The confirmation is performed based on the deviation between the edge point of the image processing object on the seek line and the midpoint of the seek line. For example, when the search object is a rectangular chip, the same number of seek lines of the same length are set for two sides parallel to each other, and the position of these seek lines is determined between the search object and the measurement template. If there is no deviation, it is determined that the midpoint of all seek lines matches the edge point of the search object. If the sum of the deviations considering the directionality between the edge point of the object and the midpoint of the seek line is 0, the image processing object can be confirmed to be the search object.
According to the invention of this section, it is possible to confirm whether or not the image processing object is a search object, and an effect of improving the reliability of the determination is obtained compared to the determination using only the search template. In addition, when another image process is performed after the determination, it is possible to avoid performing the process unnecessarily for the object that is not the search target.
(6) Furthermore, it includes an object calculation means for calculating at least one of the size, position and rotation angle of the image processing object based on the plurality of edge point coordinates calculated by the edge point coordinate calculation means, and The object calculation means calculates at least one of the size, position and rotation angle of the image processing object based on a distance between an edge point of the image processing object on the seek line and a midpoint of the seek line. Includes midpoint reference type object calculation means
(Four) Term or (Five) An image processing apparatus according to claim 6 (claim 6).
In the image processing apparatus according to the present invention, at least one of the size, position, and rotation angle of the image processing object is calculated based on the edge point coordinates. The necessary size, position and rotation angle are calculated. This calculation is performed on the seek line. This is performed based on the distance between the edge point of the physical object and the midpoint of the seek line.
If the number of coordinates of the edge point is so large that the shape of the image processing object can be specified, any of the size, position, and rotation angle of the image processing object can be calculated using only the coordinates of the edge point. Even when the number of coordinates of the edge point is small, for example, the size of the image processing object can be calculated by using auxiliary data such as data indicating the type of the image processing object. When the shape of the image processing object is one type such as a rectangle or a circle, the size of the image processing object can be calculated without auxiliary data.
According to the invention of this section, at least one of the size, position, and rotation angle of the image processing object can be known. As a result, for example, identification of image processing objects having the same shape but different sizes, pass / fail determination of positioning of the image processing object, calculation of misalignment correction amount, pass / fail determination of rotation angle determination, and rotation angle deviation correction A quantity calculation or the like can be performed.
Moreover, since the size of the image processing object is calculated based on the distance between the edge point of the image processing object and the midpoint of the seek line, the value used for the calculation can be small on average and the error is small. As a result, an accurate calculation result can be obtained.
(7) Further, automatic seek line setting that automatically sets the plurality of seek lines to positions where the expected positions of the edge points on the seek lines are expected to coincide with the edge points calculated by the edge point coordinate calculation means. Setting means;
Repeating means for repeating the automatic setting of the seek line by the automatic seek line setting means and the calculation of the edge point coordinates by the edge point coordinate calculating means on the automatically set seek line a plurality of times
including (Four) Term or (6) An image processing apparatus according to any one of Claims (Claim 7).
In the image processing apparatus according to the present invention, the seek line is automatically set by the seek line automatic setting means. The simplest example is that an image processing apparatus having a determination unit and an edge point coordinate calculation unit configures a point pair of a search template in which it is determined that the image processing target is a search target that matches the search template 2 This is a case where a straight line connecting points is automatically set as a seek line.
It is also possible to automatically set a seek line based on the coordinates of the edge point calculated by the edge point coordinate calculation means and further calculate the edge point using this as a second measurement template. If the number of seek lines in the second measurement template is set larger than the number of seek lines in the first measurement template, the coordinates of the edge points become more accurate and numerous, and the size, position, rotation angle, etc. are more accurate. Is measured.
In this way, when the seek line is automatically set for the second and subsequent times, based on the edge point coordinates calculated immediately before, the planned edge point position (for example, the midpoint) on the seek line matches the actual edge point position. If the seek line is automatically set at the expected position, the greater the number of operations, the smaller the error between the expected edge point position on the seek line and the actual edge point of the image processing object, and the coordinates of the edge point. Can be obtained with high accuracy.
According to the invention of this section, since the seek line is automatically set by the seek line automatic setting means, it is possible to obtain an effect that the amount of work required by the user of the image processing apparatus can be reduced. Further, an image processing apparatus that can calculate the coordinates of the edge points of the image processing object with higher accuracy is obtained.
(8) The image pickup device generates an electric signal corresponding to the light receiving state of each of the multiple image pickup devices, and the image data storage means stores the electric signal data of each of the image pickup devices. Is stored in association with the position of the image processing apparatus, and the image processing apparatus
Furthermore, a point designating unit for designating an arbitrary point on the virtual screen corresponding to the physical screen formed by the image data of the image data storage unit,
Virtual point data calculating means for calculating an optical characteristic value of a specified point based on optical characteristic value data on the physical screen for each point specification by the point specifying means;
including (Four) Term or (7) An image processing apparatus according to any one of Claims (Claim 8).
In the image processing apparatus according to the invention of this section, the optical characteristic value of an arbitrary point on the virtual screen is calculated every time a point is designated by the point designating means. A virtual screen is a collection of innumerable points, of which optical characteristic values of only necessary points specified by the point specifying means are calculated. The
In this way, every time a point is designated by the point designating means, the optical characteristic value is calculated, and the calculation is performed only for the necessary points, so that the time required for the entire image processing can be reduced. Further, the storage capacity of the optical characteristic value storage means for storing the calculated optical characteristic value may be small.
(9) The point designating unit designates one of a plurality of division points set at a pitch smaller than the arrangement pitch of the imaging elements on the seek line as the arbitrary point, and the virtual point data The calculation means includes a division point characteristic value acquisition means for acquiring the optical characteristic value of the designated division point, and the image processing apparatus acquires the optical characteristic of the division point acquired by the division point characteristic value acquisition means. Includes edge point search means for searching for the sharpest change point of the optical characteristic value on the seek line based on the value as an edge point (8) An image processing apparatus according to claim 9 (claim 9).
In the image processing apparatus according to the invention of this section, when the screen on which the seek line is superimposed is a physical screen, the optical characteristic value of the dividing point is the image data storage means that stores the image data of the pixel corresponding to the position of the dividing point. The edge point is searched for based on the optical characteristic values. Edge points are searched on a pixel-by-pixel basis.
When the screen on which the seek line is superimposed is a virtual screen, the dividing point can be set on the virtual screen regardless of the pixel size of the physical screen, and the optical characteristic value of the dividing point is the optical value on the physical screen. Calculation is performed based on characteristic value data (image data).
According to the invention of this section, the search for the edge point is performed only on the seek line, and the edge point is determined based on the optical characteristic values at the limited number of division points. Edge points can be determined. Further, the division points on the virtual screen are set at a pitch smaller than the pitch of the image sensor, and the optical characteristic value is calculated based on the optical characteristic value data on the physical screen. Changes in the optical characteristic values are searched at a finer pitch on the seek line, and the resolution of the image processing is increased.
(10) Furthermore, an input device;
A monitoring device;
Input related data display means for causing the monitor device to display input related data related to the input of the input device;
Image processing data display means for displaying on the monitor device image processing data such as progress and results of image processing by the image processing device;
Input related data priority display means for giving priority to display by the input related data display means over display by the image processing data display means;
including (1) Term or (9) An image processing apparatus according to claim 10 (claim 10).
In the image processing apparatus according to the invention of this section, the display unit of the input related data and the display unit of the image processing data are used together, and when the display of both data is necessary at the same time, the display of the input related data has priority. Is done.
“Input-related data” is not limited to the data itself input by the input device, but is used to input data such as data obtained by performing an operation based on the input data and data read from the storage means. Including data obtained accordingly. By displaying this input-related data and image processing data on the monitor device, the operator can know whether or not the data input has been performed correctly and the progress and result of the image processing. Etc. can be performed.
According to the invention of this section, it is possible to display both input-related data and image processing data using a single monitor device, and an image processing device that has a simple configuration and is inexpensive can be obtained. Moreover, since display of input related data is given priority, data input is not hindered by display of image processing.
(11) The imaging device is preliminarily selected from at least one of an electronic component mounted on a printed circuit board to form an electronic circuit board, a reference mark provided on the printed circuit board, and a reference mark provided on a screen of a screen printing machine. Imaging a defined object as the image processing object The search template data storage means stores the search template set in advance for an edge of the predetermined one, and the determination means is imaged by the imaging device. It is determined that the image processing object is the predetermined object. (1) Term or (Ten) An image processing apparatus according to any one of the items.
Search objects are predetermined ones among electronic components mounted on a printed circuit board to constitute an electronic circuit board, reference marks provided on the printed circuit board, and reference marks provided on the screen of a screen printing machine. In this case, since the shape and dimensions are known in advance, it is easy to generate and set search templates, re-search templates, measurement templates, etc. Since it is required to perform detection, these are very suitable as an application object of the present invention.
(12) The edge point coordinate calculation means uses the point having the largest absolute value of the change gradient of the image data on the seek line based on the image data stored in the image data storage means as the edge point. Is to calculate coordinates (Four) Term or (9) An image processing apparatus according to any one of the items.
As the image data, optical characteristic values of each part of the image, for example, brightness value data can be used. At the edge point, the absolute value of the change gradient of the optical characteristic value is usually maximized.
(13) When the absolute value of the change gradient of the image data on the seek line is equal to or less than a set value, the edge point coordinate calculation means determines that an edge point cannot be detected for the seek line. Includes fail processing means for preventing the calculation of the coordinates of the edge points. (12) The image processing apparatus according to item.
For example, if dirt is attached near the edge point of an image processing object whose luminance value is larger than the surrounding area, or there is a defect, the luminance value in the vicinity of the edge point decreases and differs from the background luminance value. Becomes smaller. Even in such a case, if the point where the absolute value of the change gradient of the image data on the seek line is the largest is the edge point, the boundary between the normal surface of the image processing object and the dirt or missing portion is erroneously edged. There is a possibility of being pointed. This aspect is effective in avoiding such inconvenience.
(14) Furthermore,
Object computing means for computing at least one of the size, position and rotation angle of the image processing object based on a plurality of edge point coordinates computed by the edge point coordinate computing means;
When the number of seek lines in which the failure has occurred is smaller than a set number, the object calculation means performs at least one calculation of the size, position, and rotation angle, and the number of seek lines in which the failure has occurred is set to the set number. If this is the case, object calculation means control means for preventing at least one calculation of the size, position and rotation angle by the object calculation means;
including (13) The image processing apparatus according to item.
(15) The object calculation means includes actual edge point coordinates that are the coordinates of the actual edge point on the seek line on which the edge point coordinate calculation has been performed by the edge point coordinate calculation means, and the target position of the edge point. Image processing object position deviation calculation means for calculating the position of the representative point of the image processing object as the direction and amount of deviation from the target representative point position based on a set of deviations from the target edge point coordinates. (14) The image processing apparatus according to item.
The calculation of the direction and amount of deviation of the representative point position of the image processing object from the target representative point position takes into account whether or not multiple seek lines have failed and which seek line has failed. Done.
(16) A size calculation unit that calculates a size factor that is a ratio between a size calculated by the size calculation unit and a normal size by the size calculation unit that calculates the size of the image processing target. And the image processing object position deviation calculating means includes a size factor calculated by the size factor calculating means and a coordinate of a normal edge point on the seek line of the image processing object having a normal dimension. Including target edge point coordinate calculation means for calculating the target edge point coordinates based on certain normal edge point coordinates (15) The image processing apparatus according to item.
(17) The size factor calculating unit includes a single size factor acquiring unit that acquires a simple average value of a ratio between an actual dimension and a normal dimension in different directions of an image of the image processing object as a size factor. (16) The image processing apparatus according to item.
(18) A plurality of size factor obtaining means for obtaining a plurality of size factors by separately calculating a ratio between an actual dimension and a normal dimension in different directions of the image of the image processing object. including (16) Term or (17) The image processing apparatus according to item.
(19) Until the determination unit determines that the image processing object is a search object that matches the search template, the determination is performed on a plurality of types of search templates having different positions and rotation angles. What to do (1) Term or (18) An image processing apparatus according to any one of the items.
(20) Until the determination unit determines that the image processing object is a search object that matches the search template, the determination is performed on a plurality of types of search templates having different positions and rotation angles. The image processing apparatus performs each pair of search templates determined by the determining means that the image processing object is a search object that matches the search template among the plurality of types of search templates. Including first seek line setting means for setting the seek line by connecting the point pairs in a straight line. (1) Term or (19) An image processing apparatus according to any one of the items.
(21) Further, the apparatus further includes second seek line setting means for setting the seek line based on the edge point coordinates calculated by the edge point coordinate calculation means, and the edge point coordinate calculation means is set by the second seek line setting means. The edge point is calculated again for the determined seek line. (20) The image processing apparatus according to item.
(22) The second seek line setting means is configured by the first seek line setting means based on the edge point coordinates calculated by the edge point coordinate calculation means and data of a master measurement template set in advance. Set more seek lines than the set number of seek lines. (twenty one) The image processing apparatus according to item.
(23) (1) Term or (twenty two) An electronic component in which the image processing apparatus according to any one of the items is engaged with a plurality of parts of the intermittently rotating intermittently rotating body at the same angular interval as the intermittently rotating angle and sequentially moved to a plurality of positions. Provided in an electronic component transport apparatus that includes a holder and transports an electronic component held by the electronic component holder from a start point that is one of the plurality of positions to an end point that is another one, and the start point and the end point The image pickup device is arranged at a midpoint between and the image of the electronic component held by the electronic component holder is picked up, and the image data storage means includes the image data from the midpoint to the end point. An image data memory for storing image data of an image processing object obtained by imaging a plurality of times equal to or less than the number of intermittent movements of the electronic component holder, and the image processing apparatus includes at least By the electronic component image reaches the end point, completing the electronic component transporting apparatus processing the image data corresponding to each electronic component.
In the electronic component conveying apparatus having this configuration, when an image processing object that requires a long time for image processing and an image processing object that requires a short time are mixed and conveyed, the image processing apparatus has a short processing time. However, the remaining time can be used to perform processing that requires a long processing time, and the intermittent rotation time (sum of the stop time and the rotation time) of the intermittent rotating body can be shortened.
(24) The start point is a receiving position where the electronic component holder receives the electronic component from the component supply device, and the end point is a mounting position where the electronic component holder mounts the electronic component on a mounting object. is there (twenty three) The electronic component conveying apparatus according to the item.
[0048]
DETAILED DESCRIPTION OF THE INVENTION
  Less than,It is an embodiment of the present inventionAn image processing apparatus of an electronic component mounting apparatus will be described in detail with reference to the drawings.
  1 and 2, reference numeral 10 denotes a bed, and a pair of support walls 12 are erected on the bed 10. The support wall 12 is provided in parallel to the Y-axis direction (left-right direction in FIG. 2) orthogonal to the X-axis in a horizontal plane at a position spaced apart in the X-axis direction, which is the printed board conveyance direction. These support walls 12 support the cam box 14 at their upper ends, and a front wall 16 is provided on the apparatus front end side of the support wall 12, and covers the apparatus together with an openable / closable cover 17. . FIG. 2 shows a state in which the support wall on the front side is removed.
[0049]
A base plate 18 is stretched between the pair of support walls 12 in the vertical direction so as to be parallel to the X-axis direction. As shown in FIG. 3, a cylindrical cam 20 is fixed to the cam box 14, and the cam box 14 and the cylindrical cam 20 allow the rotation shaft 22 to be rotated around a vertical axis by bearings 24 and 26 and not to move in the axial direction. It is supported. The upper end portion of the rotary shaft 22 is protruded above the bottom wall 28 of the cam box 14, and a plurality of rollers 32 are rotated around a vertical axis on a large-diameter disk portion 30 provided at the protruding end portion. It is attached so that it can be sequentially engaged with the groove 36 of the barrel cam 34.
[0050]
Further, the lower end portion of the rotary shaft 22 is projected from the cylindrical cam 20 to form a large-diameter fitting portion 40 and to be fitted into the fitting hole 44 of the index table 42. The index table 42 is supported by the base plate 18 via bearings 46 and 48 so as to be rotatable about a vertical axis and not movable in the axial direction, and is provided in the fitting portion 40 so as to extend downward. The pin 50 and the pin hole 52 are engaged to transmit the rotation of the rotary shaft 22. Therefore, when the barrel cam 34 is rotated by a motor (not shown), the rollers 32 are sequentially engaged with the grooves 36 along with the rotation, and when the rotary shaft 22 is rotated, the index table 42 is intermittently rotated by a certain angle. It is done.
[0051]
On the index table 42, 20 sets of component mounting heads 56 (only two sets are shown in FIG. 3) are attached at equal angular intervals on a circumference around the rotation axis. As shown in FIG. 5, there are 20 work positions such as a component suction position, a component posture detection position, a component posture correction position, a component mounting position, and a nozzle return position. The component mounting head 56 is sequentially moved to 20 work positions by intermittent rotation of the index table 42.
[0052]
The component mounting head 56 is fitted to the index table 42 in a pair of guide rods 58 (only one is shown in FIG. 3) so as to be slidable in the vertical direction. The lower ends of the pair of guide rods 58 are fixed to the support member 60, and the head main body 62 is supported by the support member 60. The upper ends of the pair of guide rods 58 are connected by a connecting plate 64, and a roller 66 is attached to the connecting plate 64 so as to be rotatable about an axis extending in the radial direction of the index table 42. It is engaged with the formed groove 68. The groove 68 has a height gradually changed in the circumferential direction as shown in FIG. 2, and when the index table 42 is rotated, the component mounting head 56 has the highest component suction position and the lowest component mounting position. Can be raised and lowered between.
[0053]
The support member 60 is also fixed with a lower end portion of a pipe 72 fitted in an airtight and axially movable manner in a fitting hole 70 formed in the index table 42 in the vertical direction. The fitting hole 70 is connected to a negative pressure source by a passage 74 formed in the index table 42, a passage 76 formed in the rotary shaft 22, a port 78 formed in the cylindrical cam 20, and a hose (not shown). The negative pressure is supplied from 72 to the component mounting head 56, and the negative pressure supply state is maintained by the lifting and lowering of the pipe 72 even when the component mounting head 56 is raised and lowered.
[0054]
A suction nozzle 80 is fitted to the head body 62 so as to be rotatable about its own axis and movable in the axial direction. The suction nozzle 80 sucks the electronic component 82 by a vacuum supplied through the pipe 72 and the like, and is biased upward with respect to the head main body 62 by a spring 84 and is moved to the component suction position and the component mounting position. The electronic parts 82 are sucked and mounted on the printed circuit board 90 (see FIG. 2) by being pushed down by the pressing members 88 of the respective pressing devices 86 provided.
[0055]
As shown in FIG. 2, an electronic component supply device 100 is provided at the component suction position. The electronic component supply device 100 is in contact with the rotation trajectory of the index table 42 and is provided with a movable table 102 that is movable along a straight line extending in a direction parallel to the X axis. A plurality of component supply cartridges 104 arranged in the moving direction, and the moving table 102 is moved by a moving table moving device (not shown), so that one of the plurality of component supply cartridges 104 becomes a component supply position. Moved to.
[0056]
The component supply cartridge 104 holds the electronic component 82 on a component holding tape and supplies it as a taped electronic component. The component holding tape is supplied in the X-axis direction in a horizontal plane by a tape feeder using an air cylinder as a drive source. Of the at least one electronic component 82 that is sent in the orthogonal Y-axis direction and the cover film that covers the component receiving recess of the component holding tape is peeled off, the leading electronic component 82 is positioned at the electronic component extraction position.
[0057]
At the component mounting position, as shown in FIG. 2, a printed circuit board moving device 108 that supports the printed circuit board 90 as a material to be mounted and moves it in the X-axis direction and the Y-axis direction is a substrate conveyor 109 (see FIG. 1). It is provided below. The X-axis table 110 of the printed circuit board moving device 108 is guided by the guide rail 116 and moved in the X-axis direction by an X-axis moving device 114 composed of an X-axis servo motor 112, a ball screw and a nut (not shown). .
[0058]
A Y-axis table 118 is mounted on the X-axis table 110 so as to be movable in the Y-axis direction. A Y-axis moving device 122 configured by a Y-axis servo motor, a ball screw 120, a nut (not shown), and the like (not shown). , Guided by the guide rail 124 and moved in the Y-axis direction. Although not shown on the Y-axis table 118, a substrate support lifting device that receives the printed circuit board 90 from the substrate conveyor 109, supports the printed circuit board 90 from above and below by the substrate pressing member and the substrate support member, and moves up and down. Is provided. The substrate conveyor 126 conveys the printed circuit board 90 in the X-axis direction. The printed circuit board 90 supported by the substrate support lifting device is a substrate by combining the movement of the X-axis table 110 and the Y-axis table 118. It is moved to an arbitrary position in the horizontal plane at a position below the conveyor 109.
[0059]
The printed circuit board 90 is provided with fiducial marks at two locations separated on a diagonal line, and images are taken by a CCD camera 128 (see FIG. 6) provided on the bed 10. After the printed circuit board 90 supported by the substrate supporting lifting device is lowered, it is moved by the printed circuit board moving device 108 to move the reference mark to the imaging position that coincides with the axis of the CCD camera 128, and the reference mark is imaged. .
[0060]
The printed board moving device 108 is provided at a position lower than the board conveyor 109 and lower than the electronic component supply apparatus 100. The board support lifting / lowering device is provided at such a height that the board support lifting / lowering device can be inserted into the lower side of the component supply cartridge 104 while supporting the printed board 90 on the board support surface. Therefore, the printed circuit board moving device 108 can be provided so as to overlap the electronic component supply device 100 in the horizontal direction, and the electronic component mounting device can be configured compactly.
[0061]
An image acquisition device 130 is provided at the component posture detection position as shown in FIG. The image acquisition device 130 includes an illumination device 132 and a CCD camera 134 as an imaging device. The lighting device 132 includes a first surface light emitter 138 and a second surface light emitter 140. The first surface light emitter 138 has a plate-like shape, is provided with a large number of light-emitting diodes that are horizontally disposed and covered with a milky white diffuser, and emits light upward from the diffuser. The first surface light emitter 138 is provided at a position that is concentric with the component mounting head 56 that has been moved to the component posture detection position, and a through hole 142 that passes through the center line in the vertical direction is provided. The second surface light emitter 140 is provided perpendicular to the lower surface of the first surface light emitter 138 and includes a number of light emitting diodes covered with a diffuser plate, from the diffuser plate to the center line side of the first surface light emitter 138. Emits light.
[0062]
A half mirror 144 is provided below the first surface light emitter 138 so as to be inclined by 45 degrees with respect to the center line of the first surface light emitter 138. A CCD camera 134 is provided on the opposite side of the first surface light emitter 138 with the half mirror 144 therebetween. The CCD camera 134 is provided with a lens 148 on a main body 146 and is disposed vertically and upward. In the main body 146, an imaging surface is formed by arranging a large number of solid-state imaging elements in a single plane. .
[0063]
The electronic component 82 is irradiated with light emitted from the first surface light emitter 138, and light emitted from the second surface light emitter 140 is reflected by the half mirror 144 and irradiated through the through hole 142. Illumination is performed as if light was emitted from the CCD camera 134, and the entire electronic component was irradiated with light evenly. The first surface light emitter 138, the second surface light emitter 140, and the half mirror 144 constitute the illumination device 132. The reflected light from the electronic component 82 passes through the half mirror 144 and enters the CCD camera 134, and an image is formed on the imaging surface.
[0064]
The CCD camera 134 is controlled by the control device 150 shown in FIG. 6 together with the CCD camera 128 for imaging the reference mark on the printed circuit board 90. The CCD cameras 128 and 134 and the control device 150 constitute an image processing device. The control device 150 is mainly a computer, and includes a CPU 154, a DRAM (dynamic ram) 156, an SRAM (static ram) 158, a PROM (programmable rom) 160, a kanji ROM 162, a frame grabber memory 164, and an overlay for four sides. A display memory 166 is included, and these are connected to each other by an internal bus (not shown) on the substrate 167.
[0065]
A two-channel serial interface 170 is also connected to the internal bus, and an input device 172 is connected. The input device 172 is a device for inputting information necessary for image processing, such as the type and number of objects to be processed, together with information and commands necessary for the operation of the entire electronic component mounting apparatus, and has a numeric keypad, alphabet keys, and the like. However, it is provided on the front wall 16 as shown in FIG. An Ethernet interface 174 and a memory card interface 176 are also connected to the bus.
[0066]
The Ethernet interface 174 is an interface for performing communication with a computer that controls a part other than the image processing apparatus of the electronic component mounting apparatus. For example, an external control device (if any option is required) can be connected to the control device 150, and the Ethernet interface 174 exchanges data via the P1 connector 168. In the electronic component mounting apparatus, the control device for controlling the index table 42, the component mounting head 56, the pressing device 86, the electronic component supply device 100, the printed board moving device 108, the board conveyor 109, etc. is also an image mainly using a computer. It is provided separately from the control device 150 of the processing apparatus, and is connected to the P1 connector 168 via an external bus (not shown). Since this other control device is not related to the present invention, its illustration and description are omitted. The memory card stores a program created in advance for performing image processing. If the memory card is set in the control device 150, the CPU 154 uses the PROM 160 to execute the program and the program in the memory card. Data is read through the memory card interface 176 and stored in the DRAM 154.
[0067]
Further, a CCD camera interface 180 is connected to the bus, and a CCD camera 128 for imaging a reference mark on the printed circuit board 90 and a CCD camera 134 of the image acquisition device 130 are connected. The reference marks obtained by the imaging of the CCD cameras 128 and 134 and the image data of the electronic component 82 are stored in the frame grabber memory 164 via the CCD camera interface 180, respectively. As described above, four frame grabber memories 164 are provided, and the image data of the four electronic components 82 that are continuously mounted are sequentially stored in each frame grabber memory 164. Further, since the image data of the reference mark is acquired and used at a different timing from the image data of the electronic component 82, one of the frame grabber memories 164 for storing the image data of the electronic component 82 is also used. The
As in the case where the image processing object is particularly large, it is necessary that the imaging device has a large number of imaging elements in one line, or it is necessary to image the moving image processing object. In some cases, a line sensor is used instead of the CCD camera. Therefore, the line sensor board interface 184 is provided on the board 167 and is connected to the frame grabber memory 164. When the line sensor is used, an optional line sensor board (not shown) is connected to the line sensor board interface 184 via the line sensor bus 182, and image data obtained by imaging the line sensor is stored in the frame grabber memory 164.
[0068]
Further, a CRT interface 186 is connected to the bus, and a monitor CRT device 188 is connected. As shown in FIG. 1, the monitor CRT device 188 is provided on the front wall 16 along with the input device 172, and is capable of both color display and monochrome display.
As described above, image data of four monochrome images obtained by imaging of the electronic component 82 are stored in parallel in the frame grabber memory 164, while the overlay display memory 166 stores 16 images. There are four memories for storing color image data to be displayed in color. On the monitor CRT device 188, the color image corresponding to the monochrome image among the four color images is superimposed on any one of the four monochrome images, and the progress and result of the image processing are displayed. .
Data input using the input device 172 on the same monitor CRT device 188 is also displayed in color. During this display, the kanji ROM 162 is used.
[0069]
When the electronic component 82 is mounted on the printed circuit board 90 in the electronic component mounting apparatus, the component mounting head 56 takes out the electronic component 82 from the component supply cartridge 104 at the component suction position as the index table 42 rotates, and then the component The electronic component 82 is imaged by the CCD camera 134 at the posture detection position. The image data of the electronic component 82 is processed by the control device 150, and the amount of horizontal displacement (position error in the X-axis and Y-axis directions) of the electronic component 82 and the amount of rotation angle deviation around the axis of the suction nozzle 80 (rotation angle). Error) is calculated, the positional deviation is corrected by the movement of the printed circuit board 90, and the rotational angle deviation is corrected at the component posture correction position. Although not shown, a nozzle rotation driving device that engages with the suction nozzle 80 and rotates the suction nozzle 80 is provided at the component posture correction position, and the suction nozzle 80 corrects the rotational angle deviation of the electronic component 82. It can be rotated by the necessary angle. After correcting the rotational angle deviation, the electronic component 82 is mounted at a predetermined position on the printed circuit board 90 at the component mounting position. After mounting, the suction nozzle 80 is rotated in the opposite direction by the amount rotated for correcting the rotational angle deviation at the nozzle return position, and returned to the rotational position before the rotational angle deviation correction.
[0070]
The printed circuit board 90 is conveyed by the substrate conveyor 109, and the reference mark is imaged by the CCD camera 128 while being supported by the substrate supporting lifting device. Based on the image data, the amount of horizontal displacement (horizontal position error) and the vertical axis The amount of rotation angle deviation around (rotation angle error) is calculated. When the electronic component 82 is mounted on the printed circuit board 90, the printed circuit board 90 is moved by the printed circuit board moving device 108, and the horizontal position error is corrected together with the horizontal position error of the electronic component 82. Further, the rotation angle error of the printed circuit board 90 is corrected together with the rotation position error of the electronic component 82 by rotating the suction nozzle 80.
[0071]
Hereinafter, processing of image data obtained by imaging with the CCD cameras 128 and 134 will be described.
The program and data for image processing are stored in the memory card as described above. If the memory card is set, it is read and stored in the DRAM 156. Image processing programs read from the memory card are shown in FIGS.
[0072]
The program shown in FIG. 7 is a preprocessing program. The preprocessing program is executed when one production program is started, that is, after the preprocessing program is stored in the DRAM 156. First, it is determined whether or not to perform pattern matching for one of all image processing objects necessary for execution of one production program. If so, a search template is generated based on the master search template. Stored in the DRAM 156. Similar processing is sequentially performed for all the image processing objects.
[0073]
The master search template has a plurality of point pairs each having two points as a pair, and a coordinate plane (referred to as a master search template coordinate plane) that defines the point pairs coincides with a reference coordinate plane of the image processing apparatus. It is what. That is, the origin and coordinate axis directions of the master search template coordinate plane coincide with the origin and coordinate axis directions of the reference coordinate plane in which the origin is set at the center of the field of view of the CCD cameras 128 and 134. The master search template is created in advance based on the shape and size of the image processing object and stored in the memory card, and is read into the DRAM 156 together with the preprocessing program.
[0074]
FIG. 10 shows an example of master search template setting data when the image processing object is a square electronic component, and FIG. 11 shows a master search template 200 set by the data. In the data of FIG. 10, the data of the seventh, eighth, tenth, and eleventh lines and hs (half span) = 5.5 of the fifth line are the setting data of the master search template. Pair means that another point pair is set symmetrically with respect to the center line of the image processing object on the extension line of two points constituting the point pair. For example, each of (7) ′, (8) ′, (10) ′, (11) ′ for each point pair 202 of (7), (8), (10), (11) shown in FIG. A point pair 202 is set. The numbers in parentheses attached to these point pairs 202 coincide with the line numbers in FIG. Reference numeral 204 denotes an electronic component.
[0075]
In the master search template, for a master image processing object having no error in size, position, or rotation angle, one of two points constituting each pair of point pairs is inside the image processing object, and the other is It is created such that it is located outside and the midpoint of the two points of the point pair is located on the edge of the master image processing object. If it is shown in the figure, it will be as shown in FIG.
In general, it is not indispensable that the midpoint of the two points of the point pair is positioned on the edge of the master image processing object, and the two points are respectively located inside the edge of the image processing object. It suffices to be designated as outside.
In the example shown in FIG. 11, one of the two points of the point pair 202 is shared with one point of another point pair 202, but this is not essential.
Further, in FIG. 11, two points constituting the point pair 202 are connected by a straight line in order to easily show which point constitutes the point pair 202. However, the straight line data is not actually set.
[0076]
FIG. 14 shows the data of the master search template when the image processing object is the disc 206 of FIG. 15 with a part cut away. In this master search template 208, the point pairs 210 of (15) to (17) provided in the circumferential portion are paired with the point pairs 210 of (15) ′ to (17) ′. Point pair 210 is not paired with another point pair.
[0077]
In the electronic component mounting apparatus, the image processing object is an electronic component or a reference mark, and master search template data is created in advance and stored in a memory card for all types of electronic components to be mounted and different types of reference marks. And stored in the DRAM 156 when image processing is executed. Therefore, when the search template is generated, the master search template data is read from the DRAM 156 according to the type of the image processing object for which the search template is to be generated. As described above, the master search template is a search template with a rotation angle of 0 degrees, and the master search template is rotated at a set pitch as shown by a two-dot chain line in FIG. A search template is generated and the data is stored in DRAM 156.
[0078]
Data specifying the generation angle range and set pitch of the search template are stored together with the data of the master search template as shown in FIG. The pitchA = 4.5 in the 15th line is the set pitch data, and startA = −45 in the 16th line and endA = 45 in the 17th line are data defining the generation angle range of the search template. The angle range and pitch for generating the search template are set according to the image processing object. For example, when the rotation angle of the image processing object is expected to be greatly shifted, the generation angle range is widened. Incidentally, in the example of the pre-processing program shown in FIG. 7, the generation angle range is set to −45 degrees to +45 degrees, and the set pitch is set to 5 degrees.
[0079]
When the generation of the search template for one image processing object is completed, the execution of the program returns to the beginning, and whether to perform pattern matching for the next image processing object and generation of the search template if it is performed. Is done. When pattern matching is not performed, program execution returns to the beginning, and it is determined whether or not pattern matching is performed for the next image processing object. If it is determined whether or not pattern matching is to be performed for all image processing objects, and search templates are generated for the image processing objects to be subjected to pattern matching, the execution of the program in FIG. 7 ends.
[0080]
Next, the execution processing program shown in FIG. 8 will be described. This program is executed after electronic components or reference marks are imaged by the CCD cameras 128 and 134 and image data is stored in the frame grabber memory 164. First, if the object to be processed can be processed only by pattern matching, such as a square chip, image processing is performed according to the pattern matching program shown in FIG.
[0081]
Next, the image processing target is a QFP (Quad Flat Package), PLCC (Plastic Leaded Chip Carrier), BGA (Ball Grid Array), or other electronic component having a complicated shape with leads and solder bumps. It is determined whether it is necessary to activate a pattern matching manager that combines pattern matching for processing. The combination of pattern matching will be described later.
If the pattern matching manager does not need to be operated, it is determined whether image processing on the virtual screen should be performed. If the determination result is NO, it is determined whether image processing on the physical screen should be performed. Is done.
The physical screen has an optical characteristic value obtained for each pixel, and is an image screen in which image data actually exists. The virtual screen has an optical optical characteristic value at an arbitrary point not restricted by the pixel as necessary. This screen is obtained by calculation. As will be described later, both the pattern matching and the pattern matching manager are processes performed on the virtual screen. However, in the present embodiment, in addition to these, physical processing on the virtual screen is performed independently of pattern matching. Image processing can be performed on the screen. The determination of whether or not to perform image processing on the virtual screen and whether or not to perform image processing on the physical screen is instructed to perform the latter two image processing. It is a judgment
[0082]
The pattern matching program shown in FIG. 9 will be described.
First, a search window is set, and a search area for searching for an image processing target is set. The search window is set by designating part or all of the imaging surfaces of the CCD cameras 128 and 134 by coordinate values. Whether the image processing object is a reference mark or an electronic component, or what type is an electronic component is known from the mounting data during the work procedure, and the image processing object formed on the imaging surface Since the position of the image is roughly known, the search window is set to a sufficient size suitable for including the image processing object even if the position is slightly shifted. In this way, the search area can be narrow and the search can be performed in a short time.
[0083]
The full pattern matching process consists of a search step for searching for a search object, a re-search step for searching for an approximate edge point of the search object, a measurement step for calculating the edge point of the search object, and a measurement step that repeats the measurement step. It includes four steps of measurement steps. Normally, pattern matching is completed when all four steps are completed. If there is even one abnormal step, the next step is not executed and the pattern matching is immediately terminated.
[0084]
First, the search step will be described.
In the search step, search templates are sequentially read out from the DRAM 156 one by one and superimposed on a screen 221 on which an image including an image 220 of an image processing object and a background exists as shown in FIG. Optical characteristic values (luminance in this embodiment) of two points (hereinafter referred to as point pair constituent points) constituting the plurality of point pairs 224 are calculated. In the example shown in FIG. 10, the search is sequentially performed from the search template whose rotation angle is −45 degrees.
[0085]
The point pair composing point is a point on the virtual screen, and the luminance of the point pair composing point is obtained by interpolation calculation from the luminance as image data of a plurality of pixels on the physical screen. The optical characteristic value of the point on the virtual screen specified by the data of the search template is obtained based on the image data of the physical screen, and this is visually represented in FIG. This means “superimpose the search template on the screen on which the image represented by the image data is present”. The screen 221 in FIG. 17 is a virtual screen, and the image 220 of the image object in the screen 221 is only virtual if it exists at this position, and there is actually no image data representing this image 220.
The same applies to FIGS. 19, 20, and 27 described later.
[0086]
Interpolation calculation of the luminance of the point pair constituent points can be performed using a curved surface such as a bicubic spline curved surface defined by image data of 4 × 4 control points on the X, Y coordinate plane, for example. However, in the present embodiment, it is performed by the simplest linear interpolation based on the image data of four pixels adjacent to the point pair constituent points. In FIG.₀ , V₀ ) Is the point pair constituent point, f (u₀ , V₀ ) Is the luminance of the point pair constituent points, and (u ′, v ′), (u ′ + 1, v ′), (u ′, v ′ + 1), and (u ′ + 1, v ′ + 1) are used for linear interpolation, respectively. The center positions of the four pixels to be processed are f (u ′, v ′), f (u ′ + 1, v ′), f (u ′, v ′ + 1), and f (u ′ + 1, v ′ + 1). It is each brightness | luminance of four pixels, and the brightness | luminance of a point pair constituent point is calculated by (1) Formula.
f = (u₀ , V₀ ) = F (u ′, v ′) (1-α) (1-β) + f (u ′ + 1, v ′) α (1-β) + f (u ′, v ′ + 1) (1-β) β + f (U ′ + 1, v ′ + 1) αβ (1)
[0087]
The above calculation is performed by the physical screen / virtual screen conversion driver 300 shown in FIG. As shown in the figure, the physical screen / virtual screen conversion driver 300 is configured separately from general image processing application software 302, and is based on image data on the physical screen 304 in the image processing application software 302. Whenever it becomes necessary to calculate the image data on the virtual screen 306, the physical screen / virtual screen conversion driver 300 is called to calculate the image data on the virtual screen 306.
[0088]
Each time the luminance is calculated for the two component points of each pair of point pairs 224, the luminance of the two point pair component points is compared. At the time of imaging by the CCD camera 128 or 134, the light applied to the image processing object such as the electronic component 82 or the reference mark is reflected more than the light applied to the background, so the portion corresponding to the image processing object and the background There is a difference in the charge amount of the solid-state imaging device from the portion corresponding to. The image of the image processing object is bright and the background is dark. Therefore, if one of the two point pair constituent points is located within the edge of the image processing object and the other point is located within the background, the luminance of the two point pair constituent points is preset. There is a difference greater than the set value (when the set value is positive) or less than the set value (when the set value is negative).
[0089]
The set value of the brightness difference is stored together with the master search template data. For example, in FIG. 10, the setting value diff is set to −20 as described in the fifth line. In this case, if the luminance of the point pair constituent point within the edge of the image processing object is greater than 20 gradations than the luminance of the point pair constituent point outside the edge, the set value or less. It is determined that there is a difference.
On the contrary, if the set value diff is set to 20, if the luminance of the point pair constituent point outside the edge of the image processing object is 20 gradations or more higher than the luminance of the point pair constituent point in the edge, the set value or higher It is determined that there is a difference.
In either case, these two point pair constituent points straddle the edge of the image processing object, and are in an adapted state. In this case, it is expressed as “two point pair constituent points satisfy the set luminance difference condition”.
[0090]
(1) The image processing object is not the search object, and the two point pair constituent points do not straddle the edge, (2) The suction nozzle 80 does not suck the electronic component 82 due to a suction error, or (3 ) Due to the reason that dust or the like adheres to the solid-state image pickup device and image data cannot be obtained, the two point pair constituent points do not satisfy the set luminance difference condition and cannot be said to be in an adapted state. This state is called a point pair failure. The number of failures for determining that there is no search object that matches the search template is preset. For example, in FIG. 10, the number of failures is set to 0 as shown in the third line, and if the two point pair constituent points do not satisfy the set brightness difference condition for all point pairs, the search template is matched. It is determined that there is an object to be searched.
[0091]
If the number of failures is set to 1 or more, if there are more than a set number of point pairs that do not satisfy the set brightness difference condition among multiple point pairs, there is a search target that matches the search template. It is determined not to.
If it is determined that there is a search object at a rotation angle of 0 degrees, the search step ends and a re-search step is executed. If it is determined that there is no search object, search templates having different rotation angles are read out. The search object is searched.
[0092]
Until it is determined that the search target exists, a plurality of types of search templates are sequentially read and the search target is searched. Even if searching using all types of search templates, if it is not determined that there is a search object that matches the search template, the search template is shifted and then searched. The search object is searched using a plurality of types of search templates that are shifted by a certain pitch in the X-axis direction and the Y-axis direction and have different rotation angles at each position.
[0093]
This movement pitch is preset and stored in the memory card together with data defining the master search template. In FIG. 10, pitchX = 2.2 and pitchY = 2.2 shown in the 13th and 14th lines are the movement pitches. First, the set pitch is moved in the Y-axis direction. Specifically, coordinate conversion is performed so that the coordinates of each point pair of a plurality of types of search templates are shifted from the Y-axis direction by a set pitch in the positive direction. The search object is searched using this search template. If it is not determined that a search target exists even if all types of search templates with different rotation angles are used at this position, the search template is then shifted in the positive direction by the set pitch in the X-axis direction. Furthermore, if it is not determined here that the search object exists, the search template is shifted from the Y-axis direction to the negative direction by the set pitch. If it cannot be determined that the search object exists again, then the search template is further shifted in the Y-axis direction by the set pitch in the negative direction, and further, it can be determined that the search object exists. For example, the search template is then shifted in the negative direction by the set pitch in the X-axis direction. The search template is moved in the search window so as to draw a square spiral.
[0094]
Even if the search template is moved, it cannot be determined that the search target exists, and if a point pair that protrudes from the search window occurs when coordinate conversion is performed, the search template cannot be moved. Thus, it is determined that there is no search object that matches the search template, and the search step is terminated abnormally. The occurrence of abnormality is displayed on the monitor CRT device 188, and the occurrence of abnormality is stored. If the image processing object is the electronic component 82, the suction nozzle 80 that has been determined to have an abnormal image processing result is prevented from mounting the electronic component 82. The pressing device 86 is prevented from operating at the component mounting position. If the search object is a reference mark, the printed circuit board 90 is carried out without the electronic component 82 mounted.
[0095]
If the two point pair composing points of all the point pairs 224 are inside and outside the edge of the image 220 as in the image 220 of the image processing object shown in FIG. 17, and if the set luminance difference condition is satisfied, The position and rotation angle of the current search template are stored in the DRAM 156, and the re-search step is executed. In the re-search step, a re-search template coordinate plane (search template coordinate plane) in which the edge point of the image 220 of the image processing object is the coordinate plane of the re-search template 228 using the re-search template 228 as shown in FIG. Is searched). The re-search template 228 includes a plurality of seek lines 230. The seek line 230 is set based on the search template in which the image 220 of the image processing object is found in the search step. The two point pair composing points of the point pair are points that define both ends of the seek line 230, respectively.
[0096]
The edge point of the image 220 of the image processing object is searched for each of the set plurality of seek lines 230. As shown in FIG. 20, this search is performed by dividing the seek line 230 at a predetermined pitch (for example, 0.05 mm) and calculating the luminance for each of the plurality of division points P1 to P15. This pitch is set to a length shorter than the diagonal line of the solid-state imaging device 232 of the CCD cameras 128 and 134. Therefore, three to four division points are included in one solid-state imaging device 232. The division points are also points on the virtual screen, and linear interpolation is performed in the same manner as in the search step, and the luminance of the division points P1 to P15 is calculated.
[0097]
An example of each brightness | luminance of 15 division | segmentation points P1-P15 calculated by linear interpolation is shown in FIG. Note that the luminance value is represented by a positive value, and the value increases as the luminance value acquisition object becomes brighter. The image acquisition device 130 according to the present embodiment is configured to irradiate the surface of the image processing object with light and acquire an image based on the reflected light from the surface. It is assumed that the brightness value is large and the brightness value is large, the background is dark and the brightness value is small.
From the luminance value calculated by linear interpolation, it cannot be determined where the luminance changes the most as shown in the graph of FIG. Therefore, the differential value of the luminance value is obtained using a difference filter. The result of obtaining the differential value using the differential filter shown in FIG. 22 is shown in the graph of FIG. This difference filter has a negative value for the luminance of a point located upstream of two adjacent points from one point that defines a seek line toward the other point, and points that are located downstream. This is a filter that takes a positive value and calculates the sum of these two values. This differential value is not the value of the dividing point, and in the graph of FIG. 25, the position where the luminance differential value is obtained is indicated by “0.5” which is the center position of the adjacent two dividing points. As is clear from this graph, the magnitude of the luminance change is known, but it is not known where it is the maximum.
Depending on the calculation direction, that is, whether the calculation is performed from the division point outside the image processing object (within the background) toward the division point inside the image processing object or vice versa, the luminance differentiation is performed. The sign of the value is reversed. In the former case, the luminance differential value becomes a positive value, and the position where the luminance differential value is the maximum is the position where the absolute value of the change gradient of the luminance is the maximum (the luminance differential value of the position where the absolute value of the change gradient is the maximum). Called maximum). In the latter case, the luminance differential value is a negative value, and the position where the luminance differential value is the minimum is the position where the absolute value of the luminance change gradient is the maximum (the luminance differential value at the position where the absolute value of the change gradient is the maximum). Called the local minimum). The luminance differential values shown in FIG. 25 and the graph of FIG. 26 described next are values obtained by the former calculation.
[0098]
On the other hand, if differentiation is performed using the differential filter shown in FIG. 23, the maximum value 177 of the luminance differential value is obtained at the position of f8.5 as shown in the graph of FIG. It can be seen that the absolute value of is the maximum position. In the difference filter shown in FIG. 23, for one of the division points set on the seek line, all four division points upstream including the division point in the calculation direction are set to negative values. In this filter, the luminance values of four consecutive dividing points on the downstream side are all positive values and the sum thereof is obtained.
[0099]
If dirt or the like adheres to the part corresponding to the edge point of the solid-state image sensor and the charge amount changes, the maximum or minimum value of the luminance differential value may be obtained at a position other than the edge point. The absolute value of the luminance change gradient at such a position is small. On the other hand, in the vicinity of the edge point, there is a significant difference between the brightness of the image processing object and the background, and the absolute value of the luminance change gradient is large. Therefore, a set value is provided, and it is determined whether or not the luminance differential value at the position where the absolute value of the change gradient is the maximum is a value obtained in the vicinity of the edge point, and the former case is excluded. This set value is set to a positive value when the luminance differential value is obtained as a positive value, it is determined whether or not the maximum value of the luminance differential value is greater than or equal to the set value, and the maximum value is greater than or equal to the set value. If so, the maximum value is a value obtained in the vicinity of the edge point, and it is determined that it can be used for the calculation of the edge point, and the calculation of the edge point is performed. In addition, when the luminance differential value is obtained as a negative value, the set value is set as a negative value, and it is determined whether or not the minimum value of the luminance differential value is less than or equal to the set value. If it is below, it is determined that the minimum value can be used for the calculation of the edge point. In other words, the failure of the seek line in the re-search step is that the edge point is not calculated because the maximum value of the luminance differential value is smaller than the set value or the minimum value is larger than the set value.
[0100]
In the example shown in FIG. 10, contrary to the examples shown in FIGS. 25 and 26, it is determined that the luminance differential value is calculated from the dividing point inside the image processing object to the dividing point outward. In the position where the absolute value of the change gradient of the luminance value is the largest, the luminance differential value is minimum, and the setting value for determining whether or not the minimum value is a value obtained in the vicinity of the edge point is a negative value. The value, that is, ll = −200 as shown in the fifth line. In this example, since the image processing object is brighter than the background, the luminance differential value is obtained as a negative value, and if the value is not −200 or less, the position of the edge point is not calculated. ing.
[0101]
In addition, as shown in FIG. 10, it is also determined in advance what kind of difference filter is used for the calculation. This differential filter coefficient N is calculated according to the equation (2).
N = gUnit / pitch between division points (2)
Where gUnit is the diagonal length of the solid-state image sensor.
[0102]
If differentiation is performed using a differential filter and the minimum value (or maximum value) of the luminance differential value is obtained, the position where the absolute value of the luminance change gradient is maximum, that is, the edge point is obtained according to the following equation. Equations (3) and (4) are equations taking N = 4 as an example, and f_max, F_(max-4)~ F_(max-1), F_{(max + 1)}~ F_{(max + 4)}Is the luminance differential value (f_maxIs the maximum (small) value. As shown in FIG. 26, “f” indicates a position where the luminance differential value is obtained on the seek line by adding a number. In the expressions (3) and (4), “f” is designated by the number added to f. Represents the luminance differential value of the position to be performed. f_maxIs the luminance differential value at the position where the luminance differential value is the maximum (small) (f8.5 in the example shown in FIG. 26), and f_(max-1), F_(max-2), F_(max-3), F_(max-4)Are respectively f in the calculation direction_maxThe luminance differential values at four locations on the upstream side (f7.5, f6.5, f5.5, f4.5 in the example shown in FIG. 26) and f_{(max + 1)}, F_{(max + 2)}, F_{(max + 3)}, F_{(max + 4}Are respectively f in the calculation direction_maxThe luminance differential values at four locations on the downstream side (in the example shown in FIG. 26, f9.5, f10.5, f11.5, and f12.5).
dl = f_max× 4-(f_(max-1)+ F_(max-2)+ F_(max-3)+ F_(max-4)(3)
dr = f_max× 4-(f_{(max + 1)}+ F_{(max + 2)}+ F_{(max + 3)}+ F_{(max + 4)}(4)
edgePitch = (dl × N) / (dl + dr) −N / 2 (5)
Edge point = (Luminance differential value maximum (small) value point pitch + edgePitch) x division point pitch (6)
Equations (3) and (4) are equations in the case of N = 4. In general, when dl is obtained, it is calculated from a value obtained by multiplying the luminance differential value of the maximum (small) value point by N. When the sum of the luminance differential values of N points upstream of the local maximum (small) value point in the direction is subtracted and dr is obtained, the luminance differential value of the local maximum (small) value point is multiplied by N, The sum of the luminance differential values of N points downstream of the maximum (small) value point is subtracted. When the edge point is obtained when the calculation result shown in FIG. 21 is differentiated using the difference filter shown in FIG. 23, the brightness differential value maximum (small) value point pitch number in the equation (6) is 8. 5.
[0103]
At the time of calculating the edge point, first, the luminance of the dividing point is calculated by linear interpolation, and after differentiation is performed according to the difference filter coefficient N, the calculation is performed according to equations (3) to (6) to obtain the maximum luminance change position. That is, an edge point is obtained. In the case of the seek line 230 shown in FIG. 20, the calculation result of equation (6) is 0.403 mm, and it can be seen that there is an edge point at a position 0.403 mm from the division point P1 of the seek line 230.
[0104]
In this way, an edge point is calculated for each of the plurality of seek lines 230. The number of failures of the seek line 230 (if the number of failures of the point pair 202 is less than or equal to the set number, if it is determined that there is a search object that matches the search template, the failure of the point pair 202 If the sum of the number and the number of failures of the seek line 230) is equal to or less than the set number, it is determined that the operation is normal and the measurement step is executed. If there is a failure exceeding the set number, the process ends abnormally. The processing when an abnormality occurs is the same as the search step. In FIG. 10, as shown in the third line, the fail count is set to 0, and if there is even one failure, the re-search step is terminated abnormally.
[0105]
If the number of failures is less than or equal to the set number and the re-search step ends normally, the measurement step is executed next. The re-search template is set based on the search template determined to be the search object in the search step, and the edge point can be found on the seek line 230, but the midpoint between the edge point and the seek line 230 (see FIG. 19 is usually marked with a cross (hereinafter referred to as “ideal point”). The two points constituting the point pair are set so that the midpoint of the two points is located on the edge of the search object unless the search object has a deviation in size, position, and rotation angle. Although the ideal point and the edge point should match, there is actually a shift in the image processing object, and a shift occurs between the ideal point and the edge point obtained by the calculation.
[0106]
Therefore, if the re-search step is executed without abnormality, the measurement step is executed next, and the position of the edge point is calculated. In the measurement step, first, a measurement template 236 as shown in FIG. 27 is automatically set. The measurement template 236 has a plurality of seek lines 238, data of a master measurement template set in advance, data of a relative position of the re-search template coordinate plane with respect to the reference coordinate plane in the re-search step, and re-search It is set based on the calculation result of the edge point on the template coordinate plane.
[0107]
The master measurement template data is stored together with the master search template data and the like as illustrated in FIG. The data of the 20th line to the 33rd line in FIG. 10 is data for executing the measurement step, and the data of hs = 3.5 in the 21st line, the data in the 23rd line to the 27th line and the 29th line to the 33rd line are the master measurement. Template data. A master measurement template 240 obtained from this data is shown in FIG. Reference numeral 242 denotes a seek line. The master measurement template 240 has more seek lines than the master search template for the same electronic component.
When the image processing object is a disc 206 with a part cut away, a master measurement template 244 having a plurality of seek lines 246 is set as shown in FIG.
[0108]
Part or all of the measurement template seek line is paired. On the extension line of the seek line, another seek line is set symmetrically with respect to the center line of the image processing object. These paired seek lines are referred to as pair seek lines.
The measurement template is set by coordinate conversion of master measurement template data. Coordinate plane of the search template (The coordinate plane of this search template is the same as the search template coordinate plane of the search template when the image processing target is determined to be a search target that matches the search template in the search step. The relative position and relative rotation angle with respect to the reference coordinate plane, and the relative position and relative rotation angle of the image processing object with respect to the re-search template coordinate plane (these are based on the coordinate values of the edge points calculated in the re-search step). Since this calculation is performed, coordinate conversion corresponding to this calculation is applied to the master measurement template data (set in the master measurement template coordinate plane that matches the reference coordinate plane). is there.
[0109]
When the automatic setting of the measurement template is completed, the calculation of the edge point on each seek line of the measurement template is performed in the same manner as that in the re-search step. A dividing point is set on the seek line at a constant pitch, and the luminance is calculated by linear interpolation for each dividing point, and a differential filter is used to calculate a luminance differential value and an edge point is calculated. Also in the measurement step, the number of allowable failures is set. The fail here means that the edge point is not calculated for the seek line as in the re-search step. If the number of failures is less than or equal to the set number, the result is normal, and then a re-measurement step is executed. If the number of failures exceeds the set number, the measurement step is terminated abnormally. The processing when an abnormality occurs is the same as in the search step.
[0110]
In the remeasurement step, a remeasurement template is set and an edge point is calculated. The remeasurement template is automatically set based on the measurement template and the edge point calculated in the measurement step. Based on the edge point obtained in the measurement step, the measurement template is rotated by coordinate transformation to a position where the ideal point is expected to be located on the edge point. The calculation of edge points in the re-measurement step is performed in the same manner as in the re-search step.
[0111]
The determination of abnormality in the re-measurement step uses the allowable number of failures set for the measurement step, and if there are more seek lines from which edge points cannot be obtained than the set number, it is abnormal and the image processing is terminated. The processing when an abnormality occurs is the same as in the search step. If the number is less than the set number, the process ends normally, and then the object vector, that is, the size, position, and rotation angle of the image processing object are calculated. The greater the number of executions of the re-measurement step, the smaller the deviation between the ideal point and the edge point, and the edge point detection accuracy is improved. The set number of remeasurement steps is preset and stored.
Note that the remeasurement template used for execution of the second and subsequent remeasurement steps is automatically set from the remeasurement template at the time of execution of the immediately previous remeasurement step and the calculation result of the edge point in the remeasurement step.
[0112]
The monitor CRT device 188 displays the progress of image processing. For example, when the search step is executed, the image and background of the image processing object based on the image data stored in the frame grabber memory 164 (for example, one set of four sets of image data obtained by imaging four electronic components) A monochrome image is displayed, and the angle of the search template is changed by the set pitch and the position of the search template is changed to a rectangular spiral, and the progress of the process is shown to the operator.
[0113]
The monitor CRT device 188 can display in two modes, an automatic selection display mode and a manual selection display mode. When the automatic selection display mode is set, the display of the image processing progress is displayed. Both the input related data related to the input from the input device 172 (this data also includes data for notifying the abnormality of the electronic component mounting device and requesting the operator to take action) while giving priority to the input related data Is displayed. Therefore, when data is input by the input device 172 in the state where the automatic selection display mode is set, the display of the image processing progress is automatically switched to the display of input related data.
In the manual display selection mode, only the display selected by the operator's manual operation among the display of the image processing progress and the display of the input related data is performed.
[0114]
Next, the calculation of the object vector of the image processing target will be described. An object vector calculation program for performing the calculation described below is stored in the memory card, and is copied to the DRAM 156 when the memory card is set in the control device 150.
Size, position, and rotation angle are calculated when specified. For example, in the example of FIG. 10, P in the fifth and 21st lines vf = PA represents the position (Position), A represents the angle (Angle), and it is specified to calculate the position and the rotation angle. .
[0115]
For example, the size calculation is necessary in the following cases. (1) When the size of the image processing object is necessary, (2) When it is desired to identify the image processing object as an object similar to the search object but not the search object, (3) Fail in edge measurement. This is the case when it is desired to ensure position measurement accuracy. (2) is, for example, a case where it is necessary to distinguish between electronic parts having the same shape but slightly different sizes. The reason why the size calculation is necessary in the case of (3) is that if there is a failure, the size is required to determine the position of the image processing object in consideration of the failure as will be described later.
Also in this embodiment, the size calculation is not performed if none of the above (1) to (3) applies, but if the size calculation is necessary, prior to the calculation of the position and the rotation angle. Done.
[0116]
Hereinafter, a case where the image processing object is a square chip, size, position, and angle calculations are specified and there is a failure will be described as an example.
First, the size calculation will be described. The size calculation is performed using a pair seek line with an instruction to use for size calculation. In other words, if there is no instruction for use in size calculation in any seek line, size calculation is not performed if the seek line is not a pair seek line even if there is an instruction. Whether or not to perform size calculation by setting a flag to 0 when there is no designation to perform size calculation for a seek line in one template, and setting the flag to 1 if there is even one specification. Is determined. In this way, when size calculation is unnecessary, it is possible to know that size calculation is not required without determining whether or not each of the plurality of seek lines is instructed to be used for size calculation. And processing time is short.
[0117]
The first size calculation is a size factor calculation. The size factor is an excess rate of the size of the image processing object in each of the X-axis direction and the Y-axis direction of the measurement template coordinate plane (measurement template setting coordinate plane). The size excess ratio in the X-axis direction is the distance between the edge points of each set of seek lines (measurement span) of a plurality of pairs of seek lines set parallel to the X-axis direction, as shown in FIG. Is divided by the distance (referred to as the original span) between the ideal points (points marked with x in the figure) of the pair seek line. Further, the size excess ratio in the Y-axis direction can be obtained by averaging the values obtained by dividing the measurement span of each pair seek line of a plurality of pair seek lines set in the Y-axis direction by the original span. A broken line in FIG. 28 indicates a seek line in which a failure has occurred. For a pair seek line including a seek line in which a failure has occurred, the size excess rate is not calculated. The size is calculated by multiplying the size excess ratio in the X-axis direction and the size excess ratio in the Y-axis direction by the respective original spans of the square chip 250 in the X-axis direction and the Y-axis direction.
[0118]
If there is a failure in any one of the plurality of seek lines, a size point, which is a point obtained by correcting the ideal points with the size factor for all the seek lines having no failure, is calculated. As shown in FIG. 29, for a seek line substantially parallel to the Y-axis direction, the Y-coordinate value of the size point is calculated by multiplying the Y-coordinate value of the ideal point by the size excess rate in the Y-axis direction. Is omitted, the seek line parallel to the X-axis direction is calculated by multiplying the X coordinate value of the ideal point by the size excess rate in the X-axis direction.
[0119]
In addition, as described above, the size factor is calculated using only a pair of seek lines parallel to the X axis and the Y axis, and the size using the obtained size factor is used. The point calculation and the like are also performed for a seek line that is not parallel to the X axis and the Y axis. The calculation of the size point for the inclined seek line and the following calculation are substantially the same as the calculation of the size point in the circular reference mark described later.
[0120]
Next, the difference Diff between the edge point and the size point is calculated. The size point corresponds to an ideal point when an image processing object having a size error is considered to have been set again after approving the size error. Therefore, the difference Diff between the edge point and the size point represents the positional deviation amount of the edge point itself. The difference Diff is calculated by subtracting the size point coordinate value from the edge point coordinate value. If the difference Diff is a positive value, the edge point is shifted to the positive side on the measurement template coordinate plane. It will be.
[0121]
The reason for calculating the size point when there is a failure in this way is to reduce the influence of the failure on the calculation of the position of the image processing object and to reduce the calculation error. For example, if the size of the image processing object is larger than the original size but there is no position shift, if one seek line constituting the calculation error pair seek line has a failure and the other seek line has no failure, If the size point is not calculated and the calculation is performed with the ideal point, the sum of the difference between the edge point and the ideal point becomes larger on the non-failing side than on the failing side. In spite of this, a calculation result indicating that the image processing object is shifted to the seek line with no failure is output, and the size point is calculated to avoid this.
[0122]
Next, the calculation of the position and the rotation angle will be described.
The calculation of the rotation angle is performed only based on the seek line of 0 degree, 90 degrees, 180 degrees, and 270 degrees and instructed to be used for the calculation of the rotation angle. In addition, each process of 0 degree and 180 degrees, 90 degrees and 270 degrees is performed in a lump.
[0123]
The position and the rotation angle are calculated with respect to the rotation center RC. The rotation center RC is a calculation center of the computer, and may be different from or coincide with the designated center DC designated by the operator as the center of the image processing object. The designated center DC is a point that is used as a reference for an image processing object in creating a program for mounting the electronic component 82 on the printed circuit board 90. In the present embodiment, the master search template coordinate plane and the master measurement template coordinate plane are used. The position correction amount in the horizontal plane of the electronic component 82 is calculated for the designated center DC as described later.
[0124]
First, taking the straight line L shown in FIG. 30 as an example, the calculation of the position and the rotation angle will be described. Originally four seek lines were set for the straight line L, but three seek lines SL were set depending on whether one failed or three seek lines were set from the beginning.₁ , SL₂ , SL_Three If the above-described difference Diff is calculated, the center of rotation RC is the two left seek lines SL.₁ , SL₂ Set close. The rotation center RC is the absolute value of the sum of the distances between the plurality of seek lines on one side and the rotation center RC (the distance between the seek line and the rotation center RC when there is one seek line), and the other The absolute value of the sum of the distances between the plurality of seek lines and the rotation center (the distance between the seek line and the rotation center RC when there is one seek line) is set to a position where they are equal. In other words, if one side of the rotation center RC is positive and the other side is negative, the rotation center RC is set at a position where the sum of distances to all seek lines becomes zero. The rotation angle (radian) is calculated by equation (7).
Rotation angle = (AO · A ′ + BO · B ′ + CO · C ′) / (AO² + BO² + CO² (7)
However,
AO: Center of rotation RC and seek line SL₁ Distance to
BO: Center of rotation RC and seek line SL₂ Distance to
CO: Center of rotation RC and seek line SL_Three Distance to
A ′: difference Diff on seek line A
B ′: Difference Diff on seek line B
C ′: Diff on the seek line C
Further, the position of the rotation center RC is calculated by the equation (8).
(A '+ B' + C ') / 3 (8)
[0125]
The angle can also be obtained by calculating the angle for each of a plurality of seek lines using the difference Diff and the distance from the center of rotation of the edge point, and averaging them. If the angle deviates greatly even at one location, the error greatly affects the final angle value. On the other hand, if the angle is calculated according to the above equation (7), the influence of the error at a specific edge point can be reduced, and the calculation accuracy of the rotation angle can be improved.
[0126]
If any of the plurality of seek lines has a failure and an edge point cannot be obtained, as shown in FIG. 31, there is a deviation between the designated center DC and the rotation center RC. SL₂ Is a seek line that has failed. Generally, since the designated center DC is set at the center of the image processing object and the seek line is set symmetrically with respect to the designated center DC, a failure occurs in the seek line, and the rotation center RC corresponding to the failure is set. In other words, there is a difference between the designated center DC and the rotation center RC.
As described above, the rotation center RC is positive on one side of the rotation center RC and negative on the other side, so that the sum of the distances between the edge points of the plurality of seek lines and the rotation center RC becomes zero. Therefore, equation (9) is established, and equation (10) is obtained from this equation (9), and the deviation v between the rotation center RC and the designated center DC is calculated.
t₀ + T₁ + T_Three = 0 (9)
(S₀ −v) + {s₀ + (S₁ -S₀ ) -V} + {s₀ + (S_Three -S₀ ) -V} = 0 (10)
However,
t₀ : Rotation center RC and seek line SL₀ Distance to
t₁ : Rotation center RC and seek line SL₁ Distance to
t_Three : Rotation center RC and seek line SL_Three Distance to
s₀ : Designated center DC and seek line SL₀ Distance to
s₁ : Designated center DC and seek line SL₁ Distance to
s_Three : Designated center DC and seek line SL_Three Distance to
[0127]
As shown in FIG. 32, when the seek line is set for two sides parallel to each other of the rectangular image processing object, the equation (11) is established, and the deviation between the rotation center RC and the designated center DC is established. The quantity v is calculated based on equation (12).
t₀ + T₁ + T₂ + T_Three + T_Four + T_Five = 0 (11)
s₀ + {S₀ + (S₁ -S₀ )} + {S₀ + (S₂ -S₀ )} + {S₀ + (S_Three -S₀ )} + {S₀ + (S_Four -S₀ )} + {S₀ + (S_Five -S₀ )}-6v = 0 (12)
[0128]
In the case of the image processing object shown in FIG. 33, Expression (13) is established, and the deviation v between the rotation center RC and the designated center DC is calculated based on Expression (14).
t₀ + T₁ + T₂ + T_Three + T_Four = 0 (13)
s₀ + {S₀ + (S₁ -S₀ )} + {S₀ + (S₂ -S₀ )} + {S₀ + (S_Three -S₀ )} + {S₀ + (S_Four -S₀ )}-5v = 0 (14)
[0129]
As shown in FIG. 34, when the seek lines are set in two directions perpendicular to each other, equations (15) and (16) are established for the X-axis direction and the Y-axis direction, respectively. Each position deviation in the X-axis direction and Y-axis direction from the specified center DC_x, V_yAre calculated based on the equations (17) and (18), respectively.
t₀ + T₁ + T₂ + T_Three + T_Four + T_Five = 0 (15)
t₆ + T₇ + T₈ + T₉ + T_Ten+ T₁₁= 0 (16)
s₀ + {S₀ + (S₁ -S₀ )} + {S₀ + (S₂ -S₀ )} + {S₀ + (S_Three -S₀ )} + {S₀ + (S_Four -S₀ )} + {S₀ + (S_Five -S₀ )}-6v_x= 0 (17)
s₆ + {S₆ + (S₇ -S₆ )} + {S₆ + (S₈ -S₆ )} + {S₆ + (S₉ -S₆ )} + {S₆ + (S_Ten-S₆ )} + {S₆ + (S₁₁-S₆ )}-6v_y= 0 (18)
[0130]
Center of rotation RC based on equation (17) for X-axis direction_XDisplacement in the X-axis direction with respect to the specified center DC_x, And based on that, the center of rotation RC_XIs required. Center of rotation RC based on equation (18) for Y-axis direction_YDisplacement in the Y-axis direction with respect to the specified center DC of_y, And based on that, the center of rotation RC_yIs required. Center of rotation RC_XThrough the straight line parallel to the Y axis of the seek line setting coordinates and the rotation center RC_Y, And the intersection with the straight line parallel to the X axis is the rotation center RC of the image processing object shown in FIG.
[0131]
FIG. 35 shows another example of an image processing object in which seek lines are set in two directions perpendicular to each other. With respect to this image processing object, Equations (19) and (20) are established, and each displacement v in the X-axis direction and Y-axis direction between the rotation center RC and the designated center DC is v._x, V_yAre calculated based on the equations (21) and (22), respectively. However, the image processing object shown in FIG. 34 has no deviation between the rotation center RC and the designated center DC, and v_x= V_y= 0, and the distance between the seek line and the rotation center RC and the designated center DC is shown to be the same, and therefore, the equations (19) and (20), and (21) and (22) The same symbols are used in the equations.
s₀ + S₁ + S₂ + S_Three = 0 (19)
s_Four + S_Five + S₆ + S₇ = 0 (20)
s₀ + {S₀ + (S₁ -S₀ )} + {S₀ + (S₂ -S₀ )} + {S₀ + (S_Three -S₀ )}-4v_x= 0 (21)
s_Four + {S_Four + (S_Five -S_Four )} + {S_Four + (S₆ -S_Four )} + {S_Four + (S₇ -S_Four )}-4v_y= 0 (22)
[0132]
Next, in order to calculate the rotation angle, an angle factor shown in FIG. 36 is set in advance. The angle factor is determined by the position and polarity of the seek line. If the angle factor shown in FIG. 36 is shown on XY coordinates, it will be as shown in FIG.
[0133]
FIG. 38 shows an example of calculating the position and rotation angle of the image processing object using the rotation center RC and the angle factor. The rotation angle is calculated according to equations (23) and (24). D in equation (23)₀ t₀ Or d_Four v₁ The sign of is determined by the angle factor. The rotation angle calculated by the equation (23) is the rotation angle of the image processing object with respect to the measurement template coordinate plane.
Further, the positional deviation Δx in the X-axis direction and the positional deviation Δy in the Y-axis direction of the rotation center RC are calculated by the equations (25) and (26), respectively. These positional deviations Δx and Δy are deviations from the actual position of the rotational center RC.
Rotation angle = (− d₀ t₀ -D₁ t₁ + D₂ t₂ + D_Three v₀ -D_Four v₁ ) / T_a·····(twenty three)
t_a= T₀ ²+ T₁ ²+ T₂ ²+ V₀ ²+ V₁ ²····(twenty four)
Δx = (d_Three + D_Four ) / 2 ... (25)
Δy = (d₀ + D₁ + D₂ ) / 3 (26)
Where d₀ , D₁ , D₂ , D_Three , D_Four Are the deviations of the edge points from the size points in the five seek lines.
[0134]
The rotation angle and the rotation center RC when the image processing object is a square chip and seek lines are set on two sides parallel to the X axis and Y axis of the square chip image 250 as shown in FIG. The calculation of the positional deviations Δx and Δy is shown in equations (27) to (29).
Rotation angle = (− d₀ t₀ + D₁ t₁ + D₂ t₂ -D_Three t_Three + D_Four v₀ -D_Five v₁ -D₆ v₂ + D₇ v_Three ) / (T₀ ²+ T₁ ²+ T₂ ²+ T_Three ²+ V₀ ²+ V₁ ²+ V₂ ²+ V_Three ²(27)
Δx = (d_Four + D_Five + D₆ + D₇ / 4 (28)
Δy = (d₀ + D₁ + D₂ + D_Three / 4 (29)
d₀ ~ D_Three Is the distance from the center of rotation RC is t₀ ~ T_Three Is the amount of deviation between the edge point and the size point on the seek line, d_Four ~ D₇ Is the distance from the center of rotation RC is v₀ ~ V_Three This is the amount of deviation between the edge point and the size point on the seek line.
[0135]
Instead of mathematically proving that the above equations are valid as the calculation formulas for the rotation angle and the positional deviations Δx and Δy, the numerical values are specifically substituted to show the validity.
Assume that the image processing object having the shape shown in FIG. 40 rotates by −0.1 (radian) around the rotation center RC, and the rotation center RC is shifted by 5 mm in the Y-axis direction and 0 mm in the X-axis direction. . In this case, t₀ = 30 mm, t₁ = 20_mm, T₂ = 50_mm, V₀ = 30_mm, V₁ = 30_mmAs long as the rotation angle is small enough to be regarded as sin θ = θ, d₀ = + 8mm, d₁ = + 7mm, d₂ = 0 mm, d_Three = -3mm, d_Four = + 3mm. T_a= T₀ ²+ T₁ ²+ T₂ ²+ V₀ ²+ V₁ ²Is 5600 (mm² ) Is obtained.
Substituting these values into the equations (30), (31) and (32), the rotational angles and the positional deviations Δx and Δy of the rotational center RC are −0.1 radians, 5 mm and 0 mm, respectively, as follows: The validity of the formula is confirmed.
Rotation angle = {− (8 × 30} − (7 × 20) + (0 × 50) + (− 3 × 30) − (+ 3 × 30)} / 5600 = −0.1 (30)
Δx = (− 3 + 3) / 2 = 0 (31)
Δy = (8 + 7 + 0) / 3 = 5 (32)
[0136]
Although the rotation angle and the position shift of the rotation center RC are calculated as described above, since the rotation angle and the position shift of the designated center DC are necessary for execution of the electronic component mounting work, the rotation of the rotation center RC is performed. It is necessary to calculate those of the designated center DC from the angle and the displacement. However, since the rotation angle is the same for the rotation center RC and the designated center DC, there is no need for calculation, and only the positional deviation may be calculated.
There is no rotation angle error in the image processing object, and the positional deviation Δx is simply between the X-axis direction and the Y-axis direction.₁ , Δy₁ If both are only generated, the positional deviation of the rotation center RC and the positional deviation of the designated center DC are both Δx.₁ , Δy₁ It becomes. However, when a rotation angle error Δθ occurs in the image processing object, as shown in FIG. 41, v is respectively obtained from the rotation center RC in the X-axis direction and the Y-axis direction._x, V_yThe positional deviation of the designated center DC at a position separated by₁ −Δθ × v_y), (Δy₁ + Δθ × v_x)
[0137]
The calculated rotation angle Δθ and positional deviation (Δx₁ −Δθ × v_y), (Δy₁ + Δθ × v_x) Is the rotation angle and positional deviation of the designated center DC of the image processing object with respect to the measurement template coordinate plane, but the measurement template coordinate plane itself has a rotation angle θ and a positional deviation Δx with respect to the reference coordinate plane as shown in FIG.₂ , Δy₂ It is normal to have.
The reference coordinates are generally set as the optical axis of the CCD camera, that is, the coordinate plane having the origin at the center of the visual field, and the axis of the suction nozzle 80 coincides with the optical axis of the CCD camera at the component posture detection position. Positioned. In this case, the rotation angle θ and the positional deviation Δx with respect to the reference coordinate plane of the measurement template coordinate plane₂ , Δy₂ Is the rotation angle and positional deviation of the measurement template coordinate plane with respect to the axis of the suction nozzle 80.
Therefore, the rotation angle and positional deviation of the designated center DC of the electronic component as the image processing object with respect to the axis of the suction nozzle 80 are θ + Δθ and Δx, respectively.₂ + (Δx₁ −Δθ × v_y) Cos θ, Δy₂ + (Δy₁ + Δθ × v_x) Sin θ.
[0138]
If there is a failure, the coordinates of the center of rotation, the distance from the center of rotation to each seek line, the displacement between the center of rotation and the specified center, etc. are calculated as described above. If there is no failure, these values are calculated. Since the values are determined to be constant values for each combination of the image processing object and the template, these values are stored as default values in the memory card, and the above calculations are performed using these default values. Is called. If no size calculation is specified, all seek lines are paired, and there is no failure, the size point calculation is omitted and the edge point and the ideal are replaced with the size point. The difference from the point may be calculated, and the position and rotation angle may be calculated based on the calculation result. In this case, even if there is an error in size, the influence of the size error is canceled between the two seek lines constituting the pair seek line, and the calculation result of the position is not affected.
Furthermore, even when there is a failure, if the calculation of misregistration is not specified, the size point is unnecessary and the calculation may be omitted.
[0139]
Next, a case where the image processing object is a reference mark on a printed circuit board will be described. The reference mark is attached to the printed circuit board as described above, and the horizontal position error and the rotational position error of the printed circuit board are calculated based on the image of the reference mark. Therefore, it is not necessary to calculate the rotation angle of the reference mark.
In addition, the calculation of misalignment of fiducial marks has many parts in common with the calculation of misalignment of rectangular electronic components, but the shape is often circular or part of the circle is cut off. There is peculiarity in the operation. Hereinafter, this point will be described.
[0140]
First, a case where there is a failure in the seek line of the circular reference mark image 260 will be described with reference to FIGS. Here, as shown in FIG. 43, the 0-degree seek line indicated by the broken line is the fail seek line, and the reference mark image 260 is larger than the original size (the size indicated by the two-dot chain line in the figure). Let it be big.
The size calculation is also performed on the reference mark image 260 first. In the size calculation, the size factor is calculated as in the case of the corner chip image 250. If eight seek lines are set radially at 45 degree intervals as shown in FIG. 43, the size factors sizeFX and sizeFY in the X-axis direction and Y-axis direction are respectively sizeXM = sizeYM = 0 and baseXM = After initial setting of baseYM = 0, the calculation is performed by the equations (33) to (35).
for (i = 0; i <n; i ++) {sizeXM + = (measurement span [i] / original span [i]) * | cos angle [i] |; baseXM + = | cos angle [i] |; sizeYM + = (measurement span [i] / original span [i]) * | sin angle [i] |; baseYM + = | sin angle [i] |}; (33)
sizeFX = (sizeXM) / baseXM; (34)
sizeFY = (sizeYM) / baseYM; (35)
However, the expression (33) is written in C language, and for (i = 0; i <n; i ++) calculates the operation in {} for the i-th seek line, i from 0 to n It means the total sum of the results obtained while changing 1 by 1. In addition, since n is the number of pair seek lines and is 4 in the example of FIG. 43, i is sequentially changed from 0 to 3 in the calculation of the equation (33). When a value that specifies a pair seek line to be included is reached, the value can be changed to a value that specifies the next pair seek line without performing an operation. Therefore, in the example of FIG. 43, the calculation of Expression (33) is performed for three pairs of pair seek lines other than the pair seek line including the 0-degree seek line.
[0141]
In the case of a circle, since the seek lines are set radially, there are seek lines that are inclined with respect to the X axis and the Y axis, and these seek lines are components of an oversize factor in both the X axis direction and the Y axis direction. Have Therefore, in Equations (33) to (35), the X-axis direction component and Y-axis direction component of the size excess rate on each seek line are obtained, and each contributes to the determination of the size factor according to the component ratio. It is supposed to be made.
The sizeFX and sizeFY obtained by this calculation are oversize ratios in the X-axis direction and the Y-axis direction, respectively. In order to cope with the case where the size error occurs at different ratios in the X-axis direction and the Y-axis direction, the size factor is calculated separately in both directions.
[0142]
Next, the size point is calculated. First, as shown in FIG. 45, pairDiff, which is the difference between the ideal point and the size point, is calculated by the expressions (36) to (43) (expressions (36) and (37) are described in C language), A size point is calculated from the obtained pairDiff and the ideal point, and a shift Diff between the size point and the edge point is calculated using the edge point.
ΔL_x= PairRadius * cos θ * (sizeFX -1); ・ ・ (36)
ΔL_y= PairRadius * sin θ * (sizeFY -1); ・ ・ (37)
ΔL_x≧ 0 and ΔL_yWhen ≧ 0
pairDiff = √ {(ΔL_x)² + (ΔL_y)² }; ・ ・ (38)
ΔL_x<0 and ΔL_y<0
pairDiff = -√ {(ΔL_x)² + (ΔL_y)² }; ・ ・ (39)
ΔL_x<0 and ΔL_y≧ 0 and | ΔL_x| ≧ | ΔL_yIn case of |
pairDiff = -√ {(ΔL_x)² − (ΔL_y)² }; ・ ・ (40)
ΔL_x<0 and ΔL_y≧ 0 and | ΔL_x| <| ΔL_yIn case of |
pairDiff = √ {-(ΔL_x)² + (ΔL_y)² }; ・ ・ (41)
ΔL_x≧ 0 and ΔL_y<0 and | ΔL_x| ≧ | ΔL_yIn case of |
pairDiff = √ {(ΔL_x)² − (ΔL_y)² }; ・ ・ (42)
ΔL_x≧ 0 and ΔL_y<0 and | ΔL_x| <| ΔL_yIn case of |
pairDiff = -√ {-(ΔL_x)² + (ΔL_y)² }; ・ ・ (43)
[0143]
As described above, it is not necessary to calculate the rotation angle of the reference mark image 260, and only the position is calculated. In calculating the position, first, the position factor (posFactor) is calculated according to the equation (44).
posFactorM [i] = 1 / Σcos²(Angle R [i] −angle [n]) (44)
However, i is a value for specifying a seek line, and is 0 to 7. Also, the value of n is increased by 1 from 1 to 8, which is the number of seek lines. Σcos²The calculation of (angle R [i] −angle [n]) is the sum of the squares of the cosine of the difference between the angle R [i] of the i-th seek line and the angles R [n] of all unsuccessful seek lines. Is an operation for obtaining. Also in this calculation, when i becomes a value that specifies the seek line of the failure, the calculation can be changed to a value that specifies the next seek line without performing the calculation. Then, the calculation for the 0-degree seek line is not performed, and the position factor is calculated for each seek line of 45 to 315 degrees.
[0144]
Next, using the position factor, the position of the reference mark image 260 is calculated according to equation (45) (described in C language).
for (x = 0, y = 0, i = 0; i <n; i ++) {x = x + cos (i) * posFactorM [i] * ss [i]; y = y + sin (i) * posFactorM [i ] * Ss [i];} (45)
ss [i] is the previously calculated shift amount Diff between the size point and the edge point for each seek line, and the seek line shift amount ss [i] is set for all the seek lines having no failure. A position factor posFactorM [i] is multiplied and a component parallel to the X and Y axes is calculated and added for each of the X and Y coordinates to obtain a center position (x, y). The coordinates of the center position indicate a positional deviation of the center of the reference mark image 260 in the X-axis direction and the Y-axis direction from the origin of the measurement template coordinate plane. As described above, if there is a failure in the seek line, the rotation center RC of the image processing object is deviated from the position where there is no failure (which should be called the position of the normal rotation center). The position factor is calculated by, and the position deviation is calculated by using the position factor according to the equation (45), so that even if there is a failure in the seek line, the normal rotation of the image 260 is performed as in the case where there is no failure. The center misalignment corresponding to the center is calculated. Therefore, the value obtained by adding the positional deviation of the measurement template coordinate plane to the reference coordinate plane to the obtained positional deviation represents the positional deviation of the reference mark 260 on the reference coordinate plane.
[0145]
When the image processing object is circular and there is no failure, the calculation is performed using the default value for the position factor and the like. This is the same as in the above-described case where the image processing object is a square chip.
In the above description when the image processing object is the line segment of FIG. 30 or the square chip of FIG. 39, the term “position factor” has not been used. The position factor is calculated using the equation (44), and 1/3 in the equation (8), 1/4 in the equations (28) and (29), and the like correspond to the position factor. This can be easily confirmed by substituting an actual value into the equation (44). Equation (44) can be used generally regardless of the shape. The reason why the equation (44) is not used in the description of FIGS. 30, 39, etc. is to make it easy to understand intuitively. For the same example as the result of intuitive calculation of these simple examples ( This is because the validity of the equation (44) is confirmed by comparing the calculation result using the equation (44). Of course, the values of 1/3, 1/4, and the like change if a failure occurs in the seek line.
In addition, for example, the calculation for the circular image object can be applied as it is to an octagon in which a corner of the rectangle is obliquely cut at 45 degrees, as well as an image processing object having generally inclined sides. Size and position calculations can be performed similarly.
[0146]
As described above, the case where the image processing object is a square chip without a lead and the case where it is a circular reference mark has been described. The electronic component mounted on the printed circuit board 90 includes a quad flat package type electronic component. Some have a plurality of lead wires. When such an electronic component is an image processing object, image processing is performed by a pattern matching manager that is a combination of pattern matching processes. The pattern matching manager performs image processing by combining a pattern matching process a plurality of times.
[0147]
For example, in the case of a QFP (quad flat package electronic component) image 270 shown in FIG. 46, image processing is performed based on the lead image, not based on the outline of the entire QFP. This is because in QFP, if mounting is performed based on lead position data, the error between the lead and the pattern of the substrate can be minimized, and by combining pattern matching with each lead as an image processing target, QFP The size, position, rotation angle, etc. are calculated.
[0148]
At the time of image processing, first, one of the lead images 272 is searched. For this purpose, a search window 276 having a size suitable for including one lead image 272 is set, and the lead image 272 is obtained using a search template including a plurality of preset point pairs. Searched.
[0149]
A full set of pattern matching of the search step, re-search step, measurement step and re-measurement step is performed to measure the position and rotation angle of the lead image 272. If the position and rotation angle of the lead image 272 are known, then the lead image 272 of one side is searched by pattern matching of a subset including a search step and a re-search step. The lead pitch is known in advance, and the position and rotation angle of the search template coordinate plane in the search step for the next lead image 272 are the position and rotation angle of the lead image 272 obtained by the re-search step immediately before that. Therefore, the next lead image 272 can be searched with sufficient accuracy without performing full set pattern matching.
[0150]
When pattern matching is performed on the lead images 272 on all one side and the positions of all the lead images 272 are calculated, the X coordinate value and the Y coordinate value of the center of these lead images 272 are added. Divide by the number of leads to calculate the center coordinates of one side. Similarly, the three-side lead image 272 is searched by the pattern matching of the subset. The positional relationship between the sides is known in advance, and the position and rotation angle of the search template are set based on the position and rotation angle of the re-search template coordinate plane of pattern matching performed previously. If the center coordinates are calculated for all four sides, the intersection of two straight lines a and b connecting the centers of the two sides parallel to each other is calculated and used as the center coordinates of the electronic component. The rotation angle of the QFP image 270 can be obtained by equation (46).
(Slope of straight line a-90 degrees + slope of straight line b) / 2 (46)
[0151]
Even when the electronic component is a PLCC (plastic leaded chip carrier), a BG (ball grid array), or the like, image processing by a pattern matching manager is performed on a lead or a ball as an image processing target.
For example, when the electronic component is a PLCC, the irradiated light is reflected more by the leads. Although there is also reflected light from the main body, it is less than the lead, and a bright lead image is formed on the imaging surface of the CCD camera 134 in the dark main body image. Therefore, the present invention can be sufficiently implemented with the main body image as the background and the lead image as the image processing object.
[0152]
As described above, according to the pattern matching or the pattern matching manager, it is possible to perform image recognition of almost all types of surface-mounted electronic components (electronic components that can be mounted on the substrate surface). Although it is necessary to create a master search template and a master measurement template, image processing programs using these templates can be shared. Compared to creating an image processing program for each type of electronic component, the time required to create the program Is short. Images of vertically mounted electronic components to be soldered by inserting leads into holes in the printed circuit board can also be processed by pattern matching or a combination of pattern matching.
[0153]
Note that the image processing time becomes longer as the offset amount of the center of the image processing object from the center of the search window is larger. Therefore, it is desirable to correct the position of the center of the search window based on the position of the search template coordinate plane where the image processing object is searched. For example, in the case of the QFP image 270, after image processing for a plurality of QFPs, the average deviation from the center of the search window 276 at the center of the lead image 272 is obtained, and the average deviation is set to zero. The center position of the search window is corrected.
[0154]
Each time the component mounting head 56 is moved to the component orientation detection position and an electronic component is imaged by the CCD camera 134, pattern matching or a pattern matching manager and calculation of an object vector are performed to identify and position the electronic component. An error and a rotation angle error are calculated. As shown in the time chart of FIG. 47, during the time Tr required for one intermittent rotation of the index table 42, the electronic component 82 is irradiated with light while the component mounting head 56 is stopped at the component posture detection position. Then, imaging by the CCD camera 134 is performed. After the imaging, the image data is transferred from the CCD camera 134 to the frame grabber memory 164 and subjected to image processing. However, the transfer of the image data and the image processing are performed in parallel.
[0155]
The frame grabber memory 164 can store image data of four electronic components in parallel. Therefore, the processing time is longer than the time required for one intermittent rotation of the index table 42. Can also perform image processing. The image processing only needs to be completed before the image processing result is used. If the index table 42 serving as an intermittent rotating body allows a plurality of intermittent rotations from imaging to use of the image processing result. It is not essential to end the image processing while the index table 42 is intermittently rotated once. While the index table 42 is intermittently rotated a plurality of times, image processing of the same number of image processing objects as the intermittent rotation number is performed. Should just be done. This electronic component mounting apparatus has four work positions from the component posture detection position to the component posture correction position (see FIG. 4), and between the four electronic components mounted while the index table 42 rotates intermittently four times. In this case, the image processing time can be interchanged.
[0156]
For example, some electronic components have a simple shape such as a square chip and do not have a lead, and the required image processing time is short. If the electronic component has a large number of leads such as QFP, the required image processing time is long. There is also an electronic component, and the remaining time for an electronic component having a short required image processing time can be used for processing an electronic component having a long required image processing time.
If the image processing of all the electronic components must be completed within one rotation time of the index table 42, one intermittent rotation time (the sum of the stop time and the rotation time) of the index table 42 is It is necessary to determine the electronic component that requires the longest time from among a plurality of types of electronic components that are scheduled for image processing, and the intermittent rotation speed is kept low. In contrast, in the electronic component mounting apparatus, four pieces of image data are stored in parallel in the frame grabber memory 164, and processing of the image data is performed while the index table 42 rotates four times intermittently. The total of the image processing times of the four electronic components may be within the same number of intermittent rotation times as the number of electronic components on which image processing is performed. In other words, the average of the image processing times of the plurality of electronic components only needs to be within one rotation time of the index table 42, and the intermittent rotation speed can be increased.
[0157]
An example is shown in FIG. As is apparent from the figure, each image processing time T required for one electronic component_e1~ T_e7There is a difference in length, but the total image processing time for each four T_t1~ T_t4Are shorter than four times of one intermittent rotation time Tr, and four types of image data can be processed.
[0158]
As described above, the image data of the electronic component and the reference mark captured by the CCD cameras 128 and 134 in the electronic component mounting apparatus is processed by the pattern matching or pattern matching manager. A search template, a re-search template, a measurement template, and a re-measurement template are used to search for an image processing object, and an edge point is calculated. Only the portion where the template is set is searched and measured. Only the portion that requires image processing is processed, and the portion that does not need image processing is not processed even if image data is obtained. Therefore, image processing can be performed without being affected by most image noises (white spots, black spots, spots, etc.) as long as the electronic components are not in direct contact.
[0159]
For electronic components whose leads are inside the contour, such as PLCC, when it is not necessary to recognize the contour as in the case of recognizing only the lead, even if noise is in contact with the contour, Image processing can be performed without being affected.
In addition, an image processing target that does not need to recognize an outline, such as PLCC, can perform image processing even if the background is black, white, or a pattern, as long as there is no similar background. For example, if the main body and the background are the same color, or if the part of the background has the same color as the main body, it is difficult to recognize the outline due to confusion with the background, but the leads are glossy and the glossy object is similar to the background. As long as there is no, you can search and measure. By limiting the image processing object to the lead by setting the search template and the measurement template, it can be recognized without causing confusion with the background.
[0160]
Further, in the search step, it is determined whether or not it is in an appropriate state based on the difference in optical characteristic values such as the luminance difference between the two point pair constituent points. In the re-search step, measurement step, and re-measurement step, The edge point is determined by the change gradient of the optical characteristic value such as luminance on the seek line. Therefore, even when the image processing object is relatively large, such as QFP or PLCC, and uneven illumination is likely to occur, the optical characteristic values are compared between portions where there is almost no difference in illumination. Thus, it is possible to accurately perform image processing without being affected by the bias.
[0161]
In addition, even if the size of the solid-state image sensor of the CCD camera changes, it can be easily handled without substantially changing the program.
[0162]
As is apparent from the above description, in this embodiment, the CCD cameras 128 and 134 constitute an imaging device, the frame grabber memory 164 constitutes an image data storage means, and the DRAM 156 constitutes a search template data storage means and a measurement template. The part that constitutes the data storage means and executes the search step of the pattern matching program of the control device 150 constitutes the judging means, and the part that executes the re-search step, measurement step, and re-measurement step of the pattern matching program is the edge point Coordinate calculation means is configured. The portion of the pattern matching program of the control device 150 that executes the re-search step is also a midpoint reference type confirmation means that is a kind of confirmation means. Further, the part of the control device 150 that executes the object vector calculation program constitutes a midpoint reference type object calculation means that is a kind of object calculation means. Of the pattern matching program of the control device 150, the master search template, the master measurement template, the search template, or the part that automatically sets the seek line based on the edge point constitutes the seek line automatic setting means, The portion that repeats the re-measurement step constitutes a repeating means.
[0163]
Further, in the pattern matching program of the control device 150, a portion for designating a point for calculating the luminance in the search step, re-search step, measurement step, and re-search step constitutes a point designating means, and the luminance value of the designated point The part that performs the calculation constitutes virtual point data calculation means, the part that calculates the luminance value of the dividing point on the seek line constitutes the dividing point characteristic value acquisition means, and the luminance change gradient based on the calculation result The part that searches for the point with the largest point as the edge point constitutes the edge point searching means.
The Kanji ROM 162, the CRT interface 186, and the CPU 154 constitute input related data display means, the overlay display memory 166, the CPU 154, and the CRT interface 186 constitute image processing data display means, and the input related data of the CPU 154 is converted into image processing data. The portion that is displayed preferentially on the monitor CRT device 188 constitutes input related data priority display means.
The index table 42 constitutes an intermittent rotating body that rotates intermittently by a fixed angle, and the cylindrical cam 20, the barrel cam 34, etc. constitute a table rotation driving device as an intermittent rotating body driving device that intermittently rotates the index table 42. The nozzles 80 are provided on the intermittent rotator at an angular interval equal to the rotation angle pitch of the intermittent rotator, and constitute holding means for holding the respective image processing objects. The image processing apparatus is provided in one of the stop positions of the holding means so as to face the image processing object held by the holding means, and stores data of a plurality of images captured by the imaging device in the image data storage means. The data is stored in parallel, and the processing of the image data is completed while the intermittent rotating body performs the same number of intermittent rotations as the number of images stored in parallel at a later time. The electronic component mounting device constitutes a conveying device with an image processing device in the sense that the electronic component sequentially conveys the electronic component to 20 work positions which are the stopping positions of the holding means.
[0164]
The case where the origin of the master measurement template coordinate plane is placed at the designated center DC of the image processing object has been described above, but the origin of the master measurement template coordinate plane may be placed at a position other than the designated center DC. In this case, the positional deviation (or rotation angle) between the designated center DC and the master measurement template coordinate plane is added to the positional deviation (or rotational angle) between the rotational center RC and the designated center DC, and the value is further added. By adding a positional deviation (or rotation angle) of the master measurement template coordinate plane with respect to the reference coordinate plane, a positional deviation (or rotation angle) on the reference coordinates of the designated center DC is obtained.
[0165]
In the above embodiment, the size factor is calculated separately in both directions in order to deal with the case where the size error occurs in different ratios in the X-axis direction and the Y-axis direction. The calculation may be performed under the assumption that the size factors are the same. For example, in the square chip 250 of FIG. 28, the measurement spans for all pair seek lines in the direction parallel to the X-axis direction are divided by the original spans, except for the pair seek line including the seek line where the failure occurs. Because the value and the value obtained by dividing the measurement span for all pair seek lines in the direction parallel to the Y-axis direction by the original span are determined, and the average value of all these values is used as the size factor. is there. The size factor thus determined is commonly used for calculating the size point on the seek line parallel to the X-axis direction and for calculating the size point on the seek line parallel to the Y-axis direction.
[0166]
The same applies to the size factor of the circular reference mark image 260 in FIG. 43, and all of the pair seek lines not including the seek line in which the failure has occurred among the pair seek lines shown in FIG. Then, the value obtained by dividing the measurement span by the original span is obtained, and the size factor, which is the average value of these values, is multiplied by pairRadius for all seek lines except for the seek line where the failure occurred, and the size point is calculated. The difference Diff between the size point and the corresponding edge point is calculated. The position factor and the calculation of the position factor may be the same as in the above embodiment.
[0167]
Moreover, in the said embodiment, although the re-search step was performed once, you may make it be performed twice or more. In the first re-search step, the number of failures is less than the set number, and after the edge point of the image processing object is obtained, a re-search template is set and the re-search step is performed. On the basis of the re-search template used in the re-search step and the calculation result of the edge point, the position and angle at which the deviation from the image processing object is reduced are set. Experience has shown that the probability of a failure occurring at the next measurement step is lower when the re-search step is performed multiple times than when it is performed only once. The reason for this is not clear, but in the search step, if there is a large discrepancy between the image processing object and the search template, and it is barely determined that the search object exists, there is a discrepancy between the re-search template and the image processing object. In this case, the difference between the measurement template and the image processing object may increase in the next measurement step, and a failure may occur, while the re-search step is performed twice or more. If this is the case, it is speculated that the probability may decrease.
[0168]
  In addition, if the shape, approximate position, and rotation angle of the image processing object are known in advance, the seek line can be set to a position where the edge point can be calculated without searching using the search template. Edge points can be determined using measurement templatesThe
  In the above embodiment, after performing the re-search step using the re-search template, the measurement step is performed, but when the size error or shape defect of the image processing object is small, the search step It is also possible to execute the measurement step immediately after. In this case, it is possible to use the re-search template as a measurement template, but it is desirable to use a template having a larger number of seek lines.
[0169]
Further, in the above embodiment, the determination of abnormality in pattern matching is performed based on whether or not there is a failure exceeding the allowable number of failures set in the program. For example, if there is even one failure, it is determined as abnormal. In addition, an operator can create a program that calculates an object vector only for an image processing target that has no failure, and a program that allows at least one failure and performs an object vector calculation even when there is a failure. You may do it. Since a program that does not allow failure requires a short time for image processing, this program can be selected depending on the situation, and the time required for mounting one electronic component can be shortened.
[0170]
In the above-described embodiment, the image processing apparatus acquires a surface image of the image processing object, and the illumination apparatus is a front light type that irradiates light on the surface of the image processing object. Unlike the case of acquiring a projected image of an object, it is not necessary to provide a backlight light-emitting plate or reflector on the suction nozzle, and the suction nozzle becomes lighter and can be moved at high speed. Can be placed close to each other, and an electronic component mounting apparatus capable of mounting various types of electronic components on a printed circuit board at high speed can be obtained.
However, the present invention can also be applied to an apparatus that acquires a projection image of an image processing target and performs image processing.
[0171]
Further, the present invention provides a reference mark in an image processing apparatus and a screen printing machine provided in an apparatus for mounting an electronic component on a printed circuit board by moving a component suction nozzle linearly between the electronic component supply device and the printed circuit board. The present invention can be applied to various image processing apparatuses such as an apparatus that performs the image processing. In addition, the image processing target is not limited to an electronic component or a reference mark, and an object that requires identification or calculation of an object vector can be an image processing target.
[0172]
In addition, the present invention can be implemented in various modifications and improvements based on the knowledge of those skilled in the art without departing from the scope of the claims.
[Brief description of the drawings]
[Figure 1]It is an embodiment of the present inventionIt is a perspective view which shows the external appearance of the electronic component mounting apparatus provided with the image processing apparatus.
FIG. 2 is a side view showing an index table of the electronic component mounting device together with an electron supply device and a printed board moving device.
FIG. 3 is a front sectional view showing an index table of the electronic component mounting apparatus together with a table rotation driving device.
FIG. 4 is a plan view schematically showing 20 work positions provided in the electronic component mounting apparatus together with an electronic component supply device and a printed circuit board.
FIG. 5 is a front view showing a CCD camera provided at a component posture detection position, which is one of the work positions, together with an illumination device.
FIG. 6 is a block diagram showing a control device for controlling the CCD camera.
FIG. 7 is a diagram showing a preprocessing program stored in a DRAM of the control device.
FIG. 8 is a diagram showing an execution processing program stored in a DRAM of the control device.
FIG. 9 is a diagram showing a pattern matching program stored in a DRAM of the control device.
FIG. 10 is a diagram showing setting data for executing the pattern matching program for a square electronic component.
11 is a diagram showing a master search template set based on the setting data shown in FIG. 10 together with electronic components.
12 is a diagram showing a master measurement template set based on setting data shown in FIG. 10 together with electronic components.
FIG. 13 is a diagram illustrating generation of a search template based on the master search template.
FIG. 14 is a diagram showing setting data for executing the pattern matching program on a disc with a part cut away;
15 is a view showing a master search template set based on the setting data shown in FIG. 14 together with a disk.
16 is a view showing a master measurement template set based on the setting data shown in FIG. 14 together with a disk.
FIG. 17 is a diagram illustrating a state where a search template is superimposed on a virtual screen in the search step of the pattern matching program.
FIG. 18 is a diagram illustrating linear interpolation for calculating the luminance of a specified point on a virtual screen.
FIG. 19 is a diagram illustrating a state in which a re-search template in a re-search step of the pattern matching program is overlaid on a virtual screen.
FIG. 20 is a diagram illustrating a relationship between a division point set on a seek line of the re-search template and a solid-state imaging device constituting an imaging surface.
FIG. 21 is a chart showing the luminance calculated for the dividing points shown in FIG.
FIG. 22 is a diagram showing a difference filter for differentiating the luminance calculated for the division points.
FIG. 23 is a diagram showing another difference filter for differentiating the luminance calculated for the dividing point.
FIG. 24 is a diagram showing the calculation results shown in FIG. 21 in a graph.
25 is a graph showing the results of differentiation performed using the differential filter shown in FIG.
FIG. 26 is a graph showing the results of differentiation performed using the difference filter shown in FIG.
FIG. 27 is a diagram showing a state where a measurement template is superimposed on a virtual screen in the measurement step of the pattern matching program.
FIG. 28 is a diagram for explaining the size calculation of the image processing object performed after the execution of pattern matching, taking a square chip as an example.
FIG. 29 is a diagram for explaining calculation of a difference between a size point and an edge point in the size calculation.
FIG. 30 is a diagram for explaining the calculation of the rotation center and rotation angle of an image processing object, taking a straight line as an example.
FIG. 31 is a diagram for explaining the calculation of the deviation of the rotation center of the image processing object from the designated center when there is a failure in the seek line.
FIG. 32 is a diagram for explaining another example of the calculation of the deviation of the rotation center of the image processing target object from the designated center when there is a failure in the seek line.
FIG. 33 is a diagram illustrating still another example of the calculation of the deviation of the rotation center of the image processing target object from the designated center when there is a failure in the seek line.
FIG. 34 is a diagram for describing calculation of a rotation center of an image processing object and a shift of the rotation center with respect to a designated center when seek lines are set in two directions orthogonal to each other.
FIG. 35 is a diagram for explaining another example of the calculation of the rotation center of the image processing object and the deviation of the rotation center with respect to the designated center when the seek line is set in two directions orthogonal to each other.
FIG. 36 is a chart showing angle factors used for calculating an angle of an image processing object.
FIG. 37 is a diagram showing the angle factor on a coordinate plane.
FIG. 38 is a diagram for explaining calculation of a rotation center and a rotation angle of an image processing object.
FIG. 39 is a diagram illustrating another example of calculation of the rotation center and rotation angle of an image processing object.
FIG. 40 is a diagram illustrating still another example of calculation of the rotation center and rotation angle of an image processing object.
FIG. 41 is a diagram for explaining the calculation of the position of the designated center when there is a deviation between the designated center and the rotation center of the image processing object and the image processing object has a position and angle deviation.
FIG. 42 is a diagram for explaining the calculation of the positional deviation amount and the angular deviation amount with respect to the reference coordinates of the image processing object.
FIG. 43 is a diagram illustrating a state in which a seek line is set for a circular reference mark that is a kind of image processing object.
44 is a diagram for explaining the calculation of the size of the reference mark. FIG.
FIG. 45 is a diagram for explaining the calculation of the size point and the difference from the edge point in the size calculation.
FIG. 46 is a diagram for describing pattern matching performed for QFP.
47 is a time chart showing the relationship between the intermittent rotation time of the index table of the electronic component mounting apparatus, the exposure time of the CCD camera, and the image transfer time. FIG.
FIG. 48 is a time chart illustrating an image processing cycle in the electronic component mounting apparatus.
FIG. 49 is a diagram conceptually illustrating a physical screen / virtual screen conversion driver that calculates image data on a virtual screen from image data on a physical screen in a control device that controls the imaging apparatus.
[Explanation of symbols]
42 Index table
56 Component mounting head
82 Electronic components
90 Printed circuit board
128,134 CCD camera
150 Controller
156 DRAM
164 frame grabber memory
172 Input device
188 Monitor CRT device
250 square tip
260 Reference mark
270 QFP electronic components

Claims (10)

An imaging device having an imaging surface in which a large number of imaging elements are arranged , and imaging an image processing object;
An image data storing means for storing data of the image captured by the imaging device,
A plurality of positions on the edge having a pair of two points located between the edges of the image processing object and a fixed distance larger than the arrangement pitch of the imaging elements of the imaging device. Search template data storage means for storing data of a search template having a plurality of point pairs set in advance corresponding to
When the search template of the search template data storage means is overlaid on a screen on which an image represented by the image data of the image data storage means exists, the optical characteristic values of the pairs of points constituting the plurality of point pairs If difference state is equal to or greater than a set state is located at the edge within the side of one said image processing object point of the pair, the other is to be in conformity state located in the background, the plurality of sets points An image processing apparatus comprising: a determination unit that determines that an image processing object is a search object that conforms to a search template if a pair or more of a pair is in a conforming state. Further, the search template is generated by changing at least one of the rotation angle and the position of the master search template set for the edge of the image processing object without misalignment, and the search template data storage means Search template generation means for storing;
Repeating means for repeating the search template generation by the search template generation means and the determination by the determination means until the determination result of the determination means becomes affirmative;
The image processing apparatus according to claim 1, comprising: Further, means for generating a re-search template including a plurality of straight lines connecting two points each constituting the plurality of point pairs of the search template as a plurality of seek lines,
  Means for re-searching the search object by calculating coordinates of intersections of the plurality of seek lines of the generated re-search template and the edge of the image object;
The image processing apparatus according to claim 1, comprising: Further, based on the search template in which the determination result of the determination unit is affirmative, the edges of the image processing target object are sandwiched, and are spaced apart from each other by a certain distance larger than the array pitch of the image pickup elements of the image pickup device. Measuring template generating means for generating a measuring template having a plurality of seek lines formed by connecting two points in a straight line and corresponding to a plurality of positions on the edge;
Measurement template data storage means for storing measurement template data which is data of the measurement template generated by the measurement template generation means ;
The measurement template of the measurement template data storage means is overlaid on the screen on which the image represented by the image data of the image data storage means exists, and the coordinates of the edge points of the image processing object on each of the plurality of seek lines are set. the image processing apparatus according to the edge point coordinate calculating means into any one of including claims 1-3 for computing. Further, the image processing object includes confirmation means for confirming that the object to be searched is a search object based on a plurality of edge point coordinates calculated by the edge point coordinate calculation means, and the confirmation means is on the seek line. The image processing apparatus according to claim 4 , further comprising: a midpoint reference type confirmation unit that performs the confirmation based on a deviation between an edge point of the image processing object and a midpoint of a seek line. Furthermore, the size of the edge point coordinate computing means a plurality of image processing object based on the edge point coordinates calculated by, looking containing an object means for calculating at least one of the position and rotation angle, and the object The object calculation means calculates at least one of the size, position and rotation angle of the image processing object based on the distance between the edge point of the image processing object on the seek line and the midpoint of the seek line. The image processing apparatus according to claim 4, further comprising a point-based object calculation unit . And a seek line automatic setting means for automatically setting the plurality of seek lines to positions where the expected edge point positions on the seek lines are expected to coincide with the edge points calculated by the edge point coordinate calculation means; ,
Repeating means for repeating the automatic setting of the seek line by the automatic seek line setting means and the calculation of the edge point coordinates by the edge point coordinate calculating means on the automatically set seek line a plurality of times
The image processing apparatus according to any one of claims 4 to 6 . Is intended to generate an electrical signal that the imaging device according to the light receiving state of each of the plurality of the image pickup device, the data of the image data storage means electric signals of the imaging elements, associated with the position of each imaging element Point designation that designates an arbitrary point on the virtual screen that is assumed to correspond to the physical screen formed by the image data of the image data storage means. claim comprising means and, a virtual point data calculation means for calculating the optical characteristic value of the specified point based on the data of the optical characteristics value on the physical screen every designated point by the point specifying means 4 ~ The image processing apparatus according to any one of 7. The point designating unit designates one of a plurality of division points set on the seek line at a pitch smaller than the arrangement pitch of the imaging elements as the arbitrary point, and the virtual point data calculating unit A dividing point characteristic value acquiring unit that acquires an optical characteristic value of the designated dividing point , and the image processing apparatus is based on the optical characteristic value of the dividing point acquired by the dividing point characteristic value acquiring unit. The image processing apparatus according to claim 8 , further comprising an edge point search means for searching for the point at which the optical characteristic value on the seek line has the most abrupt change as an edge point. further,
An input device;
A monitoring device;
Input related data display means for causing the monitor device to display input related data related to the input of the input device;
Image processing data display means for displaying on the monitor device image processing data such as the progress and result of image processing by the image processing device;
The image processing apparatus according to claim 1 , further comprising: an input related data priority display unit that causes the display by the input related data display unit to be performed in preference to the display by the image processing data display unit.

Images