JP5873764B2 - Defect image presentation method - Google Patents

Description

  The present invention relates to a method for presenting a defect image obtained by imaging a defect to a user, and more particularly, to a defect image presentation method for supporting a task of finding a typical image including typical examples of defects from a large number of defect images. .

  For example, in the field of manufacturing technology such as semiconductors and printed circuit boards, in order to detect defects contained in products and analyze / evaluate them, the object to be evaluated is imaged through a microscope, etc. Research has been done on calculating the quantity and performing automatic classification. In this type of technique, many classification techniques using a learning algorithm based on a teacher image given in advance as an image including a typical defect have been proposed.

  For example, in the technique disclosed in Patent Document 1, a feature amount of a teacher image in which a defect category is designated by a user is obtained, and a discriminant tree is generated by learning using a genetic algorithm using the value, and an unknown defect image is obtained. Can be classified. In this case, the accuracy of the information given by the user greatly affects the accuracy of the learning result, and if the accuracy of the information given by the user is low, the classification accuracy may be lowered. On the other hand, in order to deal with such a problem, in the technique described in Patent Document 2, a plurality of typical images are designated by the user for each of the classification categories corresponding to the defect types, and based on these. By performing learning, the classification accuracy based on the learning result is improved.

JP 2003-317083 A JP 2010-071826 A

  All of the above-described conventional techniques are the same in that selection of a typical image (or teacher image) is left to the user's work. That is, the user needs to visually check a large number of defect images including defects and select a typical image from them, and the above-described conventional technology provides any support for this operation. I can't. For this reason, in the above-described prior art, the selection of the typical image related to the classification accuracy is still performed only by the judgment of the user, and there remains a problem that the result varies and an excessive burden is imposed on the user.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a technique capable of effectively supporting a user operation for selecting a typical image from a plurality of defect images.

The defect image presenting method according to the present invention is a user who finds a typical image by a computer selecting a typical image candidate including a typical example of a defect from a plurality of defect images obtained by imaging the defect and presenting it to the user. In order to achieve the above object , a defect image presentation method for supporting work , wherein the computer calculates a feature amount calculation step for calculating a plurality of feature amounts for each of the defect images , and the plurality of feature amounts. A grouping step of grouping the plurality of defect images by grouping the two defect images close to each other in the multi-dimensional feature amount space each having one coordinate axis, for at least one, by displaying on the display unit at least one of the defect images belonging to the group, and candidates for the typical image It is characterized by performing a process including a presentation step of presenting to the user Te.

  In the invention configured as described above, a plurality of defect images are represented by a plurality of feature amounts, respectively, and are grouped based on a positional relationship in a feature amount space having the feature amounts as coordinate axes. More specifically, the defect images are grouped so that defect images that are located close to each other in the feature amount space, that is, have similar image features, form one group. In this way, defect images having similar features are grouped in advance, and typical image candidates based on the results are presented, so that the workload of the user who tries to select a typical image from a plurality of defect images is reduced. It is greatly reduced. This is because a large number of defect images are pre-sorted into several groups, so that it is not necessary to check all of the defect images with the same weight.

  In addition, in a typical image selection operation based on a user's visual judgment in the prior art, designation that contradicts the feature of an image that is quantitatively and objectively represented by a feature amount may be made. For example, as long as each feature amount is compared, different defect types are specified for two defect images that can be regarded as similar, or vice versa, the same for two defect images that are greatly different in feature amount. A defect type may be specified. In this way, judgment based on the user's subjectivity and the quantitative analysis result may contradict each other, which reduces the learning accuracy. On the other hand, in the present invention, since it is possible to determine in advance whether or not there is similarity based on the feature amount and present it, it is possible to avoid such contradictory designations in advance and improve the accuracy of learning and classification based on it. It is possible.

  In addition, if the number of defect images belonging to one group is large, it can be said that the feature common to the group is caused by a “typical” defect having a high appearance frequency, while the group having a small number of defect images belonging to the group. It can be said that it shows a peculiar defect with a lower frequency of appearance. In this way, the size of the group, that is, the number of defect images belonging to the group, becomes information indicating the frequency of appearance of defects, and by grasping such information in advance, the user can deal with defects with high appearance frequency. It is possible to find a typical image to be detected more easily.

  As described above, according to the present invention, the user's work of selecting a typical image from a plurality of defect images is effectively supported to reduce the load on the user, and learning using the selected typical image and its Classification based on the result can be performed with high accuracy. In addition, regarding the presentation of typical image candidates to the user, for example, all or a part of representative defect images belonging to a certain group may be displayed. For example, typical image candidates from a large number of defect images may be displayed. Index information that can be easily extracted by the user may be presented.

  Here, in the grouping step of the present invention, for example, defect images are grouped by performing a search process for searching for a defect image close to the reference image for a reference image selected from a plurality of defect images. Also good. In this search process, for example, a point where the reference image is located in the feature amount space is set as a reference coordinate point, and for one coordinate axis, the value of the feature amount corresponding to the coordinate axis is determined from the value of the reference coordinate point on the coordinate axis. An extraction process for selecting a defect image within a predetermined vicinity range as a candidate image is executed for a plurality of coordinate axes, and defect images selected as candidate images in all of the plurality of coordinate axes targeted for the extraction process are selected. The neighboring images close to the reference image can be in the same group as the reference image.

  As a method for determining the degree of similarity between a plurality of images, a method for obtaining a distance between points corresponding to each image in the feature amount space has been generally used. For example, a Euclidean distance, a Manhattan distance, and a Mahalanobis are used. Distance etc. are used for this purpose. However, there are various feature quantities for representing the features of the defect image, and the dimension of the feature quantity space may exceed 100. When the dimension of the feature amount space increases in this way, the amount of computation for obtaining the above-mentioned distance between a plurality of images becomes enormous, and for example, a processor having high computation capability such as floating point processing becomes essential.

  On the other hand, in the search process of the present invention described above, the degree of similarity between two defect images is determined by comparing the values of one feature amount by the number of types of feature amounts. For this reason, the amount of calculation is greatly reduced as compared with the conventional method, and advanced calculation capability is not required.

  In this case, for example, the vicinity range may be set for one coordinate axis according to the distribution mode of the feature value corresponding to the coordinate axis in the plurality of defect images. In a plurality of defect images, the distribution of the value of a certain feature value depends on the contents of the collected images, and it is difficult to predict in advance. Moreover, there is a possibility that the degree of similarity between defect images may be inappropriately determined by using the neighborhood range that is set without reflecting the actual distribution mode. Such a problem can be avoided in advance by setting a neighborhood range in accordance with the distribution mode of the feature value values in the collected defect image and performing search processing.

  More specifically, for example, a numerical range of a section obtained by equally dividing a numerical range including a maximum value and a minimum value of a feature amount corresponding to one coordinate axis into a plurality of predetermined division numbers. Can be in the vicinity range. In this way, the distribution range from the maximum value to the minimum value of the feature amount is evenly divided into a plurality of sections, and the defect image is determined as a candidate image only by determining which section the feature amount value is in. It can be determined whether it is kimono. This further simplifies the process.

  In addition, for example, for a plurality of feature amounts whose difference between a maximum value and a minimum value between a plurality of defect images is larger than a threshold value set in advance for the feature amount, a coordinate axis corresponding to the feature amount is set. You may make it make it the object of an extraction process. When the difference between the maximum value and the minimum value of the feature amount between the plurality of defect images is small, it can be said that there is no large difference between the images regarding the feature represented by the feature amount. Therefore, even if the feature amount is excluded from the extraction processing targets, the result is hardly affected, and the necessary processing amount can be further reduced by doing so.

  Further, for example, in the grouping process, for each of a plurality of feature amounts, a table that associates the value of the feature amount with a defect image having the value is created, and the reference image is sequentially changed among the plurality of defect images. However, a search process may be performed each time to search for a neighboring image with respect to the reference image. As described above, the table in which the value and the defect image are associated with each feature amount is used to search for a neighborhood image of the reference image regardless of which of the plurality of defect images is selected as the reference image. It will be useful. This is because another defect image having a value close to the value of the feature amount in the reference image can be immediately derived by referring to the table. In particular, when the search process is repeatedly performed while sequentially changing the reference image, the same table can be referred to for all the reference images, and the effect is particularly remarkable.

  In addition, the presenting step in the present invention may be configured to present at least one defect image belonging to the group as a typical image candidate for at least one group including the group having the largest number of defect images belonging thereto. Good. As the number of defect images included in one group increases, it means that a feature common to the group is a typical one that appears frequently. Therefore, it can be said that each of the defect images belonging to the group or the whole of them is an image having a high probability of being a typical image. For this reason, presenting images belonging to such a large group has the same significance as performing on behalf of the user the task of selecting a highly probable image that should be a typical image from many defect images. . This greatly reduces the user's work.

  According to the present invention, it is possible to effectively support a user's work of selecting a typical image from a large number of defect images by presenting the defect images by grouping them in advance according to their characteristics.

It is a figure which shows the concept of the defect image which is the process target of this embodiment. It is a flowchart which shows the outline of the flow of the process in this embodiment. It is a figure which shows the example of distribution of the image in feature-value space. It is a figure which shows the range of the vicinity in this embodiment. It is a flowchart which shows the pre-processing for searching a near image. It is a figure which shows the concept of normalization and quantization of the value of a feature-value. It is a figure which shows the example of the table which linked | related the quantized feature-value and the image number. It is a flowchart which shows the grouping process of a defect image. 1 is a diagram showing a schematic configuration of an inspection system to which the present invention can be preferably applied.

  Hereinafter, a method for presenting a defect image according to an embodiment of the present invention will be described. In this embodiment, when automatic defect classification is performed in which a plurality of defect images are classified into several defect categories by an automatic learning algorithm, the process is executed as a previous stage process. More specifically, a process for supporting a user's work of finding a typical image that includes typical defects and can be a teacher image in automatic learning from a plurality of defect images collected by imaging an inspection target such as a substrate. It is. Prior to the description, the concept that is the premise of the present embodiment and the following description will be described.

  FIG. 1 is a diagram showing a concept of a defect image that is a processing target of the present embodiment. As shown in FIG. 1A, for each defect image belonging to the defect image group IG collected in advance (hereinafter, simply referred to as “image”), each image is individually distinguished. Serial numbers and unique image numbers (1, 2,...) Are assigned as identification codes. In the following description, the total number of images in the defect image group IG is P (P is a natural number of 2 or more), and any one of the images is represented as an image number p (p is a natural number, 1 ≦ p ≦ P). To do.

  The feature of the defect included in each image is expressed by a plurality of feature amounts. As shown in FIG. 1B, in this embodiment, the feature of the defect is represented by N types (N is a natural number of 2 or more) of feature amounts, and a code X1 using a capital letter X for each of these various feature amounts. , X2, ..., XN. Any one of these features is represented by a code Xn (n is a natural number, 1 ≦ n ≦ N).

  The value of the feature amount corresponding to the defect image with the image number p is represented by the symbol xpn using the lowercase letter x. Therefore, for example, N types of feature values corresponding to the defect image of image number 1 are represented by codes x11, x12,..., X1N, respectively. Further, the value of the feature quantity X1 corresponding to the image numbers 1, 2,..., N is represented by x11, x21,. Further, when paying attention to one feature amount Xn, the maximum value among the feature amount values x1n, x2n,..., XPn corresponding to each defect image is represented by a symbol Mn, and the minimum value is represented by a symbol mn.

  FIG. 2 is a flowchart showing an outline of the flow of processing in this embodiment. First, a plurality of different defect images are acquired (step S101), and a defect image group IG is constituted by these defect images. Each defect image includes defects such as pinholes and foreign matters appearing on the appearance of the semiconductor substrate to be inspected, and is captured on a plurality of images taken at different positions on the same substrate or on different substrates. In addition, the defect image group IG can be configured by collecting a large number of images.

  A plurality of feature amounts X1,..., XN are calculated for each of the plurality of defect images thus obtained (step S102). Thereby, the feature of the defect contained in each defect image is quantitatively represented by the N types of feature amounts. Assuming a multidimensional feature amount space having a plurality of feature amounts as coordinate axes, each defect image can be plotted in the feature amount space as coordinate points having the feature amount values as coordinate values. Images having similar features of defects included in the defect image are positioned at positions close to each other in the feature amount space. In other words, the defect features (eg, shape, size, etc.) of multiple images located close to each other in the feature space are similar and are likely to be defects based on the same type of origin. Can be estimated.

  In order to accurately perform automatic defect classification using a learning algorithm, it is necessary to find an image (typical image) including a typical defect that becomes a teacher image from a plurality of collected defect images. Even if the defect is the same, its appearance is slightly different, and a technique for automatically obtaining a typical image by computer image processing or the like has not been established. For this reason, the selection of typical images is left to the judgment based on the experience of skilled users. At this time, the user needs to carefully scrutinize a large number of images without any clues in advance and select a typical image, and the work load is very large. It is convenient if information on defect tendency and correlation that can be automatically grasped from the collected images is provided to the user.

  Therefore, in this embodiment, the defect images are grouped so that the images at positions close to each other in the feature amount space are in the same group (step S103). Thereby, images having similar characteristics are associated with each other. For each group formed in this manner, some of the images belonging to the group are presented to the user as typical image candidates. At this time, among some of the formed groups, a large group, that is, one having a large number of images belonging to the group is presented in priority order (step S104).

  The number of images to be presented is arbitrary, and may be all images belonging to the same group or a part thereof. As a mode of presentation, a method for displaying an image to be presented on a display unit such as a display may be used, and for example, a method for outputting a list of image numbers of defect images belonging to the same group. Also good. Further, information that can grasp the size of the group, for example, information on the number of defect images belonging to the group may be further added.

  The reason why a group having a large number of images belonging is preferentially presented is as follows. It means that the larger the group of images having similar characteristics, the more “typical” defects having the characteristics are generated. Therefore, in the learning algorithm based on the teacher image, it is desirable that a typical image corresponding to the group is included in the teacher image. For this reason, images are preferentially presented from a large group.

  By such presentation, only images having features similar to each other are narrowed down and presented from a large number of unorganized defect images that are not associated with each other at all. Therefore, the user only has to select some typical images from the presented images, and the user's workload in specifying the typical images is greatly reduced. That is, much of the work for finally selecting a typical image from an unorganized image group is automatically and objectively performed. The number of typical images selected from images belonging to one group is arbitrary. For example, all images in the same group, or all except for those apparently having different origins may be used as typical images. Good.

  In addition, the user does not need to check all of the defect images similar to each other, and looks at the characteristic part grasped from some of the presented images to determine whether the defect is typical. Just judge. On the other hand, even if an image having a special defect with few images with similar characteristics is presented as a group even though it is small, such a special defect is not overlooked and is made a typical image. It is possible to make the user determine whether or not.

  In addition, when a typical image is selected only by the user's visual observation, it is specified that contradicts the feature of the image that is quantitatively determined based on the feature amount, and learning based on this may cause confusion in the classification result. obtain. However, in this embodiment, similar images and images that are not so are sorted in advance, and the user can specify a typical image after grasping the information, and thus contradicts the quantitative judgment in this way. The possibility of making a designation is extremely low.

  Next, a method for finding a group of images having similar characteristics to each other from a large number of collected defect images will be described. As described above, in general, it can be said that a plurality of images located at close positions in the feature amount space are similar to each other. Therefore, it is possible to estimate the degree of similarity between images by calculating the mutual distance in the feature amount space for a plurality of images. Euclidean distances, Manhattan distances, Mahalanobis distances, and the like are known as parameters representing such a degree of similarity. However, the amount of calculation for obtaining these distances between a large number of images in a high-dimensional space is enormous. In addition, it is known as a “dimensional curse” that comparison of distances between defect images may make little sense in a high-dimensional feature space. In this embodiment, since there is no distance calculation in a multidimensional space, such a problem does not occur, and the degree of similarity between images is estimated by a simpler method. The method will be described below.

  FIG. 3 is a diagram illustrating an example of an image distribution in the feature amount space. Here, a two-dimensional feature amount space using two types of feature amounts X1 and X2 will be described as an example. However, the following concept can be extended and applied to a higher-dimensional feature amount space. As an example, let us consider a case where 20 defect images (P = 20) are plotted in accordance with the value of the feature quantity in a two-dimensional feature quantity space having the feature quantities X1 and X2 as coordinate axes. In FIG. 3, a circle with a number inside indicates the position of each defect image in the feature space, and the number indicates the image number of each defect image.

  The 20 images with image numbers 1 to 20 are arranged at respective positions in the coordinate space according to the values of the feature amounts X1 and X2. Here, as indicated by broken lines in FIG. 3, the images at close positions can be grouped together to divide the entire image into several groups. For example, a group of images represented by image numbers 1, 4, 7, 13, 17, and 20 are close to each other and constitute one cluster as a whole. One defect image is set as a reference image, and other defect images in the vicinity thereof are included in the same group as the reference image, and another defect image in the vicinity of the other defect image is also referred to as the reference image. Same group. It should be noted that an isolated image (for example, image number 8) that is far from other images may be regarded as one group having the isolated image as a single component.

  FIG. 4 is a diagram showing the vicinity range in this embodiment. In the concept of this embodiment, when one defect image is a reference image Is, a neighborhood range R1 and a feature amount appropriately determined on the coordinate axis corresponding to the feature amount X1 with the position of the reference image in the feature amount space as the center. An image within the range of the neighborhood range R2 appropriately determined on the coordinate axis corresponding to X2 is searched, and if there is such an image, it is determined as a neighborhood image close to the reference image.

  In this example, the images in the vicinity range R1 in the coordinate axis corresponding to the feature amount X1 with respect to the reference image Is include images indicated by symbols I1 and I2, respectively. Of these, the coordinate axis corresponding to the feature amount X2 is also included. The image I1 in the vicinity range R2 is set as the vicinity image of the reference image. Since the image I2 is out of range on the coordinate axis corresponding to the feature amount X2, it is not regarded as a neighborhood. On the other hand, the image I3 is within the vicinity range R2 on the coordinate axis corresponding to the feature amount X2, but is not considered to be near because it is outside the range on the coordinate axis corresponding to the feature amount X1. Naturally, the image I4 which is out of the range in both coordinate axes is not near.

  Although it can be said that the determination based on such a criterion is not as strict as the determination based on the Euclidean distance described above, the determination can be performed by repeating independent calculation for each coordinate axis, and the number of dimensions increases. There is an advantage that the calculation amount is not so large as compared with the above distance calculation in which the calculation amount increases exponentially. In addition, the purpose of the determination here is to support the user's determination by grouping and displaying images that are considered to be related to each other. For the purpose of searching for nearby images, the above determination method is Necessary and sufficient accuracy can be obtained. That is, here, the advantage of the small amount of processing is given priority over the accuracy of calculation.

  FIG. 5 is a flowchart showing preprocessing for searching for a neighborhood image. This process is a process in which the relationship between the feature values of each defect image is arranged in advance for each type of feature value in order to smoothly search for the neighborhood relationship between a large number of defect images. This process corresponds to steps S101 and S102 and a part of step S103 in the process shown in FIG. When a plurality of defect images are acquired (step S201), sequential image numbers are assigned as codes for identifying the defect images (step S202). Then, N types of feature amounts are calculated for each defect image (step S203).

  Next, one feature quantity Xn is selected from N types, and the respective defect images are arranged based on the magnitude relationship of the feature quantity values x1n, x2n,..., XPn. Here, for example, it is assumed that they are arranged in ascending order (step S205). In the image arrangement, it is not necessary to arrange actual defect images, and it is only necessary to obtain a sequence of image numbers arranged in the order of feature value values.

  Then, the difference between the maximum value Mn and the minimum value mn of these feature quantities is obtained (step S206). This difference is compared with a predetermined threshold value a (step S207), and if it exceeds the threshold value a, the next steps S208 to S211 are executed, while if not more than the threshold value, these steps are skipped.

  In steps S208 to S211, in order to facilitate the comparison of the feature value between the defect images, the feature value is normalized and quantized, and a table in which the value is associated with the image number is created. To do. However, when the difference between the maximum value Mn and the minimum value mn of the feature amount is equal to or less than the threshold value, it is considered that there is no significant difference between the images for the feature amount, and such work is omitted. As the threshold value a, for example, a value that is 1000 times the minimum positive number that can be expressed by a double-precision floating point type can be used. When the difference between the maximum value Mn and the minimum value mn is equal to or less than the threshold value a thus determined, the difference in value between the defect images becomes even smaller, and each coordinate axis corresponding to the feature amount is different for each coordinate axis. This is because they can be regarded as being very close without comparing between images. By excluding feature quantities that hardly affect the result in this way, the substantial number of dimensions becomes smaller than the original N dimensions, and the amount of computation can be further reduced.

  In step S208, the difference value δ between adjacent images in an ascending order of feature value values is obtained, and the difference value is averaged Δn by accumulating the value δ and dividing by the number of data. Is calculated. Thereby, the range of the spread of the feature value is obtained. At this time, if the value δ is smaller than the predetermined threshold value bn, it is excluded from the cumulative addition because the difference in feature quantity between adjacent images is substantially zero, and is also excluded from the number of data. The threshold value bn can be, for example, (Mn−mn) / 1000. In this way, in the next feature quantity quantization to be performed, it is possible to apply an appropriate quantization step that does not become an unnecessarily fine step depending on the variation in the value of the feature quantity between images. It becomes possible.

In step S209, the quantization division number Kn on the coordinate axis corresponding to the feature amount Xn is obtained. The quantization division number Kn is expressed by the following formula:
Mn ≦ Kn ・ Δn
The smallest integer that satisfies the relationship Further, in step S210, based on the quantization division number thus determined, the value xpn of the feature amount Xn in each defect image is expressed by the following equation:
ypn = Int {(xpn-mn) /Δn+0.5}
Normalized and quantized by Here, Int {x} is a function that returns the integer part of the variable x as a value. The reason for adding 0.5 to an integer is to round off the decimal part.

  As a result, the feature value xpn of each defect image distributed in the range from the minimum value mn to the maximum value Mn on the coordinate axis corresponding to the feature value Xn is normalized to take discrete values from 1 to Kn. It is converted into a quantized value ypn. As a result, information regarding a minute difference in feature value between images is lost, but this information is not necessary in the application as in this embodiment, and hardly affects the result. Then, a table in which the feature value ypn quantized in this way is associated with the image number of the defect image taking the value is created (step S211).

  Such processing is performed for all types of feature values (step S212). That is, while the feature quantities to be selected are sequentially changed, the above-described feature quantity values are normalized and quantized, and a table based on this is created. In addition, since some feature amounts may be excluded in step S207, the table is not necessarily created for all N types of feature amounts. In the following, the number of feature quantities for which a table has been created is represented by N ′ (≦ N).

  FIG. 6 is a diagram showing a concept of normalization / quantization of feature value. FIG. 7 is a diagram showing an example of a table in which quantized feature amounts are associated with image numbers. As the defect image group in these drawings and the description thereof, the same one as shown in FIG. 3 is used. The quantization of the feature value in the two-dimensional feature space means that the position of the coordinate point representing each image in the feature space is the closest among the lattice points provided at equal intervals as shown by broken lines in FIG. This corresponds to moving the object to the position. Further, normalizing the feature value corresponds to scaling the coordinate axis so that the minimum value of the quantized feature value is 1 and the maximum value is Kn in one coordinate axis. In the example of FIG. 6, the quantization division number Kn (that is, K1) is 9 on the horizontal axis corresponding to the feature quantity X1 (n = 1), and the quantization is performed on the vertical axis corresponding to the feature quantity X2 (n = 2). The number of divisions Kn (that is, K2) is 10. However, these are not limited to those shown as an example, and the number of quantization divisions is determined according to the distribution of feature values on each coordinate axis as described above. is there.

  The table illustrated in FIG. 7 indicates where each defect image with image numbers 1 to 20 is positioned in the feature amount space as a result of the normalization and quantization as described above. FIG. 7A is a table for the feature quantity X1, and FIG. 7B is a table for the feature quantity X2. In the figure, “division number” has the same meaning as the normalized and quantized feature value, and “image number corresponding to the division” has the same feature value as the division number as a result of normalization and quantization. Is the image number of the defect image. For example, referring to FIG. 7A, the value of the feature quantity normalized and quantized on the coordinate axis corresponding to the feature quantity X1 is 1 in three image numbers 4, 7, and 10. It is shown.

  As described above, if a table in which the feature value and the image number are associated with each other is created, when one image number is designated as the reference image, the reference image and the feature value are stored. It is possible to immediately read out the image numbers of other defective images close to the table.

  Specifically, this table is used as follows. For example, consider a case in which a defect image represented by image number 1 is used as a reference image and a neighboring image in the vicinity thereof is searched in the feature amount space. When, for example, image number 1 is designated as the image number corresponding to the reference image, those whose feature values are in the vicinity of the reference image value are specified as candidate images from each table. Here, the same section as the reference image and a section adjacent to the same section are set as the vicinity range. Then, as is apparent from FIG. 7A, the feature quantity X1 belongs to the same section (section number 2) as the image number 1, and 5 of image numbers 1, 11, 13, 18, 20 In addition, there are a total of five that belong to adjacent sections (section numbers 1 and 3): 4, 7, 10, 14, and 17. That is, the candidate images for the feature amount X1 are images identified by image numbers 1, 4, 7, 10, 13, 14, 17, 18, and 20, respectively.

  On the other hand, as can be seen from FIG. 7 (b), the feature quantity X2 belongs to the same partition number 2 as the reference image as image numbers 1, 7, and 16, and belongs to the adjacent partition numbers 1 and 3. Section numbers 4, 13, 17, and 20. Accordingly, the candidate images for the feature amount X2 are images specified by image numbers 1, 4, 7, 13, 16, 17, and 20, respectively.

  An image in the vicinity in the feature amount space is an image in the vicinity range of the reference image on all coordinate axes (feature amounts). Therefore, the product represented by the image numbers 4, 7, 13, 17, and 20 obtained by taking the product set of candidate images with the feature amounts X1 and X2 and excluding the reference image itself is the reference image (image number 1). It can be said that it is a neighborhood image corresponding to). Note that the feature amount space in this case is an N′-dimensional space excluding the coordinate axis that is determined to have a small difference between the maximum value Mn and the minimum value mn from the original N-dimensional space (step S207 in FIG. 5). Using this principle, defect images are grouped as follows in this embodiment.

  FIG. 8 is a flowchart showing the defect image grouping process. This process corresponds to step S103 in FIG. First, a defect image that has not yet been grouped is selected from a plurality of collected defect images, and designated as a reference image (step S301). An appropriate group label for identifying the group is assigned to the reference image (step S302). In order not to give the same label to different groups, it is assumed that the group label at this time is a new one not assigned to an existing group.

  Next, a neighborhood image is searched for the reference image. That is, based on the above principle, the N′-plane table created earlier is referred to, and each feature value whose value is in the vicinity of the reference image value is extracted as a candidate image (step S303). ). Then, those extracted as candidate images in all the tables on the N ′ plane are specified (step S304), and those obtained by excluding the reference image from them are specified as neighboring images corresponding to the reference image (step S305). ). Of the neighboring images thus identified, the same group label as that of the reference image is assigned to those that have not been given a group label (step S306). As a result, the reference image and its neighboring images are associated as the same group.

  According to the gist of the present embodiment, the same group label as the reference image is given to an image in the vicinity of the vicinity image even if the image is far from the reference image beyond the vicinity range. Need to be done. Therefore, the search for the neighborhood image and the label assignment for the neighborhood image of the reference image are subsequently performed. Specifically, the image numbers of the neighboring images with respect to the reference image obtained above are added to the stack (step S307), and those images are sequentially designated as new reference images (step S309), while the neighborhood of the image is displayed. Image search and group label assignment to the found neighboring images are performed (steps S303 to S306). If a new neighborhood image is found, it is added to the arithmetic processor stack (step S307). By repeating this until the stack becomes empty (step S308), the association between the images via the neighboring images starts from the first reference image, and the same group label is assigned to them. go.

  When the stack is empty, there are no more images left in the vicinity of the group. That is, the spread of the group converges. Therefore, the process returns to step S301 (step S310), and the above processing is repeated using an image that has not yet been grouped as a new reference image. At this time, a label different from the previous group label is given. The above process is repeated until any group label is given to all defect images. In this way, all images are grouped. The method of presenting the grouping result to the user is as described above.

  In the above-described example, the description has been made as the “neighboring range” up to the same section as the reference image or the section adjacent to the reference image in the normalized and quantized feature value. Naturally, the result of grouping differs depending on how the neighborhood range is set. That is, when the neighborhood range is widened, even images with relatively large differences are grouped into the same group. Conversely, if the neighborhood range is narrowed, even a slight difference in characteristics will be treated as another group. In particular, in a high-dimensional feature amount space, a small difference in appearance tends to be a large distance in the feature amount space and each image tends to be isolated. According to the knowledge of the inventor of the present application, in the defect image group of about Kn = 100, about ± 10 to ± 50 sections based on the same section as the reference image, although it varies depending on the number of defect images and the distribution of the feature amount. Good results have been obtained when is included in the neighborhood range.

  FIG. 9 is a diagram showing a schematic configuration of an inspection system to which the present invention can be preferably applied. This inspection system 1 is an inspection system that performs inspection of defects such as pinholes and foreign matters appearing on the appearance of a semiconductor substrate S to be inspected, and automatically classifies detected defects. The inspection system 1 performs the defect inspection based on the image data from the image pickup device 2 and the image pickup device 2 that picks up the inspection target area on the substrate S, and when the defect is detected, the defect belongs to the category to which the defect should belong. And a host computer 5 as a control unit having an inspection / classification function for automatically classifying (ADC) automatic defect classification and a function for controlling the entire operation of the inspection system 1. The imaging device 2 is incorporated in the production line of the substrate S, and the inspection system 1 is a so-called inline system.

  The imaging device 2 moves the stage 22 relative to the imaging unit 21 that acquires image data by imaging the inspection target region on the substrate S, the stage 22 that holds the substrate S, and the imaging unit 21. The imaging unit 21 includes a stage driving unit 23, and an imaging unit 21 forms an image by an illumination unit 211 that emits illumination light, an optical system 212 that guides the illumination light to the substrate S and receives light from the substrate S, and an optical system 212. An imaging device 213 that converts the image of the substrate S thus converted into an electrical signal is included. The stage drive unit 23 includes a ball screw, a guide rail, and a motor. The apparatus control unit 501 provided in the host computer 5 controls the stage drive unit 23 and the imaging unit 21, so that the inspection target region on the substrate S is changed. Imaged.

  The host computer 5 implements each functional block shown in FIG. 1 by software by executing a control program read in advance. In addition to the apparatus control unit 501, the host computer 5 includes functional blocks such as a defect detection unit 502, a defect classification unit (ADC) 503, a feature amount calculation unit 504, a teaching unit 505, a determination unit 506, and a calculation unit 507. I have. Further, the host computer 5 displays a storage unit 510 for storing various data, an input receiving unit 511 such as a keyboard and a mouse for receiving operation inputs from the user, and a display for displaying visual information for the user such as operation procedures and processing results. Part 512 and the like. Although not shown in the figure, a reading device for reading information from a computer-readable recording medium such as an optical disk, a magnetic disk, a magneto-optical disk, etc. Are suitably connected via an interface (I / F) or the like.

  The defect detection unit 502 performs defect detection by finding a unique area in the inspection target area while processing the image data of the inspection target area. When the defect detection unit 502 detects a defect from the inspection target area, image data of the defect and various data used for the inspection are temporarily stored in the storage device 510.

  On the other hand, the defect classification unit 503 performs, in software, a process of classifying detected defects using learning algorithms such as SVM (Support Vector Machine), neural network, decision tree, and discriminant analysis. To do. The feature amount calculation unit 504 calculates a feature amount that characterizes the defect based on the image data of the detected defect. A teaching unit 505 gives teaching data for causing the defect classification unit 503 to machine-learn the algorithm.

  The determination unit 506 and the calculation unit 507 each perform processing for calculating the accuracy indicating the validity of the classification result classified by the defect classification unit 503. Specifically, the determination unit 506 determines whether or not each feature value of the defect image is appropriate for each classification category. The calculation unit 507 calculates the accuracy based on the determination result of the determination unit 506.

  The present invention can be suitably applied to the inspection system configured as described above. That is, when the defect classification unit 503 performs machine learning by using some of a large number of defect images captured by the imaging device 2 as typical images, the main task is to assist the user in finding the typical image from the defect images. The invention can be used. The candidates for typical images obtained by grouping can be presented to the user by display on the display unit 512 of the host computer 5. When a typical image is designated by the user, the teaching unit 505 creates teaching data based on the information and gives it to the defect classification unit 503.

  As described above, in this embodiment, step S102 of FIG. 2 corresponds to the “feature amount calculation step” of the present invention, while the processing shown in step S103 and FIGS. It corresponds to “Process”. 2 corresponds to the “presentation step” of the present invention. Further, step S303 in FIG. 8 corresponds to “search processing” of the present invention. In the above embodiment, the point occupied by the defect image selected as the reference image in the feature amount space corresponds to the “reference coordinate point” in the present invention.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. For example, in the above-described embodiment, the value of each feature value of the defect image is normalized and quantized, and then the neighborhood image is searched for the reference image. It is not an essential requirement in the present invention to normalize or quantize the value of. Only one of these processes may be executed.

  Therefore, for example, for each feature amount, a range including the minimum value to the maximum value of the feature amount value among a plurality of defect images is equally divided into a plurality of sections, and the feature value value of each image is The same effect can be obtained even if a table as shown in FIG.

  Moreover, although the said embodiment is an image classification apparatus which test | inspects and classifies the defect of a semiconductor substrate, the image classification apparatus which is an application object of this invention is not only the apparatus which test | inspects a semiconductor substrate, but another target object, for example, a print It may be a device for inspecting a substrate, a glass substrate or the like, or a surface inspection device for inspecting the surface state of various materials.

  The present invention is suitable for a technique for automatically classifying defect images, and is particularly suitable for the purpose of assisting a user's task of finding typical images including typical examples of defects from a large number of defect images. be able to.

DESCRIPTION OF SYMBOLS 1 Inspection system 2 Imaging device 503 Defect classification | category part 504 Feature-value calculation part 505 Teaching part 506 Judgment part 507 Calculation part 512 Display part S102 Feature-value calculation process S103 Grouping process S104 Presentation process S303 Search process

Claims (7)

A defect image presentation method that supports a user operation of finding a typical image by a computer selecting a typical image candidate including a typical example of a defect from a plurality of defect images obtained by imaging a defect and presenting the typical image to a user. And
The computer is
A feature amount calculating step for calculating a plurality of feature amounts for each of the defect images ;
Grouping that groups the plurality of defect images by grouping the two defect images close to each other in the multi-dimensional feature amount space each having the plurality of feature amounts as one coordinate axis. Process,
For at least one of the groups, displaying at least one of the defect images belonging to the group on a display unit to perform a process including a presentation step of presenting the user as a candidate for the typical image. And a method for presenting a defect image. In the grouping step, the defect images are grouped by performing a search process for searching for the defect image close to the reference image for the reference image selected from the plurality of defect images.
In the search process,
A point where the reference image is located in the feature amount space is set as a reference coordinate point,
For one of the coordinate axes, an extraction process for selecting, as candidate images, the defect image whose feature value corresponding to the coordinate axis is within a predetermined vicinity range from the value of the reference coordinate point on the coordinate axis. Run on the coordinate axis,
2. The defect image selected as the candidate image on all of the plurality of coordinate axes to be subjected to the extraction processing is grouped in the same group as the reference image as a neighboring image close to the reference image. How to present a defect image.   The defect image presentation method according to claim 2, wherein the neighborhood range is set for one coordinate axis in accordance with a distribution mode of feature value values corresponding to the coordinate axes in the plurality of defect images.   A numerical value range of a section obtained by equally dividing a numerical value range including a maximum value and a minimum value between the plurality of defect images into a plurality of predetermined division numbers with respect to a feature amount corresponding to one coordinate axis, and the neighborhood range The defect image presenting method according to claim 3.   Among the plurality of feature amounts, a coordinate axis corresponding to the feature amount when the difference between the maximum value and the minimum value between the plurality of defect images is larger than a threshold set in advance for the feature amount. The defect image presenting method according to claim 2, wherein the image is a target of the extraction process. In the grouping step,
Creating a table associating the plurality of defect images with a value of the feature amount and the defect image having the value for each of the plurality of feature amounts;
The presentation of a defect image according to any one of claims 2 to 5, wherein the search process is performed each time the reference image is sequentially changed among the plurality of defect images to search for the neighboring image with respect to the reference image. Method.   7. In the presenting step, for at least one group including a group having the largest number of the defect images belonging thereto, at least one of the defect images belonging to the group is presented as a candidate for the typical image. A method for presenting the defect image described.

Images