JP2879129B2 - Article identification system and method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an article identification system for identifying an article by using image processing and identification technology, and more particularly to an automatic packaging machine for pharmaceutical tablets and capsules (hereinafter simply referred to as tablets). The present invention relates to an article identification system suitable for tablet identification for inspecting whether or not a tablet selected for an input name is correct.

[0002]

2. Description of the Related Art At present, many general hospitals use an automatic tablet packaging machine for automatically packaging a plurality of tablets of a specified brand in a packaging paper. In an automatic tablet packaging machine, a plurality of tablet brand information input by an operator from a terminal is received, and a corresponding tablet is transported from a doublet case in the apparatus and packed in the same packaging paper. For general automatic tablet packaging machines, see, for example, “Toshikazu Kiyono, Takao Orii, et al .: Use of Computers in Dispensing Operations-An Attempt of an Audit System for Automatic Dispensing Packaging Machines”, Proceedings of the 8th Medical Information Association Conference , 565-568, 1988 ".

[0003] By introducing a drug system using such an automatic tablet packaging machine into a hospital, tablet packaging, which has been performed through the hands of a pharmacist, can be automated, and the waiting time can be greatly reduced. This makes it possible to improve the quality of medical treatment.

[0004]

In the above automatic tablet packaging machine, a mechanical operation is included in a process of taking out a specified tablet from a tablet case and a process of transporting the taken tablet to a packaging position. For this reason, there is a possibility that a wrong ejection or loss of the tablet may occur. In addition, although tablets are supplied to the tablet case by a human, there is a possibility that the brand to be supplied at this time may be wrong. However, even if there is such a mechanical or artificial error, it is necessary that the tablet of the designated name is definitely contained in the package finally provided to the patient. For this reason, when operating a conventional automatic tablet packaging machine, a pharmacist has performed a visual identification test to determine whether or not the packaged tablet is a designated brand.

[0005] Therefore, the trouble of the identification test is troublesome,
There is a problem that the inspection takes time. In addition, since inspection is performed after packaging, if an incorrect tablet package is found, it is necessary to break the package and make a correct tablet package again.

Accordingly, an object of the present invention is to automatically determine whether or not a selected tablet matches a specified brand in the operation of an automatic tablet packaging machine, and thereby always specify the selected tablet from the automatic tablet packaging machine. The aim is to ensure that the correct packaging with street pills comes out.

[0007] A more generalized object of the present invention is:
In applications where items of a specified category are selected from a group of multi-category items, identification processing for confirming whether the selected items are of a specified category is performed by image processing and identification. The purpose of the present invention is to enable automatic and accurate operation using technology.

Further, another object of the present invention is to perform an article identification test using image processing and identification techniques, and to ensure that each of the articles is input even if an image containing a plurality of articles is input at the same time. In order to be able to be identified.

[0009]

According to the present invention, there is provided an article identification system comprising: image input means for inputting images of a plurality of articles; and a plurality of objects to be determined as to whether or not the articles correspond to the plurality of articles. > Category input means for inputting a category, article image extracting means for extracting an image of each article from the entire input image, and feature extraction for extracting a gray-scale value vector of each of the extracted article images as a feature vector Means and each extracted
A feature vector of an article image, various categories previously prepared
Obtains distances between the reference vector of the inputted plurality categories in Lee reference vectors, according to whether this distance is in the predetermined within tolerance, or not each item is corresponding to each input category And an identification means for determining whether or not the determination is made.

[0010] In this system, a simple feature extracting means for extracting simple features of each article from the extracted article images, and simple features of various categories prepared in advance.
A category in which a distance between the simple feature corresponding to each of the input categories in the collection and the simple feature of each article is obtained, and a category candidate to be identified for each article is selected from the input categories based on the distance. It is desirable to have candidate selection means.

[0011]

According to the present invention, images obtained by photographing a plurality of articles (for example, tablets) with a camera or the like and images corresponding to those articles are obtained.
Multiple categories (for example,
Tablet brand) is input to the system. Then, an image of each article is cut out from the entire image, and a feature vector of each article is extracted from each article image. As this feature vector, a grayscale value vector having grayscale values in an image as elements is used. A feature vector, various catheter previously prepared
The distance between the input categories and the reference vector of each category is obtained from the reference vector of Gory, and whether or not the input category corresponds to the article by determining whether or not the distance is within a predetermined allowable value. Is determined.

By performing the above-described processing, it is possible to automatically perform the identification inspection of the article without looking at the person. In addition, identification of a plurality of articles can be performed at once.

In this system, prior to the identification process, a simple feature of each article is extracted, and a category candidate to be identified for each article is selected based on the extracted feature. Since the distance calculation only needs to be performed for the category candidates, the processing amount is reduced and the processing speed is increased. In addition, a high identification accuracy can be expected by skillfully selecting parameters used as simple features.

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 shows a functional configuration of a tablet identification system according to one embodiment of the present invention.

The tablet identification system operates on a computer 100 having the NTSC television camera 1 as an external device. The computer 100 includes an image input unit 10
1. Name input unit 102, inverse γ correction unit 103, RGB gray conversion unit 104, position / direction correction unit 105, simple feature extraction unit 106, simple feature learning unit 107, simple feature scatter degree calculation unit 108, simple feature file 109, an identification target extraction unit 110, a feature extraction unit 111, a feature learning unit 112, a dictionary file 112, a feature dispersion degree calculation unit 114, and an identification unit 115.

The operation of the present system can be roughly divided into a learning phase and an identification phase. Hereinafter, the functions and operations of each unit will be described separately for the (A) learning phase and the (B) identification phase.

(A) Learning Phase The system must first pass through a learning phase as initialization or preparation before starting operation. In the learning phase, for each brand of tablet to be identified, multiple samples and the exact brand name are entered,
Based on this, the simple feature vector of each brand and its scatter degree, and the reference vector and its scatter degree are calculated, respectively, and stored in the simple feature file 109 and the dictionary file 113, respectively. Hereinafter, the processing of the learning phase will be described in detail.

First, one tablet of one brand is placed on a plain black background placed in front of the NTSC television camera 1, and a color analog image of the tablet taken by the camera 1 is input to the image input unit 101. Is done. Image input unit 101
Converts this color analog image into an RGB digital image.

In parallel with this image input, a brand input unit 10
By step 2, the correct brand name of the tablet whose image has been input is input to the system.

Next, the inverse γ correction unit 103 removes the effect of the γ correction applied to each of the RGB planes and performs the inverse γ correction so that the input / output characteristics of the camera 1 become substantially linear. At this time, for example, 2.2 is used as the value of γ. This is because the NTSC television camera 1 is normally configured to apply γ correction of γ = 2.2 to a captured image. By performing the inverse γ correction, the contrast between the tablet region and the background region (black background) in the image increases, and the subsequent processing can be performed effectively.

Next, the RGB gray conversion unit 104
Using the gray value of the G component in the B component as it is, RGB
The image is converted to a gray image having only the gray components. Where G
The reason for using the component is that the G
This is because the component has a wider dynamic range than the other two components, and it is considered that in a tablet with an overwhelmingly white color, a subtle difference in white color can be reflected in shades. By converting the RGB image into the gray image, the following processing amount can be reduced to one third.

Next, the position / orientation correction unit 105 performs processing so that the position and orientation of the tablet in the image are always the same if they have the same name. FIG. 2 shows a detailed processing flow of the position / orientation correction.

As shown in FIG. 2, first, a gray image is subjected to a binarization process for the entire image using a predetermined threshold value.
Only the tablet area is extracted (step 201). At this time, as a method of selecting a threshold for binarization, for example, Mr. Otsu's method based on a discrimination standard is used. This method is very effective for an image that can be clearly divided into two regions, a background region and a tablet region. For details, see “Nobuyuki Otsu: Automatic Threshold Selection Method Based on Discrimination and Least Square Criterion, IEICE Transactions Vol. J63-DN.
o. 4, pp. 349-356, 1980 ".

Next, the center of gravity of the tablet area in the binarized image is obtained, and the center of gravity is moved to the center of the image (step 202). Subsequently, the area of the identification number code printed or stamped on the tablet surface is extracted by binarizing the gray image using a different threshold from the above in the tablet area (step 203). At this time, the method of selecting a threshold value for binarization can also use the method of Mr. Otsu based on the above-described determination criteria.

Next, the second moment of the extracted identification code area is obtained, and the entire image is rotated by affine transformation so that the direction in which the second moment becomes maximum coincides with the X axis of the image (step 204). .

By correcting the position and direction of the tablet area in this way, the accuracy of identification described later is improved. This is because the characteristics of the external shape of the tablet depend on the position and the direction. In addition, since the same position and direction are always used for the same tablet, the exact position of the tablet with respect to the camera
It is not necessary to adjust the direction.

Referring again to FIG. 1, the simple feature extracting unit 106 extracts features that can be extracted relatively quickly from the tablet image as simple features. For example, geometric features such as the area and perimeter of the tablet region in the image, color information obtained from the RGB digital image, and the like are extracted, and a simple feature vector using these as elements is created. For details of the method of extracting geometric features and color information, see “Hideyuki Tamura: Introduction to Computer Image Processing, published by Soken Publishing, 1985”.

Next, the simple feature learning unit 107 calculates, for each brand, the average of the simple feature vectors of a large number of tablets (hereinafter, referred to as learning samples) input so far in the learning phase, and calculates the average vector as the average. It is stored in the simple feature file 109 as a simple feature vector of the brand.

Next, the simple feature dispersion degree calculation unit 108 sets the average of each brand to 0 based on the distribution of the distance between the simple feature vector of the learning sample input so far and the average vector thereof. Find the standard deviation σ1 in the case. Here, the distribution of the distance of the average vector simple feature vector is as shown in FIG. 7, for example. The standard deviation σ1 of such a distance distribution is obtained for each brand, and this is stored in the simple feature file 109 as the simple feature distribution of each brand.

Next, the feature extracting unit 111 performs a process of extracting a tablet feature vector from the gray image whose position and orientation have been corrected. FIG. 3 shows a detailed processing flow of this feature extraction. 4 to 6 show examples of images for explaining the feature extraction processing step by step.

First, a rectangular area 500 of a predetermined size including the whole or a part of the tablet area is cut out from the gray image 400 whose position and orientation have been corrected as shown in FIG. Step 301). Here, the rectangular area 5
The size of 00 is determined by a statistical method as the size at which the most accurate feature extraction can be performed based on the results of feature extraction performed on tablets of various brands in advance.

Next, as shown in FIG.
00 is divided into small sections 600 of an arbitrary size N × M (step 302). Here, the size N of the small section 600
XM is determined in proportion to the processing capacity of the computer.

Next, for each small section 600, a representative value is calculated from the gray scale values of all the pixels (step 303). As the representative value, for example, a median value is used. The median is even if the distribution of gray values is clearly biased like the tablet area and the background,
This is because the entire distribution can be well reflected.
Thereafter, a gray-scale value vector having a representative value of each small section 600 as an element is extracted as a feature vector (step 30).
4).

In the learning phase, a large number of learning samples of various brands of tablets to be identified are supplied to the present system, and the above-described processing from photographing to feature extraction is performed on each learning sample.

Referring again to FIG. 1, next, the feature learning unit 1
12 performs, for each brand, principal component analysis on all gray value vectors extracted from a large number of learning samples of the brand to obtain eigenvalues and eigenvectors. Then, a contribution rate is calculated based on the eigenvalues, several principal components are selected in descending order of the contribution rate, and the full gray value vector is converted into a principal component vector using the eigenvector. As described above, by selecting a significant feature based on the result of the principal component analysis, the dimension of the feature vector can be reduced, and the amount of subsequent processing can be reduced. Next, the obtained principal component vectors are averaged for each issue to obtain a reference vector, and this reference vector is used as a reference vector for each issue in the dictionary file 11.
3 is stored. Further, the unique vector of each brand is also stored in the dictionary file 113.

Next, the characteristic dispersion degree calculation unit 114 sets the average to 0 based on the distribution of the distance between the reference vector and the principal component vectors of a large number of training samples processed so far for each brand. To find its standard deviation σ2. here,
The distribution of the distance of the learning sample with respect to the reference vector is also, for example, as shown in FIG. 7 (however, this does not mean that it is the same as the distance distribution of the simple feature vector, but a general pattern of the distance distribution is as shown in FIG. 7). Meaning). The standard deviation σ2 of such a distance distribution is obtained for each brand, and this is stored in the dictionary file 113 as the characteristic distribution of each brand.

Hereinafter, the reference vectors of all the brands and the degree of feature distribution stored in the dictionary file 113 are collectively called a dictionary.

(B) Identification Phase In the identification phase, a check is made to determine whether or not tablets and brands whose correspondence is unknown are compatible. That is, using the dictionary created in the learning phase, an identification test is performed to determine whether the tablets supplied to the system correspond to the brands. In this phase, the tablet to be identified and the brand name are input to the system.

Hereinafter, the identification face will be described in detail.

Referring to FIG. 1, first, camera shooting and brand name input are performed on the input tablet in the same manner as in the learning phase. At this time, the difference from the learning phase is that
A plurality of tablets packaged in one package by the automatic tablet packaging machine are supplied to the system at the same time, and a plurality of brand names are inputted thereto.

Next, a series of image processing from the image input unit 101 to the position / direction correction unit 105 is performed. At this time, for each of the plurality of tablets, the tablet region in the image is individually specified, and the position and direction of each are corrected.
As a method of correcting the position and orientation at that time, one method is to correct the position and direction in consideration of the mutual positional relationship so that a plurality of tablets do not overlap each other in the same image. is there. Also, as a second method, focusing on one tablet region in the image, considering other tablets as the same black as the background, and placing the focused tablet region in the center of the image in a certain direction. The operation of creating one corrected image by performing position / orientation correction is repeated for all tablets. That is, in the second method, the same processing (FIG. 2) as the position / orientation correction processing in the learning phase is repeatedly performed for each of the plurality of tablets. As in the case of inputting one tablet at a time, the same number of tablets as the number of tablets is obtained for each tablet.

Next, the simple feature extraction unit 106 extracts a simple feature vector from each of the plurality of tablets from the corrected image by the method described above.

Next, the identification target extracting section 110 specifies what brand each of the plurality of tablets should be identified as. That is, the average simple feature vector corresponding to each of the plurality of brands is read out from the simple feature file 109, and the distance between the average simple feature vector and the simple feature vector of each tablet extracted earlier is calculated. . At this time, the U-grid distance, the similarity, and the like are used as the distance function. For a general distance function, see, for example, “Noboru Funakubo: Processing and Recognition of Visual Patterns, Published by Keigaku Shuppan, 1990”.

The distance obtained between the simple feature vector of a certain tablet and the average simple feature vector of a certain brand is within an allowable value k · σ1 determined from the simple feature dispersal degree σ1 corresponding to the brand (here, , K is a predetermined coefficient of approximately 1 or more and 3 or less), it is determined that the tablet needs to be identified as the brand, and the brand is registered in the brand list for the tablet. By performing such a process between the simple feature vectors of all the input tablets and the average simple feature vector corresponding to all the brands, one or more identifications should be performed for each tablet. Two or more issues (hereinafter referred to as issue candidates) are registered in the issue list.

If there is a brand which is not registered as a brand candidate for any tablet among the plurality of brands that have been input, the tablet corresponding to the brand has not been entered or is not shown in the image. Can be determined. In that case, the processing for the input tablet may be stopped here, and the same measures (described later) as in the case where the identification results do not match may be performed.

After the brand candidates have been specified for all the tablets as described above, the feature extracting unit 11 next calculates the inputted tablets from the corrected images in the same manner as described above. Calculate the gray value vector.

Next, the identification unit 115 performs a process of identifying each tablet as each brand candidate. First, for each tablet, an eigenvector, a reference vector, and a characteristic dispersion degree σ2 corresponding to each of the registered brand candidates are stored in the dictionary file 1.
13 is read. Then, using the read eigenvector of each brand candidate, the gray-scale value vector of the tablet is converted into a principal component vector.

Subsequently, the distance between the principal component vector obtained using the eigenvector of each brand candidate and the reference vector of each brand candidate is calculated. At this time, a Euclidean distance, a similarity, or the like is used as the distance function.

Next, the identification unit 115 compares the distance obtained for each brand candidate with an allowable value r · σ2 determined from the characteristic dispersion degree σ2 corresponding to each brand candidate. as a result,
This distance is the allowable value r ·
If it is within σ2, it is determined that the tablet is a tablet corresponding to the brand candidate. If such a positive determination result is obtained for all tablets, and all tablets and all brands are completely one-to-one and there is no overlapping correspondence, the plurality of tablets and It is finally determined that the brand completely corresponds to the brand.

On the other hand, when a complete one-to-one correspondence as described above cannot be obtained, for example, a determination result that one tablet corresponds to a plurality of brand candidates, In the case where a judgment result indicating that the tablet does not correspond to any of the tablets or a brand not corresponding to any of the tablets occurs, a negative identification result is finally output. When a negative identification result is obtained, the target tablet is discharged without packaging in the automatic tablet packaging machine, or a mark indicating the negative identification result is printed on the paper on which the tablet is packaged. It is desirable to make a note and a pharmacist check later.

The allowable value r used in the identification unit 115
The coefficient r for determining σ and the coefficient k used in the identification target extraction unit 110 can be arbitrarily set by the user. For example, when the allowable value is small such as r = 1.0,
The criterion for identification is stricter, and tablets that do not correspond to brands can be repelled easily, but even for the corresponding tablets, the distance from the reference vector is large due to scratches, defects, or other reasons on the tablet surface Sometimes, it is determined that the tablet does not correspond, and it is repelled. Conversely, if the allowable value is large, such as r = 3.0, the opposite result to the above example will occur. As described above, the behavior of the present system greatly differs depending on the allowable value. For this reason, it is necessary for the user to set an appropriate allowable value depending on the purpose, for example, rejecting a slightly different tablet with emphasis on safety.

As described above, the tablet identification system according to the present embodiment can perform a brand and tablet identification test without relying on human eyes. Moreover, even if a plurality of tablets are supplied at the same time, they can be identified with high accuracy.

Further, in this embodiment, since the gray-scale value vector is used as the basic feature vector, the distance calculation process is performed by superimposing the reference image and the identification target image to calculate the degree of coincidence. Is substantially equivalent to
Such a feature vector is rarely used in a conventional pattern identification system, but is effective when processing a large category because it can make maximum use of image information.

The present invention is not limited to the above embodiment, but can be implemented in various different forms without departing from the scope of the invention. For example, not only tablets,
For example, the present invention can be applied to uses such as identification inspection of various parts in an automatic manufacturing line.

[0056]

According to the present invention, it is possible to automatically identify an article. In particular, when used for the operation of an automatic tablet packaging machine, only correct tablets can be reliably provided to patients, so that high-quality medical treatment can be realized.
Further, since identification tests between a plurality of articles and a plurality of categories are also possible, it is possible to exhibit high practicability for a normal use where a plurality of articles are handled as a set like a tablet package. .

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a tablet identification system according to one embodiment of the present invention.

FIG. 2 is a flowchart showing processing of a position / direction correction unit in the embodiment.

FIG. 3 is a flowchart showing processing of a feature extracting unit in the embodiment.

FIG. 4 is a view showing an example of a gray image after a position / direction correction unit is applied in the embodiment.

FIG. 5 is a view showing a rectangular area cut out from the image of FIG. 4 in the embodiment.

FIG. 6 is a diagram showing a state where the image of FIG. 5 is divided into small sections in the embodiment.

FIG. 7 is a view showing an example of a distance distribution of learning samples with respect to a reference vector in the embodiment.

[Explanation of symbols]

 1 NTSC TV camera 100 Computer 101 Image input unit 102 Brand input unit 103 Inverse γ correction unit 104 RGB gray conversion unit 105 Position / direction correction unit 106 Simple feature extraction unit 107 Simple feature learning unit 108 Simple feature dispersion degree calculation unit 109 Simple feature File 110 Identification target extraction unit 111 Feature extraction unit 112 Feature learning unit 113 Dictionary file 114 Feature dispersion degree learning unit 115 Identification unit

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-5-159064 (JP, A) JP-A-4-316282 (JP, A) JP-A-5-159063 (JP, A) JP-A-3-315 134779 (JP, A) JP-A-2-292676 (JP, A) JP-A-63-138472 (JP, A) JP-T-56-500107 (JP, A) (58) Fields investigated (Int. ⁶ , DB name) G06T 5/00-7/00

Claims (6)

(57) [Claims] 1. An article identification system for determining whether a plurality of articles input to a system belong to an input category, an image input means for inputting an image of the plurality of articles, and whether or not the article corresponds to the plurality of articles . Multiples subject to judgment
And category input means for inputting a category, an article image clipping means for cutting out the image of each article from in total of the input image, extracts a gray value vector of each article image the cut out as a feature vector A feature extraction means, and a feature vector of each of the extracted article images prepared in advance.
The input duplicates in various categories of reference vectors
Obtains the distance between each of the reference vector number categories, according to whether this distance is in the predetermined within tolerance, multiple each of the inputted plurality article is the input
An identification means for determining whether or not each of the categories corresponds to each of the categories. 2. A system of claim 1, from each article image the cut out, and a simplified feature extraction means for extracting a simple feature of each article, in the simple features of the various categories that are prepared in advance The distance between the simple feature corresponding to each of the plurality of categories inputted above and the simple feature of each article is obtained, and a category candidate to be identified for each article is determined based on the distance. and category candidate selection means for selecting from a plurality categories in which are the input, further wherein the identification means, the said characteristic vector of each article, before
The distance between the reference vector corresponding to the category candidate selected for each of the articles in the reference vectors of various categories is determined, and whether or not this distance is within a predetermined allowable value is determined. Is determined as to whether or not the item corresponds to the selected category candidate. 3. The system according to claim 2, wherein the category candidate selecting means determines that a distance between a simple feature of an article and a simple feature corresponding to a certain category is a predetermined allowable value corresponding to the certain category. Wherein the certain category is a category candidate of the certain article. 4. The system according to claim 1, wherein said identifying means finally determines when all of said plurality of articles correspond to all of said input categories completely one-to-one. An article identification system characterized by outputting a unique identification result. 5. An article identification method for determining whether a plurality of articles input to a system belongs to an input category, a step of inputting an image of the plurality of articles, and determining whether the plurality of articles correspond to the plurality of articles . The target of the judgment of
A step of inputting a number category, a step of cutting out an image of each article from the whole of the input image, a step of extracting a gray-scale value vector of each cut-out article image as a feature vector, and Feature vector of each article image and prepared in advance
Determining distances between reference vectors of each of the plurality of input categories in reference vectors of various categories ;
Determining whether each of the input plurality of articles corresponds to each of the input plurality of categories by determining whether the distance is within a predetermined allowable value; and An article identification method, comprising: 6. The method according to claim 5, wherein a simple feature of each article is extracted from each of the extracted article images, and various categories prepared in advance.
The distance between the simple feature corresponding to each of the plurality of input categories in the simple features and the simple feature of each article is determined, and based on this distance, a category candidate to be identified for each article is determined. Selecting from among the plurality of input categories, wherein the determining includes selecting the feature vector of each of the articles and a category candidate selected for each of the articles in the reference vectors of the various categories. And determining whether or not each of the articles corresponds to the selected category candidate by determining whether or not the distance is within a predetermined allowable value. Article identification method characterized by the above-mentioned.