JP3570862B2 - Method and apparatus for detecting object in image and recording medium - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a method and apparatus for detecting an object in an image and a recording medium, and more particularly, to an image processing for detecting an object having a change in shape.
[0002]
Detecting an object reflected in an image captured by a camera is a basic technique in image processing, and has various applications. For example, if the shape or position of the eyes, mouth, nose, etc. reflected in an image obtained by photographing a human face can be confirmed, the machine can recognize the human expression or posture. Enabled and extremely useful for human-machine communication.
[0003]
Under such circumstances, a method for accurately detecting an object having a change in shape in a short time is desired.
[0004]
[Prior art]
2. Description of the Related Art Conventionally, as a method of detecting a target object from an image, template matching that scans a template representing the object on the image and searches for a position having a high degree of matching between the two to detect the object has been used. .
[0005]
For example, as shown in FIG. 19, the template TP representing the object BJ is moved in the horizontal direction and sequentially shifted in the vertical direction from the upper left end to the lower right end of the image FG, and the position of the object bj having the highest degree of coincidence during that time. Is detected.
[0006]
In this case, when the object bj has a fixed shape such as a bolt, a nut, or a hole of a drill, the template TP representing the object BJ may have a fixed shape.
[0007]
However, if the object bj is the same person, such as eyes, mouth, nose, etc., the shape of which changes depending on the expression, the template TP representing the object BJ must also have a variable shape. Must be used. Further, even if the shape itself is a fixed object, if the imaging direction of the object changes, the apparent shape changes accordingly, so it is necessary to use a variable-shaped template TP.
[0008]
When the template TP having a variable shape is used, the shape of the template TP is changed at each position when the template TP is scanned on the image FG, and the position of the object BJ having the highest matching degree and the shape of the template TP are changed. Is detected.
[0009]
[Problems to be solved by the invention]
However, in the conventional template matching technique using the variable-shaped template TP, it is necessary to create the template TP in anticipation of a change in the shape of the object bj. Conventionally, an artificially created template is used for such a template TP. Therefore, the template TP cannot always follow the shape change of the target object bj, and the detection accuracy is reduced.
[0010]
In addition, when the template TP is scanned on the image FG and all the shapes of the changing object BJ are collated and the position and the shape having the highest matching degree are detected, the amount of information processing is reduced. There is a problem that it is enormous and takes a lot of time for detection and is not practical.
[0011]
The present invention has been made in view of the above-described problems, and has a method and apparatus capable of accurately and quickly detecting an object whose shape changes from an image and a recording for executing such processing. The purpose is to provide a medium.
[0012]
[Means for Solving the Problems]
The method according to claim 1 is a method for detecting an object having a change in shape from an image, wherein the object has a shape. Digitized representing the object shape from multiple images containing A first step of extracting an object feature amount that is a feature amount, a second step of analyzing the object feature amount using a principal component analysis method, From a given image by a predetermined method A third step of extracting image features; Expressing the shape of the object with a small number of principal component values from the analyzed object feature amount, The image feature quantity and With variable shape template matching Collation Do A fourth step, wherein the shape and shape of the object Position A fifth step of outputting the position.
[0013]
The method according to claim 2 uses coordinate values of a plurality of feature points as the image feature amount. I In the first step, a plurality of images having similar shapes of the object are Shidu Specify number of feature points And that specified The coordinates of the feature points Request You.
[0014]
Claim 3 The method according to the invention is characterized in that, in the fourth step, the value of each principal component of the analyzed object feature is changed, and the analyzed object feature and the image feature are well matched. A set of values of each principal component is set as a matching result.
[0015]
Claim 4 In the method according to the present invention, in the fourth step, the value of each principal component is adjusted so that a part of the analyzed object feature and a part of the image feature are matched, and the adjustment is performed. Among a plurality of sets of values of each principal component, a set of values of each principal component when the analyzed object feature value and the image feature value match well is defined as a matching result.
[0016]
Claim 5 In the method according to the invention, in the fourth step, when there are a plurality of sets of the values of the principal components that best match, the set having the smallest absolute value of the values of the principal components is determined as the matching result.
[0017]
Claim 6 In the method according to the invention, in the fourth step, a minimum value is set to an evaluation function having a degree of inconsistency between the analyzed object feature amount and the image feature amount and an absolute value of each principal component value as variables. A set of the values of each principal component to be given is set as a matching result.
[0018]
Claim 7 In the method according to the invention, in the fifth step, the shape of the object determined by the value of each principal component as the matching result is output.
Claim 8 An apparatus according to the invention of the above is an apparatus for detecting an object having a change in shape from an image, wherein the object Digitized representing the object shape from multiple images containing First means for extracting an object feature amount as a feature amount, second means for analyzing the object feature amount using a principal component analysis technique, From a given image by a predetermined method Third means for extracting an image feature amount; Expressing the shape of the object with a small number of principal component values from the analyzed object feature amount, The image feature quantity and With variable shape template matching Collation Do Fourth means, and the shape and shape of the object according to the result of the verification. Position And a fifth means for outputting the position.
[0019]
Claim 9 The recording medium according to the invention is configured to detect an object having a shape change from an image. Digitized representing the object shape from multiple images containing A first step of extracting an object feature amount that is a feature amount, a second step of analyzing the object feature amount using a principal component analysis method, From a given image by a predetermined method A third step of extracting image features; Expressing the shape of the object with a small number of principal component values from the analyzed object feature amount, The image feature quantity and With variable shape template matching Collation Do A fourth step, wherein the shape and shape of the object Position And a computer-readable recording medium storing a program for causing a computer to execute the fifth step of outputting the position.
[0020]
In the detection method of the present invention, shape data of an object whose shape changes is analyzed by a principal component analysis method used in statistics. By designing a template having a variable shape based on the analysis result, the detection following the shape change of the object in the image is performed.
[0021]
The detection device of the present invention is realized using, for example, a personal computer or a workstation. A program for executing the method according to the present invention is stored in a recording medium such as a semiconductor memory, a hard disk, a CD-ROM, a floppy disk, or a magneto-optical disk. The program stored in the recording medium is loaded onto the main memory as appropriate, and is executed by the processing device. At that time, a drive device such as a CD-ROM drive, a floppy disk drive, or a magneto-optical disk drive is used as necessary. When the recording medium is provided in a server connected by a communication line such as a network, the program is read or downloaded from the server through the communication line. When the program is transferred to the processing device via a communication line, the communication line itself also corresponds to a recording medium. The program can be supplied to operate under various OSs, platforms, system environments, or network environments.
[0022]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 is a flowchart showing the flow of the processing of the detection method according to the present invention.
In FIG. 1, an object feature is extracted (# 1), the extracted object feature is analyzed (# 2), an image feature is extracted (# 3), and a matching process is performed (# 4). Is output (# 5). The order and timing of the extraction and analysis of the object feature amount and the extraction of the image feature amount are arbitrary as long as they are before the collation processing.
[0023]
Hereinafter, a method of expressing the shape of the object BJ will be described first, and an example of the contents of the extraction and analysis of the object feature amounts in steps # 1 and # 2 will be described. Next, an example of a method of calculating the feature amount of the image FGS will be described, and thereby an example of the processing content of the image feature amount extraction in step # 3 will be described. Next, the collation processing in step # 4 will be described.
[0024]
First, a method of expressing the object BJ will be described.
In the present invention, the shape data of the object BJ whose shape changes is represented by some feature amounts. As the feature amount, for example, a coordinate value of a feature point is used. In the following description, the coordinate value of a feature point is used as a feature value. However, the present invention is applicable as long as the shape of an object is represented by some numerical value, and thus other feature values can be used.
[0025]
FIG. 2 is a diagram illustrating an example of an object image FGA1 in which a target object BJ1 is expressed by five feature points CP1 to CP5.
In FIG. 2A, the object BJ1 is represented by five characteristic points CP1 to CP5 of the shape. In this case, the object BJ1 is represented as a polygon having the feature points CP1 to CP5 as vertices. Each of the feature points CP1 to CP5 is represented by a coordinate value in the coordinate system of the object image FGA1, as shown in FIG. The upper left end of the object image FGA1 is the origin (0, 0), and a positive X coordinate extends rightward from the origin and a positive Y coordinate extends downward from the origin. The coordinate values of each of the feature points CP1 to CP5 are stored in the memory.
[0026]
The feature points CP1 to CP5 are determined by the operator designating each of the feature points CP1 to CP5 with a pointing device such as a mouse while the object image FGA1 is displayed on the screen of the display device. The coordinate values are obtained by designating the feature points CP1 to CP5 and stored in the memory. In specifying the feature points CP1 to CP5, the operator may select points that are considered to be the feature points CP1 to CP5 based on his or her own criteria, but often select an inflection point of the appearance shape of the object BJ1. Instead of specifying the feature points CP1 to CP5 from the object image FGA1 displayed on the screen, the operator may input the coordinate values of the feature points CP1 to CP5 from a keyboard or the like.
[0027]
The above-described operation and processing are performed on the same object BJ1 for many object images FGA2 to 4 having different shapes as shown in FIG. For example, when the object BJ1 is a "mouth" or a "lip", for each of a large number of variation images taken of the mouth or lips of the same person, the respective feature points CP1 to CP5 are selected and coordinate values are acquired. In addition, for images obtained by photographing the mouths of different persons, the respective feature points CP1 to CP5 are selected to acquire coordinate values. That is, by this processing, the coordinate values of the respective feature points CP1 to CP5 of the object images FGA2 to FGA4 having different shapes are stored in the memory for the same object BJ1. What kind of image to use, such as whether the image is for the same person or an image for a different person, may be determined depending on the object detection purpose.
[0028]
If template matching is performed using all the object images FGA1 to FGA4 obtained as templates as templates, it is possible in principle to detect an object following a change in the shape of the object. The volume is enormous, requiring a great deal of processing time, which is not practical. Therefore, in the present embodiment, it is considered that changes in the feature amount are analyzed to arrange these many object images FGA1 to FGA4. In the present embodiment, since the object images FGA1 and FGA3 are the same, only the object images FGA2 to FGA4 will be considered.
[0029]
In the case of the object images FGA2 to FGA4 shown in FIG. 3, the change of the feature points CP1 to CP5 in the three object images FGA2 to F4 is, as shown by the arrow in FIG. It can be analyzed that the other characteristic points CP1 to 3 and 5 have not changed since they are only moving linearly.
[0030]
Therefore, such analysis is performed mechanically using a principal component analysis technique of statistics. Principal component analysis is a method of expressing, for a large number of data, their respective feature amounts in the form of an average value and a sum of typical deviations from the average value. Specifically, it is an operation or a process of obtaining an eigenvector of a covariance matrix when the feature amount is regarded as a random variable. When the coordinate values of the feature points are used as the feature amounts, the coordinate values of each feature point are represented by an average position and a sum of representative deviations from the average position. The most typical deviation is referred to as a first principal component, the second most representative deviation is referred to as a second principal component, and similarly, a third principal component, a fourth principal component, and so on. When the principal component analysis is applied to the feature point sequence in FIG. 4, the first principal component in which only the upper right feature point CP4 moves is obtained, and the second principal component and below are 0 (do not move).
[0031]
Principal component analysis will be described with an actual example.
5 and 6 are diagrams for explaining the principal component analysis method. FIG. 5 is a diagram showing a case where two object images FGB1 and FGB2 are input, and FIG. 6 is a diagram showing the average value and main data in FIG. It is a figure which shows a component vector with a numerical value. In these examples, each object image FGB is represented by the coordinate values of two feature points CP1 and CP2.
[0032]
In FIG. 5, in the object image FGB1 and the object image FGB2, the X coordinate of each of the characteristic points CP1 and CP2 does not change, and only the Y coordinate changes. The average value (average position) of each feature point CP1, CP2 is the average value of the X coordinate or Y coordinate of each feature point CP1, CP2. The principal component vector exists only for the first principal component. The feature point CP1 is a vector having a size of “50” in the negative direction (upward) along the Y axis, and the feature point CP2 is the Y axis. Is a vector having a size of “50” heading in the positive direction (downward) along. That is, the average value and the principal component vector of each of the feature points CP1 and CP2 are common to all the object images FGB1 and FGB2. On the other hand, the magnification (principal component value) of each principal component vector changes for each of the input object images FGB1 and FGB2. In the examples shown in FIGS. 5 and 6, the values of the first principal components of the object images FGB1 and FGB2 are “1.0” and “−1.0”, respectively.
[0033]
In the example shown in FIG. 5, two object images FGB1 and FGB2 are input. For example, when three object images FGB1 to 3 are input, the values of the second principal component vector and the second principal component are changed. Exists. In general, when a plurality of object images FGB are input, the coordinate position of each of the feature points 1, 2,... Is calculated using the average value of each feature point, the value of each principal component, and each principal component vector. Is represented as
[0034]
X coordinate of feature point 1 = average value of X coordinate of feature point 1+ (value of first principal component × X coordinate component of feature point 1 of first principal component vector) + (value of second principal component × second value (X coordinate component of feature point 1 of principal component vector) + ...
Y coordinate of feature point 1 = average value of Y coordinate of feature point 1+ (value of first principal component × Y coordinate component of feature point 1 of first principal component vector) + (value of second principal component × second value Y coordinate component of feature point 1 of principal component vector) + ...
X coordinate of feature point 2 = average value of X coordinate of feature point 2+ (value of first principal component × X coordinate component of feature point 2 of first principal component vector) + (value of second principal component × second value X coordinate component of feature point 2 of principal component vector) + ...
Y coordinate of feature point 2 = average value of Y coordinate of feature point 2+ (value of first principal component × Y coordinate component of feature point 2 of first principal component vector) + (value of second principal component × second value Y coordinate component of feature point 2 of principal component vector) + ...
The average value and the principal component vector are represented inside the computer as an array as shown in FIG.
[0035]
The average value of the coordinates of each feature point CP of the object BJ obtained as a result of the principal component analysis, the principal component vector, and the value of the principal component are analyzed feature amounts of the object BJ. By using the average value of the coordinates of these feature points CP and the principal component vector, it is possible to express various shapes of the object BJ by changing the value of each principal component. Since the principal component vector represents a typical deviation from the average value, it is possible to accurately represent the shape of the object BJ only by considering a small number of principal component vectors. Therefore, by performing object detection by using the feature amount subjected to the principal component analysis, the amount of processing is reduced, and accurate detection can be performed in a short time. For details of the principal component analysis, see, for example, "Image Analysis Handbook, 4th Edition, pp. 40-44, University of Tokyo Press, July 15, 1992".
[0036]
Next, an example of a method of calculating the feature amount of the image FGS will be described.
The image FGS is generally a grayscale image captured by a camera or the like. The image FGS, which is a grayscale image, is represented as a two-dimensional array in which the brightness is represented for each pixel by a non-negative integer.
[0037]
FIG. 7 is a diagram illustrating an example of the image FGS1.
In FIG. 7, each cell indicates a pixel, and the numerical value in each cell indicates the density value of the pixel. The feature amount is calculated from the image FGS1. In the following, a process in the case of using the coordinate values of the feature points as the feature amounts will be described.
[0038]
One method of calculating a feature point from the image FGS is to differentiate the image FGS and use a point (pixel) having a derivative value larger than a predetermined set value as a feature point. This regards all points on the object boundary where the brightness changes drastically as feature points. The procedure of feature point calculation using this method will be described below.
(1) Initialization
The target pixel is set at a position separated by one pixel from the upper left end of the image FGS to the lower right.
(2) Calculation of X derivative
The grid at the center of the X differential window shown in FIG. 8A is placed at the position of the pixel of interest in the image FGS, and the numerical value described in each grid of the window is multiplied by the pixel value at that position to obtain 9 grids. Are added to each other, and the result is used as the X differential value for the pixel of interest.
(3) Calculation of Y differential value
The grid at the center of the Y differential window shown in FIG. 8B is placed at the position of the pixel of interest in the image FGS, and the numerical value described in each grid of the window is multiplied by the pixel value at that position to obtain nine pixels. Are added to each other, and the result is used as the Y differential value for the pixel of interest.
(4) Calculation of differential value
For the pixel of interest, the sum of the absolute values of the X differential value and the Y differential value is determined.
(5) Output of feature points
For the target pixel, the obtained differential value is compared with a predetermined threshold value, and if the differential value is equal to or larger than the set value, the coordinate value of the target pixel is output as the feature point coordinates.
(6) Pixel movement
The target pixel is moved right by one pixel. If the pixel of interest is located just before the right end of the image FGS, it is moved down by one pixel and moved to the right one pixel from the left end. If it cannot move below it, it ends.
(7) Repeat
The process returns to the step of calculating the X differential value, and the subsequent steps are repeated.
[0039]
FIG. 9 is a diagram showing the result of differentiating the image FGS1 shown in FIG. 7 with the X differential window and the Y differential window shown in FIG. However, the outermost one row and one column are excluded. When the threshold value is, for example, “12” with respect to the differentiation result shown in FIG. 9, the coordinate values of eight pixels connected in a frame shape excluding the center pixel are set as feature points CQ of the image FGS1. Is output.
[0040]
As described above, the contour (edge) of the object BJ detected by differentiating the image FGS1 can be used as the feature amount. Thus, when the image FGS is, for example, a human face, the contours of the eyes, mouth, nose, and the like included in the image FGS can be used as the feature amounts. For example, when the eyes are double eyelids, the outline may have a width of two pixels or more. In such a case, for example, only the outermost contour may be used as the feature amount.
[0041]
Note that various other methods can be used to extract the feature amount of the image FGS, such as, for example, the operator tracing and specifying a contour included in the image FGS.
[0042]
Next, a method for detecting the object BJ from the image FGS by performing a matching process between the feature amount of the image FGS and the analyzed feature amount of the object BJ will be described.
As described above, since the shape of the object BJ can be expressed by a small number of transformations (principal component values) by the principal component analysis, it is applied to the object detection from the image FGS by the template matching of the variable shape. I do.
[0043]
FIG. 10 is a flowchart illustrating an example of the matching process, and FIG. 11 is a diagram illustrating a coordinate system in the matching process.
As shown in FIG. 11, the coordinate system of the object image FGA on which the principal component analysis has been performed is “template coordinate system” or “object coordinate system”, and the coordinate system of the image FGS to be detected is “image coordinate system”. I have something to say.
[0044]
In FIG. 10, the position is initialized, and the origin of the template coordinate system is matched with the origin of the image coordinate system (# 11). Here, the upper left end of each image is set as the origin.
A shape detection process is performed (# 12). In the shape detection processing, the value of each principal component is adjusted so that the feature point CQ of the image FGS and the feature point CP of the object image FGA best match with the position of the coordinate system at that time. When the degree of coincidence is sufficiently high, for example, when the degree of coincidence exceeds the threshold value Sh (Yes in # 13), it is determined that the object BJ exists at that position, and is determined by the adjusted values of the principal components. The shape is output (# 5).
[0045]
If the degree of coincidence is not sufficiently high (No in # 13), the object image is moved by one pixel (# 14). If it cannot be moved further to the right, it is moved to the left end one pixel below. Then, step # 12 and subsequent steps are repeated. If it is not possible to move to the right or below, the process ends (Yes in # 15).
[0046]
In the above-described collation processing, Step # 11 and Steps # 14 and # 15 are processing used in conventional template matching. In the above example, the shape detection was performed while moving the object image by one pixel on the image FGS. However, when the positional correspondence between the template coordinate system and the image coordinate system is known in advance, or when the positional correspondence between them is determined. If determined by another means, only the shape detection process needs to be performed.
[0047]
In the shape detection process in step # 12, since there are generally many feature points CQ extracted (or calculated) from the image FGS, it is checked whether or not the feature point CQ matches the feature point CP representing the object shape. It is not easy to judge the degree of the coincidence, and it takes time for processing.
[0048]
Note that the coincidence between the feature point CQ and the feature point CP naturally includes a case where the pixels of the respective points completely match each other, that is, a case where the coordinate values of the pixels of the respective points are equal to each other. The case where the distance between the pixels of each point is within the number of pixels determined in the X direction or the Y direction can be included. For example, if one point of the feature point CQ and one point of the feature point CP are within n pixels in the X direction or the Y direction, it is assumed that they match, and n is set to, for example, 1, 2, 3, 4, 5, 6, It may be determined by selecting from numerical values such as ....
[0049]
Next, two detection matching methods in the shape detection processing will be described.
12 is a flowchart illustrating a first detection / collation method, FIG. 13 is a diagram illustrating the first detection / collation method, FIG. 14 is a flowchart illustrating a second detection / collation method, and FIG. 15 is a flowchart illustrating the second detection / collation method. FIG. These flowcharts do not completely correspond to step # 12 in FIG.
[0050]
In the first detection and matching method, as shown in FIG. 13, the value of the principal component at the characteristic point CP of the object BJ is adjusted so that the characteristic point CP best matches the characteristic point CQ of the image FGS.
[0051]
In FIG. 12, the value of each principal component is set to “0” (# 21). By setting the value of each principal component to “0”, detection and matching is started from the average value of the characteristic points CP of the object BJ.
[0052]
The coordinates of the feature point CP of the object BJ determined by the value of the principal component at that time are calculated, and the degree of coincidence with the feature point CQ of the image FGS is calculated (# 22). In calculating the degree of coincidence, for example, the proportion of points that coincide with the characteristic point CQ of the image FGS among the number of characteristic points CP of the object BJ is set as the degree of coincidence. Since the object BJ1 shown in FIGS. 2 to 4 has five feature points CP, if three of them match the feature point CQ of the image FGS, the matching degree is 3/5. If the calculated degree of coincidence is higher than the previous one, the present degree of coincidence and the value of the principal component are stored in the memory. The memory is provided with variables for holding the degree of coincidence.
[0053]
If the degree of coincidence is greater than the preset threshold value Sh, the process ends (Yes in # 23). When the degree of coincidence is small, the value of the principal component is slightly changed (# 24), and the degree of coincidence is calculated again (# 22).
[0054]
In addition, when the value of the main component is slightly changed, for example, when the coincidence is improved, the value is slightly changed in the same direction as the direction in which the previous minutely changed, and when the coincidence is deteriorated. Is slightly changed in the direction opposite to the direction in which it was slightly changed last time.
[0055]
In the second detection and matching method, as shown in FIG. 15, the values of the principal components are calculated such that the feature point CQ selected from the many feature points CQ of the image FGS matches the feature point CP of the object BJ. And search for a good match. In selecting the feature points CQ, for example, when the principal component vectors are up to the n-th order, n / 2 feature points CQ are selected. That is, when the principal component vectors are up to the second order, one feature point CQ is selected.
[0056]
In FIG. 14, first, initialization is performed (# 31). In the initialization, for example, the feature points CQ extracted from the image FGS are arranged in a line. Further, a variable for holding the calculated degree of coincidence and the value of the principal component is provided in the memory.
[0057]
Then, some of the feature points CQ are randomly selected, and the values of the principal components are calculated so that the selected feature point CQ and the feature point CP of the object BJ match (# 32). That is, for the feature point CQ and the feature point CP, the values of the principal components are calculated by solving simultaneous equations in which the values of the principal components are unknown and the X coordinates and the Y coordinates are set to be equal. This calculation can be easily performed because it is reduced to solving a system of linear equations.
[0058]
The coordinates of the feature point CP of the object BJ determined by the value of the principal component are calculated, and the proportion of those that match the feature point CQ extracted from the image FGS is defined as the matching degree (# 33).
[0059]
If the degree of coincidence is larger than the preset threshold value Sh, the process ends (Yes in # 34). If the degree of coincidence is low, the feature point CQ is changed and step # 32 and subsequent steps are repeated again.
[0060]
By the way, when the value of the principal component is changed, the set of the value of the principal component that maximizes the degree of coincidence between the image FGS and the object image FGA is not limited to one. In the above-described first and second detection / collation methods, a set of principal component values that gives the maximum matching degree in the range calculated so far is obtained, and a set of principal component values that gives the maximum matching degree is obtained. If there are a plurality of, the first set found is selected and output.
[0061]
Therefore, as described below, for each of the first and second detection / matching methods, by taking into account an additional evaluation criterion, one of the principal component values giving the maximum matching degree is prioritized. Output. In this case, the termination conditions (steps # 23 and # 34) of the above-described detection / collation method may be changed, and for example, may be terminated when the repetition processing is performed a certain number of times.
[0062]
First, in the first method, the maximum matching degree in the range calculated so far and the value of the principal component giving the matching degree are stored in a memory or the like, and the value of the principal component giving the maximum matching degree is stored. When a plurality of pairs appear, one of them is prioritized. The priority evaluation criterion gives priority to, for example, the smaller of the sum of the absolute values of the principal component values, that is, (the absolute value of the first principal component value + the absolute value of the second principal component +...). According to this evaluation criterion, the one in which the feature point CP is closer to the average value is prioritized, which means that a safe shape is selected as the shape of the object BJ.
[0063]
The second method is a method in which the weighted sum of the degree of inconsistency between the image FGS and the object image FGA and the distance from the average is used as an evaluation criterion, and the value of the principal component that minimizes this evaluation criterion is obtained. . In other words, a method of obtaining and outputting a set of values of each principal component that gives a minimum value to an evaluation function using the degree of inconsistency between the image FGS and the object images FGA and FGB and the absolute value of each principal component value as variables. It is.
[0064]
The degree of inconsistency referred to here is, for example, a sign obtained by reversing the sign of the degree of coincidence. As the distance from the average value, for example, the sum of the absolute values of the principal component values, that is, (the absolute value of the first principal component value + the absolute value of the second principal component +...) Is used. In the case of using the second method, in the first and second detection / collation methods, the “evaluation criterion” is used instead of the “matching degree”, and the place of “maximum matching degree” is changed to “minimum matching degree”. It may be replaced with “evaluation reference value”. This evaluation criterion also has an effect of giving priority to an evaluation value close to the average value.
[0065]
Next, an example of the detection device 1 for performing the above-described detection method will be described.
FIG. 16 is a block diagram showing an example of the configuration of the detection device 1 according to the present invention.
16, the detection device 1 includes a processing device 11, a camera 12, a pre-processing unit 13, a magnetic disk device 14, a CD-ROM device 15, a floppy disk device 16, a display device 17, a printer 18, a keyboard 19, a mouse 20, It comprises a communication line 21 and the like.
[0066]
The object is photographed by the camera 12, and various image processing or editing processing is performed by the preprocessing unit 13, and then input to the processing device 11 as an image FGS.
The magnetic disk device 14 stores programs for executing the above-described detection method, images FGA, FGB, FGS, coordinates of feature points CQ and CP, average values, principal component vectors, principal component values, and the like. .
[0067]
The processing device 11 is provided with a memory, and the above-described various programs and data are loaded or generated and stored as needed. When the processing device 11 executes the program, the above-described detection method is performed. That is, the processing device 11 corresponds to first to fifth means in the present invention. A program for executing the detection method is supplied by being recorded on a recording medium such as a CD-ROM or a floppy disk, for example. Alternatively, it is loaded online via the communication line 21.
[0068]
Images FGA, FGB, FGS, and the like are displayed on the display device 17, and the user can operate the keyboard 19 or the mouse 20 to specify the feature points CQ and CP.
[0069]
Such a detection device 1 can be realized by, for example, a workstation or a personal computer in which an appropriate program is stored.
FIG. 17 is a block diagram showing only the detection function of the detection device 1.
[0070]
17, the detection device 1 includes an image input unit 51, a feature calculation unit 52, an analysis unit 53, a collation unit 54, an output unit 55, and the like.
The image input unit reads the given image FGS. The feature calculation unit 52 obtains a feature point CQ from the read image FGS. The analysis unit 53 analyzes the characteristic points CP of the object manually input in advance by using the principal component analysis method, and stores the analysis result in a memory. The matching unit 54 adjusts the value of each principal component of the analyzed feature point CP so as to best match the feature point CQ of the image FGS. The output unit 55 outputs the shape and position of the object determined by the value of each principal component.
[0071]
The output from the output unit 55 is the position of the object BJ in the image FGS, the shape of the object BJ, or the position and shape of the object BJ. The position of the object BJ is, for example, the coordinate position of the origin of the image FGA in the image FGS, and the shape of the object BJ is the value of the principal component or the coordinates of the feature point CP based on the value of the principal component.
[0072]
Alternatively, the detection device 1 may be configured to execute various application programs, and the output from the output unit 55 may be used to execute the application programs.
[0073]
FIG. 18 is a diagram showing an example of a form of a recording medium ST of a program for executing the method according to the present invention.
As shown in FIG. 18, a storage device STA such as a main memory, a RAM, a ROM, or a hard disk provided in the processing device PS, a portable medium STB such as a CD-ROM, a floppy disk, or a magneto-optical disk, a network, or a communication device A line destination medium STC such as a server or DASD connected by the line STD, or the communication line STD itself can be used as the recording medium ST.
[0074]
In the above-described embodiment, the configuration of the entire detection device 1 or each unit, the content or order of processing, and the like can be appropriately changed in accordance with the gist of the present invention.
[0075]
【The invention's effect】
Claim 1 to Claim 9 According to the invention, an object whose shape is changed can be accurately detected in an image in a short time.
[0076]
Claim 2 According to the present invention, the process is further simplified by using the coordinate values of a plurality of feature points as the image feature amount.
Claim 5 And claims 6 According to the invention, it is possible to prioritize and output any one of the values of the principal components giving the maximum coincidence.
[Brief description of the drawings]
FIG. 1 is a flowchart showing a processing flow of a detection method according to the present invention.
FIG. 2 is a diagram illustrating an example of an object image in which a target object is expressed by five feature points.
FIG. 3 is a diagram showing an example of many object images having different shapes.
FIG. 4 is a diagram showing a state of a change of a feature point of an object image.
FIG. 5 is a diagram illustrating a case where two object images are input.
FIG. 6 is a diagram showing average value of feature points and data of principal component vectors of the object shown in FIG. 5;
FIG. 7 is a diagram illustrating an example of an input image using density values.
FIG. 8 is a diagram illustrating an example of a differentiation window.
9 is a diagram illustrating a result obtained by differentiating the image illustrated in FIG. 7 with a differentiation window illustrated in FIG. 8;
FIG. 10 is a flowchart illustrating an example of a matching process.
FIG. 11 is a diagram illustrating a coordinate system in the matching processing.
FIG. 12 is a flowchart illustrating a first detection / collation method.
FIG. 13 is a diagram illustrating a first detection / collation method.
FIG. 14 is a flowchart illustrating a second detection / collation method.
FIG. 15 is a diagram illustrating a second detection / collation method.
FIG. 16 is a block diagram illustrating an example of a configuration of a detection device according to the present invention.
FIG. 17 is a block diagram showing only a detection function of the detection device.
FIG. 18 is a diagram illustrating an example of a form of a recording medium of a program.
FIG. 19 is a diagram illustrating a conventional template matching method.
[Explanation of symbols]
1 Detector
11 processing apparatus (first means, second means, third means, fourth means, fifth means)
12 Camera
51 Image input unit
52 Feature calculation unit (third means)
53 Analysis unit (second means)
54 Collation Unit (Fourth Means)
55 output section (fifth means)
BJ object
FGS image
FGA, FGB object image
CP feature point (object feature amount)
CQ feature point (image feature amount)
STA, STB, STC, STD recording media

Claims (9)

A method for detecting an object having a change in shape from an image,
A first step of extracting, from a plurality of images including the object, an object feature amount that is a digitized feature amount representing an object shape ;
A second step of analyzing the object feature using a principal component analysis technique;
A third step of extracting image features from a given image by a predetermined method ;
The shape of the object from the object feature amount said analysis represented by the value of a few main components, and a fourth step of performing matching in the template matching of the image feature amount and a variable shape,
A fifth step of outputting the shape及beauty position location of said object in accordance with the comparison result,
A method for detecting an object in an image, comprising: As the image feature amount, you have use the coordinate values of a plurality of feature points,
In the first step, to specify multiple numbers of each Shi pair image multiple feature points similar shape for the object, Ru obtains the coordinates of the designated feature points,
The method for detecting an object in an image according to claim 1. In the fourth step, a value of each principal component of the analyzed object feature is changed, and a set of values of each principal component when the analyzed object feature and the image feature are well matched with each other. Is the matching result,
Detection method of an object in an image according to claim 1 or claim 2. In the fourth step, the value of each principal component is adjusted so that a part of the analyzed object feature amount and a part of the image feature amount coincide with each other, and a plurality of adjusted principal component values are adjusted. Of the set, the set of the values of each principal component when the analyzed object feature amount and the image feature amount well match is set as a matching result.
Detection method of an object in an image according to claim 1 or claim 2. In the fourth step, when there are a plurality of sets of the values of the principal components that best match each other, the set having the smallest absolute value of the values of the principal components is set as the matching result.
Method of detecting objects in claim 3 or claim 4, wherein the image. In the fourth step, the degree of inconsistency between the analyzed object feature amount and the image feature amount and the value of each principal component that gives a minimum value to an evaluation function that uses the absolute value of each principal component value as a variable are The set is the matching result,
Method of detecting objects in claim 3 or claim 4, wherein the image. In the fifth step, outputting a shape of an object determined by a value of each principal component as the matching result,
Detection method of an object in an image according to any one of claims 3 to 6. An apparatus for detecting an object having a shape change from an image,
First means for extracting, from a plurality of images including the object, an object feature amount which is a digitized feature amount representing an object shape ;
Second means for analyzing the object feature using a principal component analysis technique;
Third means for extracting an image feature from a given image by a predetermined method ,
And fourth means for performing said parsed from the object feature amount object shape expressed by the value of a small number of principal components, matching in the template matching of the image feature amount and a variable shape,
And fifth means for outputting a shape及beauty position location of said object in accordance with the comparison result,
A device for detecting an object in an image, comprising: In order to detect objects that change shape in the image,
A first step of extracting, from a plurality of images including the object, an object feature amount that is a digitized feature amount representing an object shape ;
A second step of analyzing the object feature using a principal component analysis technique;
A third step of extracting image features from a given image by a predetermined method ;
The shape of the object from the object feature amount said analysis represented by the value of a few main components, and a fourth step of performing matching in the template matching of the image feature amount and a variable shape,
A fifth step of outputting the shape及beauty position location of said object in accordance with the comparison result,
And a computer-readable recording medium storing a program for causing a computer to execute the program.

Images