JP5519556B2 - Image analysis apparatus, server, program and method for recognizing object based on reference image from analysis target images - Google Patents

Description

  The present invention relates to an image analysis technique for recognizing an object based on a reference image from analysis target images.

  2. Description of the Related Art Conventionally, there is a technique for analyzing an analysis target image (still image or moving image) acquired by a device such as a camera and recognizing an object shown in the analysis target image. According to this technique, the reference image storage unit that stores the reference image for each object in advance is included, and the object reflected in the analysis target image is recognized by matching the reference image with the analysis target image.

  Further, there is a technique that can match a reference image even when a part of an object shown in an analysis target image is hidden (see Non-Patent Document 1, for example). According to this technique, a local region including a part of the object, not the entire object, is handled as a unit of feature analysis. For this purpose, a plurality of local regions useful for recognition are detected for both the analysis target image and the reference image. Then, an area most similar to the plurality of local feature amounts of the reference image is detected in the analysis target image to recognize the object. Of course, the position of the object in the analysis target image is also recognized. Note that this depends on how the object in the analysis target image is reflected (due to the effects of the shooting angle, shooting distance, surrounding objects, etc.) and the texture characteristics of the recognized object itself.

  Further, there is a technique for accurately recognizing a plurality of different types of objects by learning control using a learning data set (see, for example, Patent Document 1). According to this technique, a set of different types of object images is input and a local feature amount having a predetermined geometric structure is detected. The plurality of local feature quantities are clustered to select a plurality of representative local feature quantities. An object is recognized from an object image using a learning data set including the representative local feature as teacher data. According to this technique, feature quantities useful for discriminating a plurality of different types of objects are selected from local feature quantities extracted from an object image.

  Furthermore, in order to recognize a plurality of different types of objects with high accuracy, there is a technique in which corresponding points of individual local feature amounts are separated and a reference image is matched in an analysis target image (see, for example, Patent Document 2). According to this technique, first, only feature points with high similarity are left (recognition target image), and second, it is estimated that a single recognition target exists by applying robust estimation to the remaining feature points. A third region is detected (reference image), and thirdly, the matching of the estimated region and the similarity of the image features around the corresponding points are obtained. As a result, a result obtained by increasing the accuracy of the estimation region is obtained. The processing of the same procedure is further executed in a state where the feature points included in the region of the specific object obtained by this procedure are removed, and the process is repeated until no specific object is detected.

Japanese Patent Laid-Open No. 2005-215988 JP 2007-140613 A

David G. Lowe, “Distinctive image features from scale-invariant keypoints”, International Journal of Computer Vision, 60, 2 (2004), pp.91-110, [online], [searched November 23, 2010], Internet <URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.157.3843&rep=rep1&type=pdf>

  According to the above-described conventional technique, a case where a plurality of feature points having the same and similar local feature amount exist in the reference image is not sufficiently considered. For example, the object is a network device such as a hub or a router. These objects include a plurality of identical and similar feature points such as a plurality of LAN cable connection ports.

  FIG. 1 is a first image example in which recognition of an object in an analysis target image has failed.

  FIG. 1 shows an example image for recognizing a hub device. Here, an analysis target image shown in one hub device and its periphery is shown, and a reference image of the hub device is shown in the upper part. The hub device generally includes a plurality of LAN cable ports. A plurality of identical port images (identical similar feature points) exist in the reference image of the hub device. Similarly, there are a plurality of the same port images for the analysis target image in which the hub device is reflected.

  Here, the feature point of the port has the same and similar local feature amount. According to FIG. 1, the matching between the feature points of the reference image and the feature points of the analysis target image is represented by a line. Ideally, the port feature points should be connected in parallel with matching lines. However, according to FIG. 1, since there are a plurality of similar port feature points, they are mixed and matched. This means that the position and shape of the specific object based on the reference image cannot be accurately recognized from the analysis target image.

  FIG. 2 is a second image example in which recognition of an object in the analysis target image has failed.

  According to FIG. 2, as compared with FIG. 1, two hub devices are shown in the analysis target image. Both hub devices are lined with identically shaped ports. For this reason, according to FIG. 2, the feature points of the reference image are further mixed and matched across the feature points of both hub devices.

  When the same objects A and B appear in the analysis target image, for the feature point a and the feature point b in the reference image of one object, the feature point a matches a part of the feature points of the object A in the analysis target image. The feature point b is matched with a part of feature points of the object B of the analysis target image. In this case, the accuracy of recognizing the object of the reference image from the analysis target image is significantly reduced. In particular, there arises a problem that the position and shape of a specific object are detected by being significantly deformed or cannot be detected.

  According to the technique described in Non-Patent Document 1, the object recognition is an image analysis technique that is robust against object rotation, scale change, and partial hiding. However, the similarity of local feature amounts in the reference image is not considered. Further, according to the technique described in Patent Document 1, it is a technique for selecting a local feature amount in order to improve the recognition accuracy of a plurality of different types of objects. However, while the relationship of local feature amounts between a plurality of objects is considered, the similarity of a plurality of local feature amounts existing in the same object is not considered. Therefore, these techniques are not techniques that consider object recognition at a plurality of feature points having the same local feature amount.

  Furthermore, according to the technique described in Patent Document 2, object recognition is considered in the case where a single object of a reference image appears in the analysis target image. This is not a case where a plurality of feature points appear. This means that the presence of a plurality of similar feature amounts in the same object of each object makes it impossible to recognize each object. Therefore, according to the technique of Patent Document 2 that recognizes a plurality of objects in the analysis target image on the assumption that the object recognition is successful if matching is made between individual objects in the analysis target image. Object recognition will be wrong.

  Therefore, the present invention provides an image analysis that can recognize an object based on a reference image from among analysis target images even when there are a plurality of feature points having the same local feature amount in the reference image. An object is to provide an apparatus, a server, a program, and a method.

According to the present invention, an image analysis apparatus for recognizing an object based on a reference image from among analysis target images,
Reference image storage means for storing a reference image and identification information of one object based on the reference image in association with each other;
Reference feature point extraction means for extracting a large number of feature points from the reference image;
A similar feature point detection means for reference for detecting a group of similar feature points having similar feature amounts from a reference image by clustering;
A distribution density detecting means for reference for detecting a distribution density for each group of similar feature points;
A reference representative feature point selecting means for selecting one representative feature point for each group of similar feature points for the reference image;
Analysis target feature point extracting means for extracting a large number of feature points from the analysis target image;
A feature point matching means for detecting a feature point similar to the representative feature point from the analysis target image by matching;
An analysis target distribution density detecting means for detecting a distribution density in each group of feature points similar to the representative feature points detected from the analysis target image;
For each representative feature point, when the difference between the distribution density in the reference image and the distribution density in the analysis target image is within a predetermined threshold range, the object identification information based on the reference image and the object reflected in the analysis target image And an object recognition means for associating.

According to another embodiment of the image analysis apparatus of the present invention,
The reference similar feature point detecting means further detects a feature point having no other similar feature quantity in the reference image as a single feature point,
The feature point matching means detects a feature point similar to a single feature point from the analysis target image by matching,
The object recognizing means determines that there is no object based on the reference image when there is no single feature point in the image area in which the object of the reference image is recognized by the representative feature point in the analysis target image. It is characterized by recognition.

According to the present invention, an image for recognizing an object based on a reference image from among analysis target images, which includes a first device for a reference image and a second device for an analysis target image. An analysis system,
The first device is
Reference image storage means for storing a reference image and identification information of one object based on the reference image in association with each other;
Reference feature point extraction means for extracting a large number of feature points from the reference image;
A similar feature point detection means for reference for detecting a group of similar feature points having similar feature amounts from a reference image by clustering;
A distribution density detecting means for reference for detecting a distribution density for each group of similar feature points;
The reference image includes a reference representative feature point selecting unit that selects one representative feature point for each group of similar feature points, the distribution density detected by the reference distribution density detecting unit, and the reference representative feature point The representative feature points detected by the selection means are transmitted to the second device,
The second device is
Analysis target feature point extracting means for extracting a large number of feature points from the analysis target image;
A feature point matching means for detecting a feature point similar to the representative feature point from the analysis target image by matching;
An analysis target distribution density detecting means for detecting a distribution density in each group of feature points similar to the representative feature points detected from the analysis target image;
For each representative feature point, when the difference between the distribution density in the reference image and the distribution density in the analysis target image is within a predetermined threshold range, the object identification information based on the reference image and the object reflected in the analysis target image And an object recognizing means for associating with each other.

According to the present invention, there is provided an image search server including all functions in the above-described image analysis device,
Receiving the analysis target image from the client, and outputting the analysis target image to the analysis target feature point extraction unit;
It further comprises recognition result return means for returning the identification information of the recognized object detected by the object recognition means and the position information of the object in the analysis target image to the client.

According to another embodiment of the image search server of the present invention,
The apparatus may further include a reference image receiving unit that receives a reference image serving as a search key from the client and outputs the reference image to the feature point extracting unit.

According to the present invention, an image analysis program for causing a computer mounted on an apparatus to function so as to recognize an object based on a reference image from among analysis target images,
Reference image storage means for storing a reference image and identification information of one object based on the reference image in association with each other;
Reference feature point extraction means for extracting a large number of feature points from the reference image;
A similar feature point detection means for reference for detecting a group of similar feature points having similar feature amounts from a reference image by clustering;
A distribution density detecting means for reference for detecting a distribution density for each group of similar feature points;
A reference representative feature point selecting means for selecting one representative feature point for each group of similar feature points for the reference image;
Analysis target feature point extracting means for extracting a large number of feature points from the analysis target image;
A feature point matching means for detecting a feature point similar to the representative feature point from the analysis target image by matching;
An analysis target distribution density detecting means for detecting a distribution density in each group of feature points similar to the representative feature points detected from the analysis target image;
For each representative feature point, when the difference between the distribution density in the reference image and the distribution density in the analysis target image is within a predetermined threshold range, the object identification information based on the reference image and the object reflected in the analysis target image The computer is made to function as an object recognition means for associating the.

According to another embodiment of the image analysis program of the present invention,
The reference similar feature point detecting means further detects a feature point having no other similar feature quantity in the reference image as a single feature point,
The feature point matching means detects a feature point similar to a single feature point from the analysis target image by matching,
The object recognizing means determines that there is no object based on the reference image when there is no single feature point in the image area in which the object of the reference image is recognized by the representative feature point in the analysis target image. The computer may function so as to be recognized.

According to the present invention, an image analysis method for recognizing an object based on a reference image from among analysis target images using a computer mounted on the apparatus,
A reference image storage unit that stores the reference image and identification information of one object based on the reference image in association with each other;
For the reference image
A first step of extracting a number of feature points from a reference image;
A second step of detecting, by clustering, a group of similar feature points having similar feature amounts from the reference image;
A third step of detecting a distribution density for each group of similar feature points;
A fourth step of selecting one representative feature point for each group of similar feature points for the reference image;
For the image to be analyzed
A first step of extracting a large number of feature points from the analysis target image;
A second step of detecting, by matching, a feature point similar to the representative feature point from the analysis target image;
A third step of detecting a distribution density for each group of feature points similar to the representative feature points detected from the analysis target image;
For each representative feature point, when the difference between the distribution density in the reference image and the distribution density in the analysis target image is within a predetermined threshold range, the object identification information based on the reference image and the object reflected in the analysis target image And a fourth step of associating.

According to another embodiment of the image analysis method of the present invention,
In the second step for the reference image, further, a feature point having no other similar feature amount in the reference image is detected as a single feature point,
In the second step for the analysis target image, a feature point similar to a single feature point is detected from the analysis target image by matching,
Regarding the fourth step for the analysis target image, if there is no single feature point in the image region in the analysis target image that is recognized by the representative feature point as an object of the reference image, the reference image is used. You may recognize that an object does not exist.

  According to the image analysis device, the server, the program, and the method of the present invention, the reference image is based on the reference image from among the analysis target images even when a plurality of feature points having the same and similar local feature amount exist in the reference image. An object can be recognized.

It is the 1st example of an image which failed in recognition of an object in an analysis object picture. It is a 2nd image example which failed in recognition of the object in an analysis object image. It is a functional block diagram of the image analysis apparatus in this invention. It is explanatory drawing showing the detection of the feature point in a feature point extraction part. It is explanatory drawing showing clustering of the feature point in the similar feature point detection part for a reference. It is explanatory drawing showing matching of the representative feature point (circle) with respect to an analysis object image. It is explanatory drawing showing matching of the representative feature points (circle) and (square) with respect to an analysis object image. It is explanatory drawing showing matching of single feature point (DELTA) and x with respect to an analysis object image. It is a system configuration diagram having an image analysis server of the present invention. It is a functional block diagram of the image analysis apparatus which has the function to extract a reference image.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 3 is a functional configuration diagram of the image analysis apparatus according to the present invention.

  The image analysis apparatus 1 recognizes an object based on the reference image from the analysis target image. The analysis target image may be stored in advance or may be input from the outside via the interface 100. The interface 100 may be a communication interface connected to a network or an input interface from a user.

  According to FIG. 3, the image analysis apparatus 1 analyzes an analysis target image, an analysis target image receiving unit 101, an analysis target feature point extraction unit 111, a feature point matching unit 112, and an analysis target distribution density detection unit 113. And an object recognition unit 114 and an application processing unit 115. For a reference image showing an object to be detected, a reference image storage unit 120, a reference feature point extraction unit 121, a reference similar feature point detection unit 122, a reference representative feature point selection unit 123, and a reference image And a distribution density detector 124. These functional components are realized by executing a program that causes a computer installed in the image analysis apparatus to function. Further, the flow of processing comprising these functional configurations is understood as an image analysis method in the apparatus.

  First, processing for a reference image will be described.

[Reference image storage unit 120]
The reference image storage unit 120 stores the reference image and the object identification information based on the reference image in association with each other. The “reference image” is an image of an object to be recognized from the analysis target image. The reference image may be stored by being received from the user, or may be a predetermined reference image stored in advance.

[Reference Feature Point Extraction Unit 121]
The reference feature point extraction unit 121 extracts a large number of feature points from the reference image. A “feature point” is a point that represents a visual feature of a local area in an image, and can be detected using, for example, SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features). it can. The feature points are represented by, for example, feature quantities of a multidimensional vector.
SIFT is a technique for analyzing a characteristic local region using a scale space and describing a feature quantity that is invariant to scale change and rotation.
According to SIFT, the following two steps are required to detect feature points.
(S1) A key point and a scale are determined by an extreme value search of the scale space.
(S2) The key points determined are narrowed down to stable key points according to the main curvature and contrast.
There is also a technique for extracting local feature points such as corners and line intersections by applying an Interest operator (see Non-Patent Document 1, for example).
The extracted many feature points are output to the reference similar feature point detection unit 122.

  FIG. 4 is an explanatory diagram illustrating detection of feature points in the feature point extraction unit.

  FIG. 4A shows a reference image showing the hub device. Here, the reference image includes region images (local images) of a plurality of ports in the hub device, and these region images are similar to each other.

According to FIG. 4B, for example, the following many feature points are extracted from the reference image showing the hub device.
Feature point A: Feature point ○: Port Feature point B: Feature point □: Jack Feature point C: Feature point ◇: Connector Feature point D: Feature point △: Power switch Feature point E: Feature point ×: Label Here, feature Each of the points ○, □, and ◇ has a large number of the same and similar feature points, and constitutes a “group of similar feature points”. On the other hand, the feature points Δ and X are recognized as “single feature points” because there are no other similar feature points.

[Reference Similar Feature Point Detection Unit 122]
The reference similar feature point detection unit 122 detects a “group of similar feature points” (cluster) having similar feature amounts from the reference image by clustering.

  FIG. 5 is an explanatory diagram illustrating clustering of feature points in the reference similar feature point detection unit.

  Clustering is performed on a plurality of groups (for example, k clusters) based on the similarity of the feature quantities. Therefore, the number of feature points is different for each cluster. As a technique for clustering into k clusters, for example, there is a k-means method.

  According to FIG. 5, it is divided into a group including a plurality of similar feature points and a group having only a single feature point. Here, a feature point having no other similar feature amount in the reference image is detected as a “single feature point”.

[Reference representative feature point selection unit 123]
The reference representative feature point selection unit 123 selects one representative feature point for each group of similar feature points for the reference image. The representative feature point may be any feature point in the group of similar feature points. The representative feature points are output to the feature point matching unit 112.

[Reference distribution density detector 124]
The reference distribution density detector 124 detects the distribution density of each group of similar feature points detected from the reference image. “Distribution density” refers to the number (density) of feature points within a predetermined spatial range. The distribution density for each group of similar feature points is output to the object recognition unit 114.

  In the processing for the reference image described above, the representative feature points and the distribution density may be detected from the reference image in advance. Of course, the processing for the reference image may be executed by the first device, and the processing for the analysis target image may be performed by the second device. In the second apparatus, the representative feature points and distribution density in the reference image are stored in advance in the reference image storage unit, so that processing to be executed in real time is only processing in the analysis target image described below. Become.

[Analysis Target Image Receiving Unit 101]
The analysis target image receiving unit 101 receives the analysis target image via the interface 100 and outputs the analysis target image to the feature point extraction unit 112.

[Feature Point Extraction Unit 111 for Analysis Object]
Similar to the reference feature point extraction unit 121, the analysis target feature point extraction unit 111 extracts a large number of feature points from the analysis target image. The extracted many feature points are output to the feature point matching unit 112.

[Feature Point Matching Unit 112]
The feature point matching unit 112 detects a feature point similar to the representative feature point from the analysis target image by matching (see FIGS. 6 to 8 described later). Also, feature points similar to a single feature point are detected from the analysis target image by matching. The approximate position of the object based on the reference image in the analysis target image can be estimated from the distribution of the feature points detected by the matching. The coordinate information on the image detected for each representative feature point is output to the distribution density detecting unit 113 for analysis. Also, the coordinate information on the image detected for each single feature point is output to the object recognition unit 114.

[Analysis target distribution density detector 113]
Similar to the reference distribution density detection unit 124, the analysis target distribution density detection unit 113 detects the distribution density for each group of feature points similar to the representative feature points detected from the analysis target image. The distribution density of each feature point group similar to the representative feature point is output to the object recognition unit 114.

[Object recognition unit 114]
When the difference between the distribution density in the reference image and the distribution density in the analysis target image is within a predetermined threshold range for each representative feature point, the object recognition unit 114 determines that the object based on the reference image is the analysis target image. It can be presumed that it exists at that position. At this time, the object identification information based on the reference image is associated with the object shown in the analysis target image.

  A reference image in which an object is shown usually has a group of a plurality of similar feature points, and a plurality of representative feature points are detected thereby. The region where the object exists is limited in size, and there are feature points having a plurality of similar feature amounts in the limited range. Even in the case where a plurality of the same objects are shown in the analysis target image, feature points that are located at a predetermined distance or more away from each other are similarly present.

  After detecting the position range of the object shown in the reference image in the analysis target image, the object recognition unit 114 further detects the presence of a single feature point in the position range. When a single feature point existing in the reference image does not exist in the position range of the object shown in the analysis target image, it is recognized that there is no object based on the reference image. In addition, for example, when there are a large number of representative feature points in the same object, it is assumed that object recognition can be performed with sufficient accuracy using only the representative feature points. May be.

  Then, the object recognition unit 114 outputs the identification number of the object recognized as being reflected in the analysis target image to the application processing unit 115. The object recognizing unit 114 preferably also outputs to the application processing unit 115 the coordinate information of the portion of the analysis target image where the object is reflected, together with the object identification number.

  The application processing unit 115 can provide the user with various services for the analysis target image.

  Further, the processing of the [feature point matching unit 112] will be specifically described below.

  FIG. 6 is an explanatory diagram showing the matching of the representative feature points ○ to the analysis target image.

  According to FIG. 6A, one representative feature point ○ is selected in the group of similar feature points ○ surrounded by a dotted line (see the representative feature point selection unit 123 described above).

  According to FIG. 6B, a feature point similar to the representative feature point ○ is detected from the analysis target image by matching. In the analysis target image, there are a lot of feature points similar to the representative feature points in the portion where the object of the reference image is reflected. That is, the distribution density of the feature points ○ is high. It can be said that such a portion has a high possibility that an object of the reference image exists. On the other hand, among the feature points ◯ in the analysis target image, there are also feature points ◯ that exist independently at distant positions. Thus, if the distribution density of the feature points in the analysis target image is lower than the distribution density of the feature points in the reference image, it can be said that the possibility of noise is high.

  FIG. 7 is an explanatory diagram showing the matching of the representative feature points ○ and □ to the analysis target image.

  According to FIG. 7A, one representative feature point □ is selected in a group of similar feature points □ further surrounded by a dotted line.

  According to FIG. 7B, a feature point similar to the representative feature point □ is further detected from the analysis target image by matching. Compared to FIG. 6B, the recognition accuracy at the position where the object exists can be improved by matching a plurality of representative feature points.

  As described above, the distribution density of the image area in which the same feature points similar to the representative feature points are dense for the analysis target image, and the distribution density of the image area in which the similar feature points similar to the representative feature points are dense for the reference image. Can be recognized that the object of the reference image is reflected in the analysis target image. Further, assuming that a lot of occlusion occurs in the analysis target image, it may be determined by providing a certain threshold value in the comparison of distribution density. For example, the distribution density of certain similar feature points in the analysis target image is equal to or greater than a value obtained by multiplying the distribution density of the similar feature points in the reference image by a coefficient (a real number determined in advance from 0 to 1). This is to make the determination condition.

  FIG. 8 is an explanatory diagram showing matching of single feature points Δ and X to the analysis target image.

  FIG. 8A shows single feature points Δ and X in which there are no other similar feature points.

  According to FIG. 8B, feature points similar to the single feature points Δ and X are detected from the analysis target image by matching. By further matching the presence of a single feature point with the image range of the object recognized according to FIGS. 6B and 7B, the recognition accuracy at the position where the object exists can be further improved. it can.

  Here, it is not necessary to perform matching for a portion other than the image range of the object recognized by FIGS. 6B and 7B. That is, it is not necessary to match a single feature point for the entire analysis target image. The recognition accuracy can be increased by matching an object recognized by similar feature points with a single feature point.

  Further, another embodiment in the feature point matching unit 112 will be described below.

[First Embodiment in Feature Point Matching Unit]
For example, assume that three objects appear in the analysis target image (similar to FIG. 8). At this time, it is assumed that the following feature points exist in the reference image.
Representative feature points based on many similar feature points: A, B, C
Single feature points that do not have similar feature points: D, E
In this case, in the analysis target image, three image areas (for example, A-1, A-2, and A-3) where feature points matching the representative feature point A are dense should be found. Similarly, three representative features (B-1, B-2, B-3, C-1,...) Should be found for the representative feature points B and C and the single feature points D and E as well.

  Here, with respect to the analysis target image, it may be estimated that the object of the reference image exists by a group of feature points matched with the representative feature point A, and further matched with the representative feature points B and C. The presence of the object may be recognized with higher accuracy by further detecting a group of feature points.

  Furthermore, with respect to the analysis target image, the feature points that match the single feature points D and E exist around the image area of the object recognized by the representative feature points, thereby recognizing the existence of the object with higher accuracy. There may be.

  As described above, whether all of the representative feature points and the single feature points are matched or only the representative feature points are matched depends on the desired recognition accuracy and the distribution characteristics of the feature points of the object to be recognized (for example, It can be determined as a design item according to a state in which the recognition target object includes a large number of different similar feature points.

[Second Embodiment in Feature Point Matching Unit]
Even when there are a plurality of reference images and the images have the same and similar representative feature points, it is possible to distinguish which of the reference images is an object.
Reference image 1: representative feature points A, B, C
Reference image 2: representative feature points A ′, D, E
Reference image 3: representative feature points A ″, F, G
* The feature quantities A≈A′≈A ″ are similar to each other.

  Here, there may be a region in the analysis target image where feature points that match any of the representative feature points A, A ′, and A ″ are dense. In this case, it cannot be determined which of the three reference images (three objects) exists. In such a case, it can be distinguished as an AND condition by detecting an image region where feature points matching the next most representative feature point B are dense.

  For example, if there is an image region in which feature points that match any of the representative feature points A, A ′, and A ″ exist in the analysis target image, and if there is a single feature point B in the predetermined range, For example, the image region can recognize that an object based on the reference image 1 is reflected.

  In addition, when the presence of similar representative feature points A≈A′≈A ″ is known in advance for the reference image, matching of these representative feature points is also preferably summarized. For example, the representative feature points A ′ and A ″ can be included in the similar range of the representative feature point A by slightly expanding the similar range of the feature amount for the matching of the representative feature point A. As a result, only the representative feature point A can be matched, and the same effect can be obtained as when the representative feature points A, A ′, A ″ are matched.

[Third Embodiment in Feature Point Matching Unit]
As described above, the matching process for all the representative feature points and the single feature points causes inconvenient cases. This is a case where occlusion (partial hiding of the recognition target object) occurs in the analysis target image. In this case, although an object based on the reference image is shown in the analysis target image, the object cannot be recognized due to some occlusion.

For example, in the above-described second embodiment, for example, an object based on the reference image 1 (an object including all the feature point groups AX, BX, and CX matching the representative feature points A, B, and C) is included in the analysis target image. It is assumed that the feature point group BX that matches the representative feature point B is not reflected by occlusion even though it is reflected.
Analysis target image: Object 1 = A1, B1, C1
Object 2 = A2, (B2), C2
(B2 is hidden by occlusion)
Object 3 = A3, B3, C3
In this case, of the three feature point groups that match the representative feature points A and C, there are two regions (regions having all the feature point groups AX, BX, and CX) that match the reference image 1 in the analysis target image. It will only be found.

  Therefore, for example, when any two of the feature point groups AX, BX, and CX exist in this recognition condition, it may be recognized that the object exists. For example, this is a case where any one of the feature point groups AX and BX, the feature point groups AX and CX, or the combination of the feature point groups BX and CX exists. However, since the number of feature points used for matching is reduced, the accuracy of object recognition may be reduced.

[Fourth Embodiment in Feature Point Matching Unit]
As another recognition condition, there is a dense region (a region including a feature point group) that matches any one of the representative feature points A, B, and C, and a single feature point of the reference image 1 is in the vicinity thereof. If it is included, it may be recognized that the object exists. By using a single feature point, it is effective for recognizing objects having similar representative feature points as follows.
Reference image 4: representative feature points A and B
Reference image 5: representative feature points A ′ and B ′
Reference image 6: representative feature points A ″, B ″, C
* A≈A′≈A ″, B≈B′≈B ″, and these feature quantities are similar to each other.

For example, if there are three dense regions that match the representative feature points A (matches A ′ and A ″), B (matches B ′ and B ″), and C in the analysis target image, It can be said that there is a high possibility that the object of the reference image 6 exists in the region.
In the analysis target image, if there are two dense regions that match the representative feature points A (matches A ′ and A ″) and B (matches B ′ and B ″), the region is Thus, it can be said that any one of the objects of the reference images 4, 5 and 6 exists (however, the reference image 6 may also match this condition due to the influence of occlusion). In this case, it is possible to narrow down to an object of one reference image by matching single feature points.

  FIG. 9 is a system configuration diagram having the image analysis server of the present invention.

  According to FIG. 9, the image analysis server 1 is connected to the Internet and can communicate with the terminal 2 and the reference image database 3. The reference image database 3 is a database that stores reference images, and the image analysis server 1 acquires reference images from the reference image database 3. At this time, the database may hold feature point information together with the image. In this case, the feature point information is acquired by the server. Object identification information for identifying the object is also added to the reference image.

  On the other hand, the terminal 2 is connected to the Internet via an access network, and transmits an analysis target image corresponding to a user operation to the image analysis server 1. In response to this, the image analysis server 1 returns the recognized object information as the recognition result to the terminal 2.

  FIG. 10 is a functional configuration diagram of an image analysis server having a function of extracting a reference image.

  According to FIG. 10, the image analysis apparatus 1 of FIG. 3 is represented as an embodiment of an image analysis server connected to a network.

  The image analysis apparatus 1 in FIG. 11 has all the functional components of the image analysis apparatus 1 in FIG. 3, and further includes a recognition result return unit 102 and a reference image reception unit 103. These functional components are also realized by executing a program that causes a computer installed in the image analysis apparatus to function.

  The recognition result reply unit 102 can reply the identification number of the object recognized by the object recognition unit 114 to the terminal 2 based on the user operation. The terminal 2 can know the object shown in the transmitted analysis target image. Thereby, the application processing unit activated in the terminal 2 can provide various services to the user.

  The recognition result reply unit 102 preferably returns the coordinates of the object recognized by the object recognition unit 114 to the terminal 2. The terminal 2 can recognize at which coordinate position in the analysis target image the object is reflected.

  The reference image receiving unit 103 receives a reference image via the communication interface 100 and outputs the reference image to the reference image storage unit 121. Thereby, the reference image received from the terminal 2 based on the user operation can be stored in the image analysis server 1.

  As described above in detail, according to the image analysis apparatus, server, program, and method of the present invention, even when a plurality of feature points of the same local feature amount exist in the reference image, the analysis is performed. An object based on the reference image can be recognized from the target image.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Image analysis apparatus, image analysis server 100 Interface 101 Analysis object image receiving part 102 Recognition result reply part 103 Reference image receiving part 111 Analysis object feature point extraction part 112 Feature point matching part 113 Analysis object distribution density detection part 114 Object recognition Unit 115 Application processing unit 120 Reference image storage unit 121 Reference feature point extraction unit 122 Reference similar feature point detection unit 123 Reference representative feature point selection unit 124 Reference distribution density detection unit 2 Terminal 3 Reference image database

Claims (9)

An image analysis apparatus for recognizing an object based on a reference image from among analysis target images,
Reference image storage means for storing the reference image and identification information of one object based on the reference image in association with each other;
Reference feature point extracting means for extracting a large number of feature points from the reference image;
A reference similar feature point detecting means for detecting, by clustering, a group of similar feature points having similar feature amounts from the reference image;
Reference distribution density detecting means for detecting the distribution density of each group of the similar feature points;
For the reference image, reference representative feature point selection means for selecting one representative feature point for each group of similar feature points;
Analysis target feature point extracting means for extracting a large number of feature points from the analysis target image;
A feature point matching means for detecting a feature point similar to the representative feature point from the analysis target image by matching;
Analysis target distribution density detection means for detecting a distribution density for each group of feature points similar to the representative feature points detected from the analysis target image;
When the difference between the distribution density in the reference image and the distribution density in the analysis target image is within a predetermined threshold range for each representative feature point, object identification information based on the reference image, and the analysis target image An image analysis apparatus comprising object recognition means for associating an object reflected in The reference similar feature point detecting means further detects a feature point having no other similar feature quantity in the reference image as a single feature point,
The feature point matching means detects a feature point similar to the single feature point from the analysis target image by matching,
If the single feature point does not exist in the image area in which the object of the reference image is recognized by the representative feature point in the analysis target image, the object recognition unit is added to the reference image. The image analysis apparatus according to claim 1, wherein the image recognition apparatus recognizes that there is no object based thereon. An image analysis system for recognizing an object based on a reference image from among analysis target images, comprising a first device for a reference image and a second device for an analysis target image,
The first device is
Reference image storage means for storing the reference image and identification information of one object based on the reference image in association with each other;
Reference feature point extracting means for extracting a large number of feature points from the reference image;
A reference similar feature point detecting means for detecting, by clustering, a group of similar feature points having similar feature amounts from the reference image;
Reference distribution density detecting means for detecting the distribution density of each group of the similar feature points;
A reference representative feature point selecting unit that selects one representative feature point for each group of similar feature points for the reference image, the distribution density detected by the reference distribution density detecting unit, and the reference Representative feature points detected by the representative feature point selection means are transmitted to the second device,
The second device is
Analysis target feature point extracting means for extracting a large number of feature points from the analysis target image;
A feature point matching means for detecting a feature point similar to the representative feature point from the analysis target image by matching;
Analysis target distribution density detection means for detecting a distribution density for each group of feature points similar to the representative feature points detected from the analysis target image;
When the difference between the distribution density in the reference image and the distribution density in the analysis target image is within a predetermined threshold range for each representative feature point, object identification information based on the reference image, and the analysis target image An image analysis system comprising object recognition means for associating an object reflected on the screen. An image search server including all functions in the image analysis device according to claim 1,
Analysis target image receiving means for receiving the analysis target image from a client and outputting the analysis target image to the analysis target feature point extraction means;
The image further comprising: a recognition result return means for returning the identification information of the recognized object detected by the object recognition means and the position information of the object in the analysis target image to the client. Search server.   5. The image search server according to claim 4, further comprising reference image receiving means for receiving a reference image serving as a search key from a client and outputting the reference image to the feature point extracting means. An image analysis program for causing a computer mounted on the apparatus to function so as to recognize an object based on a reference image from among analysis target images,
Reference image storage means for storing the reference image and identification information of one object based on the reference image in association with each other;
Reference feature point extracting means for extracting a large number of feature points from the reference image;
A reference similar feature point detecting means for detecting, by clustering, a group of similar feature points having similar feature amounts from the reference image;
Reference distribution density detecting means for detecting the distribution density of each group of the similar feature points;
For the reference image, reference representative feature point selection means for selecting one representative feature point for each group of similar feature points;
Analysis target feature point extracting means for extracting a large number of feature points from the analysis target image;
A feature point matching means for detecting a feature point similar to the representative feature point from the analysis target image by matching;
Analysis target distribution density detection means for detecting a distribution density for each group of feature points similar to the representative feature points detected from the analysis target image;
When the difference between the distribution density in the reference image and the distribution density in the analysis target image is within a predetermined threshold range for each representative feature point, object identification information based on the reference image, and the analysis target image An image analysis program that causes a computer to function as an object recognition means for associating an object reflected in the object. The reference similar feature point detecting means further detects a feature point having no other similar feature quantity in the reference image as a single feature point,
The feature point matching means detects a feature point similar to the single feature point from the analysis target image by matching,
If the single feature point does not exist in the image area in which the object of the reference image is recognized by the representative feature point in the analysis target image, the object recognition unit is added to the reference image. The image analysis program according to claim 6, wherein the computer functions so as to recognize that there is no object based thereon. An image analysis method for recognizing an object based on a reference image from analysis target images using a computer mounted on the apparatus,
A reference image storage unit that stores the reference image and identification information of one object based on the reference image in association with each other;
For the reference image,
A first step of extracting a number of feature points from the reference image;
A second step of detecting, by clustering, a group of similar feature points having similar feature amounts from the reference image;
A third step of detecting a distribution density for each group of the similar feature points;
A fourth step of selecting one representative feature point for each group of the similar feature points for the reference image;
For the analysis target image,
A first step of extracting a large number of feature points from the analysis target image;
A second step of detecting, by matching, a feature point similar to the representative feature point from the analysis target image;
A third step of detecting a distribution density for each group of feature points similar to the representative feature points detected from the analysis target image;
When the difference between the distribution density in the reference image and the distribution density in the analysis target image is within a predetermined threshold range for each representative feature point, object identification information based on the reference image, and the analysis target image And a fourth step of associating the object reflected in the image. For the second step with respect to the reference image, further, a feature point having no other similar feature quantity in the reference image is detected as a single feature point,
For the second step on the analysis target image, a feature point similar to the single feature point is detected from the analysis target image by matching,
Regarding the fourth step for the analysis target image, when the single feature point does not exist in the image region in which the object of the reference image is recognized by the representative feature point in the analysis target image The image analysis method according to claim 8, wherein an object based on the reference image is recognized not to exist.

Images