JP4740889B2 - Parameter estimation device, parameter estimation method, program implementing the method, and recording medium recording the program - Google Patents

Description

  The present invention relates to a parameter estimation apparatus that estimates parameters of an object included in an image, a parameter estimation method, a program that implements the method, and a recording medium that records the program.

  The object imaged by the camera includes various parameters such as from which direction the object is taken and from where the illumination is applied. For example, FIG. 4 shows the relationship between an image and parameters when the shooting angle (horizontal direction) is considered as a parameter. Estimating these parameter information only from images is considered to have a wide range of applications such as industrial part classification and moving object monitoring, and has high engineering value. As a method for realizing this, a method combining a linear subspace method and support vector regression has been proposed (see Non-Patent Document 1).

  This is a technique for expressing the relationship between an image and a parameter when the parameters are changed in various ways as an input / output function in the feature space, and estimating the parameter using the function. This method is not a method of storing all the sample points as in the “parametric eigenspace method” (see Non-Patent Document 2), but only a vector (support vector) corresponding to the basis vector of the function needs to be stored. There is little worry to squeeze the space.

As a related technique, a subspace method (see, for example, Non-Patent Document 3) is widely known.
Shingo Ando, Yoshinori Kusachi, Akira Suzuki, Kenichi Arakawa “3D Object Pose Estimation Using Support Vector Regression”, Science Theory D, Vol. J89-D, No8, pp. 1840-1847, 2006 Hiroshi Murase, Shree K. NAYAR “3D Object Recognition by 2D Matching—Parametric Eigenspace Method”, IEICE Transactions D-II, Vol. J77-D-II, No. 11, pp. 2179-2187, 1994 Kenichiro Ishii, Noriyoshi Ueda, Eisaku Maeda, Hiroshi Murase, “Intuitive Pattern Recognition”, Ohmsha, 1998

  However, the methods described in Non-Patent Documents 1 and 2 have a problem that they are weak in situations where the background changes variously or occlusion occurs.

  The present invention has been made in view of the above points, and an object of the present invention is to implement a parameter estimation apparatus, a parameter estimation method, and a method that can realize parameter estimation robust to background change and occlusion in parameter estimation of an object in an image. An object is to provide a program and a recording medium on which the program is recorded.

  In order to achieve the above object, the present invention first divides the area of the input image into a plurality of partial areas, for example, as shown in FIG. At this time, instead of completely dividing the area completely, a plurality of areas may be set by allowing overlap. In the example of FIG. 1, five areas of the upper left, lower left, upper right, lower right, and center of the input image are set, and the center area 5 overlaps with the other areas 1 to 4.

  In the learning process, the relationship between the image and the parameter in each region is expressed as a function. In the parameter estimation process, a parameter is estimated in each region, and an optimum parameter value is selected from the parameters. As the selection method, median (median value), weighted median, average value, weighted average value, and the like can be considered. In the case of using a weighted median or a weighted average value, weighting is performed with some predetermined reliability. For example, the reciprocal of the distance between the subspace created using the learning image and the input vector can be used as the reliability (of course, the method of calculating the reliability is not limited to this). The influence on the area can be reduced.

  Further, instead of using the luminance itself as the feature amount of the image, the feature amount relating to the gradient at each position is used. The feature vectors at this time are assumed to be those of the following formulas (1) to (5).

Here, x _{α, k} , x _{β, k} , x _{χ, k} and x _{δ, k} are the k-th element of the feature vector x, a _k is the lateral differential value of the k-th corresponding position, b _k Is a differential value in the vertical direction of the k-th corresponding position. In addition, _ck is a differential value in the upper right direction of the lick corresponding to the kth position, and d _k is a differential value in the lower right direction of the lick corresponding position.

  Since the differential value is originally a sparse feature, it can be said that it is basically a feature that is robust against changes in the background. However, the contour portion that is the boundary between the background and the object is not necessarily stable because the strength of the differential changes or the differential direction is reversed by the change of the background. In this respect, the feature amount is invariant with respect to the change in the strength of the differentiation and the inversion of the differentiation direction, and thus can be said to be a more robust feature amount with respect to the change in the background.

  As described above, the median or the median weighted by the reliability using the parameter estimation function derived in a plurality of local regions at the same time using the feature amount based on the differentiation in the four directions at each position of the image. If an optimal parameter value is selected by using, the influence when a part of the feature amount changes can be reduced, so that parameter estimation robust to background change and occlusion can be realized.

A parameter estimation apparatus according to claim 1 of the present invention is a parameter estimation apparatus for estimating a parameter of an object included in image data, and includes an input means for inputting digital image data, and a digital signal input by the input means. From the image data , the differential values in the vertical, horizontal, tanned and licked down directions at each position of the image are extracted, and the gradient strength is normalized at each position to obtain the absolute value. a feature extraction means for extracting as the features on the basis of the characteristics of the extracted objects by extracting means, respectively, in a plurality of regions previously determined, subspace group creation means for calculating a subspace of the object by principal component analysis And a parameter for creating a parameter for estimating the parameter by support vector regression in each of the plurality of regions. Meter estimation function group creation means, subspace group storage means for storing the subspace of the object calculated by the subspace group creation means in a memory, and parameter estimation function for the object created by the parameter estimation function group creation means Parameter estimation function group storage means for storing the image in the memory, and the feature quantity extracted from the image data obtained when the object to be parameter estimated is imaged onto the partial space stored in the partial space group storage means in each area And a parameter estimation unit that estimates the parameter of the feature amount projected by the subspace group projection unit in each region based on the function stored in the parameter estimation function group storage unit. From the parameter values estimated in the respective areas by the parameter estimation means, An estimation result integration means for selecting have value, e Bei and output means for outputting the estimation result selected by the estimation result integration means, the feature vector x representing a characteristic of the object, k-th feature vector x The elements are x _{α, k} , x _{β, k} , x _{χ, k} and x _{δ, k} , a _k is the lateral differential value of the k-th corresponding position , and b _k is the vertical position of the k-th corresponding position. A differential value in the direction, _ck is a differential value in the upper right direction of the lick corresponding to the kth position, and d _k is a differential value in the lower right direction of the lick corresponding to the position corresponding to the kth,
It is characterized by being of the formula of

The parameter estimation method according to claim 4 is a parameter estimation method for estimating a parameter of an object included in image data, wherein the input means inputs a learning image input step for inputting learning digital image data; Extraction means extracts from the digital image data input in the learning image input step the differential values in the vertical, horizontal, tanning and tanning directions at each position of the image, and the gradient strength at each position. A first feature extracting step for extracting a value obtained by normalizing and extracting the absolute value as a feature of the object, and a subspace group creating means based on the feature of the object extracted by the first feature extracting step A subspace group creation step for calculating a subspace of the object by principal component analysis in each of a plurality of predetermined regions; A parameter estimation function group creating unit creates a parameter estimation function group creating step for creating a function for estimating a parameter by support vector regression in each of the plurality of regions, and a subspace group storing unit is the subspace group creating step A subspace group storage step for storing the subspace of the object calculated by the above in a memory, and a parameter for storing the parameter estimation function for the object created by the parameter estimation function group creation step in the memory An estimation function group storage step, an input means for inputting digital image data for parameter estimation, and a feature extraction means for each of the images from the digital image data input in the target image input step Vertical, horizontal, tanned on position Extracting the differential value of the oblique downward direction, and the second feature extraction step of extracting a value taking the absolute value normalized as a feature of the object the intensity of the gradient at each position, the subspace group projection means, A subspace projection step of projecting the feature amount extracted in the second feature extraction step onto a subspace stored in the subspace group storage unit in each region; and a parameter estimation unit including the parameter estimation function group Based on the function stored in the storage means, the parameter estimation step for estimating the parameter of the feature amount projected by the subspace group projection step in each region, and the estimation result integration means include the parameter estimation step, An estimation result integration step of selecting the most probable value from the parameter values estimated in each region, and an output means, E Bei and an output step of outputting the estimation result selected by integration step,
The feature vector x representing the feature of the object is set to x _{α, k} , x _{β, k} , x _{χ, k} and x _{δ, k as} the k th element of the feature vector x , and a _k corresponds to the k th position. Differential value in the horizontal direction, b _k is the vertical differential value of the k-th corresponding position , c _k is the slanted upper right differential value of the k-th corresponding position , and d _k is the k-th corresponding position. The differential value of the licked lower right direction,
It is characterized by being of the formula of

According to the above configuration, parameter estimation that is robust against background change and occlusion and accurate can be performed on an object included in an image. Furthermore, the feature quantity of the object extracted by the feature extraction means (step) becomes invariant with respect to the change in the strength of differentiation and the inversion of the differentiation direction, and becomes a more robust feature quantity with respect to changes in the background.

The parameter estimation device according to claim 2 is characterized in that, in claim 1 , the estimation result integrating means uses a median.

The parameter estimation method according to claim 5 is characterized in that, in claim 4 , the estimation result integration step uses a median.

According to a third aspect of the present invention, there is provided the parameter estimation apparatus according to the first aspect , wherein the estimation result integration unit uses a weighted median and uses a reliability defined in advance as a weight. Yes.

The parameter estimation method according to claim 6 is characterized in that, in claim 4 , the estimation result integration step uses a weighted median, and uses a reliability defined in advance as a weight. Yes.

  According to the above configuration, the influence on the occlusion area is reduced, and a more reliable parameter value can be selected.

A parameter estimation program according to a seventh aspect causes a computer to execute the steps according to any one of the fourth to sixth aspects .

A recording medium according to an eighth aspect is characterized by a computer-readable recording medium in which a parameter estimation program for causing a computer to execute each step according to any one of the fourth to sixth aspects is recorded .

  Further, the present invention is characterized in that the plurality of predetermined areas are defined while allowing duplication. Further, the estimation result integration means and the estimation result integration step are characterized in that weighted medians are used, and a reliability defined in advance is used as the weight. The reciprocal of the distance from the feature vector obtained from the image input to “is used”.

(1) According to the first to eighth aspects of the present invention, parameter estimation that is robust against background change and occlusion and has high accuracy can be performed on an object included in an image. Further, the feature amount of the object extracted by the feature extraction means (step) becomes invariant with respect to the change in the strength of the differentiation and the inversion of the differentiation direction, and a feature amount that is more robust against the change in the background is obtained.
(2) According to the inventions described in claims 2 , 3 , 5, and 6 , the influence on the occlusion region is reduced, and a more reliable parameter value can be selected.
(3) According to the invention described in claim 7 , it is possible to provide a computer program for executing each step described in any one of claims 4 to 6 .
(4) According to the invention described in claim 8 , it is possible to provide a recording medium in which a computer program for executing each step described in any one of claims 4 to 6 is recorded.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiments. FIG. 2 is a block diagram showing the configuration of the parameter estimation apparatus according to the embodiment of the present invention, and FIG. 3 is a flowchart of the parameter estimation method of the embodiment according to the present invention.

  In FIG. 2, a learning image data input unit 101 (input unit) inputs learning object image data captured by a digital camera or the like and a correct parameter value thereof, and transmits them to a feature extraction unit 102 (feature extraction unit).

  The feature extraction unit 102 extracts the feature vector of the object from the input digital image data and transmits it to the subspace group creation unit 103 (subspace group creation means). Any feature vector may be used basically, but if it is extracted using the above-described equations (1) to (5), a feature robust to background changes can be used. Thereafter, since processing is performed in parallel in each predefined region (for example, each region in FIG. 1), the extracted feature vector is assigned to each corresponding region.

  The partial space group creation unit 103 creates a partial space in each region based on a set of feature vectors extracted from the learning object image. The partial space can be obtained by calculating (principal component analysis) eigenvalues and eigenvectors of the covariance matrix of the feature vectors in each region. By projecting all the learning feature vector sets to this subspace, the feature vector of the object is compressed to a lower dimension, which contributes to memory saving.

The parameter estimation function group creation unit 104 (parameter estimation function group creation means) generates a parameter estimation function in each region by support vector regression. In support vector regression, an estimation equation is expressed as follows using a kernel function k (x _i , x _j ) (x is a feature vector low-dimensionally compressed by the subspace creation unit 103).

Here, x _i is a feature vector for learning, and m is the number thereof. X _i where (α _i * −α _i ) ≠ 0 is called a support vector. Α _i *, α _i and b in this equation can be obtained by solving the following optimization problem.

Here, ε is the magnitude of an insensitive band (range in which an error is allowed), and y _i is a correct parameter value corresponding to x _i . The optimization can be solved using convex quadratic programming. The kernel function is a polynomial kernel

  Etc. Here, p and σ are kernel function parameters to be set in advance. The parameter estimation function is derived independently in each region, as in the subspace.

  The subspace group storage unit 105 (subspace group storage unit) stores the subspaces of the objects in all regions (eigenvectors constituting the subspace) created by the subspace group creation unit 103 in the memory.

  In the parameter estimation function group storage unit 106 (parameter estimation function group storage unit), the parameter estimation function of the object in all regions (all numerical values related to the equation (6)) created by the parameter estimation function group creation unit 104 Is stored in the memory.

  The target image data input unit 107 (input unit) inputs object image data that is a parameter estimation target imaged by a digital camera or the like, and transmits the object image data to the feature extraction unit 108 (feature extraction unit). If it is not known where the target object is on the image, the image is searched by means such as a subspace method (see Non-Patent Document 3).

  Similar to the feature extraction unit 102, the feature extraction unit 108 extracts a feature vector of an object and transmits it to the subspace group projection unit 109.

  The subspace group projection unit 109 (subspace group projection unit) reads the subspace of the object stored in the memory by the subspace group storage unit 105, and projects the feature vector of the input image in the subspace in each region.

  The parameter estimation unit 110 (parameter estimation means) reads the parameter estimation function stored in the memory by the parameter estimation function group storage unit 106 and estimates parameters in each region. The above-described equation (6) is an estimated regression equation. And the parameter value estimated in all the areas | regions is transmitted to the estimation result integration part 111 (estimation result integration means).

The estimation result integration unit 111 selects an optimum parameter value for a median or a median weighted by reliability from a set of parameter values estimated in each region. The weighted median M _f is expressed by the following equation (10).

Here, Y represents the solution space, f _t (x) represents the parameter value estimated in the region t, and C _t represents the reliability of the region t. The reliability C _t can be defined by the following equation, for example.

Here, D _t subspace in the region _{t (a t, e t,} k is the mean vector constituting the subspace each represent eigenvectors, d is the subspace rank.) And the input vector x _t Represents distance. That is, the closer the distance D _t is, the higher the reliability of the estimated parameter value f _t (x) is. If ordinary medians are used, 1 may be substituted for all C _t .

  The output unit 112 (output unit) outputs the result obtained by the estimation result integration unit 111.

  Next, the parameter estimation method according to the embodiment of the present invention will be described with reference to FIG. FIG. 3 shows a method for estimating parameters using the parameter estimation apparatus of FIG. First, in the learning process of FIG. 3A, learning object image data captured by a digital camera or the like in step S101 and correct parameter values thereof are input (learning image input step).

  Next, in step S102, feature vectors of the object are extracted from the input digital image data using, for example, the equations (1) to (5) (first feature extraction step).

  In step S103, a partial space in each area is created based on the feature vector extracted from the learning object image.

  Next, in step S104, the partial spaces in all the areas created in step S103 are stored in the memory.

  In step S105, a parameter estimation function in each region is generated by support vector regression using the above-described equations (6) to (9).

  In step S106, the parameter estimation function generated in step S105 is stored in the memory.

  Then, steps S105 and S106 are repeatedly executed until the estimation functions for all regions are created, and steps S105 and S106 are repeated until creation of all the parameter estimation functions is completed. finish.

  In the parameter estimation process of FIG. 3B, in step S107, object image data that is a parameter estimation target imaged by a digital camera or the like is input (target image input step).

  Next, in step S108, the feature vector of the object is extracted from the input digital image data using, for example, the equations (1) to (5) (second feature extraction step).

  Next, in step S109, the partial space of the object stored in the memory in step S104 is read, and the feature vector of the target input image is projected onto the partial space in each region.

  Next, in step S110, the parameter estimation function stored in the memory in step S106 is read, and the parameter is estimated using equation (6) as an estimated regression equation.

  Then, after executing step S110 until the estimation is completed for all the regions, in step S111, an optimal parameter value is calculated from the set of parameter values estimated for each region using the equations (10) to (12). select.

  Then, after steps S110 to S111 are executed until all parameters are estimated, the parameter of the estimation result is output in step S112, and this parameter estimation process is terminated.

  In the embodiment, the learning image data input unit 101 and the target image data input unit 107 are used as input means. However, the present invention is not limited to this, and the image data input unit may be configured by one image data input unit.

  Similarly, although the two feature extraction units 102 and 108 are used as the feature extraction means, the present invention is not limited to this, and a single feature extraction unit may be used.

  Further, a part or all of the functions of each means in the parameter estimation apparatus of the present embodiment can be configured by a computer program, and the program can be executed using the computer to realize the present invention. Needless to say, the procedure in the parameter estimation method can be configured by a computer program and the program can be executed by the computer. (Floppy (registered trademark) Disk), MO (Magneto-Optical disk), ROM (Read Only Memory), memory card, CD (Compact Disk) -ROM, DVD (Digital Versa ile Disk) -ROM, CD-R ,, CD-RW, HDD, and recorded in a removable disk, or stored, it is possible to or distribute. It is also possible to provide the above program through a network such as the Internet or electronic mail.

  Further, a computer program describing a method related to the parameter estimation device described above may be mounted so as to access a memory or an external storage device that stores input / output data required for the method related to the parameter estimation device.

Explanatory drawing showing the example of 1 embodiment of this invention, and dividing an input image into partial areas. The block diagram which shows one embodiment of this invention. The flowchart which shows one embodiment of this invention. Explanatory drawing which shows the relationship between an image and a parameter at the time of considering imaging | photography angle as a parameter.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Learning image data input part 102 ... Feature extraction part 103 ... Subspace group creation part 104 ... Parameter estimation function group creation part 105 ... Subspace group memory | storage part 106 ... Parameter estimation function group memory | storage part 107 ... Target image data input part 108 ... Feature extraction unit 109 ... Subspace group projection unit 110 ... Parameter estimation unit 111 ... Estimation result integration unit 112 ... Output unit

Claims (8)

A parameter estimation device that estimates parameters of an object included in image data,
Input means for inputting digital image data;
From the digital image data input by the input means, the differential values in the vertical, horizontal, tanning and tanning down directions at each position of the image are extracted, and the absolute value is obtained by normalizing the gradient strength at each position. A feature extraction means for extracting the value obtained by taking as a feature of the object;
Subspace group creation means for calculating a subspace of the object by principal component analysis in each of a plurality of predetermined regions based on the features of the object extracted by the feature extraction means;
A parameter estimation function group creating means for creating a function for estimating a parameter by support vector regression in each of the plurality of regions;
Subspace group storage means for storing the subspace of the object calculated by the subspace group creation means in a memory;
Parameter estimation function group storage means for storing the parameter estimation function of the object created by the parameter estimation function group creation means in a memory;
A subspace projection means for projecting the feature amount extracted from the image data when the parameter estimation target object is imaged onto the subspace stored in the subspace group storage means in each area;
Based on the function stored in the parameter estimation function group storage means, parameter estimation means for estimating the parameter of the feature amount projected by the subspace group projection means in each region;
An estimation result integrating means for selecting the most probable value from the parameter values estimated in the respective areas by the parameter estimating means;
E Bei and output means for outputting the estimation result selected by the estimation result integration unit,
The feature vector x representing the feature of the object is set to x _{α, k} , x _{β, k} , x _{χ, k} and x _{δ, k as} the k th element of the feature vector x , and a _k corresponds to the k th position. Differential value in the horizontal direction, b _k is the vertical differential value of the k-th corresponding position , c _k is the slanted upper right differential value of the k-th corresponding position , and d _k is the k-th corresponding position. The differential value of the licked lower right direction,
A parameter estimation device characterized by the formula The parameter estimation apparatus according to claim 1 , wherein the estimation result integration unit uses a median. The parameter estimation apparatus according to claim 1 , wherein the estimation result integration unit uses a weighted median, and uses a reliability defined in advance as a weight. A parameter estimation method for estimating a parameter of an object included in image data,
A learning image input step in which the input means inputs digital image data for learning; and
The feature extraction means extracts, from the digital image data input in the learning image input step , differential values in the vertical, horizontal, tanned, and licked down directions at each position of the image, and the gradient strength at each position. A first feature extracting step of extracting a value obtained by normalizing the value and taking an absolute value as a feature of the object;
A subspace group in which the subspace group creation means calculates a subspace of the object by principal component analysis in each of a plurality of predetermined regions based on the feature of the object extracted in the first feature extraction step Creation steps,
A parameter estimation function group creating means for creating a function for estimating a parameter by support vector regression in each of the plurality of regions; and
A subspace group storage means for storing a subspace of the object calculated in the subspace group creation step in a memory;
A parameter estimation function group storage means for storing a parameter estimation function of the object created in the parameter estimation function group creation step in a memory;
A target image input step in which the input means inputs the digital image data of the parameter estimation target;
The feature extraction means extracts, from the digital image data input in the target image input step , differential values in the vertical, horizontal, tanned, and licked down directions at each position of the image, and the gradient strength at each position. A second feature extracting step of extracting a value obtained by normalizing the value and taking an absolute value as a feature of the object;
A subspace group projecting means for projecting the feature amount extracted in the second feature extracting step onto the subspace stored in the subspace group storing means in each region;
A parameter estimation unit that estimates a parameter of the feature amount projected by the subspace group projection step in each region based on the function stored in the parameter estimation function group storage unit;
An estimation result integration means for selecting a most probable value from the parameter values estimated in the respective regions by the parameter estimation step; and
Output means, Bei example and an output step of outputting the estimation result selected by the estimation result integration step,
The feature vector x representing the feature of the object is set to x _{α, k} , x _{β, k} , x _{χ, k} and x _{δ, k as} the k th element of the feature vector x , and a _k corresponds to the k th position. Differential value in the horizontal direction, b _k is the vertical differential value of the k-th corresponding position , c _k is the slanted upper right differential value of the k-th corresponding position , and d _k is the k-th corresponding position. The differential value of the licked lower right direction,
A parameter estimation method characterized by being of the formula: The parameter estimation method according to claim 4 , wherein the estimation result integration step uses a median. The parameter estimation method according to claim 4 , wherein the estimation result integration step uses a weighted median, and uses a reliability defined in advance as a weight. The parameter estimation program which makes a computer perform each step of any one of Claims 4 thru | or 6 . A computer-readable recording medium having recorded thereon a parameter estimation program for causing a computer to execute the steps according to any one of claims 4 to 6 .