JP4463016B2 - Image analysis apparatus, image analysis method, image analysis program, and recording medium - Google Patents

Description

The present invention relates to an image analysis technique that can be applied to object extraction, similar image search, and the like, and an image analysis apparatus that analyzes a texture feature of an image and extracts a predetermined region based on a periodic variation feature of image density, The present invention relates to an image analysis method, an image analysis program, and a recording medium storing the program.

  The main problem of area division for a texture image is to divide the texture image into areas having a high degree of freedom under evaluation on uniformity. Conventionally, modeling using a Markov random field related to image density is frequently used for extracting texture features. In particular, second-order Gaussian Markov random field models are often used.

  However, texture features captured by a second-order Gaussian Markov random field model related to image density tend to be limited to a relatively short period.

  Recently, analysis in the space-frequency domain has attracted attention as a method for capturing global features composed of texture features over different scales. In particular, wavelet frames have been widely used as a shift-invariant analysis means for texture images. There is also a report that the Markov random field model in the space-frequency domain is very effective for texture recognition.

  Of the image area divisions, when the object is to cut out an object, it is not desirable that the image is divided into a large number of small areas based on local features that capture only short-period textures. This is because the area corresponding to the object cannot be divided into several small areas.

  Also, simply applying an analysis method such as wavelet frame decomposition or Markov random field model in the space-frequency domain as it is cannot always accurately cut out the entire region corresponding to the object with a high degree of freedom. This is because even if a global feature is captured, the region of the object cannot always be expressed by a single model, and the region for constructing the model itself is unknown.

The present invention relates to an image analysis technique that can be applied to such cut or similar image retrieval object, based on the periodic variation characteristics of the image density, analyzing the texture features of an image, the image analyzing device for extracting a predetermined region, An object of the present invention is to provide an image analysis method, an image analysis program, and a recording medium storing the program .

The image analysis apparatus of the present invention includes: a wavelet frame decomposition unit that converts image density fluctuations into a space-frequency domain coefficient using a wavelet frame; and a KL divergence between a coefficient converted by the wavelet frame decomposition unit and a neighboring region. predetermined integration and division of the predetermined region by performing iteratively until a predetermined end condition is satisfied, a region dividing means for dividing an area of the image, in a predetermined area obtained by the above area division based on the model parameter estimation unit and a predetermined boundary shape according to the probability of occurrence of each model parameter estimated by the model parameter estimation means for estimating the model parameters in the frequency domain - the number of introduced Markov random field model space and a boundary shape correction means for correcting the said model parameters And the boundary shape correcting means repeatedly execute the estimation of the model parameter and the recorrection of the boundary shape based on the boundary shape corrected by the boundary shape correcting means until a predetermined end condition is satisfied. It is what I did.

Alternatively, the area dividing means, after executing repeatedly the integration and division of the predetermined region until a predetermined end condition is satisfied, based on the feature vectors to the average energy of each sub-band as elements, each region better to perform repeated until a predetermined end condition is satisfied and integration division of the predetermined region by clustering.

Further, Markov probability that the model parameter estimation means, for each predetermined region, to calculate the model parameters while sequentially increasing the number of clusters, and evaluated based on the relative merits of the model parameters to a predetermined information criterion, introduced The number of field models may be determined to be the first m that gives a higher evaluation when the number of clusters is m than when m + 1 .

The image analysis method of the present invention includes a wavelet frame decomposition step for converting image density variation into a space-frequency domain coefficient using a wavelet frame, and a KL divergence between the coefficient converted by the wavelet frame decomposition step and a neighboring area. predetermined integration and division of the predetermined region by performing iteratively until a predetermined end condition is satisfied, a region dividing step of dividing an area of the image, in a predetermined area obtained by the above area division based on the introducing a Markov random field model space in the number of - the modeling step of estimating the model parameters in the frequency domain, the boundary to correct the predetermined boundary shape according to the probability of occurrence of each model parameter estimated in the modeling step and a shape modification step, the Moderupa By the meter estimation step and the boundary shape correction step, the estimation of the model parameter and the re-correction of the boundary shape based on the boundary shape corrected by the boundary shape correction step are repeatedly performed until a predetermined end condition is satisfied. It is what I did .

Alternatively, the in region dividing step, after performed iteratively and integration division of the predetermined region until a predetermined end condition is satisfied, based on the feature vectors to the average energy of each sub-band as elements, each region better to perform repeated until a predetermined end condition is satisfied and integration division of the predetermined region by clustering.

In the model parameter estimation step , a model parameter is calculated while sequentially increasing the number of clusters for each predetermined region, the superiority or inferiority of the model parameter is evaluated based on a predetermined information criterion, and the Markov random field to be introduced The number of models may be determined to be the first m that gives a higher evaluation when the number of clusters is m than when m + 1 .

The image analysis program of the present invention is an image analysis program for causing a computer to execute each step in the above-described image analysis method .
The present invention also provides a computer-readable recording medium in which the above program is recorded.

More image analyzer, image analyzing method, according to the image analysis program, and a recording medium, Ri Na As can cut the object based on the texture feature further, by utilizing the obtained area information and area features, The effect that a similar image search etc. is realizable is acquired.

  That is, in the present invention, the rough region division based on the analysis by the wavelet frame is first performed, and then a predetermined number of Gaussian Markov random field models are configured in the space-frequency region for each predetermined divided region. Further, based on those models, the object is cut out by repeatedly correcting the boundary shape of the region corresponding to the object and updating the model parameters. By using the region information and region features obtained in this way, it is possible to perform a similar image search and the like with high speed and high accuracy.

  An embodiment of the present invention will be described.

  FIG. 1 is a block diagram showing a hardware configuration of an image analysis apparatus 11 according to an embodiment of the present invention.

  As shown in FIG. 1, the image analysis apparatus 11 is a computer such as a personal computer, and these are connected to a bus 100, a CPU 111, a memory 112, a magnetic storage device 113, an input device 114, and a display device. 115, a storage medium reader 116, and a communication interface 117.

  The CPU 111 performs overall control of the entire image analysis apparatus 11, and the memory 112 includes a ROM in which system programs, various drivers, various setting values, and the like are stored, and a RAM in which a work area of the CPU 111 is secured. The magnetic storage device 113 is a hard disk, the input device 114 is a mouse or a keyboard, the display device 115 is an LCD or a CRT, and the storage medium reader 116 is a storage medium MM (optical disk such as CD or DVD, magneto-optical disk, The communication interface (I / F) 117 is a communication device with the network 200 such as the Internet.

  In the present embodiment, the image analysis program according to the present invention is stored in the magnetic storage device 113. This image analysis program can be read from the storage medium reading device 116 from the storage medium MM and installed in the magnetic storage device 113, or can be downloaded from the network 200 and installed in the magnetic storage device 113. By this installation, the image analysis apparatus 11 becomes operable. This image analysis program may be a part of specific application software, or may operate on a predetermined OS.

  Hereinafter, the contents of the processing (steps S101 to S113) performed by the image analysis apparatus 11 based on the image analysis program will be described.

  FIG. 2 is a flowchart illustrating an outline of processing performed by the image analysis apparatus 11. The CPU 111 executes a wavelet frame decomposition program, a region division program, a model parameter estimation program, and a boundary shape correction program stored in the memory 112, so that the wavelet frame decomposition unit, region division unit, model parameter estimation unit, and The function as the boundary shape correcting means can be achieved. That is, the wavelet frame decomposition means converts the fluctuation of the image density into a space-frequency domain coefficient by the wavelet frame, the area dividing means divides the area of the image by the division integration method, and the model parameter estimation means is the area dividing A predetermined number of Markov random field models are introduced into a predetermined region obtained by the above-described method to estimate model parameters in the space-frequency domain, and the boundary shape correcting means generates each model parameter estimated by the model parameter estimating means. A predetermined boundary shape is corrected according to the probability. The wavelet frame decomposition means, the area dividing means, the model parameter estimating means, and the boundary shape correcting means are realized by a wavelet frame decomposition step, a region dividing step, a model parameter estimating step, and a boundary shape correcting step.

  As shown in FIG. 2, the CPU 111 first initializes a count value n of a predetermined counter to a predetermined integer (S101). Next, wavelet frame decomposition is performed up to a predetermined decomposition level J by wavelet frame decomposition A1 (S102).

In S103, it compares the upper limit _{N 2} of the count value n and the number of repetitions. Here, N ₂ is a predetermined natural number. When the count value n has not reached _{N 2} is _(n <N 2: "Y" in S103), and divides the target image G of interest in ^{2 2n} pieces of rectangular tiles (invention region) (S104) . As a result, for each decomposition level, the same number of wavelet coefficients respectively corresponding to the four types of subbands LL, HL, LH, and HH are obtained as the number of pixels of the tile. On the other hand, when the count value n has reached N ₂ (“N” in S103), the process proceeds to S113.

In the next S105, and compares the upper bound _{N 1} of the counter n and number of repetitions. Here, N ₁ is a predetermined natural number. When n does not reach N ₁ (n <N ₁ : “Y” in S105), each tile is labeled (S106), n is incremented by 1 (S107), and the process returns to S103. . On the other hand, when n reaches N ₁ (n = N ₁ : “N” in S105), the process proceeds to S108.

In S108, it determines difference between the upper limit _{N 1} of the counter n and number of repetitions. When n is equal to N ₁ (n = N ₁ : “Y” in S108), a predetermined area considered to be included in the object of interest (hereinafter referred to as “selected area”) is selected (S109). Proceed to S110. Here, the selection area is set by selecting the area label closest to the center of the image from the area labels set in S106. However, area labels whose number of pixels is less than the predetermined number are excluded from selection area candidates, and the entire process is terminated when there is no candidate area label.

When a plurality of area labels are equidistant from the center of the image, the area label having the largest number of pixels is selected from them. Furthermore, if the distance and the number of pixels are equal, an arbitrary region label is selected from them. On the other hand, when n and N ₁ are different (“N” in S108), the process proceeds to S110.

In S110, a predetermined feature vector a _xy is extracted for each tile. Here, a _xy is a 3J + 1 dimensional number vector having the average energy of each subband in the space-frequency domain of the tile xy as an element. In next S111, m cluster centers are set from the feature vectors in the selected region. Here, m is a predetermined natural number. Subsequently, clustering is performed on all tiles (S112), and the process returns to S107.

  In S113, a boundary based on a Gaussian Markov random field model between a predetermined region (hereinafter referred to as a region of interest) obtained from the result of clustering in S112 and a predetermined region around it (hereinafter referred to as a peripheral region). The shape is corrected (S113), and the process ends.

FIG. 3 is a flowchart for explaining the labeling process in S106. The region label is set based on the degree of separation between the four neighboring tiles. In this process, first, for all tiles xy (x = 0, 1,..., N _x −1: y = 0, 1,..., N _y −1), the neighboring tiles xy + τ (τ ∈τ {(-1, 0), (0, -1), (1, 0), (0, 1)}) and the degree of separation D (x, y, τ) by the following equation (1) Obtain (S201).

Here, d _{** (j)} {h _xy | h _{xy + τ} } represents the tile xy and the neighboring tile xy + in the subband ** at level j.

Density function in tile xy with respect to wavelet coefficient c

And neighboring tiles xy +

This corresponds to a pseudo distance with the density function h _{xy + τ} (c) in FIG. Note that the same definition applies to d _{LL (j)} {h _xy | h _{xy + τ} }. That is,

Then,

It becomes.

  In S202, a threshold value θ that serves as a reference when setting the region label is set according to the condition of the following equation (5) from the moving average r of the frequency distribution (histogram) of the separation degree.

Here, Δt ₁ represents a predetermined interval of the histogram, t represents the number of each section, and α ₁ represents a predetermined integer. In next step S203, variables x and y representing the tile positions are initialized to 0, and 0 is _assigned to all tiles as an initial value of the region label l _xy .

In S204, a determination on l _xy is made. When l _xy is equal to 0 (“Y” in S204), a set of neighboring tile labels l _{xy + τ} satisfying D (x, y, τ) <θ is defined as L _xy (S205), and the process proceeds to S206. On the other hand, if l _xy is not equal to 0 (“N” in S204), the process proceeds to S208.

In _S206, it is determined for _{L xy.} If L _xy is not an empty set (“Y” in S206), a region label is set by the following equation (6) (S207).

Here, l _max represents the maximum value of used labels. When an existing region label that is not 0 is changed by Equation (6), the label of each tile having the same label is changed to min {l _{xy + τ} εL _xy }.

In S208, the determination about x is performed. When x does not reach the upper limit N _x (“Y” in S208), x is incremented by 1 (S209), and the process returns to S204. On the other hand, when x reaches the upper limit N _x (“N” in S208), the process proceeds to S210.

In S210, the determination about y is performed. If y does not reach the upper limit N _y (“Y” in S210), 0 is substituted for x, y is incremented by 1 (S211), and the process returns to S204. On the other hand, if y reaches the upper limit N _y (S210 "N") of, in terms of the number of the corresponding pixels are initialized to 0 the value of the label less than the predetermined value (S212), ends the labeling process Then, the process proceeds to S107.

  FIG. 4 is a flowchart for explaining the clustering process in S112. Clustering is performed based on the subspace method. In this process, first, for each cluster set in S111, eigenvalues and eigenvectors representing a subspace of the cluster are obtained by KL expansion based on the minimum mean square error criterion. (S301).

In S302, the similarity S _i with the partial space related to the cluster i is obtained for all tiles xy by the following equation (7).

Here, o _i is an arbitrary integer representing the number of upper eigenvalues in the partial space related to cluster i. W _iq represents an eigenvector corresponding to the qth upper eigenvalue in the partial space related to cluster i.

  In S303, a threshold value η serving as a reference for setting a cluster is set from the moving average r of the histogram of similarity according to the condition of the following equation (8).

Here, Δt ₂ represents a predetermined interval of the histogram, and α ₂ represents a predetermined integer. In the next S304, variables x and y representing tile positions are initialized to 0, and 0 is _assigned to all tiles as the initial value of the cluster number b _xy .

In S305, a determination is made for l _xy and max {S _i (a _xy )}. Here, select the label of the selected area

It will be expressed as l _xy is

And max {S _i (a _xy )} is larger than the threshold η _i (“Y” in S305), the cluster number b _xy of the tile _xy is

Is substituted (S306), and the process proceeds to S307. On the other hand, l _xy is

Or max {S _i (a _xy )} is equal to or smaller than the threshold η _i (“N” in S305), the process proceeds to S307.

In S307, the determination for x is performed. When x does not reach the upper limit N _x (“Y” in S307), 1 is added to x (S308), and the process returns to S305. On the other hand, when x reaches the upper limit N _x (“N” in S307), the process proceeds to S309.

In S309, the determination for y is performed. When y does not reach the upper limit N _y (“Y” in S309), 0 is substituted for x, 1 is added to y (S310), and the process returns to S305. On the other hand, ( "N" in S309) if y reaches the upper limit _{N y,} the process proceeds to S311.

In S311, the label value

Cluster number adjacent to 8 neighbors of tile with

The tile label value of

Change to This process is recursively repeated until there are no more tiles whose label values are to be changed. When this process is completed, the clustering process is terminated, and the process proceeds to S107.

FIG. 5 is a flowchart for explaining the boundary shape correction processing in S113. In this process, first, an attention area to be cut out as an object and its peripheral area are set (S401). For attention areas, label values

Set the tile with. For the peripheral area, a predetermined number of areas are set in order from the tile having the other label value, in the order of the area of the same label area.

  In S402, Gaussian Markov random field model parameters are estimated in the attention area and each peripheral area. At this time, the optimum number of models in each region is determined using a predetermined information criterion such as a minimum description length criterion. Specifically, first, the pixels in the region are divided into m clusters. The feature vector used at this time is a 3J + 1-dimensional number vector whose elements are wavelet coefficients of each subband. When clusters are determined, temporary model parameters are estimated for each cluster. Thereafter, based on the estimated model parameter, a cluster having the maximum occurrence probability is obtained for all the pixels in the region. According to this result, the cluster to which each pixel belongs is updated, and the processing after the provisional model parameter estimation is continued until there is no cluster update. However, when the number of repetitions reaches a predetermined value, the model parameter at that time is adopted, and the superiority or inferiority of the model parameter when divided into m + 1 clusters is determined according to a predetermined information amount criterion. If the evaluation based on the information amount criterion is high when divided into m clusters, m models are adopted. On the other hand, if the evaluation based on the information amount criterion is high when divided into m + 1 clusters, 1 is added to m and the above process is repeated again. Note that a predetermined upper limit is set for the number of clusters. The number of free parameters is the number obtained by excluding the variance parameter from the parameter set. In next step S403, a variable flag for determining the inside / outside of the boundary pixel of the region of interest is initialized to 1, a variable e indicating whether or not a label is changed is set to 1, and a variable z indicating the number of repetitions is initialized to 0.

  In S404, 0 is substituted into variables u and v representing pixel positions, and -1 is applied to flag. In the next step S405, the pixel uv is determined. Specifically, when the following condition is satisfied, uv is determined as the boundary pixel of the attention area.

Here, τ ₁ = (1, −1), τ ₂ = (1,0), τ ₃ = (1,1), τ ₄ = (0,1), τ ₅ = (− 1,1), τ ₆ = (− 1, 0), τ ₇ = (− 1, −1), and τ ₈ = (0, −1). When uv is a boundary pixel of the attention area (“Y” in S405), a model model _max (uv) that maximizes the occurrence probability based on the model parameter obtained in S402 and a corresponding area label l _new are obtained (S406). ), And proceeds to S407. On the other hand, if uv is not a boundary pixel of the attention area (“N” in S405), the process proceeds to S408.

  In S407, the label of the pixel uv is updated according to the following condition.

(1) When flag = −1 and l _new corresponds to the region of interest, l _new is substituted into l _xy and 0 is substituted into e, and the process proceeds to S408.

(2) When flag = −1 and l _new corresponds to the peripheral region, the process proceeds to S408.

(3) When flag = 1 and l _new corresponds to the peripheral region, 0 is substituted into l _xy and 0 is substituted into e, and the process proceeds to S408.

(4) When flag = 1 and l _new corresponds to the region of interest, the process proceeds to S408.

In S408, the determination for u is performed. If u does not reach the upper limit N _u (“Y” in S408), 1 is added to u (S409), and the process returns to S405. On the other hand, if _u has reached the upper limit Nu (“N” in S408), the process proceeds to S410.

In S410, the determination for v is performed. v may not reach the upper limit _{N v} (S410 "Y"), and adds 1 to v with 0 is substituted into u (S411), the flow returns to S405. On the other hand, if _v reaches the upper limit Nv (“N” in S410), the process proceeds to S412.

In S412, the variable z is compared with the upper limit Z of the number of repetitions, and the variable e is determined. Here, Z is a predetermined natural number. When z is less than Z and e is 0 (“Y” in S412), the model to which the pixel uv belongs is changed according to model _max (uv), the model parameters are re-estimated (S413), and the processing after S404 is performed. repeat. On the other hand, when z is equal to or greater than Z or e is not 0 (“N” in S412), the region boundary shape correction process is terminated, and all the processes in the proposed method are terminated.

It is a block diagram which shows the hardware constitutions of the image analysis apparatus which is one Embodiment of this invention. 2 is a flowchart illustrating an outline of processing performed by the image analysis apparatus in FIG. 1. It is a flowchart explaining the labeling process in the flowchart of FIG. It is a flowchart explaining the clustering process in the flowchart of FIG. It is a flowchart explaining the correction process of the boundary shape in the flowchart of FIG.

Explanation of symbols

11 Image Analysis Device 100 Bus 111 CPU
112 memory 113 magnetic storage device 114 input device 115 display device 116 storage medium reader 117 communication interface 117

  200 network

Claims (8)

Wavelet frame decomposition means for converting fluctuations in image density into space-frequency domain coefficients using wavelet frames;
Based on the coefficients transformed by the wavelet frame decomposing means and the KL divergence between neighboring regions, division and integration of a predetermined region are repeatedly performed until a predetermined end condition is satisfied , whereby the region of the image is Area dividing means for dividing;
Model parameter estimation means for estimating a model parameter in the space-frequency domain by introducing a predetermined number of Markov random field models into the predetermined region obtained by the region division;
Boundary shape correcting means for correcting a predetermined boundary shape according to the occurrence probability of each model parameter estimated by the model parameter estimating means,
The model parameter estimating means and the boundary shape correcting means repeat the estimation of the model parameters and the recorrection of the boundary shape based on the boundary shape corrected by the boundary shape correcting means until a predetermined end condition is satisfied. An image analysis apparatus characterized by being executed . Said area dividing means, after executing repeatedly joining and the splitting of the predetermined region until a predetermined end condition is satisfied, based on the feature vectors to the average energy of each sub-band and elements, clustering the areas The image analysis apparatus according to claim 1, wherein division and integration of a predetermined area are repeatedly performed until a predetermined end condition is satisfied. The model parameter estimation means calculates a model parameter while sequentially increasing the number of clusters for each predetermined region, evaluates the superiority or inferiority of the model parameter based on a predetermined information criterion, and introduces a Markov random field model to be introduced 3. The image analysis apparatus according to claim 1, wherein the number m is determined to be the first m that gives a higher evaluation when the number of clusters is m than when m + 1 . A wavelet frame decomposition step for converting image density variation into a space-frequency domain coefficient by wavelet frames;
By repeatedly carrying out up and integration division of the predetermined area a predetermined end condition is satisfied based on the KL divergence between the wavelet frame converted by cracking step coefficients and the neighboring region, the area of the image An area dividing step to divide;
A modeling step of estimating a model parameter in a space-frequency domain by introducing a predetermined number of Markov random field models into a predetermined region obtained by the region division;
And a boundary shape modification step of modifying the predetermined boundary shape according to the probability of occurrence of each model parameter estimated in the modeling step,
By the model parameter estimation step and the boundary shape correction step, the estimation of the model parameter and the re-correction of the boundary shape based on the boundary shape corrected by the boundary shape correction step are repeated until a predetermined end condition is satisfied. An image analysis method characterized by being executed . Wherein in the area dividing step, after the split and integration of the predetermined termination condition of the predetermined area is performed iteratively until satisfied, based on the feature vectors to the average energy of each sub-band and elements, each region 5. The image analysis method according to claim 4, wherein division and integration of a predetermined area are repeatedly performed by clustering until a predetermined end condition is satisfied. In the modeling step , model parameters are calculated while sequentially increasing the number of clusters for each predetermined region, the superiority or inferiority of the model parameters is evaluated based on a predetermined information criterion, and the Markov random field model to be introduced 6. The image analysis method according to claim 4 , wherein the number is determined to be the first m that gives a higher evaluation when the number of clusters is m than when m + 1 . An image analysis program for causing a computer to execute each step in the image analysis method according to claim 4 . A computer-readable recording medium on which the program according to claim 7 is recorded .