JP6783207B2 - Object head region extractor and method - Google Patents

Description

The present invention relates to a head region extraction device and a method for extracting a head region of an object from an arbitrary input image.

Patent Document 1 discloses a method for quickly and accurately detecting a head in an image by using a preliminary head detection model.

Non-Patent Document 1 describes a method of detecting a head by using HOG as a cascade classifier consisting of a low-resolution single-headed model classifier of the first stage and a partial-based model trained for detection of the previous stage. Is disclosed.

Non-Patent Document 2 discloses a technique for detecting the positions of the head and shoulders using Haar features and HOG features in order to count the number of people. In Non-Patent Document 2, so-called joint HOG features are created based on the detected symmetric structures of the head and shoulders, negative samples are excluded using a Haar classifier, from which the final head and shoulders are finalized. The position is estimated.

Japanese Patent Application No. 2012-522506

E. Rehder, H. Kloeden, and C. Stiller, "Head Detection and Orientation Estimate for Pedestrian Safety," IEEE 17th Int. Conf. On ITSC, 2014 L. Chen, H. Wu, S. Zhao, and J. Gu, "Head-shoulder detection using joint HOG features for people counting and video surveillance in library," IEEE Workshop on ECA, 2014.

Non-Patent Documents 1 and 2 do not examine the problem of how to identify each head when occlusion occurs between a plurality of heads. In Patent Document 1, a plurality of heads can be detected in an image, but the detection ability is lowered when the prominence of the texture / color of the heads is lower than that of the background.

An object of the present invention is to solve the above technical problems and to recognize the head region of an object at the time of object recognition, so that even if occlusion occurs between the objects and both cannot be distinguished, the position of the head region can be determined. The purpose is to be able to estimate the position and posture of each object based on the posture.

In order to achieve the above object, the present invention is characterized in that the head extraction device and method for extracting the head region of an object from an arbitrary input image have the following configurations.

(1) A head learning means for learning a texture pattern and an edge pattern of a known head image, and a texture matching means for extracting a head region candidate by performing a matching process based on the texture pattern on an input image. The head region candidate is provided with a pattern matching means for determining the head region by performing a matching process based on the texture pattern and the edge pattern.

(2) A head region candidate integration means for integrating a plurality of head region candidates that are close to each other into one based on the position information is further provided.

(3) The head learning means obtains each pixel value for each head image, calculates the pixel value average and pixel value dispersion for each pixel position, and textures the pixel value average and pixel value dispersion obtained for each pixel position. I tried to learn as a pattern.

(4) The head learning means converts the head image into the CIELab color space, and the means for calculating the pixel value averaging and pixel value variance calculates the averaging and variance of each channel value of dimension L and complementary color dimensions a and b. I tried to calculate.
apparatus.

(5) The head learning means includes a means for calculating a gradient value for each pixel block of each head image and a means for calculating a gradient value average and a gradient value variance for each position of the pixel block, and includes a gradient value. An edge pattern is learned in which the gradient value of the pixel block position where the variance is lower than a predetermined reference value is represented by the gradient value average.

(6) The texture matching means is a means for identifying whether or not the difference between each pixel value of the input image and the corresponding pixel value average of the texture pattern is less than a predetermined value in relation to the corresponding pixel value variance. , A means for sequentially shifting a predetermined search range on the input image and calculating the proportion of pixels not below the predetermined value for each search range, and the proportion of pixels not below the predetermined value is the reference value. The search range below is extracted as a head region candidate.

(7) The pattern matching means is a means for calculating the first spatial variance matrix for each head region candidate based on the texture pattern and edge pattern of each head region candidate, and the first based on the learned texture pattern and edge pattern. A means for calculating the two-spatial variance matrix is provided, and a head region candidate whose difference between the first spatial variance matrix and the second spatial variance matrix is less than a predetermined reference value is determined as the head region.

According to the present invention, the following effects are achieved.

(1) Since the head area of an object is recognized during object recognition, the position and posture of each object are estimated based on the position and posture of the head area even if occlusion occurs between the objects and the two cannot be distinguished. become able to.

(2) For any input image, the search range is limited by first extracting the head area candidates by texture matching, and then the space using the texture pattern and edge pattern only for the head area candidates. Since the head region is determined by performing a search based on the target correlation, the head region of the object can be accurately extracted with a small processing load.

It is a functional block diagram which showed the structure of the main part of the head extraction apparatus of the object which concerns on one Embodiment of this invention. It is a figure which showed the extraction example of the head image for learning. It is a flowchart which showed the learning procedure of a texture pattern. It is a flowchart which showed the learning procedure of an edge pattern. It is a flowchart which showed the procedure of texture matching. It is a flowchart which showed the procedure of pattern matching. It is a figure which showed the integration method of a head region candidate (the 1). It is a figure (No. 2) which showed the integration method of a head region candidate.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional block diagram showing a configuration of a main part of an object head region extraction device according to an embodiment of the present invention. Such a device can be configured by implementing an application (program) that realizes each function on a general-purpose computer or server. Alternatively, it can be configured as a dedicated machine or a single-purpose machine in which a part of the application is made into hardware or ROM.

The head region extraction device of the present invention is an arbitrary head learning unit 1 that learns the characteristics of the head based on a large number of head images prepared in advance known to be the head region, and an arbitrary based on the learning result. It is composed of a head identification unit 2 that identifies a head region from the input image of. As shown in FIG. 2, the database 3 stores a large number of head images extracted manually, for example, from a billboard relating to a large number of objects (humans).

In the head learning unit 1, the head pattern generating unit 101 includes a texture pattern learning unit 101a and an edge pattern learning unit 101b. The texture pattern learning unit 101a learns the texture pattern of the head by calculating the pixel value average and the pixel value variance for each of the coordinate positions x and y of the head image acquired from the database 3. The texture pattern learning result is stored in the texture pattern storage unit 102.

The edge pattern learning unit 101b performs edge detection and gradient calculation for each predetermined pixel block on the head image acquired from the database 3, and calculates the gradient average and the gradient variance to calculate the edge pattern of the head. To learn. The learning result of the edge pattern is accumulated in the edge pattern storage unit 103.

A mask image of an object extracted by applying, for example, the background subtraction method is input to an arbitrary object image in the head identification unit 2. The filtering unit 201 selectively removes only noise line segments from the input image of the object mask while preserving the edge components thereof. The color space conversion unit 202 converts the color space of the input image from the BGB space to the CIELab space in order to improve the prominence of the texture structure.

The texture matching unit 203 performs texture matching on the input image in units of predetermined pixel blocks using the learning result of the texture pattern storage unit 102, and all that are highly likely to be the head region from the input image. Extract head region candidates. In this embodiment, as a result of learning the texture pattern, as will be described in detail later, the average value u ^L , u ^a , of the dimension L of the CIELab space at each pixel position x, y and the channel values of the complementary color dimensions a, b, u ^b and variances σ ^L , σ ^a , σ ^b are used.

The head region candidate integration unit 204 integrates a plurality of head region candidates that are close to each other based on the position information. The pattern matching unit 205 determines the head region by using both the texture pattern matching and the edge pattern matching for the extracted or integrated head region candidate.

FIG. 3 is a flowchart showing a texture pattern learning procedure by the texture pattern learning unit 101a of the head learning unit 1.

In step S1, all head images are acquired from the database 3. In step S2, the head image represented in the RGB color space is converted into the CIELab color space in order to improve the prominence of the texture structure. In step S3, each head image is subjected to a filtering process to remove noise components such as background pixels.

In step S4, the channel values (L, a, b) in the CIELab color space are extracted from all the head images for each pixel position x, y, and the average value u * _xy of each channel value for each pixel position x, y. (= U ^L _xy , u ^a _xy , u ^b _xy ) is calculated based on the following equation (1). Here, F is the number of head images acquired from the database 3, and Ii * represents each channel value at each pixel position x and y of the i-th head image.

In step S5, each channel for each pixel position x, y is based on the channel value Ii * of the CIELab color space extracted for each pixel position x, y from each head image and the average value u * _xy of each of the channel values. The variance of the value σ * _xy (= σ ^L _xy , σ ^a _xy , σ ^b _xy ) is calculated based on the following equation (2).

As described above, when the mean value u * _xy and the variance σ * _xy of each channel value in the CIELab color space of the head image are obtained, in step S6, the texture pattern is based on the following equations (3) and (4). Is constructed.

FIG. 4 is a flowchart showing an edge pattern learning procedure by the edge pattern learning unit 101b of the head learning unit 1.

In step S11, all head images are acquired from the database 3. In step S12, the head image represented in the RGB color space is converted into a grayscale image. In step S13, edge information is extracted by edge filtering.

In step S14, each head image after filtering is divided into pixel blocks of 4 × 4 pixels, and an edge pattern ε _pq is constructed for each pixel block based on the HOG feature amount (here, the gradient value k). ..

In the present embodiment, the position of each pixel block is represented by a relative value from the upper left pixel block, and k _pq represents the gradient value of the p-th pixel block in the vertical direction and the q-th pixel block in the horizontal direction. Then, the relationship between the gradient value k and its variance σ is obtained for each position pq of the corresponding pixel block, and an edge pattern ε _pq is constructed for each pixel block based on the following equation (5).

In the present embodiment, for each position pq of each corresponding pixel block of each head image, it is determined whether or not all the gradient values k _pq are within ± 2σ at the position pq. The edge pattern epsilon _pq pixel blocks within ± 2 [sigma] is the average value k _'pq slope value k _pq of each pixel block in the position pq, the position of the edge pattern including a gradient value k _pq departing from ± 2 [sigma] epsilon _pq Is zero.

FIG. 5 is a flowchart showing the procedure of texture matching by the texture matching unit 203. The matching process using the texture pattern is executed to extract a head region candidate having a high probability of being the head of an object from the input image input to the head identification unit 2.

In this embodiment, when the texture patterns of the above equations (3) and (4) are given, the input image is shifted by 4 pixels with the pixel block of 24 × 24 pixels in the input image as the search range. The head candidate area is searched from all the areas of.

In step S21, the pixel value I _xy at each position (x, y) of the input image is acquired. In step S22, based on the following equation (6), the absolute value of the difference between each pixel value I _xy of the input image and the average pixel value u _xy of the texture pattern at the corresponding position is set to the texture at the corresponding position. Compare with 2σ _{xy, which} is twice the variance average of the pattern. Then, if the following equation (6) is satisfied, the process proceeds to step S23 to set the pixel value I _xy at the position (I _xy = 1), and if not, the process proceeds to step S24 and the pixel value I at the position is set. Reset _xy (I _xy = 0).

In step S25, it is determined whether or not the above processing is completed for all the pixels of the input image. If it is not completed, the process returns to step S21, and each of the above processes is executed while switching the pixel of interest.

When all the pixels of the input image are set or reset, the process proceeds to step S26, and the search range of 24 × 24 pixels is set in the upper left of the input image. In step S27, the ratio of non-zero pixels whose pixel value Ixy is not zero is calculated within the search range. In step S28, the non-zero ratio is compared with a predetermined reference value, and if the non-zero ratio is lower than the reference value, the process proceeds to step S29 and the current search range is set as a head region candidate.

In step S30, it is determined whether or not the entire area of the input image has been searched. If the search has not been completed, the process proceeds to step S31, the search range is shifted in the horizontal or vertical direction by 4 pixels to set the next search range, and then the process returns to step S27 to target the new search range. The process is repeated.

FIG. 6 is a flowchart showing a pattern matching procedure by the head region candidate integration unit 204 and the pattern matching unit 205. In the present embodiment, pattern matching based on the texture pattern and the edge pattern is performed on all the head region candidates, and finally the head region is determined.

In the pattern matching process, the texture pattern (u, σ) and the edge pattern ε learned in advance are correlated with the spatial covariance matrix ρ in pixel block units (in this embodiment, the 4 × 4 pixel pixel block).

Similarly, for each head region candidate extracted from the input image, the pixel mean value u, the pixel variance σ, and the edge pattern ε are correlated with the spatial covariance matrix ρ in the same pixel block unit. Then, each spatial covariance matrix ρ is compared for each corresponding pixel block, and a head region candidate having a low ratio of the difference becoming non-zero is determined as the head region.

In step S51, as shown in FIGS. 7 and 8, a plurality of head region candidates Hi_1 to Hi_5 having close positions are recognized by the head region candidate integration unit 204 with respect to the same head region. It is regarded as a candidate group and integrated into one. In the present embodiment, for example, the center position, the center of gravity position, or the apex coordinates are defined as the position index values (x, y) of the head region candidate. Then, all the head region candidates whose misalignment is within a predetermined range, for example, x-4 <x <x + 4 and y-4 <y <y + 4, are included in the minimum including these head region candidates. Integrate into a new head region candidate that covers the rectangular region of.

In step S52, one of the head region candidates after integration is selected. In step S53, the super pixel I _pq defined by the following equations (7) and (8) is calculated for each pixel block at the position pq of 4 × 4 pixels from the current head region candidate.

In the above equation (8), L _pq , a _pq, and b _pq represent the average value of each channel value L, a, and b in the CIELab space detected in the pixel block at the position pq. ω and Δs are the pattern coefficient size and the area coefficient distance, respectively, and in this embodiment, ω = 24 and Δs = 25 are assumed. The pattern coefficient size is determined based on the following equation (9).

Here, P is the pattern size, and S is expressed by the following equation (10). Where A is the superpixel area size and s is the superpixel size.

In step S54, the spatial covariance matrix ρ _pq is obtained by the following equation (11) as the sum of α times the edge information ε _pq and β times the texture information I _pq .

In step S55, it is determined whether or not the calculation of the spatial covariance matrix ρ _pq is completed for all the 4 × 4 pixel blocks in the head region candidate this time. If it is not completed, the process returns to step S53, and the spatial covariance matrix ρ _pq is similarly calculated for each of the remaining pixel blocks.

In step S56, based on the texture pattern (u, σ) and the edge pattern ε learned in advance, the spatial covariance matrix ρ of each pixel block calculated by the same procedure as above and the head region candidate this time were calculated. The difference between each pixel block and the spatial covariance matrix ρ _pq is calculated for each pixel block.

In step S57, it is determined whether or not the ratio of the difference between the spatial covariance matrices ρ and ρ _pq to non-zero is less than a predetermined reference value, for example, 20%. If it is less than the reference value, the process proceeds to step S58 and the current head region candidate is determined as the head region. On the other hand, if the non-zero ratio is not less than the reference value, the process proceeds to step S59 and the current head region candidate is determined as the non-head region.

In step S60, it is determined whether or not the head region determination based on the spatial covariance matrix ρ is completed for all the head region candidates. If it is not completed, the process returns to step S52, and each of the above processes is repeated for the remaining head region candidates.

According to the present embodiment, since the head area of the object is recognized at the time of object recognition, even if occlusion occurs between the objects and both cannot be distinguished, the position of each object is based on the position and posture of the head area. , You will be able to estimate the posture.

Further, according to the present embodiment, for an arbitrary input image, the search range is limited by first extracting the head region candidates by texture matching, and then the texture pattern and the texture pattern and the head region candidates are targeted only. Since the head region is determined by performing a search based on spatial correlation using an edge pattern, the head region of the object can be accurately extracted with a small processing load.

1 ... Head learning unit, 2 ... Head identification unit, 3 ... Database, 101 ... Head pattern generation unit, 101a ... Texture pattern learning unit, 101b ... Edge pattern learning unit, 102 ... Texture pattern storage unit, 103 ... Edge Pattern storage unit, 201 ... Filtering unit, 202 ... Color space conversion unit, 203 ... Texture matching unit, 204 ... Head area candidate integration unit, 205 ... Pattern matching unit

Claims (8)

In a head extraction device that extracts the head area of an object,
A head learning means for learning the texture pattern and edge pattern of the head image,
A texture matching means for extracting head region candidates by performing matching processing based on the texture pattern on an input image, and
An object head region extraction device, characterized in that the head region candidate is provided with a pattern matching means for determining a head region by performing matching processing based on the texture pattern and the edge pattern. The head region extraction device for an object according to claim 1, further comprising a head region candidate integration means that integrates a plurality of head region candidates that are close to each other into one based on the position information. .. The head learning means includes means for obtaining each pixel value for each head image and calculating the pixel value average and the pixel value variance for each pixel position.
The head region extraction device for an object according to claim 1 or 2, wherein the pixel value average and the pixel value dispersion obtained for each pixel position are learned as a texture pattern. The head learning means includes means for converting a head image into a CIELab color space.
The head region extraction device for an object according to claim 2, wherein the means for calculating the pixel value average and the pixel value variance is to calculate the average and variance of each channel value of the dimension L and the complementary color dimensions a and b. .. The head learning means
A means to calculate the gradient value for each pixel block of each head image,
A means for calculating the gradient value average and the gradient value variance for each position of the pixel block is provided.
The head region extraction device for an object according to claim 3 or 4, wherein the edge pattern in which the gradient value of the pixel block position where the gradient value dispersion is lower than a predetermined reference value is represented by the gradient value average is learned. The texture matching means
A means for identifying whether or not the difference between each pixel value of the input image and the corresponding pixel value average of the texture pattern is less than a predetermined value in relation to the corresponding pixel value variance.
It is provided with a means for sequentially shifting a predetermined search range on the input image and calculating the ratio of pixels that do not fall below the predetermined value for each search range.
The head region extraction device for an object according to claim 3, wherein a search range in which the proportion of pixels not less than the predetermined value is less than the reference value is extracted as a head region candidate. The pattern matching means
A means for calculating the first spatial variance matrix for each head region candidate based on the texture pattern and edge pattern of each head region candidate, and
A means for calculating a second spatial variance matrix based on the learned texture pattern and edge pattern is provided.
The head region extraction of the object according to claim 6, wherein a head region candidate whose difference between the first spatial variance matrix and the second spatial variance matrix is less than a predetermined reference value is determined as the head region. apparatus. In the head extraction method that extracts the head area of an object by a computer,
Learn the texture and edge patterns of the head image,
Matching processing based on the texture pattern is performed on the input image to extract head region candidates.
A method for extracting a head region of an object, which comprises performing a matching process based on the texture pattern and the edge pattern on the head region candidate to determine the head region.