JP4827403B2 - Pattern identification method and apparatus, and program thereof - Google Patents

Description

  The present invention relates to a method and apparatus for identifying a pattern of an input signal by extracting features hierarchically in image recognition, voice recognition, and the like, and a program thereof.

  There is a technique for identifying an input signal pattern by extracting features hierarchically. In this method, when extracting high-order features, feature extraction is performed using features that are lower than the features that make up the features to be extracted. There is a feature that it is possible. However, in order to increase robustness against pattern fluctuations, it is necessary to increase the types of features to be extracted, resulting in an increase in processing cost. Thus, if the number of types of features to be extracted is not increased, there is a problem that the possibility of misidentification increases.

  In response to the above problem, a feature vector of each class pattern arranged in descending order of vector components is used as a dictionary pattern, a feature vector is generated from the input pattern, and matching with the dictionary pattern up to the upper N dimensions is performed. A pattern recognition method has been proposed in which processing cost is reduced by performing matching with lower dimensions based on the result (see, for example, Patent Document 1).

Also, feature vectors are extracted from the input data, classified into clusters based on the degree of coincidence with the standard vector of each cluster, and classified into categories based on the degree of coincidence between the category standard vector and the feature vector in the cluster into which the input pattern is classified Thus, a pattern recognition dictionary creation device and a pattern recognition device that reduce the processing cost of matching have also been proposed (see, for example, Patent Document 2).
Japanese Patent Publication No.7-11819 Japanese Patent Laid-Open No. 10-11543

  However, it is desired to perform robust pattern recognition against variations in input patterns, and to perform pattern recognition with lower processing cost while reducing the possibility of erroneous identification.

  In order to solve the above-described problem, according to the present invention, a pattern identification method for hierarchically extracting features of input data and identifying a pattern of the input data includes a first feature for extracting features of the first layer. A feature extraction step, an analysis step for analyzing a distribution of feature extraction results in the first feature extraction step, and a position or likelihood of a specific feature in the first hierarchy in the distribution analyzed in the analysis step. The position or likelihood of the specific feature in order to execute the extraction of the feature in which each variation of the plurality of variation categories has occurred with respect to the feature of the second layer having the feature of the first layer as a constituent element A determination step for determining whether or not a preset condition is satisfied, and the second hierarchy feature in which the variation of the variation category determined to satisfy the condition by the determination step has occurred. And a second feature extraction step of extracting features of the hierarchy.

  In addition, according to another aspect of the present invention, a first feature extraction for extracting features of the first hierarchy in a pattern identification device that hierarchically extracts features of the input data and identifies a pattern of the input data. Means, analysis means for analyzing the distribution of the feature extraction result by the first feature extraction means, and the position or likelihood of the specific feature of the first hierarchy in the distribution analyzed by the analysis means is the first The position or likelihood of the specific feature is set in advance in order to execute the extraction of the feature in which the variation of each of the plurality of variation categories has occurred with respect to the feature of the second layer having the feature of the layer as a constituent element A determination unit that determines whether or not the determined condition is satisfied, and a feature of the second hierarchy in which the variation of the variation category determined to satisfy the condition by the determination unit is generated. And a second feature extraction means for extracting symptoms.

  According to another aspect of the present invention, a pattern identification program that causes a computer to extract features of input data in a hierarchical manner and identify a pattern of the input data is used to extract the features of the first layer. A feature extraction step, an analysis step of analyzing a distribution of feature extraction results in the first feature extraction step, and a position or likelihood of a specific feature in the first hierarchy in the distribution analyzed in the analysis step In order to execute the extraction of the feature in which each variation of the plurality of variation categories has occurred with respect to the feature of the second layer having the feature of the first layer as a constituent element, the position or likelihood of the specific feature A determination step for determining whether or not a preset condition for the degree is satisfied, and a feature of the second hierarchy in which the variation of the variation category determined to satisfy the condition by the determination step has occurred Te, and a second feature extraction step of extracting a feature of the second hierarchy.

  According to the present invention, it is possible to perform robust recognition against fluctuations in input patterns, and it is possible to perform pattern recognition with a lower processing cost while reducing the possibility of erroneous identification.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

(First embodiment)
As a first embodiment of the present invention, a method of using two-dimensional image data as input data and identifying whether the image data is in a specific category will be described.

  In the present embodiment, as the identification category, as shown in (i) to (iv) of FIG. 4, a face presence image in which the vicinity of the center of the face exists in the approximate center of the input image, and (v) of FIG. A method for identifying which of the above two categories the input image data will be described assuming a face-free image that is another image.

  In the present embodiment, whether or not the image is a face is identified. However, the present invention is not limited to this. The present invention is also applicable to cases where other image patterns and input data are audio data. Further, in this embodiment, for the sake of simplicity, only a single category of face is identified as to whether or not it is within that category. It can also be applied to identifying the.

  FIG. 1 shows a basic configuration of the pattern identification apparatus. Hereinafter, an outline of the pattern identification apparatus will be described with reference to FIG.

  The data input unit 11 in FIG. 1A inputs input data for pattern identification. The hierarchical feature extraction processing unit 12 is a processing unit that hierarchically extracts features from input data and performs pattern identification of input data, and a primary feature extraction processing unit 121 that performs primary feature extraction processing. And a secondary feature extraction processing unit 122 that performs secondary feature extraction processing. The extraction result distribution analysis unit 13 analyzes the distribution of the feature extraction results extracted by the primary feature extraction processing unit 121.

  In this pattern identification device, first, data to be subjected to identification processing is input in the data input unit 11. The hierarchical feature extraction processing unit 12 performs a hierarchical feature extraction process on the input data. In this hierarchical extraction process, first, the primary feature extraction processing unit 121 extracts a plurality of primary features hierarchically from input data. Next, at least one kind of primary feature distribution extracted by the primary feature extraction processing unit 121 is analyzed by the extraction result distribution analysis unit 13, and a secondary feature extraction process is performed based on the analysis result. In the unit 122, secondary feature extraction is performed.

  Next, FIG. 1B shows another basic configuration of the pattern identification device. Hereinafter, an outline of the pattern identification apparatus will be described with reference to FIG.

  In FIG. 1B, the data input unit 11 inputs input data for pattern identification. The hierarchical feature extraction processing unit 12 is a processing unit that hierarchically extracts features from input data and performs pattern identification of input data, and a primary feature extraction processing unit 121 that performs primary feature extraction processing. And a secondary feature extraction processing unit 122 that performs secondary feature extraction processing. The extraction result distribution analysis unit 13 analyzes the distribution of the feature extraction results extracted by the primary feature extraction processing unit 121. The category-specific likelihood calculation unit 14 is a processing unit that calculates the likelihood of each category of secondary features from the analysis result analyzed by the extraction result distribution analysis unit 13.

  In this pattern identification device, first, data to be subjected to identification processing is input in the data input unit 11. The hierarchical feature extraction processing unit 12 performs a hierarchical feature extraction process on the input data. In this hierarchical extraction process, first, the primary feature extraction processing unit 121 extracts a plurality of primary features hierarchically from input data. Next, the extraction result distribution analysis unit 13 analyzes the extraction result distribution of at least one kind of primary feature extracted by the primary feature extraction processing unit 121. Based on the result analyzed by the extraction result distribution analysis unit 13, the category-specific likelihood calculation unit calculates the likelihood of each category of the secondary features extracted by the secondary feature extraction processing unit 122. The secondary feature extraction processing unit 122 performs secondary feature extraction that belongs to a category whose calculated likelihood is equal to or greater than a predetermined value.

  FIG. 2 shows a functional configuration of the pattern identification apparatus according to the present embodiment. FIG. 3 shows the flow of processing in this embodiment. Hereinafter, processing in the present embodiment will be described with reference to FIGS. The solid arrows in FIG. 2 indicate the flow of actual signal data, and the broken arrows indicate the flow of command signals such as operation instructions rather than actual signal data. The same expression is used in FIGS. 8 and 10 described later.

  First, in step S301, image data to be identified is input from the image input unit 21. As the input image data, a grayscale image is used in the present embodiment, but an RGB color image or the like may be used.

  In step S302, the initial feature extraction unit 22 extracts at least one initial feature such as an edge in a specific direction in the input image. In step S303, the local feature extraction unit 23 uses the initial feature extracted by the initial feature extraction unit 22 to extract local features such as edge line segments having specific lengths, end points of the edge line segments, and the like. . In step S304, the partial feature extraction unit 24 extracts partial features such as eyes and mouth using the local features extracted by the local feature extraction unit 23.

  In step S305, the partial feature distribution determining unit 25 analyzes the distribution of the partial features extracted by the partial feature extracting unit 24 in the image. In step S306, the partial feature distribution determination unit 25 issues an activation command to the face extraction unit 26 according to the analysis result, and turns on the flag of the face extraction module to be activated.

  The face extraction unit 26 is a processing unit that performs face extraction using the partial features extracted by the partial feature extraction unit 24. The face extraction unit 26 includes a plurality of modules that perform face extraction corresponding to a specific size and orientation, and performs face extraction only for the module that has received an activation command. In steps S307 to S309, face extraction processing is sequentially performed by the face extraction module whose flag is turned on, and the flag of the face extraction module that performed the face extraction is turned off. When there is no face extraction module with the flag turned on, the face extraction process is terminated.

  In Steps S310 to 311, the detection result output unit 27 integrates the face extraction results by the face extraction module to determine whether the input image is a face presence image or a face absence image, and the result is obtained. Output.

  Hereinafter, processing in each processing unit after the initial feature extraction unit 22 on the image data input in the image input unit 21 will be described in detail.

  The initial feature extracted from the input image in the initial feature extraction unit 22 is preferably a feature that is a constituent element of the feature extracted in the local feature extraction unit 23 that is a higher layer. In this embodiment, filtering processing is performed at each position of the input image using a differential filter in a vertical direction, a horizontal direction, a diagonally upward right direction, and a diagonally upward left direction, and a vertical edge, a horizontal edge, a diagonal edge, etc. The four types of features are extracted. In the present embodiment, the filtering process as described above is performed. However, a feature may be extracted by performing template matching at each position of the input image using a template image or the like indicating an initial feature prepared in advance. I do not care.

  The feature extracted here is held as information such as the type of the feature, the position in the image, the likelihood of the feature to be extracted, and the feature detection level. In this embodiment, at this stage, features as shown in (a) to (d) of FIG. 5 are extracted from the input image. 5A shows the extraction result of the vertical edge, FIG. 5B shows the horizontal edge, FIG. 5C shows the right diagonal edge, and FIG. 5D shows the left diagonal edge extraction result.

  In FIG. 5, a position where the result of filtering at each position of the image is 0 is gray, a positive value is represented as a high luminance value, and a negative value is represented as a low luminance value. That is, in the image of FIG. 5, the position where the luminance value is shown is the position where the edge in the direction corresponding to the type of each filter is extracted. On the contrary, the position where the luminance value is low is a position where an edge in the direction opposite to the direction corresponding to the type of each filter exists. Further, a gray portion that is an intermediate value of luminance indicates a position where no edge is extracted.

  Here, since a differential filter is used for feature extraction, the absolute value of the filtered value indicates the sharpness of the edge. That is, in the input image, the position where the change in the luminance value is large in the direction according to the type of filter is shown as a high luminance value or a low luminance value.

  In the local feature extraction unit 23, the local feature extracted using the initial feature extraction result extracted by the initial feature extraction unit 22 is a partial feature that is a higher layer, similar to the feature extracted by the initial feature extraction unit 22. It is desirable that the feature is a constituent element of the feature extracted by the extraction unit 24.

  In the present embodiment, the partial feature extraction unit 24 extracts the eyes and mouth, and therefore, the local feature extraction unit 23 indicates the portion surrounded by a circle in (1-a) to (1-d) in FIG. As the end points of the edge line segment corresponding to the corners of the eyes, the corners of the eyes, the ends of the mouth, etc., the two types of features of the left end point and the right end point, the features corresponding to the upper part of the eye and the upper lip, Edge line segments having two specific lengths of corresponding features are extracted.

  (1-a) to (1-d) in FIG. 6 are initial feature extraction results at a position where the left end point (the left eye's head in the figure) exists. (1-a) is a vertical edge extraction result, (1-b) is a horizontal edge, (1-c) is a right diagonal edge, and (1-d) is a left diagonal edge extraction result. Others (2-a) to (2-d) are extraction results of respective initial features (in order, vertical, horizontal, right diagonal, left diagonal edges) at the position where the right end point (the end point of the mouth in the figure) exists, (3-a) to (3-d) are the upper part of the eye and upper lip (the upper part of the right eye in the figure), and (4-a) to (4-d) are the lower part of the eye and the lower lip (the lower part of the lip are shown). Is an extraction result of each initial feature (in order, vertical, horizontal, right diagonal, left diagonal edge) at the position where the

  In the present embodiment, as a feature extraction method, a unique two-dimensional mask is prepared in advance for each feature extracted by the initial feature extraction unit 22, and is shown in (a) to (d) of FIG. Filtering processing (convolution calculation) is performed using a two-dimensional mask specific to a feature to be extracted prepared in advance at each position of each feature extraction result. Then, by extracting the filtering results for each initial feature extraction result, each feature is extracted.

  The unique two-dimensional mask prepared in advance is the distribution of the initial feature extraction results at the position where the feature is to be extracted, here, for example, the feature such as the left end point, that is, (1-a ) To (1-d). That is, when the distribution of the initial feature extraction results is a distribution of the initial feature extraction results peculiar to the vicinity of the position where the features to be extracted exist, a two-dimensional mask is set so that the filtered value is high. .

  As a method for setting a two-dimensional mask, a plurality of test patterns are simply given. When the given test pattern is a feature to be extracted, each of the two-dimensional masks is set so that the result of filtering becomes a high value. If the value of the element is adjusted and the feature is not to be extracted, the value may be set by adjusting the value of each element of the two-dimensional mask so that the filtered value becomes a low value. As another method, the value of each element of the two-dimensional mask may be set by using knowledge possessed in advance.

  Similar to the initial feature extraction unit 22, the features extracted by performing the above processing are stored as information such as the type of extracted features, the position in the image, the likelihood of the features to be extracted, and the feature detection level. In the present embodiment, for each of the four types of features of the two types of end points and the two types of edge line segments having specific lengths, the extracted feature positions and the two-dimensional mask specific to each feature are used. Each initial feature is filtered, and the result is integrated and recorded as the likelihood of the feature.

  The processing in the partial feature extraction unit 24 is the same as the processing in the local feature extraction unit 23. From the extraction results of a plurality of local features extracted in the local feature extraction unit 23, which is the feature extraction result of the lower layer, Extract features. Similarly, the partial feature to be extracted is desirably a feature extracted by the face extraction unit 26 which is an upper layer, that is, a feature which is a constituent element of the face in the embodiment.

  In the present embodiment, as described above, the partial feature extraction unit 24 extracts eyes, mouths, and the like. The extraction process is the same as the extraction method in the local feature extraction unit 23, and the feature may be extracted by filtering using a specific two-dimensional mask. Or, simply by extracting the eyes and the mouth depending on whether or not the features having the likelihood of a certain value or more among the feature extraction results in the local feature extraction unit 23 are in a specific spatial arrangement relationship. Also good.

  The eyes and mouth extracted as described above are also stored as information such as the type of extracted feature, the position in the image, the likelihood of the feature to be extracted, and the feature amount. In this embodiment, for each eye and mouth, the filtering result for the local feature extraction result using a unique two-dimensional mask is integrated at each position in the image, and the likelihood at each position of each partial feature is obtained. Hold as.

  The partial feature distribution determination unit 25 performs a simple distribution analysis on the feature extraction result extracted by the partial feature extraction unit 24, and based on the result, the face extraction unit 26 performs predetermined face extraction. Give a start command to the module (s).

  The analysis performed here is different from the processing performed from the initial feature extraction unit 22 to the partial feature extraction unit 24, and extracts necessary conditions for each predetermined face extraction module that gives an activation command. For example, in the present embodiment, an analysis for determining whether or not an eye has been extracted by the process of the partial feature extraction unit 24 near a predetermined coordinate of the input image, or a process of the partial feature extraction unit 24 is performed. Analysis for determining whether or not the center of gravity position of the mouth extraction result by the method is in the vicinity of a predetermined coordinate, or analysis for determining whether or not the cumulative likelihood of the eye, which is the processing result of the partial feature extraction unit 24, is greater than or equal to a predetermined value Etc.

  In the analysis as described above, conditions corresponding to modules that constitute the face extraction unit 26 and perform face extraction corresponding to a plurality of variations may be set in advance. Here, the fluctuation is a change in characteristics obtained by, for example, affine transformation such as rotation transformation or size transformation, transformation corresponding to when the face is turned sideways, or the like. For example, in the case of a face extraction module that supports clockwise in-plane rotation fluctuations, the center of gravity position of the mouth extraction result is present at the lower left position from the center of the image, and the center of gravity position of the eye extraction result is the mouth extraction result. The condition that the center of gravity position exists at the position in the upper right part is set as one of the necessary conditions of the face extraction module corresponding to the rotation variation in the clockwise direction.

  Several such analyzes are performed, and an activation command is issued to a predetermined face extraction module that satisfies the analysis conditions. The analysis of the center of gravity, the analysis of the cumulative likelihood, and the like may be performed within a predetermined range such as a position where the eye is predicted to exist. In addition, a comparison regarding the cumulative likelihood of two or more features may be performed. By selecting a module for performing feature extraction through such simple analysis of necessary conditions, it is possible to reduce processing costs and to reduce misidentification.

  In the face extraction unit 26, only the predetermined face extraction module that receives the activation command from the partial feature distribution extraction unit 25 is used to extract the partial feature extraction unit 24 using the eye and mouth extraction results extracted by the partial feature extraction unit 24. A feature extraction process similar to the above is performed. Modules corresponding to specific fluctuations include, for example, fluctuations due to size (ii in FIG. 4), fluctuations due to in-plane rotation (iii in FIG. 4), lateral shaking of the face (iv in FIG. 4), fluctuations due to vertical shaking, etc. Prepare specialized modules.

  In the present embodiment, a specific two-dimensional mask is prepared in advance for each module corresponding to the above-described variation, and only the module that has received the activation command performs a filtering process using the specific two-dimensional mask. . The setting of the two-dimensional mask is the same as that described in the local feature extraction unit 23, and a face having a specific variation corresponding to the module is given as a test pattern so as to specialize the variation corresponding to each module. Set by.

  In this face extraction, since a face centered around the center of the image is extracted as a target, unlike the feature extraction processing up to the partial feature extraction unit 24, it is not necessary to perform filtering at each position of the image. It is only necessary to perform filtering within the range for extracting the face.

  The detection result output unit 27 performs category classification of the final input image based on the result of filtering by the module corresponding to the variation that received the activation command and performed the face extraction process. Here, it is simply determined whether or not the output value of the activated face extraction module exceeds the threshold set for each module, and the output value of at least one module exceeds the threshold At this time, it is determined that the input image is a face present image, and otherwise, it is determined that the input image is a face absent image.

  The determination here is not limited to this method. For example, the final determination may be performed by integrating the output values of the activated modules. Specifically, the module output value corresponding to the clockwise in-plane rotation fluctuation is given a predetermined weight to the module output value corresponding to the counterclockwise in-plane rotation fluctuation, which is a category opposite to the fluctuation. It is possible to reduce misidentification by suppressing mutual outputs between modules whose fluctuations are contradictory, such as subtraction.

  In addition, the module output value corresponding to a face of a specific size is added with a predetermined weight applied to the output value of the module corresponding to a face of a size slightly larger than the specific size, which is a similar category as a variation. By promoting mutual output between modules having similar variations such as, etc., a threshold for identification can be set high, and as a result, misidentification can be reduced.

  Alternatively, a value obtained by weighted addition of the output values of two or more modules of similar categories as described above, or simply by performing an arithmetic mean operation, etc. By newly setting as an output value of a typical feature extraction module, it is possible to perform highly accurate identification with little processing cost and no identification omission.

  As described above, the first embodiment uses two-dimensional image data as input data, and in the method for identifying whether the image data is in a specific category, as the identification category, the vicinity of the center of the face is the input image. Assuming a face presence image that is present at approximately the center and a face absence image other than that, an example of a method of identifying which of the above two categories for input image data has been described.

(Second Embodiment)
In the second embodiment, as a modification of the first embodiment, two-dimensional image data is used as input data, and a method for detecting where a face is in the image is shown. In this embodiment, a process of detecting a face in an image is performed. However, as in the first embodiment, the present invention is not limited to this, and other image patterns and input data are audio data. It is also applicable to cases. Further, the present invention can also be applied to a case where objects of a plurality of categories are detected.

  In the present embodiment, as a method for detecting a specific pattern robustly against fluctuations by extracting hierarchical features from two-dimensional image data, the basic configuration of the Convolutional Neural Network (hereinafter referred to as CNN) is changed and used. FIG. 7 shows a basic CNN configuration. The basic processing of CNN will be described with reference to FIG. In FIG. 7, the process flow is processed in the right direction with the left end as an input.

  Reference numeral 71 in FIG. 7 denotes a pixel value distribution corresponding to the luminance value and the like of the input image. In the figure, reference numerals 72, 74, 76, 78 denote feature detection layers, and L7 · 21, L7 · 22, L7 · 23, L7 · 24, L7 · 41, L7 · 42, L7 · 43 in each layer. , L7 · 44, L7 · 61, L7 · 62, and L7 · 81 are feature detection cell surfaces. On the other hand, 73, 75 and 77 are feature integration layers, and L7 · 31, L7 · 32, L7 · 33, L7 · 34, L7 · 51, L7 · 52, L7 · 53 and L7 · 54 in each layer. , L7 · 71, L7 · 72 are feature-integrated cell surfaces.

  In the CNN, a feature detection layer and a feature integration layer are set as one set, and are hierarchically configured. Each feature detection cell surface in the feature detection layer has a feature detection neuron that detects a specific feature. Each feature detection neuron is a local range corresponding to the position of the feature detection neuron, and if it is a feature detection result of the previous stage layer, that is, a feature detection neuron in the feature detection layer 74, from L7 · 31 to L7 · 34 In the feature extraction result, if it is a feature detection neuron in the feature detection layer 72, it is coupled to the input image 71 with a unique weight distribution for each feature detection cell surface.

  This weight corresponds to the differential filter for extracting an edge and the two-dimensional mask for extracting a specific feature described in the first embodiment, and as described in the first embodiment. It may be set using prior knowledge, or may be set by learning by giving a plurality of test patterns. In addition, it may be set using a known neural network learning method such as learning by the error back-propagation method or self-organized learning by Hebb's learning rule.

  Each feature detection neuron performs weighted addition with a predetermined weight to the feature extraction result on each feature cell surface that is the connection destination or the luminance value of the input image in the case of the feature detection layer 72, and the calculation result The feature is detected by converting the value of で with a nonlinear function such as a hyperbolic tangent function into the output value of the feature detection neuron.

  For example, if L7 · 21 is a cell surface for detecting a vertical edge, each feature detection neuron in L7 · 21 performs weighting addition corresponding to the differential filter on the luminance value of the input image, and the vertical direction in the input image. At the position where the edge exists, the value of the result of calculation performed by the feature detection neuron in L7 · 21 becomes large and shows a high output value, that is, the feature is detected.

  Similarly, with respect to other feature detection cell planes, the feature detection neuron has a high output value at a position where a specific feature is detected on each feature detection cell plane. Regarding the calculation of the output value, the nonlinear conversion is generally performed as described above, but the present invention is not limited to this.

  Each feature integration cell surface in the feature integration layer is combined with one feature detection cell surface of the feature detection layer that is the previous stage layer, and is combined in a local range with respect to the feature detection result of the previous stage. It has a feature integration neuron that performs (integration) processing to blur the detection result. Each feature integration neuron basically performs the same operation as the above-described feature detection neuron, but is characterized in that the weight distribution corresponding to a specific two-dimensional mask is a Gaussian Filter or a Low-Pass Filter.

  The network structure of the CNN is to gradually detect higher-order features from the initial features using such hierarchical feature detection and feature integration processing, and finally categorize the input. A specific image can be detected by detecting higher-order features from the input image by the processing as described above. The CNN is characterized by being able to identify robustly against variations of various patterns by hierarchical feature extraction and feature integration layer blurring.

  In the present embodiment, the CNN will be described as a basic hierarchical feature extraction processing configuration. FIG. 8 shows the configuration of the processing unit in this embodiment. FIG. 9 shows the flow of processing in this embodiment. Hereinafter, processing in the present embodiment will be described with reference to FIGS.

  The image input unit 801, initial feature extraction unit 802, local feature extraction unit 803, and partial feature extraction unit 804 in FIG. 8 are the image input unit 21, initial feature extraction unit 22, and local feature extraction unit 23, respectively, in the first embodiment. This is the same as the partial feature extraction unit 24. And the process of step S901-904 is the same as the process of step S301-304 in FIG.

  In the present embodiment, an RGB color image is used in the image input unit 801, and the RGB color image converted into a grayscale image is input to the initial feature extraction unit 802 as the next layer. Further, the processing by the CNN is used for feature extraction, and each feature extraction unit performs feature detection by the feature detection layer and integration of the detected feature by the feature integration layer. The types of features extracted by the local feature extraction unit 803 and the partial feature extraction unit 804 are the same as those in the first embodiment. The unique weight distribution for each feature detection cell surface for detecting each feature is also given a plurality of test patterns as inputs, as in the method for setting the unique two-dimensional mask described in the first embodiment. The one set by learning is used.

  In this embodiment, the feature extracted by the initial feature extraction unit 801 does not use a previously limited feature, and the local feature is extracted by using the error back-propagation method when learning the feature detected by the local feature extraction unit 802. Each feature detection cell surface specific weight distribution for detection is learned, and each feature cell surface specific weight distribution for detecting an initial feature is automatically set. As a result, the initial feature extraction unit 801 extracts the input image 71 so as to extract the features constituting the local features detected by the local feature extraction unit 802 and necessary to detect the local features. The connection weight distribution can be set automatically.

  In step 905, the first face extraction unit 805 performs the same processing as the above feature extraction method on the eye and mouth extraction results extracted by the partial feature extraction unit 804, and the face in the image is extracted. Extract.

  When the output value of the first face extraction unit 805 exceeds a predetermined threshold, the face candidate presence determination unit 806 determines that there is a face candidate (step S906), and sets the number of face candidates to Count. After setting (step S907), the coordinates of the face candidate existence position determined to have a face candidate are sequentially output, and an activation command is issued to the skin color area extraction unit 807 and the partial feature distribution determination unit 808 (step S908). ).

  The skin color region extraction unit 807 receives the activation command from the face candidate presence determination unit 806, and extracts a skin color region from the input image in a range based on the face candidate presence position coordinates (step S909). The partial feature distribution determination unit 808 determines the distribution of partial feature extraction results in the range based on the face candidate existence position coordinates (step S910), and turns on the flag of the face extraction module to be activated as in the first embodiment. (Step S911).

  Unlike the partial feature distribution determination unit 25 of the first embodiment, the partial feature distribution determination unit 808 of this embodiment is not only the feature extraction result of the partial feature extraction unit 804 but also the skin color of the skin color region extraction unit 807. Using the region extraction results, simple distribution analysis is performed on the feature extraction results, and the second face extraction unit 809 configured by a face extraction module corresponding to a plurality of variations is activated. A processing unit that issues a command. Incidentally, one face extraction module in the present embodiment corresponds to one feature detection cell surface in the CNN.

  Similar to the first embodiment, the second face extraction unit 809 performs face extraction by the face extraction module corresponding to the fluctuation. That is, the face extraction module in which the flag is turned on sequentially performs face extraction processing at the face candidate position coordinates and turns off the flag of the face extraction module that has performed face extraction (steps S911 to 914).

  Unlike the first embodiment, the face extraction process in this embodiment is not only the eye and mouth feature extraction result extracted by the process in the partial feature extraction unit 804 but also the eye extracted in the local feature extraction unit 803. The face corresponding to the specific variation is extracted using the feature extraction result corresponding to the upper part of the skin and the upper part of the lip and the skin color area extraction result extracted by the skin color area extraction unit 807.

  The detection result output unit 810 outputs a result indicating which face is present in the input image based on the face extraction result in the second face extraction unit 809. That is, the output results of the modules are integrated (step S914), the detection result at the face candidate existence position is output (S915), and the process loops to detection at the next face candidate existence position (steps S917 to 918).

  Detailed processing in each processing unit after the first face extraction unit 805 in this embodiment will be described below.

  The face extraction process performed by the first face extraction unit 805 is the same as the feature extraction process performed by the local feature extraction unit 803 and the partial feature extraction unit 804. The face extraction here does not have a plurality of face extraction modules corresponding to fluctuations unlike the face extraction unit 26 in the first embodiment, and is configured by only one module. Also, in this embodiment, unlike the first embodiment, in order to detect where the face is in the image, the face is not extracted only near the center of the image, but at each position of the image. Perform face extraction.

  Here, the inherent weight distribution combined with the partial feature extraction result extracted by the partial feature extraction unit 804 of each face detection neuron used for the extraction processing is a face with various variations, that is, from (i) of FIG. Settings are made by learning with faces having various variations as shown in iv) as test data. By learning in this way, it is possible to extract a face with various fluctuations with a single module, although the accuracy is low, such as the possibility of determining a non-face as a face is high. In this processing unit, feature detection is performed using the weight distribution learned as described above, and the result is integrated by the feature integration layer.

  The face candidate presence determination unit 806 determines a portion that is an output equal to or greater than a predetermined threshold with respect to the result of the face extraction processing in the first face extraction unit 805. Then, assuming that a face candidate exists at the determined position, an activation command is issued so that the skin color partial feature distribution determination unit 807 and the partial feature distribution determination unit 808 perform processing within the range where the candidate exists.

  The skin color area extraction unit 807 receives the activation command from the face candidate presence determination unit 806 and extracts the skin color area in the vicinity of the range where the face candidates exist. In the present embodiment, the RGB color input image is converted into the HSV color system in the area where the skin color area is extracted, and only the pixels in the specific hue (H) range are extracted as the skin color area. The skin color region extraction method is not limited to this, and other generally known methods may be used. For example, extraction may be performed using saturation (S) or luminance (V). In the present embodiment, the skin color region is extracted. However, other than this, a hair region or the like may be extracted.

  The partial feature distribution determination unit 808 performs the same processing as the partial feature distribution determination unit 25 in the first embodiment. In the present embodiment, similar to the skin color region extraction unit 807, in response to an activation command from the face candidate presence determination unit 806, the distribution of a predetermined feature extraction result is analyzed in the vicinity of the range where the face candidates exist. In accordance with the result, a predetermined face extraction module of the second face extraction unit 809 composed of face extraction modules corresponding to a plurality of specific fluctuations is selected, and face extraction processing is performed at the face candidate existing position. Give the start command as follows.

  The feature extraction results analyzed by the partial feature distribution determination unit 806 are the eye and mouth extraction results extracted by the partial feature extraction unit 804 and the skin color region extraction result by the skin color region extraction unit 807. The analysis performed here is the same as that described in the first embodiment, and the necessary conditions that must be satisfied when each of the modules corresponding to the fluctuations constituting the second face extraction unit 809 has a face. The process which extracts is performed.

  In this embodiment, unlike the first embodiment, the skin color region extraction result is used. The simplest example is to analyze the area of the extracted skin color area. In addition, analyze the aspect ratio of the area extracted as skin color, or the relative position of the center of gravity position of the upper half skin color extraction area and the center of gravity position of the lower half skin color extraction area of the area determined to have a face candidate You may analyze the relationship.

  For example, the first example is one of the requirements for a face extraction module of a specific size depending on the area. Further, the second example can be set as one of the necessary conditions in the module corresponding to the horizontal or vertical swing of the face, and the third example can be set in the module corresponding to the in-plane rotation of the face. Further, by using the partial feature extraction result extracted by the partial feature extraction unit 804, the area of the area where the eyes are extracted and the area of the skin color area are compared, and conversely, the area where the eyes are not extracted and the area of the skin color area are compared. You may perform analysis, such as a comparison of an area, and the comparison of the area of the area | region where eyes are not extracted, and the area | region which is not skin color.

  Also in the analysis of the area and the like as described above, as described in the first embodiment, the analysis may be performed only in a specific region. For example, you may make it analyze the area of the area | region which is not the skin color in the area | region considered to be the position of the hair. In addition to the analysis related to the eye and mouth extraction results as performed in the first embodiment, by adding the above analysis, it is possible to issue a more accurate activation command.

  The second face extraction unit 809 is a processing unit similar to the face extraction unit 26 of the first embodiment, and includes a plurality of face extraction modules corresponding to specific fluctuations. In the present embodiment, unlike the first embodiment, not only the eye and mouth extraction results by the partial feature extraction unit 804, but also the skin color extraction results by the skin color region extraction unit 807, the first face extraction unit 805, and so on. The face extraction is performed at the position of the face candidate using the extraction result of the face with various fluctuations and the feature extraction result corresponding to the upper part of the eye and the upper lip among the features extracted by the local feature extraction unit 803.

  In this way, not only the feature extraction result of the immediately preceding hierarchy, but also the feature extraction result in the same hierarchy (here, the first face extraction result) that is a feature of the same level, or from outside the hierarchical feature extraction framework In the extracted feature extraction result (here, the skin color region extraction result), the feature extraction result in the previous hierarchy (here, the feature extraction result corresponding to the upper part of the eye and the upper lip), and further in the third embodiment described later The accuracy of feature extraction can be improved by supplementarily using the feature extraction result of the later-stage layer to be described in the feature extraction. By doing so, the processing cost increases, but the feature extraction processing of the second face extraction unit 809 is performed only at the position where the face candidate exists in the module that has received the activation command from the partial feature distribution determination unit 808. As a result, an increase in processing costs can be minimized.

  The detection result output unit 810 is a processing unit similar to the detection result output unit 27 in the first embodiment, and is a part of the face extraction module corresponding to a plurality of variations that constitutes the second face extraction unit 809. From the result of performing the feature extraction process by the activation command from the feature determination unit 808, it is determined at which position in the image the face exists, and the result is output. Here, as described in the first embodiment, highly accurate detection is possible by integrating the outputs of a plurality of modules.

  As described above, the second embodiment has described the example in which two-dimensional image data is used as input data and a face is a detection target in a method of detecting a specific target in the image.

(Third embodiment)
The third embodiment of the present invention is a modification of the second embodiment. In the present embodiment, similarly to the second embodiment, a process of detecting a face in an image is performed. However, the present invention is not limited to this, and can be applied to other image patterns and audio data. Further, the present invention can be applied to a case where objects of a plurality of categories are detected.

  FIG. 10 shows the configuration of the processing unit in this embodiment. FIG. 11 shows the flow of processing in this embodiment. The basic processing configuration of this embodiment is the same as that described in the second embodiment. Hereinafter, the processing in this embodiment will be described with reference to FIG.

  Since the processing (steps S1101 to 1109) from the image input unit 1001 to the skin color region extraction unit 1007 in FIG. 10 is exactly the same as steps S901 to 909 of the second embodiment, description thereof is omitted.

  The partial feature distribution determination unit 1008 is the same process as the partial feature distribution determination unit 808 in the second embodiment, but the second face extraction unit 1009 has a plurality of processes according to the analysis result of the distribution of the feature extraction results. A second partial feature extraction unit configured to give a start command to the face extraction module corresponding to the variation so as to perform the face extraction process at the face candidate existing position, and is configured by a partial feature extraction module corresponding to a plurality of variations An activation command is also issued to 1011. That is, the distribution of the partial feature extraction results in the range based on the face candidate presence position coordinates is determined (step S1110), and the flag of the face extraction module to be activated is turned on (step S1111).

  The second partial feature extraction unit 1011 is composed of a plurality of modules that extract partial features corresponding to specific fluctuations, and receives a startup command from the partial feature distribution determination unit 1008 and receives the startup command. The partial feature is re-extracted only at a specific position determined by the position of the face candidate. That is, if there is a partial feature extraction module corresponding to the face extraction module for which the flag is on, partial feature extraction processing is performed at a position determined by the face candidate existence position coordinates (steps S1113 to 1114).

  The second face extraction unit 1009 is a processing unit that is substantially the same as the second face extraction unit 809 of the second embodiment. However, when the partial feature re-extraction corresponding to the face extraction module activated here is performed by the second partial feature extraction unit 1011, the feature extracted by the second partial feature extraction unit 1011 is used. To extract the face. That is, the face extraction module with the flag turned on performs face extraction at the face candidate existing position, and turns off the flag of the face extraction module that performed the face extraction (steps S1115 to 1116).

  The detection result output unit 1010 is exactly the same as the detection result output unit 810 of the second embodiment, and steps S1117 to 1120 are exactly the same as steps S915 to 918 of the second embodiment. Omitted.

  Detailed processing in the partial feature distribution determination unit 1008, the second partial feature extraction unit 1011 and the second face extraction unit 1009 in this embodiment will be described below.

  As described above, the partial feature distribution determination unit 1008 is the same as the second embodiment with respect to the process of analyzing the distribution of partial feature extraction results. In the second embodiment, an activation command is issued to a module that extracts a face corresponding to a plurality of variations. However, a partial feature corresponding to the variation of the face extraction module that issued the activation command is further extracted. An activation command is also issued to the second partial feature extraction unit 1011 to be performed. Specifically, for example, when a start command is issued to start a face extraction module corresponding to a clockwise in-plane rotation variation, a partial feature extraction module corresponding to the same in-clockwise in-plane rotation variation is simultaneously issued. In contrast, an activation command is issued.

  The second partial feature extraction unit 1011 includes a plurality of modules that extract partial features corresponding to a plurality of variations. Here, the partial feature extraction module corresponding to the module for extracting a face corresponding to a plurality of fluctuations in response to the activation command from the partial feature distribution determination unit 1008 is activated and obtained as a result of the face candidate presence determination unit 1006. Partial features are extracted only within a specific range determined by the face candidate existing position. The feature extraction method is the same as that described in the second embodiment.

  Each partial feature module basically corresponds to each of the face extraction modules corresponding to a plurality of variations constituting the second face extraction unit 1009, but may not be a one-to-one correspondence. For example, there may be no partial feature extraction module corresponding to a face-oriented face extraction module. In this case, when an activation command is issued to the face-oriented face-facing module, the processing in the second partial feature extraction unit 1011 is not performed.

  Furthermore, one partial feature extraction module may be associated with a plurality of types of face extraction modules. For example, a face extraction module corresponding to an in-plane rotation fluctuation of 15 degrees clockwise and a face extraction module corresponding to an in-plane rotation fluctuation of 30 degrees clockwise perform extraction including both fluctuations in one module. The partial feature extraction module is made to correspond.

  In this way, a feedback mechanism for controlling the operation of the feature extraction module at the lower hierarchy level is introduced based on the feature extraction result output at the higher hierarchy level. In other words, the feature extraction module that corresponds to the face extraction module corresponding to the specific variation that is activated in the second face extraction re-extracts the low-order features, thereby enabling more accurate feature extraction. Become. Although the processing cost increases due to the feature re-extraction, the processing is performed only at a specific position of the module that has received the activation command, so that the increase in the processing cost can be minimized.

  In the present embodiment, the extraction of the mouth is not performed in this processing unit, and only the eye corresponding to the fluctuation is extracted. If more accurate feature extraction is desired, mouth extraction corresponding to fluctuations may be performed, or types of features other than the features extracted by the partial feature extraction unit 1004 may be extracted.

  Further, the feature extraction here is different from the partial feature extraction unit 1004, in addition to the local feature extraction result extracted by the local feature extraction unit 1003, such as the eyes and mouth extracted by the partial feature extraction unit 1004. The eyes are extracted using the partial feature extraction result and the first face extraction result extracted in the first face extraction 1005. As described in the second embodiment, the feature extraction results in the same hierarchy, which are the features of the same level, and the feature extraction results of the upper hierarchy, which are the features of the upper level, are supplementarily used, thereby further improving the accuracy. High feature extraction processing becomes possible.

  The second face extraction unit 1009 basically performs the same processing as the second face extraction unit 809 in the second embodiment. The difference from the second face extraction unit 809 in the second embodiment is that the second partial feature extraction unit 1011 performs partial feature extraction corresponding to the variation corresponding to the activated face extraction module. In this case, face extraction is not performed using the partial feature extraction result extracted by the partial feature extraction unit 1004, but the partial feature extraction result corresponding to the variation extracted by the second partial feature extraction unit 1011 is used. The point is that face extraction is performed.

  In the present embodiment, since only the eyes are extracted in the second partial feature extraction unit 1011, the extraction result of the partial feature extraction unit 1004 is used as the extraction result of the mouth. As described in the second partial feature extraction unit 1011 above, for example, when there is no partial feature extraction module corresponding to the front-facing face extraction module, an activation command is issued to the front-facing face extraction module. If it is, the second partial feature extraction unit 1011 does not perform feature re-extraction.

  In such a case, the feature extraction result of the partial feature extraction unit 1004 may be used as it is. In this embodiment, when the partial feature extraction corresponding to the variation corresponding to the face extraction module to be activated is performed, the eye extraction result extracted by the partial feature extraction unit 1004 is not used, but the accuracy is further improved. Therefore, this feature extraction result may also be used supplementarily.

  As described above, the third embodiment is described as an example of detecting a face in a method of detecting a specific target in an image using two-dimensional image data as input data as a modification of the second embodiment. did.

(Fourth embodiment)
The fourth embodiment of the present invention is a modification of the third embodiment. In the present embodiment, similarly to the third embodiment, the process of detecting a face in an image is performed. However, the present invention is not limited to this, and can be applied to other image patterns and audio data. Further, the present invention can be applied to a case where objects of a plurality of categories are detected.

  The configuration and processing flow of the processing unit in this embodiment are basically the same as the configuration and processing flow of the processing unit in the third embodiment shown in FIGS. The detailed explanation is omitted. This embodiment differs from the third embodiment in the processing of the partial feature distribution determination unit 1008 in FIG. 10, and this will be described with reference to FIGS. 10 and 11.

  The partial feature distribution determination unit 1008 in this embodiment is substantially the same process as the partial feature distribution determination unit 1008 in the third embodiment, but the processing content of the analysis of the distribution of the feature extraction result (step S1110) is the first. This is slightly different from the third embodiment. Details of this processing will be described later. The process (step S1111) of issuing the activation command, which is another process of the partial feature distribution determination unit 1008, is the same as that of the third embodiment.

  Detailed processing of the analysis of the distribution of feature extraction results (step S1110) in the partial feature distribution determination unit 1008 in the present embodiment will be described below.

  In the analysis process of the distribution of the feature extraction result in the partial feature distribution determination unit 1008, the distribution of the feature detection result of the eyes / mouths extracted in the partial feature extraction unit 1004 is analyzed as in the third embodiment. However, in order to improve the accuracy of the activation command, not only the centroid of the distribution and the total sum of the likelihoods, but also the result of filtering using a predetermined filter on the distribution of the feature detection results is used. In addition, regarding the feature extraction result for analyzing the distribution, not only the feature detection result in the partial feature extraction unit 1004 and the feature extraction result in the skin color region extraction unit 1007 but also the initial feature extraction unit 1002 and the local feature extraction unit 1003 The feature extraction result is also used, and the input image is also used as one feature extraction result.

  FIG. 13 shows a processing flow of filtering using a predetermined filter for the distribution of feature detection results performed in step S1110. This filtering will be described with reference to FIG.

  First, a target feature detection result (for example, an eye detection result, one of local feature detection results, an input image, or the like) is selected (step S1301). Next, a predetermined region of the feature detection result is cut out (step S1302). Next, a filter corresponding to the area cut out in step S1302 is selected from the filter database 1303 (step S1304). Then, a filter operation is performed on the region cut out using the selected filter (step S1305). The filter calculation value at this time is used as the analysis result of the distribution of the feature detection result. In the present embodiment, the filter operation in step S1305 performs a linear filter operation. However, for example, multistage nonlinear filtering such as a feedforward type neural network may be used.

  The filter calculation in step S1305 and the filter selected in step S1304 used for the filter calculation will be specifically described. For example, it is assumed that the region cut out from the feature detection result in step S1302 is a rectangle and has 10 × 10 elements. At this time, the filter used for the filter calculation of the present embodiment is a filter composed of 10 × 10 elements having the same size as the cut-out size. In the filter operation, the product of the 10 × 10 elements and the feature detection result of the cut out 10 × 10 elements are respectively calculated, and the sum of the products is obtained. The calculation is performed, and the result is used as the analysis result of the distribution of the feature detection result. That is, an inner product operation is performed on a 10 × 10 = 100-dimensional vector in which the elements of the cut-out region are arranged and a 10 × 10 = 100-dimensional vector in which the elements of the filter are also arranged, and the inner product value is calculated as the feature detection result. The result of distribution analysis. As a filter used in this filter operation, it is preferable to use a filter that can separate a feature category pattern to be detected in a module that issues an activation command from a pattern that is not detected as much as possible.

  A method for creating a filter used in the filter calculation will be specifically described. In the present embodiment, since a face with a predetermined fluctuation is detected in the module that issues the activation command, first, a plurality of two classes of images, that is, a face image that has the predetermined fluctuation and a pattern that does not, are prepared. Then, the processing up to the partial feature extraction unit 1004 is performed on these patterns, and a predetermined region of the predetermined feature detection result is cut out and used as a learning sample set.

  Next, using a learning sample set created from each class, an identification surface that separates the two classes is obtained by a linear discriminant method, and the normal vector of the identification surface is used as a filter. By performing a filter operation using such a filter, the filter value can be treated as a representative value suitable for separating a pattern into one of two classes. Therefore, this filter value should be evaluated. Thus, a highly accurate start command can be issued. The filter created in this way is stored in the filter database 1303, and the filter value is obtained with reference to the actual processing.

  In the above description, a filter that separates a face image having a predetermined variation and an image having a pattern that is not a predetermined variation has been created. For example, a plurality of face images having a predetermined variation, that is, images having a plurality of feature categories, Analysis results that can be used in common in multiple categories, although the accuracy is reduced by preparing multiple images of patterns that are not so, for example, two classes of images, such as images that are not faces, and creating filters using the same method Can also be obtained.

  In this embodiment, a filter generated using a linear discriminant method is used as the linear filter. However, the present invention is not limited to this. For example, a filter with a high degree of separation may be configured by least square learning or the like. good. Even when the filter is not a linear filter, a filter with a high degree of separation may be used.

  Next, a method for determining a module to be activated from these analysis results will be described in detail. In the third embodiment, necessary conditions to be satisfied by the module are determined, and an activation command is issued depending on whether the conditions are satisfied. Similarly, for example, a condition that a certain filter value is greater than or equal to a predetermined value may be set in advance, and an activation command may be issued based on the condition. In this embodiment, each filter value In addition, a plurality of analysis results such as a center-of-gravity analysis position are comprehensively determined to determine whether or not to issue an activation command. The flow of this processing in this embodiment is shown in FIG. Details of this processing will be described below with reference to FIG.

  In the present embodiment, as a comprehensive determination method, first, values of each analysis result (for example, the filter value, the x-coordinate and y-coordinate of the centroid position of the predetermined feature, the accumulated value of the feature detection level, etc.) are simply arranged. Generated vectors (step S1401). Next, the vector, a vector serving as so-called dictionary data specific to each module that is the target of issuing the activation command, and an inverse matrix for calculating the Mahalanobis distance are selected from the dictionary database 1402 (step S1403). Then, the Mahalanobis distance between the vector selected in step S1403 and the vector generated in step S1401 is calculated using the inverse matrix selected in step S1403 (step S1404). Finally, if the distance obtained in step S1404 is within a predetermined threshold, a process of issuing an activation command to the module (step S1405) is performed.

  The vector which is the dictionary data used here is the average vector data of each category, and the value of each analysis result is calculated from each of a plurality of images of the category and is simply arranged in the same manner as described above. Is generated in advance by obtaining the average. The inverse matrix for calculating the Mahalanobis distance is an inverse matrix of the variance-covariance matrix for the vector distribution of each category, and uses a vector generated from each of a plurality of images of the category and the average of those vectors. Thus, a variance-covariance matrix is generated, and its inverse matrix is obtained in advance. At the time of detection, the Mahalanobis distance is obtained using the average vector and the inverse matrix.

  In this embodiment, whether or not to start using the Mahalanobis distance is determined, but is not limited to this method. For example, determination is made using a simple Euclidean distance or linear or nonlinear discriminant analysis is used. Or may be determined using a neural network.

  As described above, in the present embodiment, in the analysis of the feature detection result distribution, the feature detection result as the analysis target is extended to use the feature detection result other than the partial feature detection result and the input image. Further, a filter operation using a predetermined filter is performed on the feature detection result, and the filter value is added as an analysis result to the analysis result in the third embodiment such as the position of the center of gravity. Further, in determining whether or not to issue a start command, not only whether each analysis result satisfies a necessary condition but also a determination process that can comprehensively determine each analysis result as described above is used. As a result, a highly accurate start command can be issued, and the overall detection accuracy can be improved.

  As described above, the fourth embodiment is described as an example of detecting a face in a method of using a two-dimensional image data as input data and detecting a specific target in the image as a modification of the third embodiment. did.

<Other embodiments using software, etc.>
Even if the present invention is applied as a part of a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), a part of a device composed of a single device (for example, a copying machine, a facsimile machine). You may apply to.

  Further, the present invention is not limited to the apparatus and method for realizing the above embodiment and the method performed by combining the methods described in the embodiment, and is not limited to the computer (CPU or MPU) in the system or apparatus. The present invention also includes a case in which the above embodiment is realized by supplying a program code of software for realizing the above embodiment and operating the various devices by the computer of the system or apparatus according to the program code. included.

  In this case, the program code of the software itself realizes the functions of the above embodiment, and the program code itself and means for supplying the program code to the computer, specifically, the program code The stored storage medium is included in the category of the present invention.

  As a storage medium for storing such a program code, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  The computer controls various devices according to only the supplied program code, so that not only the functions of the above embodiments are realized, but also the OS (operating system) on which the program code is running on the computer. In the case where the above embodiment is realized in cooperation with other application software or the like, such program code is also included in the scope of the present invention.

  Further, after the supplied program code is stored in the memory of the function expansion board of the computer or the function expansion unit connected to the computer, the program code is stored in the function expansion board or function storage unit based on the instruction of the program code. A case in which the CPU or the like provided performs part or all of the actual processing and the above-described embodiment is realized by the processing is also included in the scope of the present invention.

  FIG. 12 is a diagram illustrating a block configuration example of an information processing apparatus that implements the present invention. As shown in the figure, this information processing apparatus includes a CPU 1201, a ROM 1202, a RAM 1203, an HD (hard disk) 1204, a CD 1205, a KB (keyboard) 1206, a CRT 1207, a camera 1208, and a network interface (I / F) 1209. 1210 are communicably connected to each other via 1210.

  The CPU 1201 controls the operation of the entire information processing apparatus, and controls the entire information processing apparatus by reading and executing a processing program (software program) from an HD (hard disk) 1204 or the like.

  The ROM 1202 stores programs and various data used in the programs.

  The RAM 1203 is used as a work area for temporarily storing a processing program and information to be processed for various processes in the CPU 1201.

  The HD 1204 is a component as an example of a large-capacity storage device, and stores various data such as model data or a processing program transferred to the RAM 1203 or the like when various processes are executed.

  A CD (CD drive) 1205 has a function of reading data stored in a CD (CD-R) as an example of an external storage medium and writing data to the CD.

  A keyboard 1206 is an operation unit for a user to input various instructions to the information processing apparatus.

  The CRT 1207 displays various instruction information to the user and various information such as character information or image information.

  The camera 1208 captures and inputs an identification target image.

  The interface 1209 is used for taking in information from the network and transmitting information to the network.

It is a figure which shows the basic composition of the pattern identification apparatus concerning 1st Embodiment. It is a figure which shows the function structure of the pattern identification apparatus in 1st Embodiment. It is a flowchart which shows the flow of the process in 1st Embodiment. It is a figure which shows the face presence image as an identification category in 1st Embodiment. It is a figure which shows four types of initial feature extraction results. It is a figure which shows each initial feature extraction result in the position where each local feature which should be extracted exists. It is a figure which shows the structure of basic Convolutional Neural Network. It is a figure which shows the function structure of the pattern identification apparatus in 2nd Embodiment. It is a flowchart which shows the flow of the process in 2nd Embodiment. It is a figure which shows the function structure of the pattern identification apparatus in 3rd Embodiment. It is a flowchart which shows the flow of the process in 3rd Embodiment. It is a figure which shows the block configuration of the computer which implement | achieves this invention. It is a flowchart which shows the flow of the filtering process with respect to distribution of a feature detection result. It is a flowchart which shows the flow of the process which determines whether it issues a starting command.

Claims (15)

In a pattern identification method for hierarchically extracting features of input data and identifying a pattern of the input data,
A first feature extraction step of extracting features of the first hierarchy;
An analysis step of analyzing a distribution of feature extraction results in the first feature extraction step;
The position or likelihood of the specific feature of the first hierarchy in the distribution analyzed in the analysis step has a plurality of variation categories with respect to the feature of the second hierarchy having the feature of the first hierarchy as a component. A determination step of determining whether a predetermined condition is satisfied with respect to the position or likelihood of the specific feature in order to perform extraction of the feature in which each variation of
As a target characteristic of said determining step by the second hierarchy variation occurs in the determined variation category to satisfy the condition, to a second feature extraction step of extracting a feature of the second hierarchy A pattern identification method characterized by the above. In the second feature extraction step, a feature extraction module that is different for each variation category is used, and a feature extraction module corresponding to the variation category determined that the position or likelihood of the specific feature satisfies the condition The pattern identification method according to claim 1, wherein the pattern identification method is selectively activated to extract the characteristics of the second hierarchy.   2. The pattern identification method according to claim 1, wherein in the first or second feature extraction step, a feature obtained by giving a predetermined transformation to a predetermined feature is extracted.   The pattern identification method according to claim 1, further comprising a re-extraction step of re-extracting features of a lower hierarchy based on a feature extraction result of an upper hierarchy in the second feature extraction step.   The pattern identification method according to claim 1, wherein in the analysis step, a distribution of each of the plurality of feature extraction results is analyzed, and a relative relationship between the analysis results is analyzed.   The pattern identification method according to claim 1, wherein in the analysis step, a distribution within a specific range of at least one of the feature extraction results is analyzed.   2. The pattern according to claim 1, wherein in the analysis step, it is analyzed that the feature is extracted or not extracted within a predetermined range in the distribution of at least one of the feature extraction results. Identification method.   The pattern identification method according to claim 1, wherein, in the analysis step, a barycentric position of at least one feature extraction result distribution is analyzed.   2. The pattern identification method according to claim 1, wherein, in the analysis step, a size of a range in which the features are extracted or a range in which the features are not extracted is analyzed in at least one distribution of the feature extraction results. .   The pattern identification method according to claim 1, wherein in the analysis step, a likelihood or a total of feature detection levels of at least one feature extraction result is analyzed.   The pattern identification method according to claim 1, wherein in the analysis step, the distribution of at least one feature extraction result is analyzed by performing filtering using a predetermined filter.   The pattern identification according to claim 11, wherein the predetermined filter is a filter for separating a pattern of a feature category to be extracted and a pattern that should not be extracted in the second feature extraction step. Method.   In the second feature extraction step, the likelihood of the plurality of features in the second hierarchy is calculated by performing a predetermined discrimination process on at least one analysis result obtained in the analysis step. The pattern identification method according to claim 2, wherein: A pattern identification device for hierarchically extracting features of input data and identifying a pattern of the input data,
First feature extraction means for extracting features of the first hierarchy;
Analyzing means for analyzing the distribution of the feature extraction results by the first feature extracting means;
The position or likelihood of the specific feature of the first hierarchy in the distribution analyzed by the analysis means is a plurality of variation categories with respect to the feature of the second hierarchy having the feature of the first hierarchy as a component. Determining means for determining whether a predetermined condition for a position or likelihood of the specific feature is satisfied in order to perform extraction of the feature in which each variation has occurred ,
The characteristics of the second hierarchy variation occurs in the determined variation category and satisfying the condition as a target by the determination means, to a second feature extracting means for extracting a feature of the second hierarchy A pattern identification device. A pattern identification program that causes a computer to hierarchically extract features of input data and identify a pattern of the input data,
A first feature extraction step of extracting features of the first hierarchy;
An analysis step of analyzing a distribution of feature extraction results in the first feature extraction step;
The position or likelihood of the specific feature of the first hierarchy in the distribution analyzed in the analysis step is a plurality of variation categories with respect to the feature of the second hierarchy having the feature of the first hierarchy as a component. A determination step of determining whether a predetermined condition is satisfied with respect to a position or likelihood of the specific feature in order to perform extraction of the feature in which each variation has occurred , and
As a target characteristic of said determining step by the second hierarchy variation occurs in the determined variation category to satisfy the condition, to a second feature extraction step of extracting a feature of the second hierarchy A pattern identification program.

Images