JP4511850B2 - Person attribute identification method and system - Google Patents

Description

  The present invention relates to a person attribute identification method and system for identifying a person attribute including gender and / or age of a person photographed by a camera, and can be used for, for example, statistical processing, trend survey, database search, security system, and the like. .

  Conventionally, as a method for identifying the gender of a person, a method using only face information has been proposed. For example, there is a method of performing gender identification by discriminating and analyzing a face image (see Patent Document 1 and Non-Patent Document 1). There is also a method of performing gender identification on a face image by using a support vector machine (see Non-Patent Document 2 and Non-Patent Document 3).

  In addition, the Gaussian mixture model used when creating the person attribute identification model in the present invention is a known technique and is described in many documents (for example, see Non-Patent Document 4).

JP 2000-222572 A Hitoshi Hongo, Yohei Ishii, Yoshinori Niwa, Kazuhiko Yamamoto, “Proposal of Integrated Gender and Age Estimation from Face Images”, IEICE Technical Report, The Institute of Electronics, Information and Communication Engineers, December 20, 2002, Vol. 102, no. 532, PRMU 2002-146, p. 1-6 Baback Moghaddam, Ming-Hsuan Yang, “Gender classification with support vector machines”, IEEE International Conference on・ Automatic Face and Gesture Recognition (IEEE International Conference on Automatic Face and Gesture Recognition), USA, IEEE, March 2000, p. 306-311 Hosoi Sei, Serikawa Erina, Kawade Masato, “Gender and Age Estimation System Using Gabor Wavelet Transform and Support Vector Machine”, Proc. Proceedings of 8th Image Sensing Symposium, Image Sensing Technology Study Group, July 18, 2002, p . 243-246 Yang, M, Ahuja, N, “Gaussian Mixture Model for Human Skin Color and It's Applications in Image and Video Databases ( Gaussian mixture model for human skin color and its applications in image and video databases) ”, In Proceedings of SPIE-the International Society for Optical Engineering: Conference on Storage and Retrieval for Image and Video Databases VII (1999), USA, 1999, vol. 3656, p. 458-466

  However, since the gender identification methods described in Patent Document 1 and Non-Patent Documents 1 to 3 described above use only face information, the identification accuracy is limited. Even if a person looks at a person and tries to estimate his / her gender, there are many cases in which it cannot be estimated by face information alone. When a person actually wants to identify a person's gender, in addition to the face information, various information such as hairstyle, clothes, and body type are integrated in the head for identification. Therefore, it is desired to establish and systematize a gender identification method or age identification method corresponding to the integration process naturally performed in such a person's head.

  An object of the present invention is to provide a person attribute identification method and system capable of effectively integrating face information and information other than face information and improving the identification accuracy of person attributes such as gender and age. There is a place to do.

  The present invention relates to a person attribute identification method for identifying a person attribute including the gender and / or age of a person photographed by a camera, and a plurality of types of person attribute identification including facial data for identifying a person attribute A plurality of categories for identifying personal data are set for each person attribute identifying data, and at least one person attribute identifying model indicating the characteristics of data included in these categories is provided for each person attribute identifying data. Create each type of person attribute identification data from the frame image of the person attribute identification processing target obtained by photographing each person with the camera and then corresponding to each created person attribute identification data Using each person attribute identification model to confirm, it is ascertained in which category each person attribute identification data created is included in the data. The likelihood, attribution degree, probability, ratio, or similar index values for indexing are calculated individually for each person attribute identification data, and then calculated for each person attribute identification data individually. The information indicating the identification result of the attribute of the person is calculated by integrating a plurality of index values.

  Here, since there is a “person attribute including the gender and / or age of a person”, identification of a person attribute performed by applying the present invention may be identification of only a person's gender, or identification of only a person's age. It may be a simultaneous identification of a person's gender and age, or even a simultaneous identification of other person attributes (such as occupation, race, nationality, residence, etc.) and gender. It may be the simultaneous identification of other person attributes and age, or the simultaneous identification of other person attributes, gender and age.

  The “category” set for each person attribute identification data may be a conflict category (two categories), or may be three or more categories.

  Further, “create at least one person attribute identification model for each person attribute identification data” means that the categories for identifying each person attribute identification data are conflict categories (two categories). Is set, a person attribute identification model may be created for both of the conflict categories, or a person attribute identification model may be created for only one of the conflict categories. In this case, the human attribute identification model may be created for all categories, or the human attribute identification model may be created for some categories.

  And the meaning of “for each person attribute identification data” in “calculate likelihood value, degree of attribution, probability, ratio, or an index value similar to these for each person attribute identification data” In addition to calculating index values for each category in which the data for identifying person attributes is identified, that is, calculating index values for all categories, when conflict categories (two categories) are set, conflict categories The calculation of the index value for one of them includes the case where three or more categories are set, that is, the calculation of the index value for some categories instead of all the categories.

  “Face data” means data created from face information as element information for identifying a person's attributes. For example, the face basic data and the face compression data obtained by dimensionally compressing the face data in the embodiment described later. The concept includes both, and the same applies to “person attribute identification data” of other types (other than “face data”). The “person attribute identification data” may be any form of data such as scalars, vectors, matrices, tensors, etc., as long as it can express information that can be an identification element of person attributes such as face information.

  Furthermore, as the “person attribute identification model”, for example, a Gaussian mixture model (see Non-Patent Document 4 described above), a support vector machine (SVM), a neural network, or the like can be adopted. It is arbitrary as long as it can be identified into a plurality of categories.

  And it is premised that “a plurality of types of person attribute identification data are created from the person attribute identification process target frame image obtained by photographing a person with a camera”, but an element for identifying a person attribute It is only necessary that the information includes multiple types of person attribute identification data created from the frame image. In applying the present invention, person attributes other than the person attribute identification data created from the frame image are used. Identification data may be included. Therefore, for example, a plurality of identification categories are set for human attribute identification data obtained by analyzing odor component data acquired from an odor sensor, sound data acquired from a microphone, vibration data acquired from an acceleration sensor, and the like. A corresponding person attribute identification model may be prepared, and an index value such as a likelihood calculated using the person attribute identification model may be included in the integration target.

  In such a person attribute identification method of the present invention, for a plurality of types of person attribute identification data including face data, corresponding person attribute identification models are prepared, and each of these person attribute identification models is prepared. Using each of the individual attribute identification data, after calculating individually the index values such as likelihood for indexing the certainty about which category the data is included, a plurality of obtained By integrating the index values, information indicating a person attribute identification result is calculated. For this reason, it is possible to improve the identification accuracy as compared with the conventional case where the identification of the person attribute is performed using only the face information. Also, rather than simply collecting individual identification results based on multiple types of person attribute identification data, by integrating multiple index values obtained as individual identification results, the identification results of person attributes can be obtained. Since the information to be shown is calculated, a plurality of pieces of information including face information can be effectively integrated, and high-precision identification is realized, thereby achieving the object.

  Further, in the person attribute identification method described above, a male category and a female category are set as categories for identifying facial data, and characteristics of data included in the male category are shown as a personal attribute identification model. When creating a male model for the face and a female model for the face that shows the characteristics of the data included in the female category, and calculating the index value for each person attribute identification data individually, Using the male model for the face, calculating the index value indicating the certainty that the face data created from the frame image is data included in the male category, and using the female model for the face, It is desirable to calculate an index value indicating the certainty that the face data created from the frame image is data included in the female category.

  In this way, when a facial male model and a facial female model are prepared for facial data to identify person attributes, individual identification for identifying gender from facial information can be performed. It becomes possible.

  Further, in the above-described personal attribute identification method, the human attribute identification data includes the hair style data in addition to the face data, and the male category and the female category are classified as categories for identifying the hair style data. As a model for identifying human attributes, a male model for hairstyle that shows the characteristics of data included in the male category and a female model for hairstyle that shows the characteristics of data included in the female category are created. When calculating the index value for each person attribute identification data individually, the hairstyle data created from the frame image using the hairstyle male model is data included in the male category. In addition to calculating an index value indicating the certainty of the hair, the hairstyle data created from the frame image using the female model for the hairstyle includes data that is included in the female category. It is desirable to calculate the index value indicating the likelihood of being a.

  In this way, when a hairstyle man model and a hairstyle woman model are prepared for the hairstyle data and the person attributes are identified, individual identification for identifying the gender from the hairstyle information can be performed. It becomes possible. For this reason, it integrates with the individual identification result of the man and woman by face information, and a highly accurate identification is realized.

  In the above-described person attribute identification method, the person attribute identification data includes the tie data for identifying whether or not the tie is tightened in addition to the face data, and the tie data is identified. A category with a tie and a category without a tie are set as categories, and a tie model indicating the characteristics of data included in the category with a tie as a model for identifying human attributes, and characteristics of data included in the category without a tie When the index value is calculated separately for each person attribute identification data, the tie data created from the frame image using the tie model is the one with the tie. In addition to calculating an index value that indicates the certainty of data included in the category, With tie model, ties data generated from the frame image, it is desirable to calculate an index value that indicates the likelihood of that the data included in the category without tie.

  In this way, when a tie model and non-tie model are prepared for tie data to identify person attributes, individual identification of person attributes is based on tie presence / absence information in which a significant difference in gender appears. Can be performed. For this reason, it integrates with the individual identification result of the man and woman by face information, and a highly accurate identification is realized.

  Further, in the person attribute identification method described above, the person attribute identification data includes decollet data for identifying whether or not a chested clothes are worn in addition to the face data. A category with chest and a category without chest is set as the category to be identified, and a decollete model showing the characteristics of the data included in the category with chest and a chest without chest as the person attribute identification model When creating a non-decollete model indicating the characteristics of the data included in the category and calculating the index value individually for each person attribute identification data, a decollete model created from the frame image using the decollete model is used. An index value indicating the certainty that the data is included in the category with decollete is calculated, Using Dell neckline data generated from the frame image, it is desirable to calculate an index value that indicates the likelihood of that the data included in the category without neckline.

  In this way, when a decollet model and a non-decollete model are prepared for the decollet data and the person attributes are identified, the presence / absence information of the decollet (the chest chest of clothes) in which the difference in gender is noticeable appears. Individual identification of person attributes can be performed based on this. For this reason, it integrates with the individual identification result of the man and woman by face information, and a highly accurate identification is realized.

  Furthermore, in the case where a male model for hairstyle and a female model for hairstyle are prepared for the above-mentioned hairstyle data and the person attributes are identified, the hair color extracted from images taken in various environments Using the information, non-hair color information, skin color information, and non-skin color information, a hair color model, a non-hair color model, a skin color model, and a non-skin color model are respectively created in advance, and a frame image of a person attribute identification process target Before creating the hairstyle data from the image, update the skin color model using the person's skin color information detected from the person attribute identification target frame image, and create the hairstyle data from the person attribute identification target frame image. When using the hair color model, non-hair color model, updated skin color model, and non-skin color model, whether each pixel constituting the frame image of the person attribute identification process is a hair part. It is desirable to extract the hair by estimating how.

  When the skin color model is updated using the skin color information of the detected person before creating the hair style data in this way, the skin color model is adapted to the detected person and the hair style extraction adapted to the individual is performed. Thus, more accurate hairstyle extraction can be performed.

  In addition to updating the skin color model, the non-skin color model is also updated using the non-skin color information (for example, color information of the clothing portion) of the person detected from the frame image of the person attribute identification processing target, Hair style data may be created using the color model, the non-hair color model, the updated skin color model, and the updated non-skin color model.

  When the tie model and the non-tie model are prepared for the tie data described above and the person attribute is identified, when creating the tie data from the frame image of the person attribute identification process target, It is desirable to perform edge extraction and create an edge image.

  As described above, when the edge image is created by extracting the outline when creating the tie data, the accuracy of the individual identification of the tie / non-tie can be improved.

  In addition, when the decollete model and the non-decollete model are prepared for the above-mentioned decollete data and the person attribute is identified, the skin color information and non-skin color information extracted from images taken in various environments are displayed. Using the skin color model and the non-skin color model in advance, the skin color information of the person detected from the frame image of the person attribute identification process before creating the decollete data from the frame image of the person attribute identification process When updating the skin color model and creating the decollete data from the frame image of the person attribute identification processing target, the updated frame color model and the non-skin color model are used. It is desirable to extract the skin color part by estimating whether each pixel constituting the image is a skin color part.

  In this way, when the skin color model is updated using the skin color information of the detected person before the decollet data is created, the skin color model is adapted to the detected person, and the decollete / non-decollet identification adapted to the individual is applied. Skin color extraction can be performed, and more accurate skin color extraction can be performed.

  In addition to updating the skin color model, the non-skin color model is also updated by using the person's non-skin color information (for example, color information of the clothing part) detected from the frame image of the person attribute identification processing target. The decollet data may be created using the later skin color model and the updated non-skin color model.

  Furthermore, in the person attribute identification method described above, each person attribute identification model is characterized by principal component analysis using a large amount of sample data as prior learning for each category in which each person attribute identification data is identified. Is created by constructing a Gaussian mixture model that expresses the distribution shape of the data included in the category, and when calculating the index value for each person attribute identification data individually, Based on multiple types of human attribute identification data including face data created from the target frame image, the likelihood is calculated for each category using each human attribute identification model constructed by a Gaussian mixture model Is desirable.

  If each person attribute identification model is created by the Gaussian mixture model after the principal component analysis is performed in this way, the dimension compression is performed by the principal component analysis, and the data spread on the space of the compressed dimension. Can be expressed by a Gaussian mixture model. For this reason, for example, in the case of face information, it is possible to deal with various faces such as whether glasses are worn or whether there are wrinkles. In addition, since the likelihood is calculated using a Gaussian mixture model, if this likelihood is calculated in multiple frames for the same person, the integration of the likelihood in multiple frames can be easily realized by Bayes' law as described later It becomes.

  Then, in the case where each person attribute identification model is created by the Gaussian mixture model described above, a plurality of likelihoods are integrated by Bayes' theorem when calculating the information indicating the person attribute identification result. It is desirable.

  When multiple likelihoods are integrated by Bayes' theorem in this way, when another element (information other than face information) for identifying a person attribute such as gender or age is obtained, It is possible to construct a framework that can easily integrate the likelihoods calculated from these elements.

  In addition, in the case where each person attribute identification model is created by the Gaussian mixture model described above, a plurality of likelihoods calculated by a plurality of frames in which the same person is detected are integrated by Bayes' theorem. It is desirable to calculate information obtained by totaling the identification results of the person attributes in the frame.

  In this way, when the likelihoods calculated in a plurality of frames are integrated according to Bayes' theorem, it is possible to further improve the identification accuracy of the person attribute.

  Furthermore, in the case where each person attribute identification model is created by the Gaussian mixture model described above, it is desirable to use prior probabilities when integrating likelihoods by Bayes' theorem.

  Here, the “prior probability” can be obtained by taking statistics in an actual environment.

  In this way, when integration is performed in consideration of prior probabilities, it is possible to improve identification accuracy in a real environment.

  As a system for realizing the person attribute identification method of the present invention described above, the following person attribute identification system of the present invention can be cited.

  That is, the present invention is a person attribute identification system for identifying a person attribute including the sex and / or age of a person photographed by a camera, and includes a plurality of types of persons including facial data for identifying a person attribute Human attribute identification model storage means for storing at least one person attribute identification model indicating the characteristics of data included in a category in which attribute identification data is identified for each person attribute identification data; Stored in a person attribute identification data creation processing means for performing processing for creating each of a plurality of types of person attribute identification data from a frame image of a person attribute identification processing object obtained by photographing a person attribute, and in a person attribute identification model storage means Each person attribute identification data created by the person attribute identification data creation processing means using each corresponding person attribute identification model The likelihood, attribution degree, probability, ratio, or similar index values for indexing the certainty of whether the data is included in the category of the data is calculated individually for each person attribute identification data An index value calculation processing means for performing processing, and a plurality of index values calculated individually for each person attribute identification data by the index value calculation processing means are integrated to calculate information indicating a person attribute identification result And an identification result information calculation processing means for performing the processing.

  In such a human attribute identification system of the present invention, the actions and effects obtained by the aforementioned human attribute identification method of the present invention can be obtained as they are, thereby achieving the object.

  Further, in the above-described personal attribute identification system, the personal attribute identification model storage means stores a facial male model indicating the characteristics of data included in the male category for which facial data is identified as the personal attribute identification model. And a female model for the face indicating the characteristics of the data included in the female category for which the facial data is identified, and the index value calculation processing means is created from the frame image using the male model for the face The index value indicating the certainty that the facial data is included in the male category is calculated, and the facial data created from the frame image using the female model for the facial It is desirable that a process for calculating an index value indicating the certainty of data included in a category is performed.

  Further, in the above-described personal attribute identification system, the personal attribute identification data includes hair style data in addition to the face data, and the human attribute identification model storage means stores the personal attribute identification model as a personal attribute identification model. A male model for hairstyle showing the characteristics of the data included in the male category for which the data is identified and a female model for hairstyle indicating the characteristics of the data included in the female category for which the data for hairstyle is identified are stored. The index value calculation processing means calculates the index value indicating the certainty that the hairstyle data created from the frame image is data included in the male category, using the male model for the hairstyle, Shows the certainty that the hairstyle data created from the frame image using the female model for the hairstyle is data included in the female category. Desirably it is configured to perform a process of calculating an index value.

  In the person attribute identification system described above, the person attribute identification data includes tie data for identifying whether or not the tie is tightened in addition to the face data, and the person attribute identification model storage means Includes a tie model indicating characteristics of data included in a category with a tie in which tie data is identified and a feature of data included in a category without a tie in which tie data is identified as a person attribute identification model. The index value calculation processing means uses the tie model to indicate the certainty that the tie data created from the frame image is data included in the category with the tie. In addition to calculating the index value, using a non-tie model, Thailand for data, it is desirable that is configured to perform a process of calculating an index value that indicates the likelihood of that the data included in the category without tie.

  Further, in the above-described personal attribute identification system, the personal attribute identification data includes decollet data for identifying whether or not a chested dress is worn in addition to the face data. The model storage means includes a decollet model indicating the characteristics of the data included in the chest-with-category category in which the decollet data is identified as a person attribute identification model, and a non-chest-in category in which the decollet data is identified. A non-decollete model indicating the characteristics of the included data is stored, and the index value calculation processing means uses the decollete model and the decollete data created from the frame image is data included in a category with decollete In addition to calculating an index value indicating the certainty of the image, a non-decollete model is used to create a deco Te for data, it is desirable that is configured to perform a process of calculating an index value that indicates the likelihood of that the data included in the category without neckline.

  Furthermore, in the case where a human model for a hairstyle and a female model for a hairstyle are prepared for identifying the human attributes for the hairstyle data described above, the hair color extracted from images taken in various environments Information, non-hair color information, skin color information and hair color model, non-hair color model, skin color model and hair color model storage means for storing the non-skin color model Using the storage means, the skin color model storage means, the non-skin color model storage means, and the skin color information of the person detected from the frame image of the person attribute identification processing target, the skin color model stored in the skin color model storage means is updated. Skin color model update processing means, and the human attribute identification data creation processing means creates the hair color model when creating the hairstyle data from the frame image of the person attribute identification processing target. Hair color model stored in memory means, non-hair color model stored in non-hair color model storage means, updated skin color model stored in skin color model storage means, and non-hair color model stored in non-skin color model storage means It is desirable to use a flesh color model to perform a process of extracting a hairstyle by estimating whether each pixel constituting the frame image of the person attribute identification process target is a hair part.

  Not only the skin color model update processing means but also the non-skin color model update processing means is provided, and for the non-skin color model, the non-skin color information of the person detected from the frame image of the person attribute identification process target (for example, the color information of the clothing part) Etc.) and the hair attribute data is created using the hair color model, the non-hair color model, the updated skin color model, and the updated non-skin color model by the person attribute identification data creation processing means. It is good also as composition to do.

  In the case where a tie model and a non-tie model are prepared for the tie data described above to identify the person attribute, the person attribute identification data creation processing means is a frame image of the person attribute identification process target. When creating the tie data from the image, it is desirable that the processing is performed to extract the edge and create the edge image.

  In addition, in the case where a decollet model and a non-decollete model are prepared for the decollet data described above to identify person attributes, skin color information and non-skin color information extracted from images taken in various environments are used. Skin color model storage means and non-skin color model storage means for storing the skin color model and non-skin color model respectively created using the skin color model storage means, and using the skin color information of the person detected from the frame image of the person attribute identification processing target Skin color model update processing means for performing a skin color model update process stored in the personal attribute identification data creation processing means when creating the decollet data from the frame image of the person attribute identification processing target. Using the updated skin color model stored in the model storage means and the non-skin color model stored in the non-skin color model storage means Te, we are preferable that a configuration for performing a process of extracting a flesh color part by each of the pixels constituting the frame image of the person attribute identity processed to estimate whether a skin color portion.

  Not only the skin color model update processing means but also the non-skin color model update processing means is provided, and for the non-skin color model, the non-skin color information of the person detected from the frame image of the person attribute identification process target (for example, the color information of the clothing part) Etc.), and the decollet data may be created using the updated skin color model and the updated non-skin color model by the person attribute identification data creation processing means.

  Furthermore, in the human attribute identification system described above, each human attribute identification model is characterized by principal component analysis using a large number of sample data as prior learning for each category in which each human attribute identification data is identified. Is a learning model constructed by expressing the distribution shape of the data included in the category by a Gaussian mixture model, and the index value calculation processing means is for the face created by the human attribute identification data creation processing means. It is desirable that a process for calculating likelihood for each category is performed using each person attribute identification model constructed by a Gaussian mixture model based on a plurality of types of person attribute identification data including data. .

  In the configuration in which each person attribute identification model is created using the Gaussian mixture model described above, the identification result information calculation processing unit integrates a plurality of likelihoods calculated by the index value calculation processing unit using Bayes' theorem. It is desirable that the processing is performed.

  In addition, in the case where each person attribute identification model is created using the Gaussian mixture model described above, a plurality of likelihoods calculated by a plurality of frames in which the same person is detected are integrated by Bayes' theorem. It is desirable to include identification result total information calculation processing means for performing processing for calculating information obtained by totaling the identification results of the person attributes in the frames.

  Furthermore, in the case of a configuration in which each person attribute identification model is created using the Gaussian mixture model described above, the identification result information calculation processing means performs processing using prior probabilities when integrating likelihoods according to Bayes' theorem. It is desirable to have a configuration.

  Then, in the case where each person attribute identification model is created using the Gaussian mixture model described above, the identification result total information calculation processing means performs processing using prior probabilities when integrating the likelihood according to Bayes' theorem. It is desirable to be configured to do.

  As described above, according to the present invention, a person attribute identification model is prepared for each of a plurality of types of person attribute identification data including face data, and each of these person attribute identification models is used. After calculating index values such as likelihood separately, the obtained index values are integrated, so that the identification accuracy is improved compared to the case of identifying human attributes using only face information as in the past. In addition to collecting individual identification results based solely on multiple types of person attribute identification data, the individual attribute identification results can be obtained by integrating multiple index values obtained as individual identification results. Therefore, there is an effect that a plurality of pieces of information including face information can be effectively integrated, and high-precision identification can be realized.

  An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows the overall configuration of a person attribute identification system 10 of the present embodiment. In the present embodiment, the person attribute identification system 10 will be described as being a gender identification system that exclusively performs the process of identifying the gender of a person. FIG. 2 is a flowchart showing the overall flow of gender identification processing by the person attribute identification system 10, and FIG. 3 shows the flow of individual identification processing for each person attribute identification data by the person attribute identification system 10. Is shown in a flowchart, and FIG. 4 is a flowchart showing a flow of hairstyle pre-processing including a hair color determination process. 5 is an explanatory diagram of the normalization process, FIG. 6 is an explanatory diagram of a hairstyle extraction image creation process in the hairstyle preprocessing, and FIG. 7 is an explanation of an edge image creation process in the tie preprocessing. FIG. 8 is an explanatory diagram of a skin color extraction image creation process in the decollete pre-process.

  In FIG. 1, a person attribute identification system 10 includes a camera 20 that captures a person who is a gender identification target, a processing unit 30 that performs various processes related to gender identification, and a model storage for personal attribute identification connected to the processing unit 30. Means 50, hair color model storage means 60, non-hair color model storage means 70, skin color model storage means 80, non-skin color model storage means 90, and identification result storage means 95.

  The camera 20 is a color camera such as a CCD camera or a CMOS camera, for example.

  The processing unit 30 includes a frame image capture processing unit 31, a person detection processing unit 32, a normalization processing unit 33, a skin color model update processing unit 34, a non-skin color model update processing unit 35, and personal attribute identification data. A creation processing unit 36, an index value calculation processing unit 37, an identification result information calculation processing unit 38, an identification result total information calculation processing unit 39, and an identification result output processing unit 40 are configured.

  The frame image capture processing unit 31 performs processing for capturing the frame image captured by the camera 20 and sent from the camera 20 into the processing unit 30. The frame image captured by the frame image capture processing means 31 is a color frame image including not only a person who is a gender identification target but also a background image.

  The person detection processing unit 32 performs processing for extracting a person from the color frame image captured by the frame image capture processing unit 31 by using an existing method such as background difference, frame difference, template matching, or the like. . The image obtained by the processing by the person detection processing means 32 is a person image 100 with the background removed as shown in the left part of FIG.

  The normalization processing means 33 is a process for normalizing the person image 100 obtained by the processing by the person detection processing means 32, that is, regardless of the size (number of pixels) of the person shown in the person image 100. The process which makes it the magnitude | size of is performed. At this time, the normalization processing unit 33 determines the normalization target face area image 101 including the eyes and mouth of the person in the person image 100 shown on the left side of FIG. Conversion processing is performed so as to have a certain size (here, the horizontal direction is W pixels and the vertical direction is H pixels). More specifically, for example, conversion is performed such that W × H = 32 pixels × 32 pixels. Then, an image obtained by normalizing the person image 100 by the normalization processing unit 33 is referred to as a normalized image 200 shown in the right part of FIG. In the normalized image 200, a portion corresponding to the normalization target face area image 101 in the person image 100 before normalization is a narrow face area image 201 having a size of W pixels × H pixels. .

  In addition, the normalization processing unit 33 selects from the normalized image 200 a face narrow area image 201 indicated by a dotted line in the figure, a face wide area image 202 indicated by a one-dot chain line in the figure, and a dotted line in the figure. A process of cutting out the clothing region image 203 indicated by the above is performed.

  Further, the normalization processing unit 33 uses, for example, a skin color region image 204 for extracting the skin color information of the detected person used in the update processing by the skin color model update processing unit 34 described later from the normalized image 200. As the non-skin color region image 205 for extracting the non-skin color information of the detected person used in the update process by the non-skin color model update processing means 35 described later, while cutting out the region image of the cheek part indicated by the solid line inside, For example, a process of cutting out the region image of the trunk portion indicated by the solid line in the drawing is performed.

  The cut-out positions of the wide face area image 202, the clothes area image 203, the skin color area image 204, and the non-skin color area image 205 that are cut out from the normalized image 200 include the face cut-out area image. It is determined relative to the cutout position 201. For example, when the face narrow region image 201 is W × H = 32 pixels × 32 pixels, the size of the face wide region image 202 is 64 pixels × 64 pixels, and the base position of the face narrow region image 201 is For example, the size of the clothing region image 203 is set to 64 pixels × 64 pixels, and the top side position is lowered by 16 pixels from the bottom side position of the face wide region image 202.

  The skin color model update processing means 34 performs an update process of the skin color model 81 stored in the skin color model storage means 80 using the skin color information of the detected person. In this embodiment, the skin color model 81 is constructed by a Gaussian mixture model as will be described later. More specifically, the skin color model update processing means 34 reconstructs a Gaussian mixture model using the skin color information of each pixel constituting the skin color area image 204 of FIG.

  The non-skin color model update processing means 35 performs update processing of the non-skin color model 91 stored in the non-skin color model storage means 90 using the non-skin color information of the detected person. In the present embodiment, the non-skin color model 91 is constructed by a Gaussian mixture model in the same manner as the skin color model 81. More specifically, the non-skin color model update processing unit 35 reconstructs a Gaussian mixture model using the non-skin color information of each pixel constituting the non-skin color region image 205 of FIG.

  In the present embodiment, the personal attribute identification data creation processing means 36 uses facial data for identifying whether a male face or a female face, or a male hairstyle or female as element information for identifying a person attribute. 4 types of data: hair style data for identifying whether or not the hair style, tie data for identifying whether or not the tie is worn, and decollet data for identifying whether or not a chest is worn A process for creating person attribute identification data is performed.

  More specifically, the person attribute identification data creation processing means 36 includes face basic data creation processing means 36A and face compression data creation processing means 36B for creating face basic data and face compression data as face data, respectively. Hairstyle basic data creation processing means 36C and hairstyle compression data creation processing means 36D for creating hairstyle basic data and hairstyle compression data as hairstyle data, and tie for creating tie basic data and tie compression data as tie data, respectively. Basic data creation processing means 36E and tie compression data creation processing means 36F, decollete basic data creation processing means 36G and decollet compression data creation processing means 36H for creating decollete basic data and decollet compression data as decollet data, respectively. It is configured to include a.

  The face basic data creation processing unit 36A performs face preprocessing (step S603 in FIG. 3 described later) on the face narrow region image 201 (see FIG. 5) to create face basic data 501. In this face preprocessing, the data of the narrow face region image 201 cut out from the color frame image is converted into a monochrome image in order to absorb the difference in the camera 20 and the shooting environment, and then the distribution of each gradation in the image is substantially reduced. Histogram smoothing processing (processing to disperse the brightness of each pixel in the image substantially uniformly in 256 gradations), which is a density correction method for making equal values, is performed, and W × H dimensions (for example, 32 × 32 = 1024). Create a (dimension) vector. This vector data is the basic face data 501 (see FIG. 3).

  The face compression data creation processing unit 36B performs principal component analysis (PCA: principal component analysis) on the face basic data 501 (see FIG. 3) (step S607 in FIG. 3 described later), and creates face compression data 601. The processing is performed. In this PCA processing, for example, the principal component axis (for example, 50 × 50 = 32 × 32 = 1024-dimensional vector data) obtained by performing principal component analysis using a large number of sample data in advance. It is converted into vector data (for example, 50-dimensional vector data) having coordinate values in a compressed dimension (for example, 50 dimensions) space determined by the axis of the book. This low-dimensional vector data is the face compression data 601 (see FIG. 3). Therefore, each dimension element value (coordinate value in each axial direction) of the face compression data 601 is a scalar quantity determined as a weight for each principal component, that is, a weight for a base vector facing the axial direction of each principal component.

  The hairstyle basic data creation processing unit 36C performs a hairstyle preprocessing (step S604 in FIG. 3 described later) on the wide face region image 202 (see FIGS. 5 and 6) to create the hairstyle basic data 502. is there. In this hairstyle pre-processing, first, a hair part is extracted from the wide face area image 202 to create a hairstyle extraction image 400 as shown in FIG. When performing this hair part extraction process, after removing the narrow face area image 201 from the wide face area image 202, the hair color part is extracted from the remaining part from which the narrow face area image 201 is removed. Next, the obtained hairstyle extraction image 400 is converted into a monochrome image, as in the case of the face data, and then subjected to a histogram smoothing process to remove the influence of the camera 20 and the shooting environment. An H-dimensional vector (for example, 32 × 32 = 1024 dimensions) is created. Note that the image may be reduced to such an extent that the hairstyle can be recognized, and a lower-dimensional vector may be created. The vector created in this way is hairstyle basic data 502 (see FIG. 3).

  The hairstyle compression data creation processing unit 36D performs principal component analysis (PCA) on the hairstyle basic data 502 (see FIG. 3) (step S608 in FIG. 3 described later), and performs processing to create the hairstyle compression data 602. . In this PCA process, as in the case of the PCA process of the face basic data 501 (step S607 in FIG. 3), the dimension compression process is performed, and the hairstyle compressed data, which is vector data reduced in the hairstyle basic data 502, is obtained. 602 is created.

  The tie basic data creation processing unit 36E performs tie pre-processing (step S605 in FIG. 3 described later) on the clothing region image 203 (see FIGS. 5 and 7) to create tie basic data 503. . In this tie preprocessing, after the clothing region image 203 is converted into a monochrome image, contour extraction filter processing such as primary differentiation (gradient) and secondary differentiation (Laplacian) is performed, and an edge image 410 as shown in FIG. 7 is obtained. create. A vector representing the edge image 410 is tie basic data 503 used for tie / non-tie discrimination (see FIG. 3).

  The tie compressed data creation processing unit 36F performs principal component analysis (PCA) on the tie basic data 503 (see FIG. 3) (step S609 in FIG. 3 described later), and performs processing to create tie compressed data 603. . In this PCA process, as in the case of the PCA process of the face basic data 501 (step S607 in FIG. 3), dimensional compression processing is performed, and tie compressed data that is vector data reduced in the tie basic data 503 is reduced. 603 is created.

  The decollete basic data creation processing means 36G performs a process of creating decollete basic data 504 by performing a decollete pre-process (step S606 in FIG. 3 described later) on the clothing region image 203 (see FIGS. 5 and 8). . In this decollete pre-processing, skin color extraction is performed from the clothing region image 203 to create a skin color extraction image 420 as shown in FIG. The skin color extracted image 420 represented by a vector is decollete basic data 504 used for decollete / non-decollete discrimination (see FIG. 3).

  The decollet compressed data creation processing unit 36H performs principal component analysis (PCA) on the decollet basic data 504 (see FIG. 3) (step S610 in FIG. 3 described later), and performs a process of creating the decollet compressed data 604. . In this PCA process, similarly to the case of the PCA process of the face basic data 501 (step S607 in FIG. 3), the dimensional compression process is performed, and the decollet compressed data that is vector data reduced in dimension with respect to the decollet basic data 504 is obtained. 604 is created.

  The index value calculation processing means 37 uses the corresponding person attribute identification models 51 to 58 stored in the person attribute identification model storage means 50 to create each person created by the person attribute identification data creation processing means 36. Likelihood is used as an index value for indexing the probability that the attribute identification data 501 to 504 (or 601 to 604 may be considered as data included in which category). Individual calculation processing is performed for each person attribute identification data (in this embodiment, for each category in which each person attribute identification data is identified). At this time, in the present embodiment, as will be described later, since the person attribute identification models 51 to 58 are constructed by a Gaussian mixture model, each likelihood Pr is calculated based on the following equation (1). The Gaussian mixture model is a model expressed by a Gaussian mixture distribution formed by a plurality of Gaussian distributions (normal distributions) defined by representative vectors and variances. Further, the index value calculation processing unit 37 performs a process of storing the calculated likelihoods in the identification result storage unit 95.

_{Pr = Σ i {π i ×} p i (X; θ)} (i = 1~g) ········ (1)

Here, i is a number assigned to each Gaussian distribution, g is the number of Gaussian distributions forming the Gaussian mixture distribution, Σ _i is the sum of i = 1 to g, and π _i is , X is input data (compressed data 601 to 604 as personal attribute identification data), and p _i (X; θ) is output from each Gaussian distribution. Θ is a parameter (average value, variance-covariance matrix) representing each Gaussian distribution.

  More specifically, the index value calculation processing unit 37 includes a male face likelihood calculation processing unit 37A, a female face likelihood calculation processing unit 37B, a male hairstyle likelihood calculation processing unit 37C, and a female hairstyle likelihood calculation process. Means 37D, tie likelihood calculation processing means 37E, non-tie likelihood calculation processing means 37F, decollet likelihood calculation processing means 37G, and non-decollet likelihood calculation processing means 37H are provided.

  The male face likelihood calculation processing means 37A uses the face male model 51 stored in the person attribute identification model storage means 50 to convert the face compression data 601 (and thus the face basic data 501) into a male face. Male face likelihood 701 (see FIG. 3), which is an index value indicating the probability of being data included in the category (that is, data created from an image showing a male face). The processing is performed.

  The female face likelihood calculation processing unit 37B uses the female model 52 for the face stored in the model storage unit 50 for identifying human attributes, and the compressed face data 601 (and thus the basic face data 501) is converted into a female face. The female face likelihood 702 (see FIG. 3), which is an index value indicating the certainty of being data included in the category (that is, data created from an image showing a female face), is calculated. The processing is performed.

  The male hairstyle likelihood calculation processing unit 37C uses the hairstyle male model 53 stored in the person attribute identification model storage unit 50 to convert the hairstyle compression data 602 (and hence the hairstyle basic data 502) into a male hairstyle. The male hairstyle likelihood 703 (see FIG. 3), which is an index value indicating the certainty that the data is included in the category (that is, data created from an image showing a male hairstyle) is calculated. The processing is performed.

  The female hairstyle likelihood calculation processing unit 37D uses the hairstyle female model 54 stored in the person attribute identification model storage unit 50 to convert the hairstyle compression data 602 (hence, the hairstyle basic data 502) into a female hairstyle. The female hairstyle likelihood 704 (see FIG. 3), which is an index value indicating the certainty of being data included in the category (that is, data created from an image showing a female hairstyle), is calculated. The processing is performed.

  The tie likelihood calculation processing means 37E uses the tie model 55 stored in the person attribute identification model storage means 50, and the tie compressed data 603 (and hence the tie basic data 503) is included in the category of the tie. To calculate a tie likelihood 705 (see FIG. 3), which is an index value indicating the certainty that the data is the data (the data is created from an image showing a person wearing a tie). Is.

  The non-tie likelihood calculation processing unit 37F uses the non-tie model 56 stored in the person attribute identification model storage unit 50, and the category in which the tie compression data 603 (therefore, the tie basic data 503) does not tie. Non-tie likelihood 706 (see FIG. 3), which is an index value indicating the certainty that the data is included in the image (that is, data created from an image showing a person who is not wearing a tie). The processing is performed.

  The decollet likelihood calculation processing unit 37G uses the decollete model 57 stored in the person attribute identification model storage unit 50, and the decollete compressed data 604 (hence, the decollete basic data 504) is a chested outfit (decollete). Decollet likelihood 707 which is an index value indicating the probability that the data is included in the category wearing the item (the data is created from an image in which a person wearing a chested clothes is shown) (See FIG. 3) is calculated.

  The non-decollet likelihood calculation processing unit 37H uses the non-decollete model 58 stored in the person attribute identification model storage unit 50, so that the decollet compressed data 604 (and hence the decollete basic data 504) is not a chestless clothes ( Non-decollete) is an index value indicating the probability of being data included in a category (that is, data created from an image showing a person wearing clothes with no chest) Processing for calculating the decollet likelihood 708 (see FIG. 3) is performed.

  The identification result information calculation processing means 38 is based on the following formula (2), for each individual person attribute identification data (here, for each category in which each individual attribute identification data is identified) by the index value calculation processing means 37. By integrating a plurality of (here, 8) likelihoods 701 to 708 (see FIG. 3) individually calculated as information indicating the identification result of the person's attribute (here, gender), the male probability And the ratio of the probability of the woman (or the value of the ratio) are calculated. The following equation (2) is an equation obtained by applying the Bayes' theorem on the assumption that each element information of face, hairstyle, tie, and decollete is independent. Further, the identification result information calculation processing means 38 performs processing for storing the ratio (or ratio value) between the calculated male probability and female probability in the identification result storage means 95.

Probability of man: Probability of woman = Pr [male] × Pr [X_face | M_m_face] × Pr [X_hair | M_m_hair]
× (Pr [tie | Men] / Pr [X_tie | M_tie]
+ Pr [non-tie | Men] / Pr [X_tie | M_non-tie])
× (Pr [dec | Male] ・ Pr [X_dec | M_dec]
+ Pr [non-dec | M] / Pr [X_dec | M_non-dec])
: Pr [female] × Pr [X_face | M_f_face] × Pr [X_hair | M_f_hair]
× (Pr [tie | female] / Pr [X_tie | M_tie]
+ Pr [non-tie | female] / Pr [X_tie | M_non-tie])
× (Pr [dec | female] ・ Pr [X_dec | M_dec]
+ Pr [non-dec | female] ・ Pr [X_dec | M_non-dec])
(2)

  Here, X_face is face compression data 601 created by performing principal component analysis (PCA) on the face basic data 501 by the face compression data creation processing unit 36B (see FIG. 3). M_m_face is a male model 51 for face, M_f_face is a female model 52 for face, and Pr [X_face | M_m_face] and Pr [X_face | M_f_face] are obtained by inputting X_face into M_m_face and M_f_face, respectively. The male face likelihood 701 and the female face likelihood 702 (see FIG. 3).

  Similarly, X_hair is hairstyle compression data 602 created by performing principal component analysis (PCA) on the hairstyle basic data 502 by the hairstyle compression data creation processing unit 36D (see FIG. 3). M_m_hair is a male model 53 for hairstyle, M_f_hair is a female model 54 for hairstyle, and Pr [X_hair | M_m_hair] and Pr [X_hair | M_f_hair] are obtained by inputting X_hair to M_m_hair and M_f_hair, respectively. The male hairstyle likelihood 703 and the female hairstyle likelihood 704 (see FIG. 3).

  X_tie is tie compression data 603 created by performing principal component analysis (PCA) on the tie basic data 503 by the tie compression data creation processing unit 36F (see FIG. 3). M_tie is a tie model 55, M_non-tie is a non-tie model 56, and Pr [X_tie | M_tie] and Pr [X_tie | M_non-tie] input X_tie to M_tie and M_non-tie, respectively. The resulting tie likelihood 705 and non-tie likelihood 706 are obtained (see FIG. 3).

  Similarly, X_dec is decollet compressed data 604 created by performing principal component analysis (PCA) on the decollet basic data 504 by the decollet compressed data creation processing unit 36H (see FIG. 3). M_dec is a decollet model 57, M_non-dec is a non-decorte model 58, and Pr [X_dec | M_dec] and Pr [X_dec | M_non-dec]) input X_dec to M_dec and M_non-dec, respectively. The decollet likelihood 707 and the non-decollet likelihood 708 obtained in this way (see FIG. 3).

  Furthermore, Pr [male] is the appearance probability of a man, Pr [tie | male] is the probability of a tie when a man, and Pr [non-tie | male] is a man The probability of not having a tie, Pr [dec | male] is the probability of being a decollete (with chest), and Pr [non-dec | male] is the decollete of a man. There is no probability (no chest), Pr [female] is the appearance probability of a woman, Pr [tie | female] is the probability of a tie when a woman, Pr [non-tie ｜ Woman is the probability that she is not a tie when she is a woman, Pr [dec | Woman] is the probability that she is a decollete when she is a woman, and Pr [non-dec | ] Is the probability of not being a decollete (no chest) when she is a woman. These are prior probabilities and are obtained by taking statistics in a real environment.

The identification result total information calculation processing means 39, when the same person is continuously detected in a plurality of frames, obtains information that totals the identification results of the attributes (here, gender) of the persons in the plurality of frames. The calculation process is performed. More specifically, the identification result total information calculation processing unit 39 includes data obtained from the frame currently being processed, and data obtained up to the previous frame stored in the identification result storage unit 95. Is used to calculate the ratio (or ratio value) between the product of the male probabilities calculated for each frame by the identification result information calculation processing means 38 and the product of the female probabilities calculated for each frame. Do. For example, the frame number n = 1, 2, 3, ..., when the same person is detected in each frame of N, the probability of man calculated for each frame by the identification result information calculation processing means 38 and Pm ^n, Woman When the probability and ^{Pf n, (Pm 1 × Pm} 2 × Pm 3 × ... × Pm n) :( Pf 1 × Pf 2 × Pf 3 × ... × Pf n) of the ratio or (Pm ¹ × Pm ² ×, ^{Pm 3 × ... × Pm N)} / (Pf 1 × Pf 2 × Pf 3 × ... × Pf N) that may be a value (reciprocal of the ratio.) is calculated. Further, as in the latter case, when the ratio value is calculated, the ratio value (or reciprocal number) of the ratio between the male probability and the female probability calculated for each frame by the identification result information calculation processing unit 38 may be used. The same thing can be said when calculating the product. For example, in the above example, (Pm ¹ / Pf ¹ ) × (Pm ² / Pf ² ) × (Pm ³ / Pf ³ ) ×... × (Pm ^N / Pf ^N ) may be calculated. Here, the term probability Pm ^n, probability Pf ⁿ of the woman of the man is not intended to necessarily mean probability of 1 by adding ^{(Pm n + Pf n = 1} ), and compares the magnitude of both the value, either Any value can be used as long as it is possible to relatively determine how high the probability is. Further, the identification result total information calculation processing means 39 is a ratio (or a value of the ratio) between the product of the calculated male probability and the product of the female probability, or the value of the ratio between the calculated male probability and the female probability. A process for storing the product of (may be an inverse number) in the identification result storage unit 95 is performed.

  Accordingly, the identification result total information calculation processing means 39 is a plurality of (here, 1) calculated by the index value calculation processing means 37 for each of a plurality of frames (n = 1, 2,..., N) in which the same person is detected. The processing is performed by integrating the likelihoods 701 to 708 (see FIG. 3) of 8 per frame, and the total of N frames is 8 × N.

  The identification result output processing unit 40 displays the calculation result by the identification result information calculation processing unit 38 and / or the identification result total information calculation processing unit 39 on a screen of a display device such as a CRT display or a liquid crystal display (not shown). Alternatively, output processing is performed by an output device such as a printer or plotter (not shown).

  The person attribute identification model storage means 50 stores the person attribute identification model created for each person attribute identification data (in this embodiment, for each category in which each person attribute identification data is identified). It is something to remember. These human attribute identification models show the characteristics of the data included in each category in which each human attribute identification data is identified. In this embodiment, these models are obtained from a large number of sample data included in each category. Each distribution is expressed by a Gaussian mixture model.

  More specifically, in the person attribute identification model storage means 50, a facial man indicating characteristics of data included in a male category for which facial data (face basic data 501 and face compression data 601) is identified. The model 51, the female model 52 for the face showing the characteristics of the data included in the female category for which the facial data is identified, and the male model for which the hairstyle data (hairstyle basic data 502, hairstyle compression data 602) is identified. A male model 53 for hairstyle indicating the characteristics of data included in the category, a female model 54 for hairstyle indicating the characteristics of data included in the category of the woman whose hairstyle data is identified, and tie data (tie tie basic data) A tie model 55 indicating the characteristics of data included in a category with a tie from which tie compression data 603) is identified, and tie data is Characteristics of data included in a category with chest relief in which non-tie models 56 indicating characteristics of data included in the category without tie and separate decollet data (decollet basic data 504, decollet compression data 604) are identified And a non-decollete model 58 indicating the characteristics of data included in the chestless category in which the decollet data is identified.

  Each of the models 51 to 58 stored in the person attribute identification model storage unit 50 is a model created by prior learning before the person attribute identification system 10 is operated. In this pre-learning, first, a large number of sample data included in a category to be identified is collected, and principal component analysis (PCA) is performed to determine the principal component axis. For example, taking the case of facial data as an example, various male and female facial data (processing similar to the facial preprocessing in step S603 of FIG. 3 performed after the processing by the person attribute identification system 10 is started) is performed. A principal component axis is determined by performing principal component analysis (PCA) on the obtained face basic data, that is, a large number of sample data corresponding to the face basic data 501 obtained by real-time processing from a frame image to be identified. At this time, a higher-order component (for example, a cumulative contribution rate of 80% or the like) is used as a main component, and dimensional compression of face basic data is performed to form a compressed low-dimensional space. For example, when a large number of sample data corresponding to the face basic data 501 is 32 × 32 = 1024-dimensional vector data, the basis vectors facing the axial directions of the principal components are the same 1024-dimensional vectors, For example, 50 1024-dimensional basis vectors stand to form a space compressed to 50 dimensions.

  Then, a distribution of male face data and female face data is created on this compressed dimension space using a large number of sample data corresponding to the face basic data 501. This distribution is created by calculating the weight of each principal component, that is, the weight for the basis vector facing the axial direction of each principal component, for each of a large number of sample data corresponding to the face basic data 501, This is performed by obtaining a low-dimensional vector (vector data corresponding to the face compression data 601) as a dimensional element value (coordinate value in each axis direction). Thereafter, the distribution model of the male face data and the female face data created on the compressed dimension space is expressed by a Gaussian mixture model, so that the male model 51 for the face and the female model for the face are represented. 52 is created. Thus, taking the female model 52 for a face as an example, various female faces such as wearing glasses or not wearing glasses can be expressed by a Gaussian mixture model. The same applies to the hair style data, tie data, and decollet data as in the case of the face data. In addition, since the construction technique itself of the Gaussian mixture model is a known technique and is described in Non-Patent Document 4 and the like described above, detailed description thereof is omitted here.

  The hair color model storage means 60 stores the hair color model 61 used in the hair color determination process (see FIGS. 4 and 6) in the hairstyle pre-process (the process in step S604 in FIG. 3).

  The non-hair color model storage means 70 stores a non-hair color model 71 used in the hair color determination process (see FIGS. 4 and 6) in the hairstyle pre-process (the process in step S604 in FIG. 3).

  The skin color model storage means 80 performs a skin color extraction process (a process in step S606 in FIG. 3) and a hair color determination process (a process in step S606 in FIG. 3) in a hair color determination process (a process in step S604 in FIG. 3) and a decollete preprocess (a process in step S606 in FIG. 3). The skin color model 81 used in FIG. 8) is stored.

  The non-skin color model storage unit 90 performs the hair color determination process (see FIGS. 4 and 6) in the hairstyle pre-process (process in step S604 in FIG. 3) and the skin color extraction process in the decollete pre-process (process in step S606 in FIG. 3). The non-skin color model 91 used in (see FIG. 8) is stored.

  The hair color model 61, the non-hair color model 71, the skin color model 81, and the non-skin color model 91 are models created by performing prior learning before the person attribute identification system 10 is operated. In this pre-learning, hair color, non-hair color, skin color, and non-skin color are extracted from human color images in various environments, and sample data is accumulated in a three-dimensional color space formed by a pigment expression system such as RGB or HSV. The distribution shape of these data is expressed by a Gaussian mixture model.

  Furthermore, for the skin color model 81 and the non-skin color model 91, not only the color data prepared before the operation of the person attribute identification system 10 but also the color data of the person detected after the start of processing by the person attribute identification system 10 is added. Therefore, more accurate extraction can be realized. Specifically, after the processing by the person attribute identification system 10 is started, the skin color model update processing unit 34 detects the color data of the person detected from the latest frame image currently being processed (the skin color region image 204 in FIG. 5). The skin color model 81 adapted to the person can be created by performing the process of reconstructing the Gaussian mixture model (step S602 in FIG. 3) by adding the skin color information of each pixel that constitutes. Similarly, after the processing by the person attribute identification system 10 is started, the non-skin color model update processing unit 35 detects the person color data (non-skin color region image 205 in FIG. 5) detected from the latest frame image currently being processed. The non-skin color model 91 adapted to the person can be created by performing the process (step S602 in FIG. 3) for reconstructing the Gaussian mixture model by adding the non-skin color information of each pixel constituting the.

  The identification result storage unit 95 includes the likelihoods 701 to 708 (see FIG. 3) calculated in each frame by the index value calculation processing unit 37 and the males in each frame calculated by the identification result information calculation processing unit 38. The ratio (or ratio value) between the probability and the female probability, or the ratio (or ratio) between the product of the male probability and the product of the female probability in a plurality of frames calculated by the identification result total information calculation processing means 39. Or the product of the ratio between the probability of the man and the probability of the woman (may be an inverse number) in a plurality of frames is stored for each detected person.

  In the above, each of the processing means 31 to 40 included in the processing means 30 is composed of one or a plurality of computers (not only a personal computer but also a higher-level model or a general purpose) constituting the person attribute identification system 10. A central processing unit (CPU) provided inside a personal attribute identification processing device, a calculation unit incorporated in an electric product other than a personal computer, and the operation procedure of this CPU. It is realized by one or a plurality of programs to be defined.

  Further, as the person attribute identification model storage means 50, the hair color model storage means 60, the non-hair color model storage means 70, the skin color model storage means 80, the non-skin color model storage means 90, and the identification result storage means 95, for example, A hard disk, ROM, EEPROM, flash memory, RAM, MO, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, FD, magnetic tape, or a combination thereof can be employed. .

  In the present embodiment, the person attribute identification system 10 performs person gender identification processing as follows.

  First, before the person attribute identification system 10 is operated, pre-learning is performed, and a hair color model 61, a non-hair color model 71, a skin color model 81, and a non-skin color model 91 are respectively created using a Gaussian mixture model, and a hair color model is created. Each of the storage means 60, the non-hair color model storage means 70, the skin color model storage means 80, and the non-skin color model storage means 90 is stored, and the male model 51 for the face, the female model 52 for the face, and the male model for the hairstyle. 53, a female model 54 for hairstyle, a tie model 55, a non-tie model 56, a decollete model 57, and a non-decollete model 58 are each created by a Gaussian mixture model and stored in the person attribute identification model storage means 50.

  Also, the prior probabilities Pr [male], Pr [tie | male], Pr [non-tie | male], Pr [dec | male], Pr [non-dec | male], Pr [female], Pr [ tie | female, Pr [non-tie | female], Pr [dec | female], and Pr [non-dec | female] are obtained by taking statistics in an actual environment.

  Next, in FIG. 2, the person attribute identification system 10 is activated to start the gender identification process of the person (step S1), and the frame image capture processing means 31 captures a color frame image captured by the camera 20 ( Step S2).

  Subsequently, the person detection processing unit 32 performs a process of extracting a person from the color frame image captured by the frame image capture processing unit 31 by an existing method such as background difference, frame difference, template matching, or the like. (Step S3). Thereby, when a person is reflected in the frame image, a person image 100 with the background removed as shown in the left part of FIG. 5 is obtained.

  Then, it is determined whether or not a person has been detected by the processing by the person detection processing unit 32 (step S4). If it is determined that the person image 100 is obtained by detecting the person, the normalization processing unit 33 A process of normalizing the person image 100 obtained by the process by the person detection processing unit 32 is performed (step S5). Thereby, the normalized image 200 shown in the right part of FIG. 5 is obtained. Further, the normalization processing unit 33 performs a process of cutting out the narrow face area image 201, the wide face area image 202, and the clothing area image 203 from the normalized image 200, and the skin color area image 204 and the non-skin color area image 205. The process which cuts out is also performed. When a plurality of persons are detected in the frame image by the process by the person detection processing unit 32, the normalization process by the normalization processing unit 33 is performed for each detected person.

  Thereafter, individual identification processing is performed for each element information of face information, hairstyle information, tie information, and decollet information using the images 201 to 205 cut out from the normalized image 200 of FIG. 5 (step S6). ). FIG. 3 is a flowchart showing the detailed flow of this individual identification process. When a plurality of persons are detected in the frame image by the process by the person detection processing unit 32, the individual identification process in FIG. 3 is performed for each detected person.

  In FIG. 3, first, each image 201-205 cut out from the normalized image 200 and stored in a memory (not shown) is read (step S601), and the skin color model update processing means 34 uses the skin color information ( The skin color model 81 stored in the skin color model storage unit 80 is updated using the skin color information of each pixel constituting the skin color region image 204 in FIG. 5 and detected by the non-skin color model update processing unit 35. Using the non-skin color information of the person (non-skin color information of each pixel constituting the non-skin color region image 205 in FIG. 5), the non-skin color model 91 stored in the non-skin color model storage unit 90 is updated (step) S602).

  Next, the face basic data creation processing unit 36A creates face basic data 501 by performing face pre-processing using the face narrow region image 201 (step S603). Further, the hairstyle basic data creation processing unit 36C creates hairstyle basic data 502 by performing hairstyle preprocessing using the wide face area image 202 (step S604). FIG. 4 is a flowchart showing the detailed flow of the hairstyle pretreatment.

  In FIG. 4, the hairstyle basic data creation processing unit 36 </ b> C extracts the hair color portion in the remaining portion after removing the narrow face region image 201 from the wide face region image 202. That is, the color information of the pixels constituting the remaining part from which the narrow face area image 201 is removed is read (step S60401), and the color data is stored in the hair color model storage means 60 and the non-hair color model 61. Non-hair color model 71 stored in color model storage means 70, updated skin color model 81 stored in skin color model storage means 80, and updated non-skin color stored in non-skin color model storage means 90 Based on the above-described equation (1), the hair color likelihood calculation process (step S 60402), the non-hair color likelihood calculation process (step S 60403), the skin color likelihood calculation process (step S 60404), and A non-skin color likelihood calculation process (step S60405) is performed to obtain a hair color likelihood 301, a non-hair color likelihood 302, a skin color likelihood 303, and a non-skin color likelihood 3. 4 performs a process of calculating respectively.

  Subsequently, the hairstyle basic data creation processing unit 36C determines whether the hair color likelihood 301 is larger than the non-hair color likelihood 302 and the skin color likelihood 303 is smaller than the non-skin color likelihood 304 ( In step S60406), if this condition is satisfied, it is estimated that the pixel currently being processed is a hair part (step S60407). On the other hand, if this condition is not satisfied, the pixel currently being processed is other than a hair part. (Step S60408).

  Then, it is determined whether or not there is a next pixel to be processed in the remaining portion obtained by removing the narrow face region image 201 from the wide face region image 202 (step S60409). Then, the process returns to step S60401 again, and thereafter, the processes of steps S60401 to S60409 are repeated until there are no more pixels to be processed. Thus, by leaving only the pixels that are estimated to be hair parts, a hairstyle extraction image 400 as shown in FIG. 6 is obtained.

  Then, the hairstyle basic data creation processing unit 36C performs a histogram smoothing process on the obtained hairstyle extracted image 400 after performing a conversion process to a monochrome image (step S60410), as in the case of facial data. (Step S60411). Thereby, hairstyle basic data 502 is obtained.

  In FIG. 3, the tie basic data creation processing unit 36E performs tie pre-processing using the clothing region image 203 to create tie basic data 503 (step S605). At this time, the tie basic data creation processing unit 36E converts the clothing region image 203A as shown in FIG. 7 into a monochrome image, and then performs contour extraction filter processing such as primary differentiation (gradient) and secondary differentiation (Laplacian). Then, an edge image 410 as shown in FIG. 7 is created. Then, the tie basic data 503 is obtained by expressing the edge image 410 as a vector.

  Further, in FIG. 3, the decollete basic data creation processing unit 36G performs precollete preprocessing using the clothing region image 203 to create the decollete basic data 504 (step S606). At this time, the decollete basic data creation processing means 36G stores the updated skin color model 81 and non-skin color model memory stored in the skin color model storage means 80 for each pixel constituting the clothing region image 203B as shown in FIG. Using the updated non-skin color model 91 stored in the means 90, that is, using the skin color model 81 and the non-skin color model 91 used in the hairstyle extraction process described above, the skin color based on the above-described equation (1) The likelihood and the non-skin color likelihood are respectively calculated, and pixels whose skin color likelihood output from the skin color model 81 is larger than the non-skin color likelihood output from the non-skin color model 91 are extracted as skin color pixels. A skin color extraction image 420 as shown in FIG. And the decollete basic data 504 is obtained by expressing this skin color extraction image 420 with a vector.

  Subsequently, the face basic data 501, the hairstyle basic data 502, and the tie basic data are obtained by the face compression data creation processing means 36B, the hairstyle compression data creation processing means 36D, the tie compression data creation processing means 36F, and the decollet compression data creation processing means 36H. Principal component analysis (PCA) processing is performed on the 503 and the decollete basic data 504 (steps S607 to S610), and the face compression data 601, the hairstyle compression data 602, the tie compression data 603, and the decollet compression data 604 are created.

  Thereafter, the face compression data 601 is input by the male face likelihood calculation processing means 37A using the male model 51 for the face stored in the person attribute identification model storage means 50 based on the above-described equation (1). A male face likelihood calculation process is performed as data (step S611), and a male face likelihood 701 is calculated. Also, the face compression data 601 is input by the female face likelihood calculation processing means 37B using the female model 52 for the face stored in the person attribute identification model storage means 50 based on the above-described equation (1). A female face likelihood calculation process is performed as data (step S612), and a female face likelihood 702 is calculated.

  Similarly, the hairstyle compression data 602 is obtained by the male hairstyle likelihood calculation processing means 37C using the hairstyle male model 53 stored in the person attribute identification model storage means 50 based on the aforementioned equation (1). A male hairstyle likelihood calculation process is performed as input data (step S613), and a male hairstyle likelihood 703 is calculated. Also, the female hairstyle likelihood calculation processing unit 37D inputs the hairstyle compression data 602 using the female model 54 for hairstyle stored in the person attribute identification model storage unit 50 based on the above-described equation (1). A female hairstyle likelihood calculation process is performed as data (step S614), and a female hairstyle likelihood 704 is calculated.

  Similarly, the tie likelihood calculation processing unit 37E uses the tie model 55 stored in the person attribute identification model storage unit 50 based on the above-described equation (1) to tie the tie compressed data 603 as input data. Likelihood calculation processing is performed (step S615), and a tie likelihood 705 is calculated. Further, the non-tie likelihood calculation processing unit 37F uses the non-tie model 56 stored in the person attribute identification model storage unit 50 as the input data based on the above-described equation (1). A non-tie likelihood calculation process is performed (step S616), and a non-tie likelihood 706 is calculated.

  Similarly, the decollet likelihood calculation processing unit 37G uses the decollet model 57 stored in the person attribute identification model storage unit 50 based on the above-described equation (1), and uses the decollet compressed data 604 as input data. A likelihood calculation process is performed (step S617), and a decollet likelihood 707 is calculated. Further, the non-decollet likelihood calculation processing unit 37H uses the non-decollete model 58 stored in the person attribute identification model storage unit 50 as the input data based on the above-described equation (1). A non-decollet likelihood calculation process is performed (step S618), and a non-decollet likelihood 708 is calculated. This completes the individual identification process (step S6 in FIG. 2; steps S601 to S618 in FIG. 3).

  Subsequently, in FIG. 2, an integration process is performed by the identification result information calculation processing unit 38 and the identification result total information calculation processing unit 39 (step S7). That is, the identification result information calculation processing unit 38 integrates the eight likelihoods 701 to 708 based on the above-described equation (2), and information indicating the identification result of the person's attribute (here, gender) The ratio (or ratio value) of the probability and the probability of the woman is calculated.

  Further, when the same person is continuously detected in a plurality of frames (a frame currently being processed and a frame processed before that), it is stored in the identification result storage unit 95 by the identification result total information calculation processing unit 39. Product of the male probabilities calculated for each frame by the identification result information calculation processing means 38 as the total information of the identification results of the attributes (here, gender) of the person in the plurality of frames. And the ratio of the product of the probability of woman calculated for each frame (or the value of the ratio), or the probability of the man and the probability of the woman calculated for each frame by the identification result information calculation processing means 38 A process for calculating the product of the ratio values (which may be reciprocals) is performed. At this time, whether or not the detected person in each frame is the same person can be determined based on the position information of the person in each frame image, the skin color information of the face, the face feature information, or the like. When a plurality of persons are detected in the frame image by the processing by the person detection processing unit 32, the integration processing by the identification result information calculation processing unit 38 and the identification result total information calculation processing unit 39 is performed for each detected person. To do.

  Thereafter, if there is a calculation result by the identification result information calculation processing unit 38 and a calculation result by the identification result total information calculation processing unit 39 by the identification result output processing unit 40, the calculation result is displayed on a CRT display or a liquid crystal display (not shown). Or the like or output by an output device such as a printer or plotter (not shown) (step S8). The display on the screen of these display devices and the output to the output device may be numerical display / output, graph display / output, a combination thereof, or even audio output. Good. When a plurality of persons are detected in the frame image by the process by the person detection processing unit 32, the identification result output process is performed for each detected person.

  Then, after the processing by the identification result output processing means 40 is performed, in order to move to the processing of the next frame image, the processing returns to the processing of step S2 again, and thereafter the processing of steps S2 to S8 until no person is detected. Is repeated.

  On the other hand, if it is determined in step S4 that no person has been detected, it is determined whether a person has been detected in the previous frame (step S9), and if a person has been detected in the previous frame. Determines that the person shown in the image has gone out of the shooting range of the camera 20, and based on the data stored in the identification result storage means 95 by the identification result output processing means 40, The gender identification result is displayed on a screen of a display device such as a CRT display or liquid crystal display (not shown), or is output by an output device such as a printer or plotter (not shown) (step S10). Then, in order to move to the processing of the next frame image, the processing returns to step S2 again. If no person is detected in the previous frame in step S9, it is determined that only frames in which no person is detected are consecutive, and the process returns to step S2. If the purpose of the gender identification process for a specific person or the purpose of performing only the gender identification process for a single person, etc., as indicated by the dotted arrow in FIG. After the final gender identification result output process in step S10, the series of gender identification processes may be terminated (step S11).

  According to this embodiment, there are the following effects. That is, since the person attribute identification system 10 includes the index value calculation processing means 37, it is prepared corresponding to four types of person attribute identification data: face data, hairstyle data, tie data, and decollet data. Using the eight human attribute identification models 51 to 58, the likelihood is calculated individually for each of the conflict categories in which the four types of personal attribute identification data are identified, and a total of eight likelihoods are calculated. Can do. Further, since the person attribute identification system 10 includes the identification result information calculation processing unit 38, information indicating the gender identification result of the person can be calculated by integrating the eight likelihoods.

  For this reason, compared with the case where gender identification is performed using only face information as in the prior art, the identification accuracy can be improved. Also, rather than simply collecting individual identification results based on four types of person attribute identification data, the eight likelihoods obtained as the individual identification results are integrated to show the gender identification result of the person. Since the information is calculated, a plurality of pieces of information including face information can be effectively integrated, and high-accuracy identification can be realized.

  In addition to face information and hairstyle information, the presence / absence information of ties and the presence / absence of decollete (chest of clothes), in addition to facial information and hairstyle information, are used as element information for gender identification. The improvement effect of identification accuracy by integration can be further enhanced.

  Furthermore, since the person attribute identification system 10 includes the skin color model update processing means 34 and the non-skin color model update processing means 35, the detection is performed before creating the hairstyle data (hairstyle basic data 502, hairstyle compression data 602). The skin color model 81 and the non-skin color model 91 can be updated using the person's skin color information and non-skin color information. For this reason, since the skin color model 81 and the non-skin color model 91 can be adapted to the detected person and hairstyle extraction adapted to the individual can be performed, more accurate hairstyle extraction can be performed.

  The tie basic data creation processing unit 36E performs a process of creating an edge image 410 (see FIG. 7) by performing contour extraction in the process of creating the tie basic data 503 from the frame image to be processed. The accuracy of non-tie individual identification can be improved. For this reason, the improvement effect of the identification accuracy by the integration of likelihood can be further enhanced.

  Further, since the person attribute identification system 10 includes the skin color model update processing unit 34 and the non-skin color model update processing unit 35, the detection is performed before the decollet data (decollet basic data 504, decollet compression data 604) is created. The skin color model 81 and the non-skin color model 91 can be updated using the person's skin color information and non-skin color information. For this reason, since the skin color model 81 and the non-skin color model 91 are adapted to the detected person and the skin color extraction process for decollete / non-decollet identification adapted to the individual can be performed, more accurate skin color extraction can be performed.

  Furthermore, the human attribute identification models 51 to 58 are constructed by performing dimensional compression by principal component analysis and expressing the distribution shape of data spreading in the compressed dimensional space by a Gaussian mixture model. For example, in the case of face information, it is possible to deal with various faces, such as whether glasses are worn or whether there are wrinkles, and in the case of hairstyle information, whether it is long hair or short hair It is possible to deal with various hairstyles such as whether or not they are bald.

  In addition, since the index value calculation processing unit 37 performs a process of calculating likelihood using a Gaussian mixture model, if this likelihood can be calculated in a plurality of frames for the same person, the likelihood integration in a plurality of frames is performed. Can be easily realized by Bayes' law. For this reason, the calculation process of the total information of the gender identification result in a plurality of frames by the identification result total information calculation processing means 39 can be easily realized.

  Then, the identification result information calculation processing means 38 performs a process of integrating a plurality of likelihoods by Bayes' theorem when calculating information indicating the gender identification result of the person, so that another element for identifying the gender When (information other than face information) is obtained, it is possible to construct a framework that can easily integrate the likelihood calculated from this element. Therefore, as in this embodiment, not only can each likelihood calculated from face information and other hairstyle information, tie information, and decollet information be easily integrated, but other than these four types. Even when the information is added as element information for personal attribute identification (here, gender identification), the likelihood calculated from the element information can be easily integrated with other likelihoods.

  Further, since the person attribute identification system 10 includes the identification result total information calculation processing unit 39, a plurality of likelihoods calculated by a plurality of frames in which the same person is detected are integrated by using Bayes' theorem. The total information of the gender identification results in the frame can be calculated. For this reason, the gender identification accuracy can be further improved.

  And since the identification result information calculation processing means 38 uses the prior probability calculated | required by taking statistics in a real environment, when integrating likelihood by a Bayes's theorem, it can improve the identification precision in a real environment. it can.

  Further, since the identification result information calculation processing unit 38 and the identification result total information calculation processing unit 39 calculate the ratio (or value of the ratio) of the sexes as information indicating the gender identification result, The user can easily determine whether or not the identification result finally obtained by the person attribute identification system 10 is certain by looking at the ratio (or the value of the ratio).

  Note that the present invention is not limited to the above-described embodiment, and modifications and the like within a scope where the object of the present invention can be achieved are included in the present invention.

  That is, in the above embodiment, the person attribute identification models 51 to 58 are constructed by a Gaussian mixture model, but the person attribute identification model in the present invention is not limited to the Gaussian mixture model, for example, A support vector machine (SVM), a neural network, or the like may be employed.

  Moreover, in the said embodiment, although each likelihood 701-708 calculated using the Gaussian mixture model was integrated by Bayes 'theorem, the integration process by Bayes' theorem is calculated using a Gaussian mixture model. The present invention can also be applied to integration processing of index values other than likelihood.

  Furthermore, in the said embodiment, although the person attribute identification system 10 was made into the system which identifies exclusively sex, the person attribute identification system of this invention is not limited to a gender identification system, For example, gender identification In addition, a system for performing age identification simultaneously or a system for performing age identification exclusively may be used.

  More specifically, in the case of a system that performs gender identification and age identification simultaneously, as a person attribute identification model, for example, a male model for younger than 10 years for a face, a teenage male model for a face, 20's male model for face, 30's male model for face ..., male model over 80 years old for face, female model under 10 years for face, teenage female model for face, 20's for face A female model, a female model for the 30s for the face ..., a female model for the face over 80 years old, as well as a male model for the face, a female model for the face, as well as each model for the hairstyle, etc. A process of identifying gender and age in parallel can be performed. At this time, the output of the identification result is, for example, “male: female = ○: ○, younger than 10 years: 10s: 20s: 30s:…: 80 years or older = ○: ○: ○: ○: ...: ○” Or “a man under 10 years old: a man in teens: a man in 20s: man in 30s:…: a man over 80 years old: a woman under 10 years old: a woman in teens: 20 women in a 30s:…: Female over 80 years old = ○: ○: ○: ○:…: ○: ○: ○: ○: ○:…: ○ ”, etc.

  As described above, the person attribute identification method and system of the present invention are suitable for use in, for example, statistical processing, trend surveys, database searches, security systems, and the like.

1 is an overall configuration diagram of a person attribute identification system according to an embodiment of the present invention. The figure of the flowchart which shows the whole flow of the gender identification process by the person attribute identification system of the said embodiment. The figure of the flowchart which shows the flow of the individual identification process about the data for each person attribute identification by the person attribute identification system of the said embodiment. The figure of the flowchart which shows the flow of the hairstyle pre-process including the hair color determination process of the said embodiment. Explanatory drawing of the normalization process of the said embodiment. Explanatory drawing of the creation process of the hairstyle extraction image in the hairstyle pre-process of the said embodiment. Explanatory drawing of the creation process of the edge image in the tie pre-process of the said embodiment. Explanatory drawing of the creation process of the skin color extraction image in the decollete pre-process of the said embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Person attribute identification system 20 Camera 34 Skin color model update process means 36 Person attribute identification data creation process means 36A Face basic data creation process means 36B Human attribute identification data creation process means which constitutes data creation process means for person attribute identification Constructing face compression data creation processing means 36C Hairstyle basic data creation processing means 36D constituting personal attribute identification data creation processing means 36D Hairstyle compressed data creation processing means 36E constituting personal attribute identification data creation processing means 36E Human attribute identification data Tie basic data creation processing means 36F constituting the creation processing means 36F Necktie compressed data creation processing means 36G constituting the human attribute identification data creation processing means 36G Decollete basic data creation processing means 36H constituting the person attribute identification data creation processing means Attribute identification data creation Decollete compressed data creation processing means 37 constituting processing means 37 Index value calculation processing means 37A Male face likelihood calculation processing means 37B constituting index value calculation processing means 37B Female face likelihood calculation processing means 37C constituting index value calculation processing means Male hairstyle likelihood calculation processing means 37D constituting index value calculation processing means 37D Female hairstyle likelihood calculation processing means 37E constituting index value calculation processing means 37E Tie likelihood calculation processing means 37F constituting index value calculation processing means 37F Index value calculation Non-tie likelihood calculation processing means 37G constituting the processing means 37G decollet likelihood calculation processing means constituting the index value calculation processing means 37H Non-decollet likelihood calculation processing means constituting the index value calculation processing means 38 Identification result information calculation processing means 39 Identification result total information calculation processing means 50 Human attribute identification model storage means 51 Human attribute identification A male model for a face that is a Dell 52 A female model for a face that is a model for identifying human attributes 53 A male model for a hairstyle that is a model for identifying human attributes 54 A female model for a hairstyle that is a model for identifying human attributes 55 Human attributes Necktie model which is a model for identification 56 Non-tie model which is a model for identifying human attributes 57 Decollete model which is a model for identifying human attributes 58 Non-decollete model which is a model for identifying human attributes 60 Hair color model storage means 61 Hair color model 70 Non-hair color model storage means 71 Non-hair color model 80 Skin color model storage means 81 Skin color model 90 Non-skin color model storage means 91 Non-skin color model 410 Edge image 501 Face basic data as face data which is a kind of human attribute identification data 502 Hair Style Base as Hair Style Data which is a Type of Person Attribute Identification Data Data 503 Basic tie data as tie data, which is a kind of personal attribute identification data 504 Basic decollet data as decollet data, which is a kind of personal attribute identification data 601 Facial data, which is a kind of personal attribute identification data 602 is a type of personal attribute identification data. 603 is a type of personal attribute identification data. 603 is a type of personal attribute identification data. 603 is a type of personal attribute identification data. Decolete compressed data as certain decollet data 701 Male face likelihood as index value 702 Female face likelihood as index value 703 Male hairstyle likelihood as index value 704 Female hairstyle likelihood as index value 705 Index value Tie likelihood 706 Non-tie likelihood that is an index value 70 Non neckline likelihood is neckline likelihood 708 index value is an index value

Claims (25)

A person attribute identification method for identifying a person attributes comprising a different sex of the person taken by the camera,
Set the male face category and female face category as the categories for identifying facial data, and tie as the category for identifying tie data to identify whether or not the tie is tightened of including setting the category and without tie category, the category in which a plurality of types of people attribute identification data said including the face data and the tie for data for identifying the attribute of the person is identified A plurality of sets are set for each person attribute identification data, and a male model for the face showing the characteristics of the data included in the male face category and a facial model indicating the characteristics of the data included in the female face category Female models, tie models that characterize data included in the category with the tie, and data included in the category without the tie. It characterized including the process of creating a non-tie model shown of the People attribute identification model showing the characteristics of the data included in the category, previously created at least one by one in advance the each person attribute identifying data ,
After creating the plurality of types of person attribute identification data including the face data and the tie data from the frame image of the person attribute identification processing target obtained by photographing the person with the camera,
By using the male model for the face, an index value indicating the certainty that the facial data created from the frame image is data included in the male facial category is calculated, and the facial data An index value indicating the certainty that the facial data created from the frame image is data included in the female face category is calculated using the female model, and the tie model is used. An index value indicating the certainty that the tie data created from the frame image is data included in the category with the tie is calculated, and created from the frame image using the non-tie model An index indicating the certainty that the data for the tie is included in the category without the tie Data including the process of calculating, by using the each person attribute identification model in which the created corresponding to each person attribute identifying data, wherein each person attribute identifying data is created, that is included in any of the categories The likelihood, the degree of attribution, the probability, the ratio, or an index value similar to these for indexing the certainty about whether or not there is individually calculated for each person attribute identification data,
Then the appearance probability of the man,
An index value indicating the certainty that the facial data is data included in the male face category;
The probability of tie when a man is multiplied by an index value indicating the certainty that the tie data is included in the category with the tie, and tie when a man. A process of multiplying a probability obtained by adding a value obtained by adding an index value indicating the certainty that the tie data is data included in the category without the tie,
And the probability of appearance of the woman,
An index value indicating the certainty that the facial data is data included in the female face category;
The probability of tie when female is multiplied by an index value indicating the certainty that the data for tie is included in the category with the tie, and tie when female. And a value obtained by adding a value obtained by adding an index value indicating the probability that the tie data is data included in the category without the tie to the probability that the tie is not included. the by integrating a plurality of the index values calculated individually for each person attribute identifying data, person attribute identification method and calculates the information indicating the identification result of the attribute of the person. A person attribute identification method for identifying a person attributes comprising a different sex of the person taken by the camera,
As a category for identifying the face data, set the male face category and the female face category, and identify the decollet data for identifying whether or not you are wearing chest-filled clothes. , including setting the category of the category and without breast perforated There Aki chest, a plurality of types of people attribute identifying data identification including the face data and the neckline for data for identifying the attribute of the person A plurality of categories for each person attribute identification data, and a male model for the face showing the characteristics of the data included in the male face category, and the data included in the female face category A female model for the face that shows features, a decollete model that shows the features of the data included in the category with chest relief, and a data that is included in the category without chest relief Including the process of creating a non-neckline model showing the symptoms, the person attribute identification model showing the characteristics of the data included in the category, previously created at least one by one in advance the each person attribute identification data,
After creating the plurality of types of person attribute identification data including the face data and the decollet data from the frame image of the person attribute identification processing target obtained by photographing the person with the camera,
By using the male model for the face, an index value indicating the certainty that the facial data created from the frame image is data included in the male facial category is calculated, and the facial data Using the female model, an index value indicating the certainty that the face data created from the frame image is data included in the female face category is calculated, and the decollete model is used. , Calculating an index value indicating the certainty that the decollet data created from the frame image is data included in the chested category, and using the non-decollete model, An index value indicating the certainty that the created decollet data is data included in the category with no chest clearance. Including the process of leaving, with each person attribute identification model in which the corresponding to each person attribute identification data generated, each person attribute identification data generated is, is the data contained in any of the categories The likelihood, the degree of attribution, the probability, the ratio, or an index value similar to these for indexing the probability of whether or not is individually calculated for each of the individual person attribute identification data,
Then the appearance probability of the man,
An index value indicating the certainty that the facial data is data included in the male face category;
Multiplying the probability of chest relief when a man is by an index value indicating the certainty that the decollet data is included in the category with chest relief, and no chest relief for a man A process of multiplying the probability obtained by adding a value obtained by adding an index value indicating the certainty that the decollet data is data included in the category without chest clearance,
And the probability of appearance of the woman,
An index value indicating the certainty that the facial data is data included in the female face category;
Multiplying the probability of chest relief for women by an index value indicating the certainty that the decollet data is included in the category for chest relief, and no chest relief for women And a value obtained by adding the value obtained by adding the index value indicating the certainty that the decollet data is included in the category without the chest clearance to the probability of the by integrating a plurality of the index values calculated individually for each person attribute identifying data, person attribute identification method and calculates the information indicating the identification result of the attribute of the person. A person attribute identification method for identifying a person attributes comprising a different sex of the person taken by the camera,
Set the male face category and female face category as the categories for identifying facial data, and tie as the category for identifying tie data to identify whether or not the tie is tightened Category and non-tie category, and a category with chest relief and a category without chest relief as categories for identifying decolleting data for identifying whether or not chested clothing is worn. including setting, the facial data for identifying an attribute of the person, the tie data, and a plurality of types of people attribute the categories which identification data are identified each person attribute that contains data for the neckline set by a plurality for each identification data, Male model for the face showing the characteristics of the data included in the category of the face of the male face of the female A female model for the face showing the characteristics of the data included in the category, a tie model indicating the characteristics of the data included in the category with the tie, and a non-tie model indicating the characteristics of the data included in the category without the tie. Data included in the category, including processing for creating a decollete model indicating the characteristics of the data included in the category with the chest opening and a non-decollete model indicating the characteristics of the data included in the category without the chest opening. At least one person attribute identification model showing the characteristics of each person attribute identification data in advance,
The plurality of types of person attribute identification data including the face data, the tie data, and the decollet data are respectively obtained from the frame image of the person attribute identification process target obtained by photographing the person with the camera. After creating
By using the male model for the face, an index value indicating the certainty that the facial data created from the frame image is data included in the male facial category is calculated, and the facial data An index value indicating the certainty that the facial data created from the frame image is data included in the female face category is calculated using the female model, and the tie model is used. An index value indicating the certainty that the tie data created from the frame image is data included in the category with the tie is calculated, and created from the frame image using the non-tie model An index indicating the certainty that the data for the tie is included in the category without the tie And using the decollet model, calculate an index value indicating the probability that the decollet data created from the frame image is data included in the category with the chest opening, and Using the decollete model, the decollete data created from the frame image includes the process of calculating an index value indicating the certainty that the decollete data is included in the chestless category. Using each person attribute identification model corresponding to the person attribute identification data, the probability that the created individual attribute identification data is included in which category is indicated. Likelihood, attribution degree, probability, ratio, or similar index value for each person attribute identification data Calculated individually,
Then the appearance probability of the man,
An index value indicating the certainty that the facial data is data included in the male face category;
The probability of tie when a man is multiplied by an index value indicating the certainty that the tie data is included in the category with the tie, and tie when a man. A value obtained by adding the value obtained by multiplying the probability that the tie data is data included in the category without the tie by an index value indicating the certainty,
Multiplying the probability of chest relief when a man is by an index value indicating the certainty that the decollet data is included in the category with chest relief, and no chest relief for a man A process of multiplying the probability obtained by adding a value obtained by adding an index value indicating the certainty that the decollet data is data included in the category without chest clearance,
And the probability of appearance of the woman,
An index value indicating the certainty that the facial data is data included in the female face category;
The probability of tie when female is multiplied by an index value indicating the certainty that the data for tie is included in the category with the tie, and tie when female. A value obtained by adding the value obtained by multiplying the probability that the tie data is data included in the category without the tie by an index value indicating the certainty,
Multiplying the probability of chest relief for women by an index value indicating the certainty that the decollet data is included in the category for chest relief, and no chest relief for women And a value obtained by adding the value obtained by adding the index value indicating the certainty that the decollet data is included in the category without the chest clearance to the probability of the by integrating a plurality of the index values calculated individually for each person attribute identifying data, person attribute identification method and calculates the information indicating the identification result of the attribute of the person. A person attribute identification method for identifying a person attributes comprising a different sex of the person taken by the camera,
The male face category and female face category are set as categories for identifying facial data, and the male hairstyle category and female hairstyle category are set as categories for identifying hairstyle data. The category with the tie and the category without the tie are set as the category for identifying the tie data for identifying whether the tie is tightened or not, and whether the chest is worn or not is identified. as a category of decollete for data is identified for that, including the fact that setting the category of categories and without breast autumn of there Aki chest, the face for the data to identify the attributes of the person, the hairstyle for data the tie data, and a plurality of types of people attribute the categories which identification data are identified each person attribute that contains data for the neckline Set by multiple to each another for data, woman model for the face that shows the man model for the face that shows the characteristics of the data that is included in the category of the face of the man, the characteristics of the data that is included in the category of the face of the woman And a male model for hairstyles showing the characteristics of the data included in the male hairstyle category, a female model for hairstyles showing the characteristics of the data included in the female hairstyle category, and the category with the tie. A tie model indicating the characteristics of the data included, a non-tie model indicating the characteristics of the data included in the category without the tie, a decollete model indicating the characteristics of the data included in the category with the chest opening, and the chest opening including the process of creating a non-neckline model showing the characteristics of the data included in the category without the person genus showing the characteristics of the data included in the category The identification model, previously created at least one by one in advance for each of the data for each person attribute identity,
The plurality of types of person attributes including the face data, the hairstyle data, the tie data, and the decollete data from the frame image of the person attribute identification process target obtained by photographing the person with the camera After creating each identification data,
By using the male model for the face, an index value indicating the certainty that the facial data created from the frame image is data included in the male facial category is calculated, and the facial data An index value indicating the certainty that the facial data created from the frame image is data included in the female face category using the female model, and the male model for the hairstyle And calculating an index value indicating the certainty that the hairstyle data created from the frame image is data included in the male hairstyle category, and using the female model for the hairstyle Calculating an index value indicating the certainty that the hairstyle data created from the frame image is data included in the female hairstyle category. Using the tie model, an index value indicating the probability that the tie data created from the frame image is data included in the category with the tie is calculated, and the non-tie model is used. An index value indicating the certainty that the tie data created from the frame image is data included in the category without the tie is calculated, and created from the frame image using the decollet model An index value indicating the certainty that the decollet data is data included in the chested category is calculated, and the decollet data created from the frame image using the non-decollet model is , The data included in the category without chest clearance Including a process of calculating an index value indicating the mustard is, with each person attribute identification model in which the created corresponding to each person attribute identifying data, wherein each person attribute identifying data is created, one of the The likelihood, the degree of attribution, the probability, the ratio, or similar index values for indexing the certainty about whether the data is included in the category is calculated individually for each person attribute identification data. ,
Then the appearance probability of the man,
An index value indicating the certainty that the facial data is data included in the male face category;
An index value indicating the certainty that the hairstyle data is data included in the male hairstyle category;
The probability of tie when a man is multiplied by an index value indicating the certainty that the tie data is included in the category with the tie, and tie when a man. A value obtained by adding the value obtained by multiplying the probability that the tie data is data included in the category without the tie by an index value indicating the certainty,
Multiplying the probability of chest relief for men by an index value indicating the certainty that the decollet data is included in the category for chest relief, and no chest relief for men A process of multiplying the probability obtained by adding a value obtained by adding an index value indicating the certainty that the decollet data is data included in the category without chest clearance,
And the probability of appearance of the woman,
An index value indicating the certainty that the facial data is data included in the female face category;
An index value indicating the certainty that the hairstyle data is data included in the female hairstyle category;
The probability of tie when female is multiplied by an index value indicating the certainty that the data for tie is included in the category with the tie, and tie when female. A value obtained by adding the value obtained by multiplying the probability that the tie data is data included in the category without the tie by an index value indicating the certainty,
Multiplying the probability of chest relief for women by an index value indicating the certainty that the decollet data is included in the category for chest relief, and no chest relief for women And a value obtained by adding the value obtained by adding the index value indicating the certainty that the decollet data is included in the category without the chest clearance to the probability of the by integrating a plurality of the index values calculated individually for each person attribute identifying data, person attribute identification method and calculates the information indicating the identification result of the attribute of the person. In the person attribute identification method of Claim 4,
When integrating a plurality of the index values calculated individually for each of the person attribute identification data, the ratio of the probability of the man and the probability of the woman or the value of the ratio is calculated by the following equation:
A person attribute identification method characterized by the above.
Probability of man: Probability of woman = Pr [male] × Pr [X_face | M_m_face] × Pr [X_hair | M_m_hair]
× (Pr [tie | Men] / Pr [X_tie | M_tie]
+ Pr [non-tie | Men] / Pr [X_tie | M_non-tie])
× (Pr [dec | Male] ・ Pr [X_dec | M_dec]
+ Pr [non-dec | M] / Pr [X_dec | M_non-dec])
: Pr [female] × Pr [X_face | M_f_face] × Pr [X_hair | M_f_hair]
× (Pr [tie | female] / Pr [X_tie | M_tie]
+ Pr [non-tie | female] / Pr [X_tie | M_non-tie])
× (Pr [dec | female] ・ Pr [X_dec | M_dec]
+ Pr [non-dec | female] ・ Pr [X_dec | M_non-dec])
Here, X_face is the face data, M_m_face is the male model for the face, M_f_face is the female model for the face, and Pr [X_face | M_m_face] inputs X_face to M_m_face Is the male face likelihood as an index value indicating the certainty that the face data is data included in the male face category, and Pr [X_face | M_f_face] It is a female face likelihood as an index value indicating the certainty that the facial data is data included in the female face category, which is obtained by inputting to M_f_face.
X_hair is the hairstyle data, M_m_hair is the male model for the hairstyle, M_f_hair is the female model for the hairstyle, and Pr [X_hair | M_m_hair] inputs X_hair to M_m_hair. Male hairstyle likelihood as an index value indicating the certainty that the hairstyle data is data included in the male hairstyle category, and Pr [X_hair | M_f_hair] represents X_hair as M_f_hair. Is the female hairstyle likelihood as an index value indicating the certainty that the hairstyle data is data included in the female hairstyle category.
Further, X_tie is the tie data, M_tie is the tie model, M_non-tie is the non-tie model, and Pr [X_tie | M_tie] is obtained by inputting X_tie to M_tie. Tie likelihood as an index value indicating the certainty that the tie data is data included in the category with the tie, and Pr [X_tie | M_non-tie] sets X_tie to M_non-tie. It is a non-tie likelihood as an index value indicating the certainty that the tie data obtained by inputting is data included in the category without the tie.
Further, X_dec is the decollet data, M_dec is the decollet model, M_non-dec is the non-decollet model, and Pr [X_dec | M_dec] is obtained by inputting X_dec into M_dec. , The decollet likelihood as an index value indicating the certainty that the decollet data is data included in the chest-filled category, and Pr [X_dec | M_non-dec] The non-decollet likelihood as an index value indicating the certainty that the decollet data is data included in the non-chest category, which is obtained by inputting the decollet data.
Furthermore, Pr [male] is the appearance probability of a man, Pr [tie | male] is the probability of a tie when a man, and Pr [non-tie | male] is a man The probability of not having a tie, Pr [dec | male] is the probability of having a chest when he is a man, and Pr [non-dec | male] is the probability of having no chest when a man. , Pr [female] is the appearance probability of a woman, Pr [tie | female] is the probability of tie when a woman, Pr [non-tie | female] is a tie when a woman Pr [dec | female] is the probability of chesting when she is a woman, Pr [non-dec | female] is the probability of no chesting when she is a woman, These are prior probabilities and are probabilities obtained by taking statistics in the real environment. In the person attribute identification method according to claim 4 or 5 ,
Using the hair color information, non-hair color information, skin color information, and non-skin color information extracted from images taken in various environments, a hair color model, a non-hair color model, a skin color model, and a non-skin color model are respectively stored in advance. Create it,
Before creating the hairstyle data from the frame image of the person attribute identification processing target,
Using the skin color information of the person detected from the person attribute identification processing target frame image, updating the skin color model,
When creating the hairstyle data from the person attribute identification processing target frame image,
Using the hair color model, the non-hair color model, the updated skin color model, and the non-skin color model, it is estimated whether or not each pixel constituting the person attribute identification processing target frame image is a hair portion. A human attribute identification method characterized by extracting a hairstyle by performing In the person attribute identification method in any one of Claims 1, 3, and 4,
A person attribute identification method, wherein, when creating the tie data from the frame image to be subjected to the person attribute identification process, an edge image is created by extracting a contour. In the person attribute identification method in any one of Claims 2-4 ,
Using skin color information and non-skin color information extracted from images taken in various environments, a skin color model and a non-skin color model are created in advance,
Before creating the decollete data from the person attribute identification processing target frame image,
Using the skin color information of the person detected from the person attribute identification processing target frame image, updating the skin color model,
When creating the decollete data from the person attribute identification processing target frame image,
Using the updated skin color model and the non-skin color model, the skin color portion is extracted by estimating whether or not each pixel constituting the frame image of the person attribute identification processing target is a skin color portion. Person attribute identification method. In the person attribute identification method in any one of Claims 1-8,
Each human attribute identification model is included in the category after performing feature extraction by principal component analysis using a lot of sample data as pre-learning for each category in which each human attribute identification data is identified. By creating a Gaussian mixture model that represents the distribution shape of the data
When calculating the index value individually for each person attribute identification data,
Based on the plurality of types of person attribute identification data including the face data created from the person attribute identification processing target frame image, using each of the person attribute identification models constructed by the Gaussian mixture model, A likelihood attribute is calculated for each category. In the person attribute identification method according to claim 9,
When calculating information indicating the identification result of the person attribute,
A person attribute identification method, wherein a plurality of likelihoods are integrated according to Bayes' theorem. In the person attribute identification method according to claim 9 or 10,
By combining the likelihoods calculated for each of a plurality of frames in which the same person is detected according to Bayes' theorem, information summing up the identification results of the attributes of the person in the plurality of frames is calculated. A person attribute identification method. The person attribute identification method according to claim 10 or 11,
A person attribute identification method using prior probabilities when integrating the likelihoods according to the Bayes' theorem. A person attribute identification system for identifying the person attribute containing a different sex of the person taken by the camera,
A male model for the face showing the characteristics of the data included in the category of the male face for which the facial data is identified, and for the face indicating the characteristics of the data included in the category of the female face for which the facial data is identified A tie model that shows the characteristics of data included in a category with a tie that identifies tie data for identifying whether or not the tie is tightened, and no tie that identifies the tie data non tie including a model, categories plurality of persons attribute identifying data including the face data and the tie for data for identifying the attribute of the person is identified indicating the characteristics of the data contained in category A personal attribute identification model indicating the characteristics of the data included in the personal attribute identification data is stored at least one for each personal attribute identification data. And use the model storage means,
A person who performs a process of creating each of the plurality of types of person attribute identification data including the face data and the tie data from a frame image of a person attribute identification process target obtained by photographing the person with the camera Attribute identification data creation processing means;
By using the male model for the face, an index value indicating the certainty that the facial data created from the frame image is data included in the male facial category is calculated, and the facial data An index value indicating the certainty that the facial data created from the frame image is data included in the female face category is calculated using the female model, and the tie model is used. An index value indicating the certainty that the tie data created from the frame image is data included in the category with the tie is calculated, and created from the frame image using the non-tie model An index indicating the certainty that the data for the tie is included in the category without the tie Including processing for calculating, using said each person attribute identifying the model corresponding stored in the person attribute identification model storage unit, wherein each person attribute identification created by the person attribute identifying data creation processing unit Likelihood, degree of attribution, probability, ratio, or similar index values for indexing the certainty about which category the data is included in which category, the individual attribute identification data Index value calculation processing means for performing processing to calculate individually for each;
Identification result information for performing processing for calculating information indicating the identification result of the person attribute by integrating a plurality of the index values individually calculated for each individual person attribute identification data by the index value calculation processing means Calculation processing means ,
This identification result information calculation processing means
The appearance probability of a man,
An index value indicating the certainty that the facial data is data included in the male face category;
The probability of tie when a man is multiplied by an index value indicating the certainty that the tie data is included in the category with the tie, and tie when a man. A process of multiplying a probability obtained by adding a value obtained by adding an index value indicating the certainty that the tie data is data included in the category without the tie,
And the probability of appearance of the woman,
An index value indicating the certainty that the facial data is data included in the female face category;
The probability of tie when female is multiplied by an index value indicating the certainty that the data for tie is included in the category with the tie, and tie when female. And a value obtained by adding a value obtained by adding an index value indicating the probability that the tie data is data included in the category without the tie to the probability that the tie is not included. A person attribute identification system , wherein information indicating the identification result of the person attribute is calculated by integrating a plurality of the index values . A person attribute identification system for identifying the person attribute containing a different sex of the person taken by the camera,
A male model for the face showing the characteristics of the data included in the category of the male face for which the facial data is identified, and for the face indicating the characteristics of the data included in the category of the female face for which the facial data is identified The decollete model showing the characteristics of the data included in the category with chestbreaks, in which the data for decollete for identifying whether or not she is wearing chested clothes is identified, and the data for decollete are identified non neckline including a model, a plurality of types of people attribute identifying data including the face data and the neckline for data for identifying the attribute of the person indicated by the characteristic of the data contained in the category without breast Aki is A personal attribute identification model storing at least one personal attribute identification model indicating the characteristics of the data included in the category for which each is identified. And use the model storage means,
A person who performs a process of creating each of the plurality of types of person attribute identification data including the face data and the decollet data from a frame image of a person attribute identification process target obtained by photographing the person with the camera Attribute identification data creation processing means;
By using the male model for the face, an index value indicating the certainty that the facial data created from the frame image is data included in the male facial category is calculated, and the facial data Using the female model, an index value indicating the certainty that the face data created from the frame image is data included in the female face category is calculated, and the decollete model is used. , Calculating an index value indicating the certainty that the decollet data created from the frame image is data included in the chested category, and using the non-decollete model, An index value indicating the certainty that the created decollet data is data included in the category with no chest clearance. Including the process of leaving, with each person attribute identifying the model corresponding stored in the person attribute identification model storage unit, wherein each person attribute identifying data created by the person attribute identifying data creation processing unit Is the likelihood, the degree of attribution, the probability, the ratio, or similar index values for indexing the certainty about which category the data is included in each of the data for identifying each person attribute Index value calculation processing means for performing processing to calculate individually,
Identification result information for performing processing for calculating information indicating the identification result of the person attribute by integrating a plurality of the index values individually calculated for each individual person attribute identification data by the index value calculation processing means Calculation processing means ,
This identification result information calculation processing means
The appearance probability of a man,
An index value indicating the certainty that the facial data is data included in the male face category;
Multiplying the probability of chest relief when a man is by an index value indicating the certainty that the decollet data is included in the category with chest relief, and no chest relief for a man A process of multiplying the probability obtained by adding a value obtained by adding an index value indicating the certainty that the decollet data is data included in the category without chest clearance,
And the probability of appearance of the woman,
An index value indicating the certainty that the facial data is data included in the female face category;
Multiplying the probability of chest relief for women by an index value indicating the certainty that the decollet data is included in the category for chest relief, and no chest relief for women And a value obtained by adding the value obtained by adding the index value indicating the certainty that the decollet data is included in the category without the chest clearance to the probability of A person attribute identification system , wherein information indicating the identification result of the person attribute is calculated by integrating a plurality of the index values . A person attribute identification system for identifying the person attribute containing a different sex of the person taken by the camera,
A male model for the face showing the characteristics of the data included in the male face category for which the facial data is identified, and a facial model for indicating the characteristics of the data included in the female face category for which the facial data is identified A tie model that shows the characteristics of data included in a category with a tie that identifies tie data for identifying whether or not the tie is tightened, and no tie that identifies the tie data The characteristics of the data included in the category with chest is identified with the non-tie model indicating the characteristics of the data included in the category and the decollet data for identifying whether or not the user is wearing the chest with clothes. including the neckline model, and a non-neckline model the neckline for data indicating the characteristics of the data contained in the category without breast Aki identified, the The face data for identifying the attribute of the object, for the person attribute identifying indicating characteristics of data contained in a plurality of types of person category attribute identification data are identified, including the tie data, and the neckline data A model storage unit for identifying personal attributes for storing at least one model for each data for identifying personal attributes;
The plurality of types of person attribute identification data including the face data, the tie data, and the decollet data are respectively obtained from the frame image of the person attribute identification process target obtained by photographing the person with the camera. Personal attribute identification data creation processing means for performing the creation process;
By using the male model for the face, an index value indicating the certainty that the facial data created from the frame image is data included in the male facial category is calculated, and the facial data An index value indicating the certainty that the facial data created from the frame image is data included in the female face category is calculated using the female model, and the tie model is used. An index value indicating the certainty that the tie data created from the frame image is data included in the category with the tie is calculated, and created from the frame image using the non-tie model An index indicating the certainty that the data for the tie is included in the category without the tie And using the decollet model, calculate an index value indicating the probability that the decollet data created from the frame image is data included in the category with the chest opening, and The person attribute identification including a process of calculating an index value indicating the certainty that the decollet data created from the frame image is data included in the chestless category using a decollet model Data in which each of the person attribute identification data created by the person attribute identification data creation processing means using each corresponding person attribute identification model stored in the model storage means is included in any of the categories Likelihood, attribution, probability, ratio, or this to index the likelihood of being The index values similar to al, the index value calculation processing means for performing processing for calculating individually the each person attribute identification data,
Identification result information for performing processing for calculating information indicating the identification result of the person attribute by integrating a plurality of the index values individually calculated for each individual person attribute identification data by the index value calculation processing means Calculation processing means ,
This identification result information calculation processing means
The appearance probability of a man,
An index value indicating the certainty that the facial data is data included in the male face category;
The probability of tie when a man is multiplied by an index value indicating the certainty that the tie data is included in the category with the tie, and tie when a man. A value obtained by adding the value obtained by multiplying the probability that the tie data is data included in the category without the tie by an index value indicating the certainty,
Multiplying the probability of chest relief when a man is by an index value indicating the certainty that the decollet data is included in the category with chest relief, and no chest relief for a man A process of multiplying the probability obtained by adding a value obtained by adding an index value indicating the certainty that the decollet data is data included in the category without chest clearance,
And the probability of appearance of the woman,
An index value indicating the certainty that the facial data is data included in the female face category;
The probability of tie when female is multiplied by an index value indicating the certainty that the data for tie is included in the category with the tie, and tie when female. A value obtained by adding the value obtained by multiplying the probability that the tie data is data included in the category without the tie by an index value indicating the certainty,
Multiplying the probability of chest relief when a woman is by an index value indicating the certainty that the decollet data is included in the category with chest relief and no chest relief for a woman And a value obtained by adding the value obtained by adding the index value indicating the certainty that the decollet data is included in the category without the chest clearance to the probability of A person attribute identification system , wherein information indicating the identification result of the person attribute is calculated by integrating a plurality of the index values . A person attribute identification system for identifying the person attribute containing a different sex of the person taken by the camera,
A male model for the face showing the characteristics of the data included in the category of the male face for which the facial data is identified, and for the face indicating the characteristics of the data included in the category of the female face for which the facial data is identified Of the female model, the male model for the hairstyle showing the characteristics of the data included in the male hairstyle category for which the hairstyle data is identified, and the data included in the female hairstyle category for which the hairstyle data is identified A female model for a hairstyle showing characteristics, a tie model showing characteristics of data included in a category with a tie for identifying data for tie for identifying whether or not a tie is tightened, and the data for tie are Non-tie model showing the characteristics of data included in the category without tie identified, and data for decollet to identify whether or not you are wearing chest-filled clothes Including the neckline model showing the characteristics of the data contained in the chest Available categories are identified, and a non-neckline model showing the characteristics of the data included in the category without breast Aki said neckline data is identified, the person shows the facial data for identifying an attribute, the hairstyle data, the tie for data, and the characteristics of the data in which a plurality of types of people attribute identifying data is included in the category identified including data for the neckline Personal attribute identification model storage means for storing at least one personal attribute identification model for each of the personal attribute identification data;
The plurality of types of person attributes including the face data, the hairstyle data, the tie data, and the decollete data from the frame image of the person attribute identification process target obtained by photographing the person with the camera Personal attribute identification data creation processing means for performing processing for creating each identification data;
By using the male model for the face, an index value indicating the certainty that the facial data created from the frame image is data included in the male facial category is calculated, and the facial data An index value indicating the certainty that the facial data created from the frame image is data included in the female face category using the female model, and the male model for the hairstyle And calculating an index value indicating the certainty that the hairstyle data created from the frame image is data included in the male hairstyle category, and using the female model for the hairstyle Calculating an index value indicating the certainty that the hairstyle data created from the frame image is data included in the female hairstyle category. Using the tie model, an index value indicating the probability that the tie data created from the frame image is data included in the category with the tie is calculated, and the non-tie model is used. An index value indicating the certainty that the tie data created from the frame image is data included in the category without the tie is calculated, and created from the frame image using the decollet model An index value indicating the certainty that the decollet data is data included in the chested category is calculated, and the decollet data created from the frame image using the non-decollet model is , The data included in the category without chest clearance Including a process of calculating an index value indicating the mustard is, with each person attribute identifying the model corresponding stored in the person attribute identification model storage unit, created by the person attribute identifying data creation processing unit Likelihood, attribution, probability, ratio, or similar index values for indexing the certainty about which category the individual attribute identification data is data included in, Index value calculation processing means for performing processing for calculating each person attribute identification data individually;
Identification result information for performing processing for calculating information indicating the identification result of the person attribute by integrating a plurality of the index values individually calculated for each individual person attribute identification data by the index value calculation processing means Calculation processing means ,
This identification result information calculation processing means
The appearance probability of a man,
An index value indicating the certainty that the facial data is data included in the male face category;
An index value indicating the certainty that the hairstyle data is data included in the male hairstyle category;
The probability of tie when a man is multiplied by an index value indicating the certainty that the tie data is included in the category with the tie, and tie when a man. A value obtained by adding the value obtained by multiplying the probability that the tie data is data included in the category without the tie by an index value indicating the certainty,
Multiplying the probability of chest relief for men by an index value indicating the certainty that the decollet data is included in the category for chest relief, and no chest relief for men A process of multiplying the probability obtained by adding a value obtained by adding an index value indicating the certainty that the decollet data is data included in the category without chest clearance,
And the probability of appearance of the woman,
An index value indicating the certainty that the facial data is data included in the female face category;
An index value indicating the certainty that the hairstyle data is data included in the female hairstyle category;
The probability of tie when female is multiplied by an index value indicating the certainty that the data for tie is included in the category with the tie, and tie when female. A value obtained by adding the value obtained by multiplying the probability that the tie data is data included in the category without the tie by an index value indicating the certainty,
Multiplying the probability of chest relief for women by an index value indicating the certainty that the decollet data is included in the category for chest relief, and no chest relief for women And a value obtained by adding the value obtained by adding the index value indicating the certainty that the decollet data is included in the category without the chest clearance to the probability of by integrating a plurality of the index value, the person attribute identification system, characterized by being configured to calculate information indicating the identification result of the attribute of the person. The person attribute identification system according to claim 16, wherein
The identification result information calculation processing means includes:
When integrating a plurality of the index values individually calculated for each of the person attribute identification data, the ratio between the probability of the man and the probability of the woman or the value of the ratio is calculated by the following formula: Have
A personal attribute identification system characterized by that.
Probability of man: Probability of woman = Pr [male] × Pr [X_face | M_m_face] × Pr [X_hair | M_m_hair]
× (Pr [tie | Men] / Pr [X_tie | M_tie]
+ Pr [non-tie | Men] / Pr [X_tie | M_non-tie])
× (Pr [dec | Male] ・ Pr [X_dec | M_dec]
+ Pr [non-dec | M] / Pr [X_dec | M_non-dec])
: Pr [female] × Pr [X_face | M_f_face] × Pr [X_hair | M_f_hair]
× (Pr [tie | female] / Pr [X_tie | M_tie]
+ Pr [non-tie | female] / Pr [X_tie | M_non-tie])
× (Pr [dec | female] ・ Pr [X_dec | M_dec]
+ Pr [non-dec | female] ・ Pr [X_dec | M_non-dec])
Here, X_face is the face data, M_m_face is the male model for the face, M_f_face is the female model for the face, and Pr [X_face | M_m_face] inputs X_face to M_m_face Is the male face likelihood as an index value indicating the certainty that the face data is data included in the male face category, and Pr [X_face | M_f_face] It is a female face likelihood as an index value indicating the certainty that the facial data is data included in the female face category, which is obtained by inputting to M_f_face.
X_hair is the hairstyle data, M_m_hair is the male model for the hairstyle, M_f_hair is the female model for the hairstyle, and Pr [X_hair | M_m_hair] inputs X_hair to M_m_hair. Male hairstyle likelihood as an index value indicating the certainty that the hairstyle data is data included in the male hairstyle category, and Pr [X_hair | M_f_hair] represents X_hair as M_f_hair. Is the female hairstyle likelihood as an index value indicating the certainty that the hairstyle data is data included in the female hairstyle category.
Further, X_tie is the tie data, M_tie is the tie model, M_non-tie is the non-tie model, and Pr [X_tie | M_tie] is obtained by inputting X_tie to M_tie. Tie likelihood as an index value indicating the certainty that the tie data is data included in the category with the tie, and Pr [X_tie | M_non-tie] sets X_tie to M_non-tie. It is a non-tie likelihood as an index value indicating the certainty that the tie data obtained by inputting is data included in the category without the tie.
Further, X_dec is the decollet data, M_dec is the decollet model, M_non-dec is the non-decollet model, and Pr [X_dec | M_dec] is obtained by inputting X_dec into M_dec. , The decollet likelihood as an index value indicating the certainty that the decollet data is data included in the chest-filled category, and Pr [X_dec | M_non-dec] The non-decollet likelihood as an index value indicating the certainty that the decollet data is data included in the non-chest category, which is obtained by inputting the decollet data.
Furthermore, Pr [male] is the appearance probability of a man, Pr [tie | male] is the probability of a tie when a man, and Pr [non-tie | male] is a man The probability of not having a tie, Pr [dec | male] is the probability of having a chest when he is a man, and Pr [non-dec | male] is the probability of having no chest when a man. , Pr [female] is the appearance probability of a woman, Pr [tie | female] is the probability of tie when a woman, Pr [non-tie | female] is a tie when a woman Pr [dec | female] is the probability of chesting when she is a woman, Pr [non-dec | female] is the probability of no chesting when she is a woman, These are prior probabilities, which are probabilities obtained by taking statistics in a real environment. The person attribute identification system according to claim 16 or 17 ,
Hair color model, non-hair color model, skin color model, and non-skin color model created using hair color information, non-hair color information, skin color information, and non-skin color information extracted from images shot in various environments Hair color model storage means, non-hair color model storage means, skin color model storage means, and non-skin color model storage means;
Skin color model update processing means for performing update processing of the skin color model stored in the skin color model storage means using skin color information of a person detected from the frame image of the person attribute identification processing target,
The human attribute identification data creation processing means creates the hair color model stored in the hair color model storage means and the non-hair when creating the hairstyle data from the frame image of the person attribute identification processing target. Using the non-hair color model stored in the color model storage means, the updated skin color model stored in the skin color model storage means, and the non-skin color model stored in the non-skin color model storage means, A human attribute identification system characterized in that a process for extracting a hairstyle is performed by estimating whether or not each pixel constituting a frame image to be processed for human attribute identification is a hair part. In the person attribute identification system in any one of Claim 13,15,16 ,
The person attribute identification data creation processing means is configured to perform edge extraction by performing contour extraction when creating the tie data from the frame image of the person attribute identification process target. A personal attribute identification system characterized by that. The person attribute identification system according to any one of claims 14 to 16 ,
Skin color model storage means and non-skin color model storage means for storing a skin color model and a non-skin color model respectively created using skin color information and non-skin color information extracted from images taken in various environments;
Skin color model update processing means for performing update processing of the skin color model stored in the skin color model storage means using skin color information of a person detected from the frame image of the person attribute identification processing target,
The human attribute identification data creation processing means, when creating the decollet data from the person attribute identification processing target frame image, the updated skin color model and the non-skin color model stored in the skin color model storage means. Using the non-skin color model stored in the skin color model storage means, a process of performing a process of extracting a skin color part by estimating whether each pixel constituting the frame image of the person attribute identification processing target is a skin color part A human attribute identification system characterized by The person attribute identification system according to any one of claims 13 to 20,
Each human attribute identification model is included in the category after performing feature extraction by principal component analysis using a lot of sample data as pre-learning for each category in which each human attribute identification data is identified. Is a learning model constructed by expressing the distribution shape of the data to be expressed by a Gaussian mixture model,
The index value calculation processing means includes the individual persons constructed by the Gaussian mixture model based on the plurality of types of person attribute identification data including the face data created by the person attribute identification data creation processing means. A person attribute identification system, wherein an attribute identification model is used to perform processing for calculating likelihood for each category. The person attribute identification system according to claim 21,
The identification result information calculation processing means is configured to perform processing for integrating a plurality of the likelihoods calculated by the index value calculation processing means by Bayes' theorem. The person attribute identification system according to claim 21 or 22,
By integrating the likelihood calculated in each of a plurality of frames in which the same person is detected according to Bayes' theorem, a process of calculating information totaling the identification results of the attributes of the person in the plurality of frames is performed. A person attribute identification system comprising identification result total information calculation processing means. The person attribute identification system according to claim 22,
The identification result information calculation processing means is configured to perform processing using prior probabilities when integrating the likelihoods according to the Bayes' theorem. The person attribute identification system according to claim 23.
The identification result total information calculation processing means is configured to perform processing using prior probabilities when the likelihoods are integrated according to the Bayes' theorem.