JP7613699B2 - Information processing device and information processing method - Google Patents

Description

The present invention relates to an information processing device and an information processing method.

Conventionally, various machine learning models, including deep learning models, are known as classification models that classify data to be classified with high accuracy. Conventionally, various techniques are known for increasing the size of a machine learning model (hereinafter, sometimes referred to as model size) in order to improve the classification accuracy of a classification model. For example, a technique is known in which the number of channels, the number of layers, and the resolution of each layer of a convolutional neural network (CNN) used as an image classification model are scaled up in a complex manner using a single parameter.

Mingxing Tan, Quoc V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”, [online], May 2019, [Retrieved April 17, 2020], Internet <URL: https://arxiv.org /pdf/1905.11946.pdf＞

However, in the above-mentioned conventional technology, the number of channels, the number of layers, and the resolution of each layer of a convolutional neural network (CNN) used as an image classification model are merely scaled up in a complex manner using a single parameter, and it is not necessarily possible to perform detailed classification using a compact classification model.

The present application aims to provide an information processing device and an information processing method that can perform detailed classification using a compact classification model.

The information processing device according to the present application includes a class generation unit that generates a higher class that is a higher class than the multiple different classes, the higher class being a class that corresponds to each of the hierarchical clusters based on the multiple class features that are hierarchically clustered based on the similarity of the multiple class features that are representative values of each of the multiple different classes, and a model generation unit that generates a classification model for each of the multiple higher classes, which is a machine learning model that classifies data to be classified into one of multiple lower classes that belong to the higher class.

The information processing device according to the present application also includes an acquisition unit that acquires a classification model, which is a machine learning model that classifies data to be classified into one of a plurality of lower classes that belong to a higher class that is a higher class than the plurality of different classes, and which is a feature related to data belonging to each of a plurality of different classes, based on the similarity of a plurality of class features that are representative values of each of the plurality of different classes, and which is a class that corresponds to each of the hierarchical clusters based on the plurality of class features that are hierarchically clustered based on the similarity of the plurality of class features that are representative values of each of the plurality of different classes, and the processing target data to be processed; and an estimation unit that estimates a first lower class into which the processing target data is classified using a higher classification model that is a classification model corresponding to a predetermined higher class selected in order from the classification model corresponding to the highest class, selects a lower classification model that is the classification model corresponding to the first lower class from the classification models corresponding to each of the plurality of lower classes that belong to the lower class below the predetermined higher class based on the estimation result of the higher classification model, and estimates a second lower class into which the processing target data is classified using the selected lower classification model.

According to one aspect of the embodiment, it is possible to perform detailed classification using a compact classification model.

FIG. 1 is a diagram showing the relationship between the model size of a classification model and the number of classification classes. FIG. 2 is a diagram illustrating an example of the configuration of the information processing apparatus according to the embodiment. FIG. 3 is a diagram for explaining a hierarchical class generation process according to the embodiment. FIG. 4 is a diagram for explaining the process of generating a hierarchical classification model according to the embodiment. FIG. 5 is a diagram illustrating a training data set of a hierarchical classification model according to the embodiment. FIG. 6 is a diagram for explaining the class estimation process according to the embodiment. FIG. 7 is a diagram for explaining a case where the estimation result of a lower class becomes incorrect by adopting only one estimation result by a higher-level classification model. FIG. 8 is a diagram for explaining a case where a prediction result of a lower class becomes the correct answer by adopting a plurality of prediction results of higher-level classification models. FIG. 9 is a diagram for explaining a normal multi-class classification model and a multi-class classification model capable of detecting an unknown class. FIG. 10 is a diagram for explaining a class estimation process according to a modified example. FIG. 11 is a diagram for explaining an example of a case where the estimation process according to the modified example is stopped. FIG. 12 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device.

Below, a detailed description will be given of a form for implementing the information processing device and information processing method according to the present application (hereinafter, referred to as an "embodiment") with reference to the drawings. Note that the information processing device and information processing method according to the present application are not limited to this embodiment. Furthermore, the same components in each of the following embodiments are given the same reference numerals, and duplicated descriptions will be omitted.

(Embodiment)
1. Introduction
Conventionally, it is known that as the number of classification classes by a classification model increases, the complexity of the learning data set used to learn the classification model increases, and the model size of the classification model (e.g., deep learning model) that has learned those features tends to become large. For example, in the case of a behavior recognition model, which is a machine learning model that recognizes the type of behavior of a person contained in a video, even for a single action of "grabbing", the type of the behavior (i.e., classification class) of the grab is different for each type of object being grabbed. For example, the behavior recognition model is trained to classify the action of grabbing a person's hand and the action of grabbing a person's foot as different actions. In this way, for example, if the action is between a person and an object, such as the action of "grabbing", the behavior recognition model is trained to classify the action by the type of object that is the target of the person's action. In addition, the types of human actions are not limited to actions between a person and an object, and there are a wide variety of types such as actions between people and people or actions of a person (single person). Therefore, it is generally known that the model size of the behavior recognition model tends to be very large.

Using Figure 1, the relationship between the model size of a classification model and the number of classification classes is explained. Figure 1 is a diagram showing the relationship between the model size of a classification model and the number of classification classes. As shown in Figure 1, it is generally known that as the number of classification classes in a classification model increases, the model size of the classification model increases rapidly. In addition, a size limit, which is the upper limit of the model size, is determined depending on the machine specifications of the information processing device that implements the classification model and the upper limit of the amount of computer resources. In addition, once the size limit is determined, a classifiable limit, which is the upper limit of the number of classification classes that can be classified by the classification model, is determined depending on the size limit.

As described in Figure 1, the larger the model size of the classification model, the greater the number of classification classes that can be classified by the classification model. However, data classification processing using a classification model with a large model size (hereinafter sometimes referred to as a "large classification model") requires higher performance machine specifications and more abundant computer resources. Therefore, for example, it is conceivable to implement a large classification model in cloud computing (hereinafter sometimes referred to as the cloud), which has more abundant computer resources than on-premise information processing devices, and classify data.

However, some data to be classified cannot be stored in the cloud due to privacy reasons. In such cases where the cloud cannot be used, the classification model is implemented in an on-premise information processing device and the data is classified. However, there is an upper limit to the machine specifications and amount of computer resources of on-premise information processing devices (particularly edge devices). Even when the cloud is used, there is generally an upper limit to the machine specifications and amount of computer resources that can be used due to the financial resources of the user, etc.

As such, there is generally an upper limit to the level of machine specifications and the amount of computer resources that can be used. Therefore, when enlarging a classification model to enable more detailed classification, there is an upper limit to the model size of the usable classification model (the size limit shown in Figure 1). In addition, because there is an upper limit to the model size of the usable classification model, there is also an upper limit to the number of classification classes that can be classified by the classification model (the classification limit shown in Figure 1). Therefore, it has sometimes been difficult to enable detailed classification using a larger classification model.

In contrast, the information processing device according to the present embodiment performs hierarchical clustering on class features, which are average values for each class of features corresponding to data belonging to each of a plurality of different classes obtained from a large-scale classification model generated by a known technique. The information processing device also generates a plurality of upper classes, which are classes corresponding to each of the hierarchical clusters based on the class features hierarchically clustered by hierarchical clustering and are classes higher than the plurality of different classes. In this way, the information processing device generates a hierarchical class, which is a hierarchical class including a plurality of different classes and a plurality of upper classes that are classes higher than the plurality of different classes. Note that the plurality of different classes corresponds to the lowest class in the hierarchical class. The information processing device also generates a tree diagram, which is a graph that represents the process of generating hierarchical clusters by hierarchical clustering, and obtains a number of classification models corresponding to the number of branch points included in the tree diagram. Here, each of the branch points included in the tree diagram corresponds to each of the upper classes included in the hierarchical class. The information processing device also generates a classification model for each upper class that has been trained to classify the classification target data to be classified into one of a plurality of lower classes that belong to the lower class.

As a result, the information processing device according to this embodiment can increase the number of classification classes while keeping the model size of each of the multiple classification models generated for each higher class small. Therefore, the information processing device can realize a miniaturized classification model that enables detailed classification. Furthermore, since the information processing device can realize a miniaturized classification model that enables detailed classification, it can make it possible to perform detailed classification using a miniaturized classification model. For example, the information processing device can make it possible to perform detailed behavior classification using a miniaturized behavior recognition model.

The information processing device according to this embodiment also acquires a plurality of classification models generated for each upper class and the processing target data to be processed. Next, the information processing device selects a top classification model corresponding to the top class from among the plurality of classification models. Next, the information processing device uses the selected top classification model to estimate a first top class (hereinafter, may be referred to as the first top class) in which the processing target data is classified from among the plurality of upper classes belonging to the lower level of the top class. Next, based on the estimation result of the top classification model, the information processing device selects a first top classification model corresponding to the first top class from among the classification models corresponding to each of the plurality of upper classes belonging to the lower level of the top class. Next, the information processing device uses the selected first top classification model to estimate a second top class (hereinafter, may be referred to as the second top class) in which the processing target data is classified from among the plurality of upper classes belonging to the lower level of the first top class. Next, the information processing device selects a second higher-level classification model corresponding to the second higher-level class from among the classification models corresponding to each of the multiple higher-level classes below the first higher-level class based on the estimation result of the first higher-level classification model (hereinafter, this may be referred to as a "selection process"). Next, the information processing device uses the selected second higher-level classification model to estimate a class into which the data to be processed is classified from among the multiple classes below the second higher-level class (hereinafter, this may be referred to as an "estimation process"). In this way, the information processing device repeatedly executes the selection process and the estimation process until the class into which the data to be processed is classified matches one of the multiple different classes that are the lowest class.

As a result, the information processing device according to this embodiment can perform detailed classification using a compact classification model. For example, the information processing device can perform detailed classification of behaviors using a compact behavior recognition model.

Note that in this specification, the terms higher class and lower class indicate relative relationships in a hierarchical class. For example, a higher class in this specification refers to a class that is higher than multiple different classes that are the lowest class included in the hierarchical class. Also, a lower class in this specification refers to a class that is lower than a specific higher class included in the hierarchical class. Therefore, of the multiple classes included in the hierarchical class, classes other than the highest class and the lowest class (multiple different classes) may be higher classes or lower classes.

The classification model in this specification may be, for example, a classification model that classifies images (also referred to as an image classification model), a classification model that classifies videos (also referred to as a video classification model), a classification model that classifies the types of actions of people included in videos (also referred to as a behavior recognition model), or a classification model that classifies sentences (also referred to as a sentence classification model or text classification model).

2. Configuration of information processing device
An example of the configuration of the information processing device 100 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. The information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130.

(Communication unit 110)
The communication unit 110 is realized by a network interface card (NIC), an antenna, etc. The communication unit 110 is connected to various networks via wired or wireless communication, and transmits and receives information to and from other information processing devices other than the information processing device 100, for example.

(Memory unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. Specifically, the storage unit 120 stores various data. For example, the storage unit 120 stores information about a learning data set used for learning a classification model. The storage unit 120 also stores various programs. For example, the storage unit 120 stores information about the classification model generated for each upper class by the model generation unit 133 and information capable of identifying the upper class in association with each other.

(Control unit 130)
The control unit 130 is a controller, and is realized, for example, by a central processing unit (CPU) or a micro processing unit (MPU) executing various programs stored in a storage device inside the information processing device 100 using a RAM as a working area. The control unit 130 is also a controller, and is realized, for example, by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

The control unit 130 has an acquisition unit 131, a class generation unit 132, a model generation unit 133, and an estimation unit 134 as functional units, and may realize or execute the information processing actions described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 2, and may be other configurations that perform the information processing described below. Also, each functional unit indicates a function of the control unit 130, and does not necessarily have to be physically distinct.

(Acquisition unit 131)
The acquisition unit 131 may acquire various information. Specifically, the acquisition unit 131 may acquire features related to data belonging to a plurality of different classes. The acquisition unit 131 may acquire features corresponding to data belonging to a plurality of different classes as features related to data belonging to a plurality of different classes. For example, the acquisition unit 131 may acquire features corresponding to data belonging to each of a plurality of different classes extracted by feature extraction provided in the large-scale classification model from another information processing device implementing a large-scale classification model generated by a known technology via the communication unit 110. For example, the acquisition unit 131 may acquire features corresponding to data belonging to each of a plurality of different classes (hereinafter, sometimes referred to as "multiple different classes") that are all classes that can be classified by the large-scale classification model. In other words, the data belonging to a plurality of different classes is data that is classified into a plurality of different classes using the large-scale classification model.

For example, when the classification model is an image classification model, large classification models generated by known techniques include AlexNet (Krizhevsky et al., 2012), ZFNet (Matthew et al., 2013), VGGNet (Simonyan et al., 2014), GoogLeNet (Szegedy et al., 2014), ResNet (Residual Network) (Kaiming He et al., 2015), DenseNet (Gao Huang et al., 2016), MobileNet (Andrew G. Howard et al., 2017), SENet (Squeeze-and-Excitation Networks) (Jie Hu et al., 2018), EfficientNet (Tan et al., 2019), or EfficientNetV2 (Tan et al., 2020). al., 2021) etc.

(Class Generation Unit 132)
The class generation unit 132 may calculate class features that are average values, for each of the multiple different classes, of features corresponding to data belonging to each of the multiple different classes acquired by the acquisition unit 131. The class generation unit 132 may calculate class features that are average values for each of the multiple different classes of features of data belonging to each of the multiple different classes. Here, the class features are representative values of the features of data belonging to each of the multiple different classes.

A hierarchical class generation process according to the embodiment will be described with reference to FIG. 3. FIG. 3 is a diagram for describing a hierarchical class generation process according to the embodiment. The horizontal and vertical axes shown in the leftmost, second and third leftmost diagrams of FIG. 3 roughly show the feature space of the class features. Also, the circles shown in the leftmost, second and third leftmost diagrams of FIG. 3 respectively show the class features of each of a plurality of different classes. In the leftmost, second and third leftmost diagrams of FIG. 3, the class generation unit 132 generates hierarchical clusters by performing hierarchical clustering on the class features based on the similarity of the class features. In the rightmost diagram of FIG. 3, the class generation unit 132 generates a tree diagram, which is a graph representing the process of generating hierarchical clusters. For example, the class generation unit 132 may generate a tree diagram, which is a tree structure of a binary tree. For simplicity, only 12 class features are shown in Figure 3, but in reality, 13 or more class features may exist.

In the leftmost diagram of FIG. 3, the class generation unit 132 generates multiple clusters having each of the multiple class features as an element. Hereinafter, a cluster having each of the multiple class features as an element may be referred to as a lowest-level cluster. In the leftmost diagram of FIG. 3, the class features and the lowest-level cluster are indicated by the same circle. In addition, when the class generation unit 132 generates multiple lowest-level clusters, it generates multiple lowest-level classes that are classes corresponding to each of the multiple lowest-level clusters. Here, each of the multiple lowest-level classes corresponds to each of the multiple different classes described above. In the rightmost diagram of FIG. 3, each of the multiple white circles shown at the bottom of the tree diagram GH1 indicates each of the multiple lowest-level classes. In addition, in the rightmost diagram of FIG. 3, for simplicity, the illustration of four lowest-level classes is omitted from the illustration of the 12 lowest-level classes, and only eight lowest-level classes are illustrated. In addition, in reality, 13 or more lowest-level classes may exist.

Next, the class generation unit 132 calculates the similarity between the class features for every pair of class features. For example, the class generation unit 132 may calculate the similarity between the class features. Next, the class generation unit 132 combines the lowest-level clusters in order, starting from the two lowest-level clusters whose elements are the two class features with the highest similarity between the class features. For example, the class generation unit 132 generates a plurality of first top clusters C11 to C16 by combining two lowest-level clusters whose elements are two class features whose similarity between the class features exceeds the first similarity. The first top cluster is a cluster whose elements are a pair of two lowest-level clusters whose elements are two class features whose similarity between the class features exceeds the first similarity. Furthermore, when the class generation unit 132 generates a plurality of first higher-level clusters C11 to C16, it generates a plurality of first higher-level classes L11 to L16 that correspond to each of the first higher-level clusters C11 to C16. In the diagram at the right end of FIG. 3, each of the circles shown in the second layer from the bottom of the tree diagram GH1 represents each of the first higher-level classes L11 to L14. For simplicity, the circles corresponding to each of the first higher-level classes L15 and L16 are omitted in the diagram at the right end of FIG. 3. In reality, seven or more first higher-level clusters and first higher-level classes may exist.

Note that calculating the similarity between class features for every pair of class features corresponds to calculating the distance between every pair of lowest-level clusters. The class generation unit 132 may calculate the distance between every pair of lowest-level clusters. For example, the class generation unit 132 may calculate the Euclidean distance or Manhattan distance between the lowest-level clusters. Next, the class generation unit 132 may generate multiple first higher-level clusters C11 to C16 by combining two lowest-level clusters whose distance between the lowest-level clusters is less than the first distance.

In the second and third diagrams from the left in FIG. 3, the class generation unit 132 calculates the similarity between the first top clusters for every pair of first top clusters. For example, the class generation unit 132 may calculate the similarity between the first top clusters based on the average value of the two class features included in the first top clusters. Next, the class generation unit 132 generates multiple second top clusters C21 to C23 by combining two first top clusters whose similarity between the first top clusters exceeds a second threshold. The second top clusters are clusters whose elements are pairs of two first top clusters whose similarity between the first top clusters exceeds a second threshold. In FIG. 3, the class generation unit 132 generates a second higher-level cluster C21 having elements of the pair of first higher-level clusters C11 and C12, a second higher-level cluster C22 having elements of the pair of first higher-level clusters C13 and C14, and a second higher-level cluster C23 having elements of the pair of first higher-level clusters C15 and C16. When the class generation unit 132 generates a plurality of second higher-level clusters C21 to C23, the class generation unit 132 generates a plurality of second higher-level classes L21 to L23, which are classes corresponding to the plurality of second higher-level clusters C21 to C23. In the rightmost diagram of FIG. 3, each of the plurality of circles shown in the third layer from the bottom of the tree diagram GH1 represents each of the plurality of second higher-level classes L21 to L22. Note that, for simplicity, the circle corresponding to the second higher-level class L23 is omitted in the rightmost diagram of FIG. 3. In reality, four or more second higher-level clusters and classes may exist.

Note that calculating the similarity between the first top clusters for every pair of first top clusters corresponds to calculating the distance between the first top clusters for every pair of first top clusters. The class generation unit 132 may calculate the distance between the first top clusters for every pair of first top clusters. For example, the class generation unit 132 may calculate the Euclidean distance or Manhattan distance between the first top clusters. Next, the class generation unit 132 may generate multiple second top clusters C21 to C23 by combining two first top clusters whose distance between the first top clusters is less than the second distance.

As described above, the class generation unit 132 generates a higher class that is a class corresponding to each of the hierarchical clusters based on the hierarchically clustered class features and is a higher class than the multiple different classes, based on the similarity of the multiple class features that are features related to data belonging to each of the multiple different classes and are representative values of each of the multiple different classes. Specifically, the class generation unit 132 generates the higher class by using the average value of the features of the data belonging to each of the multiple different classes as the representative value. Note that the class generation unit 132 may generate the higher class by using the median or mode of the features of the data belonging to each of the multiple different classes as the representative value.

The class generation unit 132 also generates higher-level classes by generating a plurality of first higher-level classes that are classes corresponding to a plurality of first higher-level clusters, which are clusters generated by combining each of a plurality of lowest-level clusters, which are clusters having each of a plurality of class features as elements, based on the similarity of the plurality of class features, and a plurality of second higher-level classes that are classes corresponding to a plurality of second higher-level clusters, which are clusters generated by combining each of the plurality of first higher-level clusters, based on the similarity of the plurality of first higher-level clusters.

The class generation unit 132 may perform hierarchical clustering on the class features until the number of newly generated top clusters is several to 10 or less. In this way, the class generation unit 132 hierarchically clusters multiple class features until the number of newly generated top clusters falls below a predetermined threshold. For example, the class generation unit 132 may perform hierarchical clustering on the class features until the number of newly generated top clusters is one. When the class generation unit 132 performs hierarchical clustering on the class features until the number of newly generated top clusters is one, the class generation unit 132 may set the top class corresponding to the last generated cluster as the top class.

(Model Generation Unit 133)
The model generation unit 133 generates a classification model, which is a machine learning model that classifies data to be classified into one of a plurality of lower classes that belong to a lower class below the upper class, for each upper class. A process for generating a hierarchical classification model according to an embodiment will be described with reference to FIG. 4. FIG. 4 is a diagram for describing a process for generating a hierarchical classification model according to an embodiment. The model generation unit 133 generates classification models in a number corresponding to the number of branch points included in the tree diagram generated by the class generation unit 132. Specifically, the model generation unit 133 may acquire classification models in a number corresponding to the number of branch points included in the tree diagram generated by the class generation unit 132. For example, the model generation unit 133 may acquire a classification model that is smaller than a large classification model. For example, the model generation unit 133 may acquire classification models such as MobileNet (Andrew G. Howard et al., 2017) and ShuffleNet (Xiangyu Zhang et al., 2018). In the diagram at the left end of Fig. 4, the model generation unit 133 may acquire classification models corresponding to branch points P11, P12 (not shown), P13, and P21 included in the tree diagram GH2 generated by the class generation unit 132. For example, the model generation unit 133 may acquire four classification models from the storage unit 120 that stores information about classification models. Note that the diagram at the left end of Fig. 4 shows the tree diagram GH2 generated by the class generation unit 132, which is composed of layer B, which is the lowest layer, and layer A, which is the layer one layer above layer B. Note that the tree diagram generated by the class generation unit 132 may be composed of three or more layers.

As described above, each of the branch points included in the tree diagram corresponds to each of the higher classes included in the hierarchical classes. In FIG. 4, branch point P21 corresponds to the top class, and each of branch points P11 to P13 corresponds to each of the three higher classes 1 to 3. FIG. 4 shows a hierarchical class including six lowest classes 4 to 9, three higher classes 1 to 3, and one highest class generated by the class generation unit 132. The number of the lowest classes generated by the class generation unit 132 may be seven or more. The number of the higher classes generated by the class generation unit 132 may be four or more.

Furthermore, when the model generation unit 133 acquires a number of classification models corresponding to the number of branch points included in the tree diagram, the model generation unit 133 generates classification models corresponding to each of the higher-level classes corresponding to each of the branch points included in the tree diagram. In the central diagram in FIG. 4, the model generation unit 133 may generate classification model A_1 corresponding to the top class corresponding to branch point P21 included in the tree diagram GH2, and classification models B_1 to B_3 corresponding to each of higher-level classes 1 to 3 corresponding to branch points P11 to P13 included in the tree diagram GH2.

Specifically, the model generation unit 133 generates a classification model A_1 trained based on a first learning data set that is a set of data sampled from each of the three top classes 1 to 3 that belong to the lower level of the highest class and a label corresponding to each of the three top classes 1 to 3. For example, when data included in the first learning data set is input to the classification model A_1 as input information, the model generation unit 133 generates a classification model A_1 trained to output a label corresponding to the data as output information. For example, when data included in the first learning data set is input to the classification model A_1 as input information, the model generation unit 133 may generate a classification model A_1 trained to estimate a probability value of the data being classified into each of the three top classes 1 to 3, and output a label corresponding to the top class with the largest estimated probability value as output information.

The model generation unit 133 also generates a classification model B_1 trained based on a second learning data set, which is a set of data sampled from each of the two lowest classes 4 and 5 below the higher class 1 and labels corresponding to each of the two lowest classes 4 and 5. For example, when data included in the second learning data set is input to the classification model B_1 as input information, the model generation unit 133 generates a classification model B_1 trained to output a label corresponding to the data as output information. For example, when data included in the second learning data set is input to the classification model B_1 as input information, the model generation unit 133 may generate a classification model B_1 trained to estimate a probability value of data being classified into each of the two lowest classes 4 and 5, and output a label corresponding to the lowest class with the largest estimated probability value as output information.

The model generation unit 133 also generates a classification model B_2 trained based on a third learning data set, which is a set of data sampled from each of the two lowest classes 6 and 7 that belong to the lower level of the higher class 2, and labels corresponding to each of the two lowest classes 6 and 7. For example, when data included in the third learning data set is input to the classification model B_2 as input information, the model generation unit 133 generates a classification model B_2 trained to output a label corresponding to the data as output information. For example, when data included in the third learning data set is input to the classification model B_2 as input information, the model generation unit 133 may generate a classification model B_2 trained to estimate a probability value of the data being classified into each of the two lowest classes 6 and 7, and output a label corresponding to the lowest class with the largest estimated probability value as output information.

The model generation unit 133 also generates a classification model B_3 trained based on a fourth learning data set, which is a set of data sampled from each of the two lowest classes 8 and 9 that belong to the lower level of the upper class 3, and labels corresponding to each of the two lowest classes 8 and 9. For example, when data included in the fourth learning data set is input to the classification model B_3 as input information, the model generation unit 133 generates a classification model B_3 trained to output a label corresponding to the data as output information. For example, when data included in the fourth learning data set is input to the classification model B_3 as input information, the model generation unit 133 may generate a classification model B_3 trained to estimate a probability value of the data being classified into each of the two lowest classes 8 and 9, and output a label corresponding to the lowest class with the largest estimated probability value as output information.

As described above, when data sampled from each of a plurality of lower classes is input as input information to the machine learning model, the model generation unit 133 generates a classification model for each upper class, which is a machine learning model trained to output, as output information, a lower label that is a label corresponding to the lower class to which the data belongs. For example, when data sampled from each of a plurality of lower classes is input as input information to the machine learning model, the model generation unit 133 generates, for each upper class, a classification model that is a machine learning model trained to estimate a probability value for classifying data into each of the lower classes and output, as output information, a lower label that is a label corresponding to the lower class with the largest estimated probability value. In this way, the model generation unit 133 generates a first hierarchical classification model that is a collection of multiple classification models having a hierarchy corresponding to the hierarchical structure of the hierarchical classes.

The learning data set of the hierarchical classification model according to the embodiment will be described with reference to FIG. 5. FIG. 5 is a diagram for describing the learning data set of the hierarchical classification model according to the embodiment. The model generation unit 133 generates a classification model for each upper class, which is a machine learning model trained based on a learning data set that is a pair of data sampled from each of a plurality of lower classes so that the number of data sampled from each of the plurality of lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs. In FIG. 5, the model generation unit 133 generates a second learning data set that is a pair of data sampled from each of the lowest classes 4 and 5 described in FIG. 4 so that the number of data sampled from each of the lowest classes 4 and 5 described in FIG. 4 is as equal as possible, and each of the lowest labels 4 and 5 corresponding to each of the lowest classes 4 and 5.

(Estimation unit 134)
A class estimation process according to the embodiment will be described with reference to FIG. 6. FIG. 6 is a diagram for describing a class estimation process according to the embodiment. The acquisition unit 131 acquires a first hierarchical classification model generated by the model generation unit 133 and processing target data to be processed. In FIG. 6, the acquisition unit 131 acquires the top classification model A_1 included in the first hierarchical classification model generated by the model generation unit 133, the first top classification models B_1 to B_3 belonging to the lower level of the top classification model A_1, the second top classification models C_1 to C_20 belonging to the lower level of the first top classification models B_1 to B_3, ..., and the Nth (N is a natural number of 3 or more) top classification models X_1 to X_100 belonging to the lower level of the (N-1)th top classification model (not shown), and input data to be processed.

The estimation unit 134 selects the top classification model from the first hierarchical classification models acquired by the acquisition unit 131. In FIG. 6, the estimation unit 134 may select the top classification model A_1 from the first hierarchical classification models acquired by the acquisition unit 131. Next, the estimation unit 134 uses the selected top classification model to estimate a first top class (hereinafter, may be referred to as the first top class) in which the data to be processed is classified among a plurality of top classes below the top class. In FIG. 6, the estimation unit 134 may use the selected top classification model A_1 to estimate a first top class in which the input data is classified among three top classes below the top class. For example, the estimation unit 134 may input the input data to the top classification model A_1 and obtain the first top class corresponding to the label output from the top classification model A_1 as the estimation result.

Then, based on the estimation result of the top-level classification model, the estimation unit 134 selects a first-level classification model corresponding to the first-level class from among the classification models corresponding to each of the multiple top-level classes subordinate to the top-level class. In FIG. 6, the estimation unit 134 may select the first-level classification model B_3 corresponding to the first-level class from among the first-level classification models B_1 to B_3 corresponding to each of the multiple top-level classes subordinate to the top-level class.

Next, the estimation unit 134 uses the selected first higher-level classification model to estimate a second higher-level class (hereinafter, may be referred to as a second higher-level class) into which the data to be processed is classified, among the multiple higher-level classes that belong to the lower level of the first higher-level class. In FIG. 6, the estimation unit 134 may use the selected first higher-level classification model B_3 to estimate a second higher-level class into which the input data is classified, among the 20 higher-level classes that belong to the lower level of the first higher-level class. For example, the estimation unit 134 may input the input data to the first higher-level classification model B_3, and obtain the second higher-level class corresponding to the label output from the first higher-level classification model B_3 as the estimation result.

Then, based on the estimation result of the first higher-level classification model, the estimation unit 134 selects a second higher-level classification model corresponding to the second higher-level class from among the classification models corresponding to each of the multiple higher-level classes subordinate to the first higher-level class. In FIG. 6, the estimation unit 134 may select the second higher-level classification model C_11 corresponding to the second higher-level class from among the second higher-level classification models C_1 to C_20 corresponding to each of the multiple higher-level classes subordinate to the first higher-level class.

The estimation unit 134 also selects an Nth higher classification model corresponding to the Nth higher class from among the classification models corresponding to each of the multiple higher classes subordinate to the (N-1)th higher class, based on the estimation result of the (N-1)th higher classification model. In FIG. 6, the estimation unit 134 may select the Nth higher classification model X_50 corresponding to the Nth higher class from among the Nth higher classification models X_1 to X_100 corresponding to each of the multiple higher classes subordinate to the (N-1)th higher class.

Then, the estimation unit 134 uses the selected Nth highest classification model to estimate the lowest class in which the data to be processed is classified, among the multiple lowest classes that belong to the lower level of the Nth highest class. In FIG. 6, the estimation unit 134 may use the selected Nth highest classification model X_50 to estimate the lowest class in which the data to be processed is classified, among the 200 lowest classes that belong to the lower level of the Nth highest class. For example, the estimation unit 134 may input the input data to the Nth highest classification model X_50 and obtain the lowest class corresponding to the label output from the Nth highest classification model X_50 (corresponding to the output data shown in FIG. 6) as the estimation result. In this way, the estimation unit 134 repeatedly executes the selection process and the estimation process until the class in which the data to be processed is classified matches one of the multiple different classes that are the lowest class.

As described above, the estimation unit 134 estimates a first lower class into which the data to be processed is classified using a higher-level classification model that is a classification model corresponding to a predetermined higher class selected in order from the classification model corresponding to the highest-level class, and selects a lower-level classification model that is a classification model corresponding to the first lower class from among the classification models corresponding to each of the multiple lower classes that belong to the lower level of the predetermined higher class based on the estimation result of the higher-level classification model, and estimates a second lower class into which the data to be processed is classified using the selected lower-level classification model. Furthermore, the estimation unit 134 repeatedly executes the process of selecting a lower-level classification model from among the classification models corresponding to each of the multiple lower classes based on the estimation result of the higher-level classification model, and estimating the second lower class using the selected lower-level classification model, until the data to be processed is classified into one of multiple different classes.

3. Modifications
The processing according to the above-described embodiment may be implemented in various different forms other than the above embodiment.

In the above-described embodiment, a case has been described in which the model generation unit 133 generates a first hierarchical classification model that is a collection of classification models that classify data to be processed into two or more classes (hereinafter, these may be referred to as "normal multi-class classification models"). In contrast, in the modified example, a case will be described in which the model generation unit 133 generates a second hierarchical classification model that is a collection of classification models that classify data to be processed into three or more classes including an unknown class (hereinafter, these may be referred to as "multi-class classification models capable of detecting unknown classes") instead of a two-class classification model.

In the above embodiment, the estimation unit 134 adopts only one estimation result from a higher-level classification model until the data to be processed is classified into one of the multiple lowest classes, selects one classification model from among the classification models corresponding to each of the multiple lower classes based on the one estimation result, and repeatedly executes the process of estimating the lower class using the selected one classification model. In contrast, in the modified example, the estimation unit 134 adopts two or more estimation results from a higher-level classification model until the data to be processed is classified into one of the multiple lowest classes, selects two or more classification models from among the classification models corresponding to each of the multiple lower classes based on the two or more estimation results, and repeatedly executes the process of estimating each of the lower classes using each of the selected two or more classification models. In the modified example, the estimation unit 134 determines to stop the process after the process of selecting the classification model when the estimation results of each of the selected two or more classification models satisfy a predetermined condition.

First, a case where the estimation result of the lower class is incorrect by adopting only one estimation result by a higher-level classification model will be described with reference to FIG. 7. FIG. 7 is a diagram for describing a case where the estimation result of the lower class is incorrect by adopting only one estimation result by a higher-level classification model. In FIG. 7, the correct classification destination of the input data input to the classification model is the lowest class 9. At this time, when the input data is input to the highest-level classification model A_1, the estimation unit 134 estimates the probability value of the data being classified into each of the higher classes 1 to 3, and outputs the label corresponding to the higher class 1, which has the maximum estimated probability value (0.5 in FIG. 7), as output information. Next, when the input data is input to the classification model B_1 corresponding to the higher class 1, the estimation unit 134 estimates the probability value of the data being classified into each of the lowest classes 4 or 5, and outputs the label corresponding to the lowest class 5, which has the maximum estimated probability value (0.6 in FIG. 7), as output information. Thus, in Figure 7, by adopting only one estimation result from a higher-level classification model, the estimation result for the lower class becomes incorrect.

Next, a case where the estimation result of the lower class becomes the correct answer by adopting multiple estimation results by the higher-level classification model will be described with reference to FIG. 8. FIG. 8 is a diagram for explaining a case where the estimation result of the lower class becomes the correct answer by adopting multiple estimation results by the higher-level classification model. In FIG. 8, as in FIG. 7, the correct classification destination of the input data input to the classification model is the lowest class 9. At this time, when the input data is input to the highest-level classification model A_1, the estimation unit 134 estimates the probability value that the data is classified into each of the higher-level classes 1 to 3, and selects the classification model B_1 and the classification model B_3 corresponding to the two higher-level classes 1 and 3 in descending order of the estimated probability value. Next, the estimation unit 134 uses the two selected classification models B_1 and B_3 to estimate the probability value that the input data is classified into each of the lowest classes belonging to the lower levels of the higher-level classes 1 and 3 corresponding to the two selected classification models B_1 and B_3. For example, when input data is input to classification model B_1, the estimation unit 134 estimates the probability value that the data will be classified into the lowest classes 4 and 5, respectively, and outputs the label corresponding to the lowest class 5, which has the largest estimated probability value (0.6 in FIG. 8), as output information. When input data is input to classification model B_3, the estimation unit 134 estimates the probability value that the data will be classified into the lowest classes 8 and 9, respectively, and outputs the label corresponding to the lowest class 9, which has the largest estimated probability value (0.9 in FIG. 8), as output information.

As described above, the estimation unit 134 uses a higher-level classification model to estimate a probability value that the data to be processed will be classified into each of a plurality of lower classes, and selects a plurality of classification models corresponding to each of a predetermined number of two or more lower classes in descending order of probability value from among the classification models corresponding to each of the plurality of lower classes based on the estimation result of the higher-level classification model, and estimates a probability value that the data to be processed will be classified into each of the classes further below the lower classes corresponding to each of the selected plurality of classification models, using the selected classification models. In this way, the information processing device 100 can prevent the estimation result of the lower classes from being incorrect by adopting only one estimation result from the higher-level classification model.

In FIG. 8, the estimation unit 134 may calculate the product of the probability values of the input data being classified into each class in the classification paths (1) and (2) from the highest class to the two lowest classes 5 and 9. For example, the estimation unit 134 may calculate 0.5×0.6=0.3, which is the product of the two probability values 0.5 and 0.6 corresponding to the classification path (1). The estimation unit 134 may also calculate 0.4×0.9=0.36, which is the product of the two probability values 0.4 and 0.9 corresponding to the classification path (2). Next, the estimation unit 134 may adopt the classification path (2) with the largest value of the product of the probability values of 0.36 based on a comparison between 0.3 and 0.36, which are the product values of the probability values in each of the classification paths (1) and (2), and estimate that the input data will be classified into the lowest class 9. In this way, in Figure 8, by adopting multiple estimation results from higher-level classification models, the estimation result for the lower class becomes the correct answer.

As described above, when the data to be processed is classified into two or more of a plurality of different classes, the estimation unit 134 estimates that the data to be processed will be classified into the class corresponding to the classification path in which the product of the probability values is the largest, based on a comparison of the values of the product of the probability values for classification into each class in the classification path from the highest class to the class corresponding to each of the two or more classes among the plurality of different classes.

Next, a normal multi-class classification model and a multi-class classification model capable of detecting unknown classes will be described with reference to FIG. 9. FIG. 9 is a diagram for explaining a normal multi-class classification model and a multi-class classification model capable of detecting unknown classes. The left side of FIG. 9 shows a normal multi-class classification model. The normal multi-class classification model estimates a probability value that input data is classified into each of a plurality of classes previously learned, regardless of the type of input data. The normal multi-class classification model shown on the left side of FIG. 9 (hereinafter, referred to as the "first multi-class classification model") is a classification model that has been trained to classify data to be classified into either a dog class or a cat class. In this case, when input data that should originally be classified into a bird class is input to the first multi-class classification model, the first multi-class classification model estimates the probability that the input data is classified into the dog class to be 70% and the probability that the input data is classified into the cat class to be 30%.

The right side of FIG. 9 shows a multi-class classification model capable of detecting an unknown class. The multi-class classification model capable of detecting an unknown class can classify input data that is not classified into multiple classes previously learned into an unknown class. The multi-class classification model capable of detecting an unknown class shown on the right side of FIG. 9 (hereinafter referred to as the "second multi-class classification model") is a classification model trained to classify data to be classified into one of a dog class, a cat class, or an unknown class. In this case, when input data to be classified into a bird class is input to the second multi-class classification model, the second multi-class classification model estimates that the probability that the input data will be classified into a dog class is 10%, the probability that the input data will be classified into a cat class is 15%, and the probability that the input data will be classified into an unknown class is 75%. In this way, the multi-class classification model capable of detecting an unknown class can classify data to be processed that should be classified into a class other than the multiple classes previously learned into an unknown class.

In the modified example, the model generation unit 133 generates a classification model, which is a machine learning model that classifies the data to be classified into a plurality of lower classes or an unknown class that is not one of the plurality of lower classes, for each upper class. The model generation unit 133 generates a classification model, which is a machine learning model that classifies the data to be classified into a plurality of lower classes or an unknown class that is not one of the plurality of lower classes, for each upper class. For example, the model generation unit 133 may obtain a multi-class classification model capable of detecting unknown classes as a number of classification models corresponding to the number of branch points included in the tree diagram generated by the class generation unit 132. Then, the model generation unit 133 may generate a classification model in the same manner as described in FIG. 4 and FIG. 5. For example, when data sampled from each of the plurality of lower classes is input to the machine learning model as input information, the model generation unit 133 generates a classification model, which is a machine learning model that has been trained to estimate a probability value at which the data is classified into each of the plurality of lower classes, and output the estimated probability value as output information, for each upper class. In this way, the model generation unit 133 generates a second hierarchical classification model that is a collection of multiple classification models having a hierarchy corresponding to the hierarchical structure of the hierarchical classes. The model generation unit 133 also generates a second hierarchical classification model that is a collection of multi-class classification models capable of detecting unknown classes. This allows the information processing device 100 to stop processing when, for example, the data to be processed is classified into an unknown class by a higher-level classification model, thereby preventing the estimation result of a lower-level class from being incorrect due to an error in the estimation result of the higher-level classification model.

Next, the class estimation process according to the modified example will be described with reference to FIG. 10. FIG. 10 is a diagram for explaining the class estimation process according to the modified example. In FIG. 10, the description of the contents overlapping with FIG. 6 will be omitted. The acquisition unit 131 is a classification model that is a machine learning model that classifies data to be classified into a plurality of lower classes or an unknown class that is not one of the plurality of lower classes, and acquires a classification model generated for each upper class and data to be processed. In FIG. 10, the acquisition unit 131 acquires a second hierarchical classification model generated by the model generation unit 133 and data to be processed. In FIG. 10, the acquisition unit 131 acquires the top classification model A_1 included in the second hierarchical classification model generated by the model generation unit 133, the first top classification models B_1 to B_3 subordinate to the top classification model A_1, the second top classification models C_1 to C_20 subordinate to the first top classification models B_1 to B_3, ..., and the Nth (N is a natural number equal to or greater than 3) top classification models X_1 to X_100 subordinate to the (N-1)th top classification model (not shown), as well as the input data to be processed.

In FIG. 10, the estimation unit 134 may use the selected top-level classification model A_1 to estimate the probability values of the input data being classified into three upper classes and an unknown class that are subordinate to the top-level class. Based on the estimation result of the top-level classification model, the estimation unit 134 determines whether to stop the process after the process of selecting a plurality of classification models corresponding to each of a predetermined number of lower classes, from among the classification models corresponding to each of the plurality of lower classes, in order from the one with the highest probability value. Here, the predetermined number of two or more may be a number smaller than the number of the plurality of lower classes. Hereinafter, the process of determining whether to stop the process may be referred to as a process continuation determination or a continuation determination process. For example, when the classification target data is classified into an unknown class, when the difference in the probability values of the processing target data being classified into each of the plurality of lower classes or the unknown class exceeds a first threshold value, or when the value of the product of the probability values of the processing target data being classified into each class in the classification path from the top-level class to the class corresponding to each of two or more classes among the plurality of different classes falls below a second threshold value, the estimation unit 134 determines to stop the process after the process of selecting a plurality of classification models corresponding to each of the predetermined number of lower classes. In FIG. 10, the estimation unit 134 determines to continue processing based on the result of the processing continuation determination.

When the estimation unit 134 decides to continue the process, it selects multiple classification models corresponding to each of two or more predetermined lower classes in descending order of probability value from among the classification models corresponding to each of the multiple lower classes based on the estimation result of the higher classification model. Hereinafter, the process of selecting multiple classification models may be referred to as a multiple selection process. In FIG. 10, the estimation unit 134 may select two first higher classification models B_2 and B_3 corresponding to each of the two first higher classes in descending order of probability value from among the first higher classification models B_1 to B_3 corresponding to each of the multiple first higher classes belonging to the lower class of the highest class based on the estimation result of the highest classification model. Next, the estimation unit 134 may estimate the probability value of the input data being classified into each of the multiple higher classes and the unknown class belonging to the lower class of each of the two first higher classes using the two selected first higher classification models B_2 and B_3, respectively. The estimation unit 134 repeatedly executes the continuation determination process, the multiple selection process, and the estimation process until the class into which the processing target data is classified matches one of multiple different classes that is the lowest class.

Next, an example of a case where the estimation process according to the modified example is stopped will be described with reference to FIG. 11. FIG. 11 is a diagram for describing an example of a case where the estimation process according to the modified example is stopped. In FIG. 11, the estimation unit 134 determines to stop the process after the process of selecting the classification models corresponding to each of the predetermined number of lower classes because the difference in the probability value of the data to be processed being classified into each of the plurality of lower classes or the unknown class exceeds the first threshold. In addition, when the estimation unit 134 determines to stop the process before the class into which the data to be processed is classified matches any one of the plurality of different classes that is the lowest class, the condition of the continuation determination process may be relaxed and the process may be executed again from the beginning. For example, the estimation unit 134 may increase the predetermined number when selecting the classification models corresponding to each of the plurality of lower classes in order from the one with the highest probability value from among the classification models corresponding to each of the plurality of lower classes. In addition, when the classification target data is classified into an unknown class, the estimation unit 134 may increase the first threshold when stopping the process when the difference in the probability value of the data to be processed being classified into each of the plurality of lower classes or the unknown class exceeds the first threshold. Furthermore, the estimation unit 134 may reduce the second threshold value used to stop processing if the product of the probability values of classification into each class along the classification path from the highest class to the class corresponding to each of two or more of the multiple different classes falls below the second threshold value.

[4. Effects]
As described above, the information processing device 100 according to the embodiment includes a class generation unit 132 and a model generation unit 133. The class generation unit 132 generates upper classes that are classes higher than the multiple different classes and correspond to each of the hierarchical clusters based on the multiple class features that are hierarchically clustered based on the similarity of the multiple class features that are feature amounts related to data respectively belonging to multiple different classes and are representative values of each of the multiple different classes. The model generation unit 133 generates a classification model, which is a machine learning model that classifies data to be classified into one of multiple lower classes that belong to the upper class, for each upper class.

This allows the information processing device 100 to increase the number of classification classes while keeping the model size of each of the multiple classification models generated for each higher class small. Therefore, the information processing device 100 can realize a miniaturized classification model that enables detailed classification. Furthermore, since the information processing device 100 can realize a miniaturized classification model that enables detailed classification, it is possible to perform detailed classification using a miniaturized classification model. For example, the information processing device 100 can perform detailed classification of behavior using a miniaturized behavior recognition model. Furthermore, since the information processing device 100 can perform detailed classification using a miniaturized classification model, it can contribute to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilience, innovate and innovate."

In addition, the class generation unit 132 generates a higher class by using the average feature values of the data belonging to each of the multiple different classes as a representative value.

This allows the information processing device 100 to appropriately generate a higher class based on the average value of the features of data belonging to multiple different classes.

The class generation unit 132 also hierarchically clusters multiple class features until the number of newly generated clusters falls below a predetermined threshold.

This allows the information processing device 100 to generate the desired number of higher-level classes.

The class generation unit 132 also generates higher-level classes by generating a plurality of first higher-level classes that are classes corresponding to a plurality of first higher-level clusters, which are clusters generated by combining each of a plurality of lowest-level clusters, which are clusters having each of a plurality of class features as elements, based on the similarity of the plurality of class features, and a plurality of second higher-level classes that are classes corresponding to a plurality of second higher-level clusters, which are clusters generated by combining each of the plurality of first higher-level clusters, based on the similarity of the plurality of first higher-level clusters.

This allows the information processing device 100 to appropriately generate a first higher class based on the similarity of the class features. Also, the information processing device 100 can appropriately generate a second higher class based on the similarity of the first higher cluster.

The class generation unit 132 also generates a tree diagram, which is a graph that represents the process of generating hierarchical clusters. The model generation unit 133 generates classification models whose number corresponds to the number of branch points included in the tree diagram.

This allows the information processing device 100 to generate a classification model that corresponds to the number of generated higher classes.

The model generation unit 133 also generates a classification model for each higher-level class, which is a machine learning model trained based on a learning dataset that is a set of data sampled from each of the multiple lower-level classes such that the number of data sampled from each of the multiple lower-level classes is similar, and a lower-level label that is a label corresponding to the lower-level class to which the data belongs.

This allows the information processing device 100 to avoid bias in the lower class data included in the training dataset, thereby generating a highly accurate classification model.

In addition, when data is input as input information to the machine learning model, the model generation unit 133 generates a classification model for each upper class, which is a machine learning model trained to output lower labels as output information.

This allows the information processing device 100 to appropriately generate a classification model.

In addition, the model generation unit 133 generates a classification model for each higher-level class, which is a machine learning model that classifies the data to be classified into one of multiple lower classes or an unknown class that is not one of the multiple lower classes.

As a result, the information processing device 100 can stop processing if, for example, the data to be processed is classified into an unknown class by a higher-level classification model, thereby preventing the estimation result of a lower class from being incorrect due to an error in the estimation result of the higher-level classification model.

The information processing device 100 also includes an acquisition unit 131 and an estimation unit 134. The acquisition unit 131 acquires a classification model, which is a machine learning model that classifies the classification target data to be classified into one of a plurality of lower classes belonging to a higher class that is a higher class than the plurality of different classes, and which corresponds to each of the hierarchical clusters based on a plurality of class features that are hierarchically clustered based on the similarity of the plurality of class features that are representative values of each of the plurality of different classes, and the processing target data to be processed. The estimation unit 134 estimates a first lower class in which the processing target data is classified using a higher classification model that is a classification model corresponding to a predetermined higher class selected in order from the classification model corresponding to the highest class, and selects a lower classification model that is a classification model corresponding to the first lower class from the classification models corresponding to each of the plurality of lower classes belonging to the lower class based on the estimation result of the higher classification model, and estimates a second lower class in which the processing target data is classified using the selected lower classification model.

This enables the information processing device 100 to perform detailed classification using a miniaturized classification model. For example, the information processing device 100 can perform detailed classification of behavior using a miniaturized behavior recognition model. Furthermore, since the information processing device 100 can perform detailed classification using a miniaturized classification model, it can contribute to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure and promote inclusive and sustainable industrialization."

The estimation unit 134 also repeatedly executes the process of selecting a lower classification model from among the classification models corresponding to each of the multiple lower classes based on the estimation result of the higher-level classification model, and estimating a second lower class using the selected lower-level classification model, until the data to be processed is classified into one of multiple different classes.

This enables the information processing device 100 to perform detailed classification using a compact classification model.

The estimation unit 134 also uses the higher-level classification model to estimate a probability value that the data to be processed will be classified into each of a plurality of lower classes, and based on the estimation result of the higher-level classification model, selects a plurality of classification models corresponding to each of two or more predetermined number of lower classes in descending order of probability value from among the classification models corresponding to each of the plurality of lower classes, and uses the selected plurality of classification models to estimate a probability value that the data to be processed will be classified into each of the classes further below the lower classes corresponding to each of the selected plurality of classification models.

In this way, the information processing device 100 can prevent the estimation result of a lower class from being incorrect by adopting only one estimation result from a higher-level classification model.

In addition, when the data to be processed is classified into two or more of a plurality of different classes, the estimation unit 134 estimates that the data to be processed will be classified into the class corresponding to the classification path in which the product of the probability values is maximum, based on a comparison of the values of the product of the probability values for classification into each class in the classification path from the highest class to the class corresponding to each of the two or more classes among the plurality of different classes.

This allows the information processing device 100 to obtain appropriate classification results even when using estimation results from multiple classification models.

The acquisition unit 131 acquires the classification model, which is a machine learning model that classifies the data to be classified into a plurality of lower classes or an unknown class that is not one of the plurality of lower classes, generated for each upper class, and the data to be processed. The estimation unit 134 determines whether to discontinue processing subsequent to the processing of selecting, from among the classification models corresponding to each of the plurality of lower classes, a plurality of classification models corresponding to each of a predetermined number of lower classes, based on the estimation result of the upper classification model.

As a result, the information processing device 100 can stop processing if, for example, the data to be processed is classified into an unknown class, thereby preventing the estimation result of a lower class from being incorrect due to an error in the estimation result of a higher-level classification model.

In addition, the estimation unit 134 determines to discontinue processing after the process of selecting multiple classification models corresponding to each of a predetermined number of lower classes when the data to be classified is classified into an unknown class, when the difference in the probability values for the data to be processed being classified into each of multiple lower classes or an unknown class exceeds a first threshold value, or when the product of the probability values for the data to be processed being classified into each class along the classification path from the highest class to a class corresponding to each of two or more of the multiple different classes falls below a second threshold value.

As a result, the information processing device 100 can stop processing if an error in the estimation result of a higher-level classification model is likely to occur, thereby preventing the estimation result of a lower class from being incorrect due to an error in the estimation result of a higher-level classification model.

5. Hardware Configuration
Moreover, the information processing device 100 according to the embodiment described above is realized by a computer 1000 having a configuration as shown in Fig. 12, for example. Fig. 12 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, a HDD 1400, a communication interface (I/F) 1500, an input/output interface (I/F) 1600, and a media interface (I/F) 1700.

The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400, and controls each component. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000, etc.

HDD 1400 stores programs executed by CPU 1100 and data used by such programs. Communication interface 1500 receives data from other devices via a specified communication network and sends it to CPU 1100, and transmits data generated by CPU 1100 to other devices via the specified communication network.

The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input/output interface 1600. The CPU 1100 acquires data from the input devices via the input/output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input/output interface 1600.

The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, when the computer 1000 functions as the information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 130. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a specified communication network.

Although several embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the disclosure section of the invention.

[6. Other]
Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

For example, in the above embodiment, the information processing device 100 has the acquisition unit 131, the class generation unit 132, the model generation unit 133, and the estimation unit 134 as functional units, but each unit can be distributed and configured in a separate device. For example, the information processing device 100 can have the acquisition unit 131 and the estimation unit 134 as functional units as an information processing device that estimates the class into which the processing target data is classified. In addition, an information processing device other than the information processing device 100 (hereinafter referred to as a "generation device") can have the acquisition unit 131, the class generation unit 132, and the model generation unit 133 as functional units as an information processing device that generates a classification model. At this time, the information processing device 100 and the generation device may be connected to various networks by wire or wirelessly and may transmit and receive information to each other. For example, the information processing device 100 may receive information about the classification model generated by the generation device from the generation device. In addition, the information processing device 100 may estimate the class into which the processing target data is classified using the classification model received from the generation device.

The above-described embodiments and variations can be combined as appropriate to the extent that they do not cause inconsistencies in the processing content.

REFERENCE SIGNS LIST 100 Information processing device 110 Communication unit 120 Storage unit 130 Control unit 131 Acquisition unit 132 Class generation unit 133 Model generation unit 134 Estimation unit

Claims (15)

a class generation unit that generates a higher class that is a higher class than the plurality of different classes, the higher class corresponding to each of the hierarchical clusters based on the plurality of class features hierarchically clustered based on a similarity between the plurality of class features relating to data respectively belonging to a plurality of different classes and being representative values of the plurality of different classes;
a model generation unit that generates a classification model, which is a machine learning model that classifies data to be classified into one of a plurality of lower classes that are lower than the upper class, for each of the upper classes;
Equipped with
The model generation unit
generating, for each of the upper classes, the classification model, which is the machine learning model trained based on a training data set that is a set of data sampled from each of the plurality of lower classes such that the number of data sampled from each of the plurality of lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs;
Information processing device. The class generation unit
generating the higher class by using an average value of the feature amounts of the data belonging to each of the plurality of different classes as the representative value;
The information processing device according to claim 1 . The class generation unit
hierarchically clustering the plurality of class features until the number of newly generated clusters falls below a predetermined threshold.
The information processing device according to claim 1 . The class generation unit
generating a plurality of first higher order classes which are classes corresponding to a plurality of first higher order clusters which are clusters generated by combining each of a plurality of lowest-level clusters which are clusters having each of the plurality of class features as elements based on the similarities of the plurality of class features, and a plurality of second higher order classes which are classes corresponding to a plurality of second higher order clusters which are clusters generated by combining each of the plurality of first higher order clusters based on the similarities of the plurality of first higher order clusters, thereby generating the higher order classes;
The information processing device according to claim 1 . The class generation unit
generating a tree diagram, which is a graph representing the process of generating the hierarchical clusters;
The model generation unit
generating the classification models in a number corresponding to the number of branch points included in the tree diagram;
The information processing device according to claim 1 . The model generation unit
generating, for each of the upper classes, the classification model, which is the machine learning model trained to output the lower label as output information when the data is input as input information to the machine learning model;
The information processing device according to claim 1 . The model generation unit
generating, for each of the upper classes, the classification model, which is the machine learning model that classifies the data to be classified into one of the plurality of lower classes or an unknown class that is not one of the plurality of lower classes;
The information processing device according to claim 1 . a classification model which is a machine learning model that classifies data to be classified into a lower class among a plurality of lower classes that belong to a higher class that is a higher class than the plurality of different classes, the lower class being a class corresponding to each of the hierarchical clusters based on a plurality of class features hierarchically clustered based on a similarity between the plurality of class features that are representative values of the plurality of different classes, the classification model being a machine learning model that classifies data to be classified into a lower class among a plurality of lower classes that belong to a lower class that is a higher class than the plurality of different classes, the classification model being trained based on a learning dataset that is a set of data sampled from each of the plurality of lower classes such that the number of data sampled from each of the plurality of lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs, and an acquisition unit that acquires the classification model which is the machine learning model generated for each of the higher classes , and processing target data to be processed;
an estimation unit that estimates a first lower class into which the data to be processed is classified using a higher-level classification model, which is the classification model corresponding to a predetermined higher class selected in order from the classification model corresponding to the highest-level class, selects a lower-level classification model, which is the classification model corresponding to the first lower class, from among the classification models corresponding to each of a plurality of lower classes below the predetermined higher-level class based on a result of the estimation of the higher-level classification model, and estimates a second lower class into which the data to be processed is classified using the selected lower-level classification model;
An information processing device comprising: The estimation unit is
repeatedly performing a process of selecting the lower classification model from among the classification models corresponding to each of the plurality of lower classes based on an estimation result of the higher classification model, and estimating the second lower class using the selected lower classification model, until the processing target data is classified into one of the plurality of different classes.
The information processing device according to claim 8 . The estimation unit is
using the higher-level classification model, estimating a probability value that the data to be processed will be classified into each of the plurality of lower classes; selecting a plurality of the classification models corresponding to each of two or more predetermined number of lower classes in descending order of probability value from among the classification models corresponding to each of the plurality of lower classes based on an estimation result of the higher-level classification model; and estimating a probability value that the data to be processed will be classified into each of classes further below the lower classes corresponding to each of the selected plurality of classification models.
The information processing device according to claim 8 . The estimation unit is
When the data to be processed is classified into two or more of the plurality of different classes, it is estimated that the data to be processed will be classified into the class corresponding to the classification path having the largest value of the product of the probability values based on a comparison of the product of the probability values of the data to be classified into each class in a classification path from the highest class to a class corresponding to each of the two or more classes among the plurality of different classes.
The information processing device according to claim 10 . The acquisition unit is
The classification model is a machine learning model that classifies the data to be classified into one of the plurality of lower classes or an unknown class that is not one of the plurality of lower classes, and the classification model generated for each of the upper classes and the data to be processed are acquired;
The estimation unit is
determining whether to discontinue processing subsequent to a processing of selecting a plurality of the classification models corresponding to each of the predetermined number of lower classes from the classification models corresponding to each of the plurality of lower classes based on an estimation result of the higher-level classification model;
The information processing device according to claim 10 . The estimation unit is
when the classification target data is classified into the unknown class, when a difference in probability values for the processing target data being classified into each of the plurality of lower classes or the unknown class exceeds a first threshold value, or when a value of a product of probability values for the processing target data being classified into each class in a classification path from the highest class to a class corresponding to each of two or more classes among the plurality of different classes falls below a second threshold value, determining to discontinue processing subsequent to the processing of selecting a plurality of the classification models corresponding to each of the predetermined number of lower classes.
The information processing device according to claim 12 . An information processing method implemented by a program executed by an information processing device, comprising:
a class generating step of generating higher classes that are higher than the plurality of different classes, the higher classes corresponding to the hierarchical clusters based on the plurality of class features hierarchically clustered based on a similarity between the plurality of class features relating to data respectively belonging to a plurality of different classes and that are representative values of the plurality of different classes;
a model generation step of generating, for each upper class, a classification model which is a machine learning model that classifies data to be classified into one of a plurality of lower classes that belong to the lower class;
Including,
The model generation step includes:
generating, for each of the upper classes, the classification model, which is the machine learning model trained based on a training data set that is a set of data sampled from each of the plurality of lower classes such that the number of data sampled from each of the plurality of lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs;
Information processing methods. An information processing method implemented by a program executed by an information processing device, comprising:
a classification model which is a machine learning model that classifies data to be classified into a lower class among a plurality of lower classes that belong to a higher class that is a higher class than the plurality of different classes, the lower class being a class corresponding to each of the hierarchical clusters based on a plurality of class features hierarchically clustered based on a similarity between the plurality of class features that are representative values of the plurality of different classes, the classification model being a machine learning model that classifies data to be classified into a lower class among a plurality of lower classes that belong to a lower class that is a higher class than the plurality of different classes, the classification model being trained based on a learning dataset that is a set of data sampled from each of the plurality of lower classes such that the number of data sampled from each of the plurality of lower classes is similar, and a lower label that is a label corresponding to the lower class to which the data belongs, and an acquisition step of acquiring the classification model which is the machine learning model generated for each of the higher classes , and the processing target data to be processed;
an estimation step of estimating a first lower class into which the data to be processed is classified using a higher-level classification model which is the classification model corresponding to a predetermined higher class selected in order from the classification model corresponding to the highest-level class, selecting a lower-level classification model which is the classification model corresponding to the first lower class from among the classification models corresponding to each of a plurality of lower classes below the predetermined higher-level class based on a result of the estimation of the higher-level classification model, and estimating a second lower class into which the data to be processed is classified using the selected lower-level classification model;
An information processing method comprising:

Images