JP7568054B2 - Learning device, learning method, inference device, inference method, and program - Google Patents

Description

The present disclosure relates to a method for training a machine learning model.

In recent years, recognition technologies based on machine learning have demonstrated extremely high performance, particularly in the field of image recognition. The high accuracy of such recognition technologies based on machine learning is supported by a large amount of data with correct answers. In other words, high accuracy is achieved by preparing a large amount of data with correct answers and performing learning. For example, Patent Document 1 discloses a method for learning to identify categories with a hierarchical structure.

International Publication No. WO2006/073081

On the other hand, depending on the application of image recognition technology, there is a demand for low-cost, highly accurate machine learning without preparing large amounts of data with correct answers.

One objective of this disclosure is to generate machine learning models with high accuracy at low cost.

In one aspect of the present disclosure, a learning device includes:
a feature extraction means for converting input data into a first feature representation;
a projection means for transforming the first feature representation into a second feature representation representing points in a hyperbolic space;
a classification means for performing classification based on the second feature representation and outputting a score indicating the likelihood that the input data belongs to each class;
a loss calculation means for calculating a hierarchical loss based on knowledge of a hierarchical structure to which each of the classes belongs, a correct label assigned to the input data, and the score;
an update means for updating parameters of the feature extraction means, the projection means, and the classification means based on the hierarchical loss;
Equipped with
The classification means outputs a score for each of the terminal classes of the hierarchical structure;
The loss calculation means integrates the scores of the leaf classes to calculate the loss of a layer above the layer of the leaf class, and calculates a weighted sum of the losses of each layer as the hierarchical loss.

In another aspect of the present disclosure, a method of learning comprises:
Transforming the input data into a first feature representation using feature extraction means;
using a projection means to transform the first feature representation into a second feature representation indicative of points in hyperbolic space;
using a classification means to perform classification based on the second feature representation and output a score indicative of the likelihood that the input data belongs to each class;
Calculating a hierarchical loss based on knowledge of the hierarchical structure to which each class belongs, the correct label assigned to the input data, and the score;
updating parameters of the feature extraction means, the projection means and the classification means based on the hierarchical loss ;
The classification is performed by outputting a score for each of the terminal classes of the hierarchical structure;
The hierarchical loss is calculated by integrating the scores of the terminal classes to calculate the loss of the layers above the terminal class, and calculating the weighted sum of the losses of each layer as the hierarchical loss .

In yet another aspect of the disclosure, a program includes:
Transforming the input data into a first feature representation using feature extraction means;
using a projection means to transform the first feature representation into a second feature representation indicative of points in hyperbolic space;
using a classification means to perform classification based on the second feature representation and output a score indicative of the likelihood that the input data belongs to each class;
Calculating a hierarchical loss based on knowledge of the hierarchical structure to which each class belongs, the correct label assigned to the input data, and the score;
updating parameters of the feature extraction means, the projection means and the classification means based on the hierarchical loss ;
The classification is performed by outputting a score for each of the terminal classes of the hierarchical structure;
The calculation of the hierarchical loss involves causing a computer to execute a process of integrating the scores of the terminal classes to calculate the loss of the hierarchy above the hierarchy of the terminal class, and calculating the weighted sum of the losses of each hierarchy as the hierarchical loss .

In yet another aspect of the present disclosure, a reasoning apparatus includes:
a feature extraction means for converting input data into a first feature representation;
a projection means for transforming the first feature representation into a second feature representation representing points in a hyperbolic space;
and a classification means for performing classification based on the second feature representation and calculating, for each hierarchy, a score indicating the likelihood that the input data belongs to each class, using knowledge of the hierarchical structure to which each class belongs.

In yet another aspect of the present disclosure, a method of inference includes:
Transforming the input data into a first feature representation;
Transforming the first feature representation into a second feature representation that represents points in a hyperbolic space;
A classification is performed based on the second feature representation, and knowledge of the hierarchical structure to which each class belongs is used to calculate a score for each hierarchical level indicating the likelihood that the input data belongs to each class.

In yet another aspect of the disclosure, a program includes:
Transforming the input data into a first feature representation;
Transforming the first feature representation into a second feature representation that represents points in a hyperbolic space;
The computer is caused to perform a process of performing classification based on the second feature representation, and calculating, for each hierarchy, a score indicating the likelihood that the input data belongs to each class, using knowledge of the hierarchical structure to which each class belongs.

According to the present disclosure, by using knowledge of class structure, it is possible to generate machine learning models with high accuracy at low cost.

FIG. 2 is a block diagram showing a hardware configuration of the learning device according to the first embodiment. FIG. 2 is a block diagram showing the functional configuration of the learning device according to the first embodiment. Here is an example of a class hierarchy: 4 is a flowchart of a learning process performed by the learning device of the first embodiment. 1 is a block diagram showing a functional configuration of an inference device according to a first embodiment; 4 is a flowchart of an inference process performed by the inference device of the first embodiment. FIG. 11 is a block diagram showing the functional configuration of a learning device according to a second embodiment. 1 shows an example of sharing among multiple classifiers constituting a hierarchical hyperbolic classifier. Another example of sharing of work among multiple classifiers constituting a hierarchical hyperbolic classifier is shown below. 13 is a flowchart of a learning process performed by a learning device according to a second embodiment. FIG. 11 is a block diagram showing the functional configuration of an inference device according to a second embodiment. 13 is a flowchart of an inference process performed by the inference device of the second embodiment. FIG. 13 is a block diagram showing the functional configuration of a learning device according to a third embodiment. 1 shows a schematic configuration of a hierarchical hyperbolic projection unit. FIG. 1 is a diagram conceptually explaining feature representation and difference. 13 is a flowchart of a learning process performed by a learning device according to a third embodiment. FIG. 13 is a block diagram showing the functional configuration of an inference device according to a third embodiment. 13 is a flowchart of an inference process performed by the inference device of the third embodiment. FIG. 13 is a block diagram showing the functional configuration of a learning device according to a fourth embodiment. 13 is a flowchart of a learning process performed by a learning device according to a fourth embodiment. FIG. 13 is a block diagram showing the functional configuration of an inference device according to a fifth embodiment. 13 is a flowchart of an inference process performed by the inference device of the fifth embodiment.

Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings.
<Concept explanation>
As described above, a highly accurate recognition model can be obtained by performing learning using a large amount of learning data with correct answers, but there are cases where it is required to generate a highly accurate model at low cost from a small amount of data. In order to learn a highly accurate model from a small amount of data, it is essential to use information other than the learning data. When performing multi-class classification, knowledge regarding the hierarchical structure of classes is highly versatile and often easily available. Therefore, in the following embodiment, a learning method is provided that makes it possible to obtain a highly accurate classification model even with a small amount of data by using knowledge indicating the hierarchical structure of the classes to be classified.

First Embodiment
[Learning device]
First, the learning device of the first embodiment will be described.
(Hardware configuration)
1 is a block diagram showing the hardware configuration of a learning device 100 according to a first embodiment. As shown in the figure, the learning device 100 includes an interface (I/F) 11, a processor 12, a memory 13, a recording medium 14, and a database (DB) 15.

The interface 11 inputs and outputs data between the interface 11 and an external device. Specifically, the data with correct answers used for learning is input through the interface 11.

The processor 12 is a computer such as a CPU (Central Processing Unit) and controls the entire learning device 100 by executing a program prepared in advance. The processor 12 may be a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array). The processor 12 executes the learning process described below.

The memory 13 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 13 is also used as a working memory while the processor 12 is executing various processes.

The recording medium 14 is a non-volatile, non-temporary recording medium such as a disk-shaped recording medium or semiconductor memory, and is configured to be detachable from the learning device 100. The recording medium 14 records various programs executed by the processor 12. When the learning device 100 executes various processes, the programs recorded on the recording medium 14 are loaded into the memory 13 and executed by the processor 12. The DB 15 stores data with correct answers for learning as necessary.

(Functional configuration)
2 is a block diagram showing the functional configuration of the learning device 100 according to the first embodiment. The learning device 100 includes a feature extraction unit 21, a hyperbolic projection unit 22, a hyperbolic classification unit 23, a hierarchical loss calculation unit 24, a gradient calculation unit 25, and an update unit 26.

The data with correct answers includes input data and a correct answer label corresponding to the input data. For example, when training an image recognition model, the input data is an image used for training, and the correct answer label is information indicating the class of the object contained in the image. Of the data with correct answers, the input data is input to the feature extraction unit 21, and the correct answer label is input to the hierarchical loss calculation unit 24.

The feature extraction unit 21 converts the input data into a pre-feature representation. The feature representation output by the feature extraction unit 21 is called the "pre-feature representation" to distinguish it from the feature representation output by the hyperbolic projection unit 22 described later. Both the "pre-feature representation" and the "feature representation" are information that represents the features of the input data. Specifically, when training an image recognition model, the feature extraction unit 21 is composed of a deep convolutional neural network (CNN) or the like, and outputs a sequence (vector) of real values that indicates the features of the input image to the hyperbolic projection unit 22 as the pre-feature representation.

The hyperbolic projection unit 22 converts the previous feature representation into a feature representation. Here, a "feature representation" is a point on some manifold, and selecting a specific projection unit is equivalent to selecting the manifold (feature space) to which the feature representation belongs. In general, a linear space (Euclidean space) is used as the feature space and a linear projection unit is used as the projection unit, or a high-dimensional hypersphere is used as the feature space and a spherical projection unit is used as the projection unit. In contrast, this embodiment uses a hyperbolic space as the feature space.

As described above, this embodiment uses knowledge about the hierarchical structure of classes to obtain a highly accurate model with a small amount of training data, but the hierarchical structure (tree structure) has the property of expanding exponentially. Generally, Euclidean space or a hypersphere is used as a feature space, but Euclidean space or a hypersphere only expands polynomially, so it is not suitable for embedding a tree structure. In other words, when expressing a hierarchical structure on a Euclidean space or a hypersphere, distortion is unavoidable in low dimensions. Therefore, in order to express a hierarchical structure (tree structure) on a Euclidean space or a hypersphere without distortion, it is necessary to use a feature space that is exponentially high in terms of the number of classes.

From this point of view, in this embodiment, a hyperbolic space is used as the feature space. A tree structure can be embedded efficiently in the hyperbolic space. A tree structure can be embedded without distortion even in two dimensions in the exponentially expanding hyperbolic space. Therefore, the hyperbolic projection unit 22 converts the previous feature representation into a feature representation in the hyperbolic space and outputs it to the hyperbolic classification unit 23. The feature representation is also a sequence (vector) of real values like the previous feature representation, but can be regarded as a coordinate value in the hyperbolic space, which is the feature space. The hyperbolic projection unit 22 can use Poincaré projection, Lorentz projection, etc. depending on the specific model of the hyperbolic space.

The hyperbolic classification unit 23 performs classification from one feature representation in the hyperbolic space output by the hyperbolic projection unit 22, and outputs the scores of each class obtained for that feature representation to the hierarchical loss calculation unit 24. The hyperbolic classification unit 23 outputs only the scores of the terminal classes in the hierarchical structure of classes. A hyperbolic hyperplane classifier or a hyperbolic nearest neighbor classifier can be used as the hyperbolic classification unit 23. The hyperbolic hyperplane classifier is a classifier that extends a linear classifier to hyperbolic space and uses a hyperplane in hyperbolic space as the discrimination plane. The hyperbolic nearest neighbor classifier is a classifier that follows the nearest neighbor rule in hyperbolic space. The specific expression of the hyperbolic classification unit 23 is determined by the model of hyperbolic space selected by the hyperbolic projection unit 22.

The hierarchical loss calculation unit 24 calculates a loss function from the scores of each class input from the hyperbolic classification unit 23 and the correct label included in the data with correct answers. At this time, the hierarchical loss calculation unit 24 uses knowledge of the hierarchical structure of the classes to be classified. Specifically, the hierarchical loss calculation unit 24 calculates a score for each layer from the scores of each class output by the hyperbolic classification unit 23, and calculates a loss for each layer so that the score for each layer can predict the correct class in each layer. The hierarchical loss calculation unit 24 can use a general loss function for multi-value classification, such as cross-entropy loss.

Here, the method of calculating losses by the hierarchical loss calculation unit 24 will be described with an example. FIG. 3 shows an example of a hierarchical structure of classes. This example shows a hierarchical structure (tree structure) with the root node "product" and has the first to third hierarchies. The first hierarchical structure includes three classes "food", "beverage", and "medicine" as child nodes of "product". The second hierarchical structure includes three classes "lunch", "bread", and "rice ball" as child nodes of "food", and includes three classes "tea", "juice", and "water" as child nodes of "drink". The third hierarchical structure includes "lunch A" to "lunch C" as child nodes of "lunch", "bread A" to "bread C" as child nodes of "bread", and "rice ball A" to "rice ball C" as child nodes of "rice ball". For convenience, the second and third hierarchical structures of medicine and the third hierarchical structure of beverages are omitted from the illustration.

As mentioned above, the hyperbolic classification unit 23 outputs only the scores of the leaf classes in the hierarchical structure of classes. In the example of Figure 3, the hyperbolic classification unit 23 outputs only the scores of leaf classes such as "Lunch A" to "Lunch C", "Bread A" to "Bread C", and "Onigiri A" to "Onigiri C". Now, assume that some input data is input and its correct label is "Lunch B". In this case, the hierarchical loss calculation unit 24 calculates a loss for the third hierarchical layer, which is the hierarchy of leaf classes, that maximizes the score of the correct class, "Lunch B", and sets this as the loss for the third hierarchical layer.

In addition, when calculating the loss of a hierarchy higher than the terminal class, the hierarchical loss calculation unit 24 integrates the scores of the terminal classes that are descendants of each node and uses them in the loss calculation. Specifically, if the score output by the hyperbolic classification unit 23 is the probability of the terminal class, the score of each class in the higher hierarchy is the sum of the probability of its descendant terminal class.

For example, in the example of Figure 3, the score of "Lunch" in the second hierarchy is the sum of the scores of its child nodes "Lunch A" to "Lunch C". Similarly, the score of "Bread" in the second hierarchy is the sum of the scores of its child nodes "Bread A" to "Bread C", and the score of "Onigiri" in the second hierarchy is the sum of the scores of its child nodes "Onigiri A" to "Onigiri C". The hierarchical loss calculation unit 24 then calculates a loss for the second hierarchy that maximizes the score of "Lunch", which has the correct class "Lunch B" as a descendant node.

The score of "Food" in the first hierarchical layer is the sum of the scores of its grandchild nodes, the leaf classes "Lunchbox A" to "Lunchbox C," "Bread A" to "Bread C," and "Onigiri A" to "Onigiri C." Similarly, the scores of "Drinks" and "Medicines" in the first hierarchical layer are the sum of the scores of their grandchild nodes, the leaf classes. For the first hierarchical layer, the hierarchical loss calculation unit 24 calculates a loss that maximizes the score of "Food," which has the correct class "Lunchbox B" as a descendant node. The hierarchical loss calculation unit 24 then calculates a weighted sum of the losses calculated for each hierarchical layer, and outputs the result to the gradient calculation unit 25 as the hierarchical loss.

The gradient calculation unit 25 calculates the gradient of the hierarchical loss input from the hierarchical loss calculation unit 24 and outputs it to the update unit 26. The update unit 26 uses the gradient to update the parameters of the feature extraction unit 21, the hyperbolic projection unit 22, and the hyperbolic classification unit 23.

(Learning process)
4 is a flowchart of the learning process by the learning device 100 of the first embodiment. This process is realized by the processor 12 shown in FIG. 1 executing a program prepared in advance and operating as each element shown in FIG.

First, the feature extraction unit 21 converts the input data into a pre-feature representation (step S11). Next, the hyperbolic projection unit 22 converts the pre-feature representation into a feature representation in hyperbolic space (step S12). Next, the hyperbolic classification unit 23 calculates the score of each class from the feature representation (step S13). Next, the hierarchical loss calculation unit 24 calculates the hierarchical loss from the score of each class and the correct label using knowledge of the hierarchical structure of the classes (step S14). Next, the gradient calculation unit 25 calculates the gradient of the hierarchical loss (step S15). Next, the update unit 26 updates the parameters of the feature extraction unit 21, the hyperbolic projection unit 22, and the hyperbolic classification unit 23 based on the gradient (step S16). The above process is repeated until a predetermined learning end condition is met, and the learning process ends.

As described above, according to the learning device 100 of the first embodiment, it is possible to learn a highly accurate model even with a small amount of training data by utilizing knowledge of the hierarchical structure of classes.

[Inference device]
Next, the inference device of the first embodiment will be described.
(Hardware configuration)
The hardware configuration of the inference device 200 of the first embodiment is similar to that of the learning device 100 shown in FIG. 1, and therefore a description thereof will be omitted.

(Functional configuration)
5 is a block diagram showing the functional configuration of the inference device 200 of the first embodiment. The inference device 200 includes a feature extraction unit 21, a hyperbolic projection unit 22, and a hyperbolic classification unit 23. Note that the feature extraction unit 21, the hyperbolic projection unit 22, and the hyperbolic classification unit 23 are set with parameters obtained by the previous learning process.

Input data is input to the feature extraction unit 21. This input data is data such as an image that is actually the subject of class classification. The feature extraction unit 21 converts the input data into a pre-feature representation and outputs it to the hyperbolic projection unit 22. The hyperbolic projection unit 22 converts the pre-feature representation into a feature representation in hyperbolic space and outputs it to the hyperbolic classification unit 23. The hyperbolic classification unit 23 calculates scores for terminal classes in the hierarchical structure of classes and outputs them as inference results. In this way, the input data is classified into classes.

(Inference Processing)
6 is a flowchart of an inference process performed by the inference device 200 of the first embodiment. This process is realized by the processor 12 shown in FIG. 1 executing a program prepared in advance and operating as each element shown in FIG.

First, the feature extraction unit 21 converts the input data into a pre-feature representation (step S21). Next, the hyperbolic projection unit 22 converts the pre-feature representation into a feature representation in hyperbolic space (step S22). Next, the hyperbolic classification unit 23 calculates the score of each terminal class from the feature representation and outputs it as an inference result (step S23). The above process is performed for each input data.

Second Embodiment
Next, a second embodiment will be described. In the second embodiment, the hyperbolic classifier is also hierarchical using knowledge of the hierarchical structure of classes.

[Learning device]
First, the learning device of the second embodiment will be described.
(Hardware configuration)
The hardware configuration of the learning device 100a of the second embodiment is similar to that of the learning device 100 shown in FIG. 1, and therefore a description thereof will be omitted.

(Functional configuration)
7 is a block diagram showing the functional configuration of a learning device 100a according to the second embodiment. As can be seen from a comparison with the learning device 100 according to the first embodiment shown in FIG. 2, the learning device 100a according to the second embodiment has a hierarchical hyperbolic classifier 23x instead of the hyperbolic classifier 23.

The hierarchical hyperbolic classification unit 23x uses knowledge of the hierarchical structure of classes to output a score at each level of the hierarchical structure from one feature representation in hyperbolic space input from the hyperbolic projection unit 22. Figure 8 shows an example of a sharing method by multiple classifiers that constitute the hierarchical hyperbolic classification unit 23x. Each of the frames 91 to 93 shown in bold lines indicates a part corresponding to one classifier. In the example of Figure 8, one classifier is provided for each level in the hierarchical structure of classes. That is, the hierarchical hyperbolic classification unit 23x is composed of three classifiers corresponding to the first to third levels, respectively. Each classifier is a classifier that identifies nodes belonging to the same level across subtrees. In this example, the hierarchical hyperbolic classification unit 23x outputs the classification results for each level by the three classifiers.

Figure 9 shows another example of a sharing method by multiple classifiers constituting the hierarchical hyperbolic classification unit 23x. Each of the bold boxes 91 to 93 indicates a part corresponding to one classifier. In the example of Figure 9, in the third layer, as shown in box 93, multiple classifiers are provided to identify sibling nodes belonging to the same parent node. That is, one classifier is prepared corresponding to the nodes "Lunch A" to "Lunch C" belonging to the same parent node "Lunch", and one classifier is prepared corresponding to the nodes "Bread A" to "Bread C" belonging to the same parent node "Bread". In addition, for the third layer, one classifier is also prepared for sibling nodes belonging to all parent nodes other than "Lunch" and "Bread" in the second layer, but is omitted from the illustration for convenience. In this example, the hierarchical hyperbolic classification unit 23x outputs the classification results by each of the multiple classifiers. That is, the hierarchical hyperbolic classification unit 23x outputs a classification result corresponding to frame 91 for the first layer, outputs a classification result corresponding to frame 92 for the second layer, and outputs classification results corresponding to multiple frames 93 for the third layer.

With any of the above configurations, the hierarchical hyperbolic classification unit 23x calculates classification results (scores) for each hierarchy and outputs them to the hierarchical loss calculation unit 24. The hierarchical loss calculation unit 24 calculates losses for the classification results of each hierarchy input from the hierarchical hyperbolic classification unit 23x, and outputs the weighted sum of the losses as the hierarchical loss to the gradient calculation unit 25. As described above, in the second embodiment, the hierarchical hyperbolic classification unit 23x outputs not only the scores of the terminal classes but also the scores of the classes in the upper hierarchy, so that the hierarchical loss calculation unit 24 does not need to integrate the scores of the terminal classes to calculate the scores of the upper hierarchy as in the first embodiment.

In addition, since the configurations and operations of the feature extraction unit 21, gradient calculation unit 25 and update unit 26 in the learning device 100a of the second embodiment are the same as those in the first embodiment, their explanations are omitted.

(Learning process)
10 is a flowchart of the learning process by the learning device 100a of the second embodiment. This process is realized by the processor 12 shown in FIG. 1 executing a program prepared in advance and operating as each element shown in FIG.

First, the feature extraction unit 21 converts the input data into a pre-feature representation (step S31). Next, the hyperbolic projection unit 22 converts the pre-feature representation into a feature representation in hyperbolic space (step S32). Next, the hierarchical hyperbolic classification unit 23x calculates the score of each class for each layer from the feature representation using knowledge of the hierarchical structure of the classes (step S33). Next, the hierarchical loss calculation unit 24 calculates the hierarchical loss from the score of each class for each layer and the correct answer label (step S34). Next, the gradient calculation unit 25 calculates the gradient of the hierarchical loss (step S35). Next, the update unit 26 updates the parameters of the feature extraction unit 21, the hyperbolic projection unit 22, and the hierarchical hyperbolic classification unit 23x based on the gradient (step S36). The above process is repeated until a predetermined learning end condition is met, and the learning process ends.

[Inference device]
Next, an inference device according to a second embodiment will be described.
(Hardware configuration)
The hardware configuration of the inference device 200 is similar to that of the learning device 100 shown in FIG. 1, and therefore a description thereof will be omitted.

(Functional configuration)
11 is a block diagram showing the functional configuration of an inference device 200a according to the second embodiment. The inference device 200a includes a feature extraction unit 21, a hyperbolic projection unit 22, and a hierarchical hyperbolic classification unit 23x. Note that parameters obtained by the previous learning process are set in the feature extraction unit 21, the hyperbolic projection unit 22, and the hierarchical hyperbolic classification unit 23x.

Input data is input to the feature extraction unit 21. This input data is data such as an image that is actually the subject of class classification. The feature extraction unit 21 converts the input data into a pre-feature representation and outputs it to the hyperbolic projection unit 22. The hyperbolic projection unit 22 converts the pre-feature representation into a feature representation in hyperbolic space and outputs it to the hierarchical hyperbolic classification unit 23x. The hierarchical hyperbolic classification unit 23x uses knowledge of the hierarchical structure of classes to calculate a score for each class in each hierarchy and outputs it as an inference result. In this way, the input data is classified into classes.

(Inference Processing)
12 is a flowchart of an inference process performed by the inference device 200a of the second embodiment. This process is realized by the processor 12 shown in FIG 1 executing a program prepared in advance and operating as each element shown in FIG 11.

First, the feature extraction unit 21 converts the input data into a pre-feature representation (step S41). Next, the hyperbolic projection unit 22 converts the pre-feature representation into a feature representation in hyperbolic space (step S42). Next, the hierarchical hyperbolic classification unit 23x uses knowledge of the hierarchical structure of classes to calculate the score of each class for each hierarchy from the feature representation and outputs it as an inference result (step S43). The above process is performed for each input data.

Third Embodiment
Next, a third embodiment will be described. In the third embodiment, the hyperbolic projection unit 22 is also hierarchical using knowledge of the hierarchical structure of classes.

[Learning device]
First, the learning device of the third embodiment will be described.
(Hardware configuration)
The hardware configuration of the learning device 100b of the third embodiment is similar to that of the learning device 100 of the first embodiment shown in FIG. 1, and therefore a description thereof will be omitted.

(Functional configuration)
Fig. 13 is a block diagram showing the functional configuration of a learning device 100b according to the third embodiment. As can be seen from a comparison with the learning device 100a according to the second embodiment shown in Fig. 7, the learning device 100b according to the second embodiment has a hierarchical hyperbolic projection unit 22x instead of the hyperbolic projection unit 22.

The hierarchical hyperbolic projection unit 22x uses knowledge of the hierarchical structure of classes to output feature representations at each level of the hierarchical structure from the previous feature representation input from the feature extraction unit 21. Figure 14 shows a schematic configuration of the hierarchical hyperbolic projection unit 22x. The hierarchical hyperbolic projection unit 22x includes first to third embedded networks (NWs) and adders 31 and 32.

The first to third embedding networks receive the previous feature representation from the feature extraction unit 21. The first embedding network uses knowledge of the hierarchical structure of classes to output, as feature representation C1, a vector indicating a point in hyperbolic space of a class corresponding to a node in the first hierarchy.

For a node in the second layer, the second embedded network outputs a difference D1 between the feature representation C1 of the class corresponding to the parent node of the node and the feature representation of the node. The adder 31 then outputs the sum of the feature representation C1 of the parent node and the difference D1 as the feature representation C2 corresponding to the node in the second layer. Like the feature representation C1, the feature representation C2 is a vector indicating a point in hyperbolic space.

Similarly, for a node in the third layer, the third embedded network outputs a difference D2 between the feature representation C2 of the class corresponding to the parent node of the node and the feature representation of the node. The adder 32 then outputs the sum of the feature representation C2 of the parent node and the difference D2 as the feature representation C3 corresponding to the node in the third layer. Like feature representation C1, feature representation C3 is a vector indicating a point in hyperbolic space.

Figure 15 is a diagram conceptually explaining the above characteristic representations C1 to C3 and the differences D1 to D2. In Figure 15, the hyperbolic space is shown as a two-dimensional space for convenience. Assuming the hierarchical structure of classes shown in Figure 3, a circle (●) indicates the characteristic representation C1 of the first-level class, a square (■) indicates the characteristic representation C2 of the second-level class, and a triangle (▲) indicates the characteristic representation C3 of the third-level class. In this case, the difference D1 can be considered as a vector pointing from the first-level class "food" shown by a circle to the second-level classes "bento", "bread", and "onigiri" shown by squares. Similarly, the difference D2 can be considered as a vector pointing from the second-level class "bread" shown by a square to the third-level classes "bread A" to "bread C" shown by triangles. Mathematically, the above "difference" is a tangent vector of the hyperbolic space in the characteristic representation of the parent node class, and the "sum" is realized by an exponential map.

In this way, the hierarchical hyperbolic projection unit 22x outputs the feature representations C1 to C3 for each layer for one input data to the hierarchical hyperbolic classification unit 23x. The hierarchical hyperbolic classification unit 23x receives the feature representations for each layer, classifies them for each layer, and outputs the classification results to the hierarchical loss calculation unit 24.

In addition, since the configurations and operations of the feature extraction unit 21, gradient calculation unit 25 and update unit 26 in the learning device 100b of the third embodiment are the same as those in the first embodiment, their explanations are omitted.

(Learning process)
Fig. 16 is a flowchart of the learning process by the learning device 100b of the third embodiment. This process is realized by the processor 12 shown in Fig. 1 executing a program prepared in advance and operating as each element shown in Fig. 13.

First, the feature extraction unit 21 converts the input data into a previous feature representation (step S51). Next, the hierarchical hyperbolic projection unit 22x converts the previous feature representation into a feature representation in hyperbolic space for each layer (step S52). Next, the hierarchical hyperbolic classification unit 23x calculates the score of each class for each layer from the feature representation for each layer input from the hierarchical hyperbolic projection unit 22x (step S53). Next, the hierarchical loss calculation unit 24 calculates the hierarchical loss from the score of each class for each layer and the correct answer label (step S54). Next, the gradient calculation unit 25 calculates the gradient of the hierarchical loss (step S55). Next, the update unit 26 updates the parameters of the feature extraction unit 21, the hierarchical hyperbolic projection unit 22x, and the hierarchical hyperbolic classification unit 23x based on the gradient (step S56). The above process is repeated until a predetermined learning end condition is met, and the learning process ends.

[Inference device]
Next, an inference device 200b according to the third embodiment will be described.
(Hardware configuration)
The hardware configuration of the inference device 200b is similar to that of the learning device 100 shown in FIG. 1, and therefore a description thereof will be omitted.

(Functional configuration)
17 is a block diagram showing the functional configuration of an inference device 200b according to the third embodiment. The inference device 200b includes a feature extraction unit 21, a hierarchical hyperbolic projection unit 22x, and a hierarchical hyperbolic classification unit 23x. Note that parameters obtained by the previous learning process are set in the feature extraction unit 21, the hierarchical hyperbolic projection unit 22x, and the hierarchical hyperbolic classification unit 23x.

Input data is input to the feature extraction unit 21. This input data is data such as an image that is actually the subject of class classification. The feature extraction unit 21 converts the input data into a pre-feature representation and outputs it to the hierarchical hyperbolic projection unit 22x. The hierarchical hyperbolic projection unit 22x uses knowledge of the hierarchical structure of classes to convert the pre-feature representation into a feature representation in hyperbolic space for each layer and outputs it to the hierarchical hyperbolic classification unit 23x. The hierarchical hyperbolic classification unit 23x calculates a score for each class in each layer based on the feature representation for each layer and outputs it as an inference result. In this way, the input data is classified into classes.

(Inference Processing)
18 is a flowchart of an inference process performed by the inference device 200b of the third embodiment. This process is realized by the processor 12 shown in FIG. 1 executing a program prepared in advance and operating as each element shown in FIG.

First, the feature extraction unit 21 converts the input data into a pre-feature representation (step S61). Next, the hierarchical hyperbolic projection unit 22x converts the pre-feature representation into a feature representation in hyperbolic space for each layer (step S62). Next, the hierarchical hyperbolic classification unit 23x calculates the score of each class for each layer from the feature representation for each layer, and outputs it as an inference result (step S63). The above process is performed for each input data.

Fourth Embodiment
19 is a block diagram showing the functional configuration of a learning device according to the fourth embodiment. The learning device 70 includes a feature extraction unit 71, a projection unit 72, a classification unit 73, a loss calculation unit 74, and an update unit 75.

Figure 20 is a flowchart of the learning process by the learning device 70 of the fourth embodiment. First, the feature extraction means 71 converts the input data into a first feature representation (step S71). Next, the projection means 72 converts the first feature representation into a second feature representation indicating a point on a hyperbolic space (step S72). Next, the classification means 73 performs classification based on the second feature representation and outputs a score indicating the possibility that the input data belongs to each class (step S73). Next, the loss calculation means 74 calculates a hierarchical loss based on knowledge of the hierarchical structure to which each class belongs, the correct label assigned to the input data, and the score (step S74). Next, the update means 75 updates the parameters of the feature extraction means, the projection means, and the classification means based on the hierarchical loss (step S75). According to the fourth embodiment, by using knowledge of the hierarchical structure of the classes, it is possible to generate a highly accurate model even with a small amount of input data.

Fifth Embodiment
21 is a block diagram showing the functional configuration of an inference device according to the fifth embodiment. The inference device 80 includes a feature extraction unit 81, a projection unit 82, and a classification unit 83.

Figure 22 is a flowchart of the inference process by the inference device 80 of the fifth embodiment. First, the feature extraction means 81 converts the input data into a first feature representation (step S81). Next, the projection means 82 converts the first feature representation into a second feature representation indicating a point in hyperbolic space (step S82). Next, the classification means 83 performs classification based on the second feature representation, and calculates a score indicating the possibility that the input data belongs to each class for each hierarchy using knowledge of the hierarchical structure to which each class belongs (step S83). According to the fourth embodiment, it is possible to perform inference with high accuracy using a model learned using knowledge of the hierarchical structure of classes.

Some or all of the above embodiments may also be described as, but are not limited to, the following notes:

(Appendix 1)
a feature extraction means for converting input data into a first feature representation;
a projection means for transforming the first feature representation into a second feature representation representing points in a hyperbolic space;
a classification means for performing classification based on the second feature representation and outputting a score indicating the likelihood that the input data belongs to each class;
a loss calculation means for calculating a hierarchical loss based on knowledge of a hierarchical structure to which each of the classes belongs, a correct label assigned to the input data, and the score;
an update means for updating parameters of the feature extraction means, the projection means, and the classification means based on the hierarchical loss;
A learning device comprising:

(Appendix 2)
The classification means outputs a score for each of the terminal classes of the hierarchical structure;
The learning device according to claim 1, wherein the loss calculation means integrates the scores of the terminal classes to calculate the loss of a hierarchy above the hierarchy of the terminal class, and calculates a weighted sum of the losses of each hierarchy as the hierarchical loss.

(Appendix 3)
The learning device described in Appendix 2, wherein the loss calculation means calculates a loss that maximizes the score of the correct class for the hierarchy of the terminal classes, and calculates a loss that maximizes the score of the class to which the correct class belongs among the classes in the hierarchy above the hierarchy of the terminal classes.

(Appendix 4)
The classification means uses knowledge of the hierarchical structure to output the score for each hierarchical level;
The learning device according to any one of claims 1 to 3, wherein the loss calculation means calculates the hierarchical loss based on the score output for each hierarchy.

(Appendix 5)
said projection means for outputting said second feature representation for each hierarchical level based on knowledge of said hierarchical structure;
The learning device according to claim 4, wherein the classification means outputs the score for each hierarchy based on the second feature representation output for each hierarchy.

(Appendix 6)
Transforming the input data into a first feature representation using feature extraction means;
using a projection means to transform the first feature representation into a second feature representation indicative of points in hyperbolic space;
using a classification means to perform classification based on the second feature representation and output a score indicative of the likelihood that the input data belongs to each class;
Calculating a hierarchical loss based on knowledge of the hierarchical structure to which each of the classes belongs, the correct label assigned to the input data, and the score;
A learning method that updates parameters of the feature extraction means, the projection means, and the classification means based on the hierarchical loss.

(Appendix 7)
Transforming the input data into a first feature representation using feature extraction means;
using a projection means to transform the first feature representation into a second feature representation indicative of points in hyperbolic space;
using a classification means to perform classification based on the second feature representation and output a score indicative of the likelihood that the input data belongs to each class;
Calculating a hierarchical loss based on knowledge of the hierarchical structure to which each of the classes belongs, the correct label assigned to the input data, and the score;
A recording medium having a program recorded thereon that causes a computer to execute a process of updating parameters of the feature extraction means, the projection means, and the classification means based on the hierarchical loss.

(Appendix 8)
a feature extraction means for converting input data into a first feature representation;
a projection means for transforming the first feature representation into a second feature representation representing points in a hyperbolic space;
a classification means for performing classification based on the second feature representation, and calculating, for each hierarchy, a score indicating the likelihood that the input data belongs to each class, using knowledge of a hierarchical structure to which each class belongs;
An inference device comprising:

(Appendix 9)
said projection means for outputting said second feature representation for each hierarchical level based on knowledge of said hierarchical structure;
The learning device according to any one of claims 1 to 4, wherein the classification means outputs the score for each hierarchy based on the second feature representation output for each hierarchy.

(Appendix 10)
Transforming the input data into a first feature representation;
Transforming the first feature representation into a second feature representation that represents points in a hyperbolic space;
An inference method that performs classification based on the second feature representation, and calculates a score for each hierarchy indicating the likelihood that the input data belongs to each class using knowledge of the hierarchical structure to which each class belongs.

(Appendix 11)
Transforming the input data into a first feature representation;
Transforming the first feature representation into a second feature representation that represents points in a hyperbolic space;
A recording medium having a program recorded thereon that causes a computer to execute a process of performing classification based on the second feature representation and calculating, for each hierarchy, a score indicating the likelihood that the input data belongs to each class, using knowledge of the hierarchical structure to which each class belongs.

Although the present disclosure has been described above with reference to embodiments and examples, the present disclosure is not limited to the above embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

21 Feature extraction unit 22 Hyperbolic projection unit 22x Hierarchical hyperbolic projection unit 23 Hyperbolic classification unit 23x Hierarchical hyperbolic classification unit 24 Hierarchical loss calculation unit 25 Gradient calculation unit 26 Update unit 100, 100a, 100b Learning device 200, 200a, 200b Inference device

Claims (6)

a feature extraction means for converting input data into a first feature representation;
a projection means for transforming the first feature representation into a second feature representation representing points in a hyperbolic space;
a classification means for performing classification based on the second feature representation and outputting a score indicating the likelihood that the input data belongs to each class;
a loss calculation means for calculating a hierarchical loss based on knowledge of a hierarchical structure to which each of the classes belongs, a correct label assigned to the input data, and the score;
an update means for updating parameters of the feature extraction means, the projection means, and the classification means based on the hierarchical loss;
Equipped with
The classification means outputs a score for each of the terminal classes of the hierarchical structure;
The loss calculation means is a learning device that integrates the scores of the end classes to calculate the loss of a hierarchy above the hierarchy of the end class, and calculates a weighted sum of the losses of each hierarchy as the hierarchical loss . 2. The learning device according to claim 1, wherein the loss calculation means calculates a loss that maximizes the score of a correct class for the hierarchy of the terminal classes, and calculates a loss that maximizes the score of a class to which the correct class belongs in a hierarchy above the hierarchy of the terminal classes. The classification means uses knowledge of the hierarchical structure to output the score for each hierarchical level;
The learning device according to claim 1 or 2 , wherein the loss calculation means calculates the hierarchical loss based on the score output for each hierarchy. said projection means for outputting said second feature representation for each hierarchical level based on knowledge of said hierarchical structure;
The learning device according to claim 3 , wherein the classification means outputs the score for each of the hierarchical layers based on the second feature representation output for each of the hierarchical layers. Transforming the input data into a first feature representation using feature extraction means;
using a projection means to transform the first feature representation into a second feature representation indicative of points in hyperbolic space;
using a classification means to perform classification based on the second feature representation and output a score indicative of the likelihood that the input data belongs to each class;
Calculating a hierarchical loss based on knowledge of the hierarchical structure to which each class belongs, the correct label assigned to the input data, and the score;
updating parameters of the feature extraction means, the projection means and the classification means based on the hierarchical loss ;
The classification is performed by outputting a score for each of the terminal classes of the hierarchical structure;
The hierarchical loss calculation is a learning method in which the scores of the terminal classes are integrated to calculate the loss of the hierarchy above the terminal class, and the weighted sum of the losses of each hierarchy is calculated as the hierarchical loss . Transforming the input data into a first feature representation using feature extraction means;
using a projection means to transform the first feature representation into a second feature representation indicative of points in hyperbolic space;
using a classification means to perform classification based on the second feature representation and output a score indicative of the likelihood that the input data belongs to each class;
Calculating a hierarchical loss based on knowledge of the hierarchical structure to which each class belongs, the correct label assigned to the input data, and the score;
updating parameters of the feature extraction means, the projection means and the classification means based on the hierarchical loss ;
The classification is performed by outputting a score for each of the terminal classes of the hierarchical structure;
The calculation of the hierarchical loss is a program that causes a computer to execute a process of integrating the scores of the terminal classes to calculate the loss of the hierarchy above the hierarchy of the terminal class, and calculating the weighted sum of the losses of each hierarchy as the hierarchical loss .

Images