JP7655779B2 - Learning model updating device and learning model updating method - Google Patents

Description

The present invention relates to a learning model update device and a learning model update method.

The increasing capacity of various storage devices, improved computing performance, and faster transfer speeds over networks have made it possible to store large amounts of electronic historical information in a variety of business processes. In terms of information content, it is now possible to store not only traditional bibliographic information, but also a variety of media information such as video and images.

Against this background, there is an increasing need for information processing technology to utilize this electronic history information. For example, in the examination process for patents, trademarks, and designs, a variety of information such as applied documents, related images, and bibliographic information attached in relation to the examination process is accumulated as work history information. If this history information could be utilized to improve the efficiency of the examination process and the work related to applications, it would make a significant contribution to the development of the industrial sector.

Business information is often stored with any number of keywords for images. There is also a strong demand for a system that can automatically assign keywords to images acquired during business operations.

Training data in which multiple labels have been assigned to the data is called multi-label training data. When training deep neural networks that target multi-label training data, a method that uses Cross-Entropy Loss is commonly used. With this method, a network is constructed that outputs values equal to the "number of different" labels. However, when assigning keywords, it is often difficult to predict in advance the number of different keywords that will be assigned. Furthermore, when the number of different expected keywords is extremely large, problems arise in configuring the network model and in executing the process.

On the other hand, triplet network learning, which is one of the learning methods for deep neural networks, trains the neural network to output feature vectors in which the distance between data with the same label is small and the distance between data with different levels is large.

The triplet concept is illustrated in Figure 1. In triplet network learning, data at the same level as the data of interest (anchor) 111 is treated as positive examples 112, and data at a different level is treated as negative examples 113, and a triplet 110 consisting of the anchor, positive examples, and negative examples is created.

Each piece of data that makes up the triplet is input to a neural network model 120 with the same weighting, and the neural network that makes up the neural network model 120 outputs a feature vector 130 that corresponds to each piece of data.

Note that, although FIG. 1 illustrates a CNN (Convolutional Neural Network) as an example of the neural network model 120, there is no intention to limit the neural network model 120 to a CNN, and any known neural network model can be applied. Throughout this specification, known network models can be suitably applied.

Next, a distance D ₊ 140 in the feature space between the anchor 111 and the positive example 112, and a distance D _- 150 in the feature space between the anchor 111 and the negative example 113 are calculated. A loss function, which is an objective function of the learning, is defined as the sum of the differences between these two distances calculated for each triplet. In Non-Patent Document 1, this triplet network learning is applied to learning features for performing personal authentication using face images.

When a learning set is given, there are an infinite number of triplets. Non-Patent Document 2 discusses a method for selecting triplets that achieve efficient learning by applying a similar vector search process.

Schroff, Florian, Dmitry Kalenichenko, James Philbin. "Facenet: A unified embedding for face recognition and clustering", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2015) Yasumitsu Ikeura, Koichi Okamoto, Ryohei Kashima, and Atsushi Hiroike: Multimodal Deep Learning Platform for IoT Data, Hitachi Review, 102-3, 119-123 (2020)

The triplet network learning described above is an effective method for obtaining feature vectors, but it is a method that can be applied when a single label is assigned to each piece of data. On the other hand, in the case of multi-label learning data, the relationship between positive and negative examples between the anchor data and other data cannot be uniquely defined, so triplets cannot be constructed.

The present invention has been made in consideration of the above problems, and its purpose is to provide a learning model update device and a learning model update method that enable triplet network learning for multi-label learning data.

In order to solve the above problem, a learning model updating device according to one aspect of the present invention is a learning model updating device that accepts a learning model that calculates an image feature vector of image data to which multiple labels have been assigned, and updates this learning model, and is characterized by comprising: a query, which is one image data selected from multiple image data accepted as learning data for the learning model, a label similarity calculation unit that calculates label similarity between the image data selected as the query and multiple image data different from the multiple image data; a selection processing unit that selects, from the multiple image data used in calculating the label similarity, a pair of image data whose label similarity gap satisfies a predetermined condition as a pair of positive and negative examples; and a model updating processing unit that updates the learning model based on the selected pair of positive and negative examples.

The present invention provides a learning model update device and a learning model update method that enable triplet network learning for multi-label learning data.

FIG. 1 is a schematic diagram of general triplet network learning. FIG. 1 is a configuration diagram of a learning model updating device according to a first embodiment. 1 is a flow of processing of a learning model updating device according to a first embodiment. 13 is a flow of a triplet set generation process of the learning model updating device according to the first embodiment. FIG. 2 is an explanatory diagram of a triplet generation process for each search result in the learning model update device according to the first embodiment. 13 is a flow of a triplet generation process for each search result in the learning model updating device according to the first embodiment. FIG. 11 is a configuration diagram of a learning model updating device according to a second embodiment. 13 is a flow of processing of a learning model updating device according to a second embodiment. 13 is a flow of a triplet set generation process in the learning model updating device according to the second embodiment. 13 is a flow of a triplet generation process for each anchor in the learning model updating device according to the second embodiment. FIG. 11 is a configuration diagram of a search device using a network model updated according to the first or second embodiment.

The following describes an embodiment of the present invention with reference to the drawings. Note that the embodiment described below does not limit the invention as claimed, and not all of the elements and combinations thereof described in the embodiment are necessarily essential to the solution of the invention.

In addition, in the figures explaining the embodiments, parts having the same functions are given the same reference numerals, and repeated explanations are omitted.

In the following explanation, expressions such as "xxx data" may be used as an example of information, but the information may have any data structure. That is, to show that the information does not depend on the data structure, "xxx data" may be referred to as a "xxx table." Furthermore, "xxx data" may simply be referred to as "xxx." In the following explanation, the structure of each piece of information is an example, and the information may be divided and stored, or combined and stored.

In the following explanation, the process may be described with a "program" as the subject, but since a program is executed by a processor (e.g., a CPU (Central Processing Unit)) to perform a defined process using storage resources (e.g., memory) and/or communication interface devices (e.g., ports) as appropriate, the subject of the process may also be the program. Processes described with a program as the subject may also be processes performed by a processor or a computer having that processor.

The learning model update device of this embodiment has the following configuration, as an example:

In multi-label learning data, the label similarity between data is defined based on the label assigned to each piece of data. When a certain piece of data is considered as an anchor, the set of data with the largest difference (gap) in label similarity with the anchor, or the set of data sets with a difference (gap) in label similarity equal to or greater than a certain value is extracted. Among each set of data, those with high label similarity with the anchor are considered as positive example candidates, and those with low label similarity are considered as negative example candidates. From the extracted sets of positive and negative example candidates, the necessary number of pairs are selected in order of smallest distance to the anchor in the feature vector space to form a set of triplets.

By executing the above process using each data in the training data as an anchor, a universal set of triplets to be learned is constructed. A loss function is defined for the universal set of triplets constructed, and learning for extracting feature vectors is performed by minimizing this function.

This embodiment achieves efficient learning by combining the technology of this embodiment with similar vector search technology.

Figure 2 shows the configuration of this embodiment.

The learning model update device 210 of this embodiment accepts a collection of image data to which multi-label information has been added as learning data 220. In normal operation, when learning a deep neural network, it is rare for the network weights to start in a completely random state. Learning is made more efficient by providing a network model 230 that has already been trained in another learning task as the initial state of learning.

Inside the learning model update device 210, the inference processing unit 211 extracts feature vectors of the learning data using the state of the network model at that point in the learning process. The similar vector search processing unit 212 performs a similar vector search on the set of feature vectors extracted by the inference processing unit 211.

The triplet set generation processing unit 213 constructs a set of triplets to be used for learning based on the result of the similarity vector search processing by the similarity vector search processing unit 212. More specifically, the label similarity calculation unit 213a of the triplet set generation processing unit 213 calculates the label similarity between a query, which is one image data selected from the multiple image data that are the learning data 220, and multiple image data different from the image data selected as the query, and the selection processing unit 213b selects, from the multiple image data used in calculating the label similarity, a pair of image data whose label similarity gap satisfies a predetermined condition as a pair of positive and negative cases.

The model update processing unit 214 performs a network model update process using the set of learning triplets constructed by the triplet set generation processing unit 213. The state of the updated network model 240 is saved in an external storage medium by the model saving processing unit 215 as necessary.

Figure 3 shows the processing flow of the learning model update device 210.

The learning model update device 210 first acquires the learning data and the initial state of the network model (301). In the subsequent inference process (302), a feature vector of the learning data is extracted.

The next clustering process (303) is a pre-processing step to quickly execute the similar vector search performed in the triplet set generation process (304). Similar vector search using clustering is discussed in the above-mentioned non-patent document 2 and elsewhere. The subsequent triplet set generation process (304) will be described later.

From the set of triplets obtained in the triplet set generation process (304), a learning triplet to be actually used for learning is selected (305) based on certain criteria. The content of this criterion will be described later. A learning triplet is an ordered array whose elements are triplets. The network model is updated (306) using the learning triplets obtained in this way. This updated network model will be used in the next inference process (302).

In this embodiment, the process from the inference process (302) to the update of the network model (306) is considered to be one cycle of learning. In the process of determining whether or not to end learning (307), learning is usually ended when the number of cycles reaches a preset maximum number of cycles, but learning may also be ended based on other evaluation criteria.

Figure 4 shows the process flow for triplet set generation processing (304).

First, the elements of the triplet set are cleared (401). Next, one piece of data in the training data is selected as a query (402), data having a feature vector similar to that of the query is searched for (403), and search results are obtained (404). The obtained search results are used to perform triplet generation processing for each of the next search results (405). The details of this processing will be described later.

Next, the triplet set for each obtained search result is added to the triplet set of this process (406). The processes from 402 to 406 are usually performed using all learning data as a query, but there are cases where an upper limit is set on the number of pieces of data to be used as a query due to requirements such as limits on the amount of calculation. In this case, a method is used in which a certain number of pieces of data randomly selected from the learning data are used as the query.

Next, we will explain the details of the triplet generation process (405) for each search result.

ここで、ｑは、クエリとして選択されたデータ、ｘは、検索結果中の任意のデータ、Ｑはクエリｑに付与されたレベルの集合、Ｘは検索結果中のデータｘに付与されたラベルの集合である。また、式（１）左辺の分母は、ＱとＸの和集合の要素数、分子は、ＱとＸの積集合の要素数である。ｑとｘとのラベル類似度ｓ（ｑ，ｘ）は、両者のラベルが完全に一致した場合１、共通するラベルが全く存在しない場合０となる。 First, the label similarity when multiple labels are assigned is defined as follows.

Here, q is data selected as a query, x is any data in the search results, Q is a set of levels assigned to query q, and X is a set of labels assigned to data x in the search results. The denominator on the left side of formula (1) is the number of elements in the union of Q and X, and the numerator is the number of elements in the intersection of Q and X. The label similarity s(q, x) between q and x is 1 when the labels of the two completely match, and 0 when there are no common labels.

上式の値の範囲は、－１～１となる。 Next, for a data pair (x, y) consisting of two different data in the search results, the difference (gap) g(q, x, y) in label similarity with the query q is defined as follows:

The value range of the above formula is from -1 to 1.

Figure 5 shows a schematic diagram of the change in label similarity in search results.

The horizontal axis of Figure 5 is the ranking in the search results, and the vertical axis is the label similarity of each search result. If there is a high correlation between the similarity in the feature vector space and the label similarity, it is expected that the label similarity will monotonically decrease as the search ranking increases. However, in reality, a discrepancy between the two occurs, and data with a low search ranking may show high label similarity, or data with a high search ranking may show low label similarity.

Therefore, we use the concept of label similarity gap in formula (2). When focusing on a certain piece of search ranking data, we calculate the label similarity gap between that data and data with a lower ranking. According to the variable names in formula (2), the data with a lower ranking corresponds to x, and the focused data corresponds to y.

In this embodiment, for the data pairs in the search results, the pair with the largest gap in label similarity is extracted, and the data that make up each pair with the lower ranking is selected as a positive example candidate for forming a triplet, and the data with the higher ranking is selected as a negative example candidate.

When a search result is given, there are usually multiple pairs for which the gap value is maximum. The gap and its underlying label similarity are defined as real numbers between 0 and 1, but as is clear from equation (1), the label similarity is defined as a combination of not-so-large integer values, such as the number of labels assigned to each piece of data. As a result, the gap value is not distributed as a continuous value, but as a number of discrete values.

Therefore, the pair that gives the maximum value is not unique, but usually there are many pairs. Also, if there are multiple pairs that share the same negative example candidate among the pairs with the largest gap, the pair with the higher ranked positive example candidate, i.e., the pair with the smaller gap in search ranking, is selected.

From the set of pairs of positive and negative example candidates extracted using the above method, a pre-specified number of pairs are selected in order of the negative example candidate's search ranking, and a triplet set is constructed by combining each pair with the query. If the number of pairs of positive and negative example candidates is less than the specified number, a triplet set is constructed using all pairs of positive and negative example candidates. Note that if the maximum gap value is smaller than a pre-specified threshold, no triplets are constructed. In other words, in this case, the set of triplets generated is an empty set.

Figure 6 shows the process flow for generating a triplet set for each search result (405).

First, the label similarity between the query and the data in the search results is calculated (601). Next, the pair of search results with the largest gap in label similarity is extracted as a pair of positive and negative example candidates (602). Next, from the extracted pairs of positive and negative example candidates, N negative example candidates are selected in descending order of search ranking (603). Finally, N triplets are constructed by combining the query and the selected pairs of positive and negative example candidates (604).

ここで、ｑは、クエリの特徴量ベクトル、ｘは正事例の特徴量ベクトル、ｙは負事例の特徴量ベクトルである。本実施例では、クエリからの特徴量ベクトル空間上での距離が負事例よりも正事例の方が大きいものを選択してtripletを構成しているため、上式の値は、非負となる。 For each triplet generated, we define the following value (loss function for each triplet):

Here, q is the feature vector of the query, x is the feature vector of the positive case, and y is the feature vector of the negative case. In this embodiment, a triplet is constructed by selecting positive cases whose distance from the query in the feature vector space is greater than that of negative cases, so the value of the above formula is non-negative.

In the selection of learning triplets (305) in FIG. 3, the triplet set obtained by the triplet set generation process (304) is sorted in ascending order of the value of expression (3). If the number of elements in the triplet set is greater than the specified number, the lower triplets, i.e., the triplets with relatively large values of expression (3), are discarded.

ここで、ｉは学習用tripleの配列の添え字、ｑ_ｉはｉ番目のtripletのクエリの特徴量ベクトル、ｘ_ｉはｉ番目のtripletの正事例の特徴量ベクトル、ｙ_ｉはｉ番目のtripletの負事例の特徴量ベクトルである。また、ｂは、２乗距離の差の下限で、あるtripletの正事例、負事例間での２乗距離の差がｂを下回る場合、そのtripletは最適化計算の対象から除外される。 In updating the network model (306) in FIG. 3, optimization is performed so as to reduce the loss function of the following equation.

Here, i is the subscript of the array of training triples, _qi is the feature vector of the query of the i-th triplet, _xi is the feature vector of the positive cases of the i-th triplet, and _yi is the feature vector of the negative cases of the i-th triplet. Also, b is the lower limit of the difference in squared distance, and if the difference in squared distance between the positive and negative cases of a triplet falls below b, the triplet is excluded from the optimization calculation.

In this embodiment, optimization is not performed on all training triplets at once. As with most deep neural network training, the entire array of training triplets is divided into small arrays called mini-batches, and the network is updated sequentially in mini-batch units.

Therefore, even if the loss function for each triplet in equation (3) satisfies the non-negative condition in the initial state of a certain cycle, the difference in squared distance between positive and negative examples may become negative due to fluctuations in the state of the network during the update process. The value of b in equation (4) is intended to prevent learning from failing due to instability in the numerical calculations when updating the network based on an extremely negative value. From this perspective, the value of b in equation (4) is set in advance as an appropriate scalar value.

The network update process (306) is performed in the order sorted in process (305). Therefore, learning using triplets with smaller values of equation (3) is prioritized. Learning triplets with smaller loss functions per triplet first means that triplets that are expected to require smaller corrections in the feature space, that is, triplets that are expected to be easier to learn, are prioritized. This is a measure to achieve a more stable learning process.

In the triplet set generation process for each search result described in FIG. 6, the pair of positive example candidate and negative example candidate with the largest gap in label similarity is extracted, but all pairs with a gap value equal to or greater than a certain threshold may be extracted. In addition, from the extracted pairs of positive example candidate and negative example candidate, the pair with the highest ranking of the negative example candidate is selected with priority, but the pair with the smallest loss function for each triplet in Equation (3) may be selected with priority, or may be selected randomly. In addition, in the selection of learning triplets (305) in FIG. 3, the pair with the largest gap in label similarity may be selected with priority, or may be selected randomly.

In addition, other definitions may be adopted in the method of defining the label similarity in formula (1). For example, the following formula is another definition method using set operations, similar to formula (1).

Furthermore, as a representation format for multi-labeled data, there is also a method of expressing it in vector form, assuming a vector whose dimension is the total number of labels present, and representing it as 1 if a label has been assigned to a certain piece of data, and 0 if it has not been assigned. In this case, the label similarity can be defined from the inner product between vectors, or the distance between vectors.

As described above in detail, according to this embodiment, it is possible to realize a learning model updating device 210 and a learning model updating method that enable triplet network learning targeting multi-label learning data.

In this example, we will explain the learning process that does not involve similar vector search.

Figure 7 shows the configuration of this embodiment.

Many of the processing units constituting this embodiment provide the same functions as the corresponding processing units in Example 1 shown in Figure 2.

The learning model update device 710 receives a collection of image data to which multi-label information has been assigned as learning data 720. If necessary, it also receives a network model 730 that has been trained in advance on another learning task as the initial state of learning.

Inside the learning model update device 710, a triplet set generation processing unit 711 generates a set of triplets to be used for learning, and a model update processing unit 712 performs a network model update process using the generated set of learning triplets. The state of the updated network model 740 is saved in an external storage medium by a model saving processing unit 713 as necessary.

Figure 8 shows the processing flow of the learning model update device 710.

The learning model update device 710 first acquires the learning data and the initial state of the network model (801). Next, it calculates the label similarity of equation (1) between all the learning data and stores the value (802).

Next, a triplet set generation process (803) in this embodiment is performed. The details of this process will be described later. From the set of triplets obtained in the triplet set generation process (803), a learning triplet to be actually used for learning is selected (804) based on a certain criterion. The details of this criterion will be described later. A learning triplet is an ordered array whose elements are triplets. The network model is updated (805) using the learning triplets obtained in this way.

In this embodiment, the process from triplet set generation process (803) to network model update (805) is considered to be one learning cycle. In the learning end determination process (806), learning is usually terminated when the number of cycles reaches a preset maximum number of cycles, but learning may also be terminated based on other evaluation criteria.

Figure 9 shows the process flow for triplet set generation processing (803).

First, the elements of the triplet set are cleared (901). Next, one piece of data in the training data is selected as an anchor (902). Based on the data selected as the anchor, a triplet generation process (903) is performed for each anchor. The details of this process will be described later.

Next, the triplet set for each acquired anchor is added to the triplet set of this process (904). The processes from (902) to (904) are usually performed by selecting all the training data as anchors, but there are also cases where an upper limit is set on the number of data items to be used as anchors due to requirements such as restrictions on the amount of calculation. In this case, a certain number of data items randomly selected from the training data are used as anchors.

Figure 10 shows the process flow for generating triplet for each anchor (903).

First, the set S of pairs of positive and negative example candidates, which is data temporarily held during this process, is emptied (1001). Next, a data pair (x, y) is randomly selected from the training data excluding anchor a (1002). Next, the label similarity gap g(a, x, y) in equation (2) is calculated from the label similarity between a and x, and between a and y (1003). Here, the one corresponding to query q in equation (2) is anchor a in this embodiment. If the gap g(a, x, y) is equal to or greater than a predefined threshold, (x, y) is added to set S (1005).

The process from (1002) to (1005) is repeated until a certain termination condition is met (1006). In this embodiment, this repeated process is terminated when the number of elements in set S reaches a predefined number, or when the number of times the process has been executed reaches a predefined maximum number.

Next, the set S of pairs of positive example candidates and negative example candidates extracted in this way is sorted in order of the largest gap in label similarity, and the top pairs are selected (1007). There is an upper limit to the number of pairs that can be selected, and pairs exceeding this limit, i.e. pairs with relatively small gaps in label similarity, are discarded. A set of triplets is constructed from the pairs of positive example candidates and negative example candidates selected in this way (1008).

Unlike in the first embodiment, the triplets generated in the triplet set generation process (803) of this embodiment have no constraints on the feature vector space regarding the selection of positive and negative cases. Therefore, the loss function for each triplet in formula (3) may be a negative value. In the selection of learning triplets (804), the set of generated triplets is sorted in ascending order of the absolute value of the loss function for each triplet in formula (3), and a certain number of the top triplets are selected as the triplets to be ultimately used for learning. In the subsequent network model update (805), the model is updated using each triplet according to the sorted order.

Therefore, this embodiment can achieve the same effects as the above-mentioned embodiment 1.

In this embodiment, a search device 1100 is described that searches for similar image data using network models 240, 740 updated by the learning model update devices 210, 710 of embodiments 1 and 2.

Figure 11 is a configuration diagram of a search device 1100 using a network model 240, 740 updated according to Example 1 or Example 2.

When image data that is a query (search target) is input to the image input unit 1101 of the search device 1100, the feature calculation unit 1102 calculates the feature amounts of the input image data. Next, the search unit 1103 inputs the calculated feature amounts to the network models 240, 740, and obtains image data that is similar (or judged to be similar) to the query, which is the search result. Then, the search result display unit 1104 displays the image data that is the search result obtained by the search unit 1103 using a display device such as a display not shown.

The above-mentioned embodiments are detailed descriptions of the configurations in order to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. In addition, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

The above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example by designing them as integrated circuits. The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor of the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, and ROMs.

In addition, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C/C++, perl, Shell, PHP, Java (registered trademark), Python, etc.

Furthermore, all or part of the program code of the software that realizes the functions of each embodiment may be stored in advance in the storage of the machine learning system, or, if necessary, may be stored in the storage from a non-transitory storage device of another device connected to the network, or from a non-transitory storage medium via an external I/F (not shown) provided in the machine learning system.

Furthermore, the program code of the software that realizes the functions of the embodiment may be distributed over a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or storage medium.

In the above-mentioned embodiment, the control lines and information lines are shown as those considered necessary for the explanation, and not all the control lines and information lines are shown in the product. All the components may be connected to each other.
(Appendix 1)
A learning model update device that accepts a learning model that calculates an image feature vector of image data to which a plurality of labels are assigned, and updates the learning model,
a label similarity calculation unit that calculates label similarity between a query, which is one of the image data selected from the plurality of image data accepted as learning data for the learning model, and a plurality of image data different from the image data selected as the query;
a selection processing unit that selects, from among the plurality of image data used in calculating the label similarity, a pair of image data in which a gap in the label similarity satisfies a predetermined condition as a pair of a positive case and a negative case;
a model update processing unit that updates the learning model based on the selected set of positive examples and negative examples;
A learning model updating device comprising:
(Appendix 2)
The learning model updating device according to (Appendix 1), wherein the predetermined condition is that the gap is maximum.
(Appendix 3)
The learning model updating device described in (Appendix 1), wherein the specified condition is that the gap is greater than or equal to a predetermined threshold.
(Appendix 4)
a similar vector search processing unit that performs a similar vector search for each of the plurality of positive examples and the plurality of negative examples for the query based on the learning model, and calculates a ranking of similar vector search results for the query;
The learning model updating device according to (Supplementary Note 2) or (Supplementary Note 3), wherein the selection processing unit selects a set of the positive examples and the negative examples based on the predetermined condition and the ranking.
(Appendix 5)
The selection processing unit extracts a plurality of candidate pairs of the positive example and the negative example in which the gap in the label similarity satisfies a predetermined condition, identifies the negative examples that fall within a predetermined rank in descending order of the ranking, and selects a pair of the positive example and the negative example that includes the identified negative example from the plurality of extracted candidate pairs.
(Appendix 6)
The selection processing unit extracts a plurality of candidate pairs of the positive example and the negative example in which the gap in the label similarity satisfies a predetermined condition, and selects the pair of the positive example and the negative example having the smallest triplet loss function from the extracted plurality of candidate pairs.
(Appendix 7)
The learning model updating device described in (Appendix 6) is characterized in that the triplet loss function is calculated based on a squared sum error of a distance between the feature vector of the query and the feature vector of the positive example, and a distance between the feature vector of the query and the feature vector of the negative example.
(Appendix 8)
The learning model updating device described in (Appendix 1) is characterized in that the label similarity is calculated based on the number of elements in the union and the number of elements in the intersection of the set of labels assigned to the query and the set of labels assigned to multiple pieces of image data other than the image data selected as the query.
(Appendix 9)
A learning model updating method using a learning model updating device that accepts a learning model that calculates an image feature vector of image data to which a plurality of labels are assigned, and updates the learning model, comprising:
Calculating label similarity between a query, which is one image data selected from the plurality of image data accepted as learning data for the learning model, and a plurality of image data different from the image data selected as the query;
selecting, from among the plurality of image data used in the calculation of the label similarity, a pair of image data in which the gap in the label similarity satisfies a predetermined condition as a pair of a positive case and a negative case;
A learning model updating method using a learning model updating device, characterized in updating the learning model based on the selected set of positive examples and negative examples.

110...triplet 111...anchor 112...positive example 113...negative example 120...deep neural network 130...feature vector 140...distance between positive example and anchor 150...distance between negative example and anchor 210, 710...learning model update device 211...inference unit 212...similar vector search processing unit 213, 711...triplet set generation processing unit 213a...label similarity calculation unit 213b...selection processing unit 214, 712...model update processing unit 215, 713...model storage processing unit 220...learning data 230, 240, 730, 740...network model

Claims (9)

A learning model update device that accepts a learning model that calculates an image feature vector of image data to which a plurality of labels are assigned, and updates the learning model,
a label similarity calculation unit that calculates label similarity between a query, which is one of the image data selected from the plurality of image data accepted as learning data for the learning model, and a plurality of image data different from the image data selected as the query;
a selection processing unit that selects, from among the plurality of image data used in calculating the label similarity, a pair of image data in which a gap in the label similarity satisfies a predetermined condition as a pair of a positive case and a negative case;
a model update processing unit that updates the learning model based on the selected set of positive examples and negative examples;
A learning model updating device comprising: The learning model update device according to claim 1, characterized in that the predetermined condition is that the gap is maximum. The learning model update device according to claim 1, characterized in that the specified condition is that the gap is equal to or greater than a predetermined threshold. a similar vector search processing unit that searches for the image data having the image feature vector similar to the image feature vector of the query, obtains a similar vector search result for the query, and calculates a ranking in the similar vector search result for the query;
4. The learning model updating device according to claim 2, wherein the selection processing unit selects the set of the positive examples and the negative examples based on the predetermined condition and the ranking. 5. The learning model updating device according to claim 4, wherein the selection processing unit extracts a plurality of pairs of the positive example and the negative example in which the gap in the label similarity satisfies a predetermined condition as candidates, identifies a pre-specified number of the negative examples in descending order of the rank, and selects a pair of the positive example and the negative example including the identified negative example from the extracted plurality of candidate pairs. 5. The learning model updating device according to claim 4, wherein the selection processing unit extracts a plurality of pairs of the positive example and the negative example, the gap of the label similarity of which satisfies a predetermined condition, as candidates, and selects, from among the extracted plurality of candidate pairs, a candidate pair having a small triplet loss function as the pair of the positive example and the negative example, by giving priority to the candidate pair. The learning model update device according to claim 6, characterized in that the triplet loss function is calculated based on the squared sum error of the distance between the feature vector of the query and the feature vector of the positive example, and the distance between the feature vector of the query and the feature vector of the negative example. 2. The learning model updating device according to claim 1, wherein the label similarity is calculated based on the number of elements in a union and a product of a set of the labels assigned to the query and a set of the labels assigned to a plurality of image data other than the image data selected as the query. A learning model updating method using a learning model updating device that accepts a learning model that calculates an image feature vector of image data to which a plurality of labels are assigned, and updates the learning model, comprising:
Calculating label similarity between a query, which is one of the image data selected from the plurality of image data accepted as learning data for the learning model, and a plurality of image data different from the image data selected as the query;
selecting, from among the plurality of image data used in the calculation of the label similarity, a pair of image data in which the gap in the label similarity satisfies a predetermined condition as a pair of a positive case and a negative case;
A learning model updating method using a learning model updating device, characterized in updating the learning model based on the selected set of positive examples and negative examples.

Images