JP7776008B2 - Learning device, learning method, and program - Google Patents

Description

The present invention relates to a learning device, a learning method, and a program for learning a classifier through machine learning.

Machine learning, particularly so-called supervised learning, is becoming increasingly popular in a wide range of fields. In supervised learning, a training dataset is prepared in advance and a classifier is trained based on the training dataset. A training dataset is the input dataset to be analyzed, with each judgment result (correct answer) for classification, regression, identity, etc. annotated. The cost required to create an input dataset (training dataset) with annotated correct answers poses a problem when building a machine learning system.

To solve this problem, active learning and weakly supervised learning have been proposed, which involve adding training datasets through computer processing.

In active learning, an existing training dataset and classifier are used to present an input dataset from a group of input datasets without a correct answer, where the performance of the classifier improves when the correct answer is known. The correct answer is annotated to the presented input dataset and added to the training dataset.

In weakly supervised learning, a function is implemented in the system that uses the knowledge of the entity that assigns the correct answer as a rule, and the system assigns the correct answer to the input dataset according to the function. The input dataset with the correct answer is added to the training dataset.

Non-Patent Document 1 describes a technique for sequentially adding rules through active learning in weakly supervised learning. This technique enables efficient rule addition.

Osamu Saisho et al., "Human Knowledge Based Efficient Interactive Data Annotation via Active Weakly Supervised Learning", 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), 2021.

However, in the real world, rules added by subjects are likely to contain errors. In fact, it has been shown that a certain percentage of human error occurs even in training datasets constructed using normal labeling, which is much simpler than adding rules. In weakly supervised learning, when rules containing errors are added, the subject is unable to discover the errors or be prompted to correct or delete them, and they accumulate as noise, which can significantly degrade the performance of the machine learning system.

Existing technology only provides a framework for efficiently adding new rules, but does not provide a way to estimate which rules that have already been added are negatively impacting the machine learning system. This means that an environment for trial and error, which is essential when building a machine learning system, is not realized.

The present invention aims to provide a learning device, learning method, and program that estimates rules that have a negative impact on a machine learning system in weakly supervised learning and presents them to a subject.

In order to solve the above problem, according to one aspect of the present invention, a learning device is provided with a group of labeling functions that assign labels to an input dataset and a classifier that performs learning based on the labeling functions, and presents labeling functions that should be modified or deleted from the group of labeling functions based on the output of the group of labeling functions and the output of the classifier.

In order to solve the above problem, according to another aspect of the present invention, a learning device includes: a labeling function processing unit that uses a labeling function group including one or more labeling functions to calculate a voting result for each labeling function for input data included in an input dataset; a second probability distribution calculation unit that uses the input data as input to a trained classifier and calculates an output probability distribution based on the trained classifier; a probability distribution calculation unit that uses the voting result to calculate the output probability distribution of the labeling function; an evaluation unit that uses the output probability distribution based on the trained classifier and the output probability distribution of the labeling function to calculate a score _s (p) that indicates the need for correction or deletion in the input data and calculates a score _s (i) that indicates the need for correction or deletion in the labeling function from the score s(p); a presentation information generation unit that generates information for presenting to a subject the labeling function whose score s(i) indicates the highest need for correction or deletion; and a labeling function correction unit that receives input from the subject to which the labeling function has been presented, and deletes or corrects the labeling function according to the input content.

The present invention has the effect of being able to estimate rules that are having a negative impact on a machine learning system in weakly supervised learning and present them to the subject.

FIG. 1 is a functional block diagram of a learning device according to a first embodiment. FIG. 4 is a diagram showing an example of a processing flow of the learning device according to the first embodiment. FIG. 4 is a diagram showing an example of a processing flow of the learning device according to the first embodiment. FIG. 1 is a diagram showing an example of the configuration of a computer to which the present technique is applied.

The following describes an embodiment of the present invention. In the drawings used in the following description, components with the same functions and steps that perform the same processing are denoted by the same reference numerals, and duplicate explanations will be omitted. In the following description, processing performed on each element of a vector or matrix is assumed to apply to all elements of that vector or matrix, unless otherwise specified.

<Features of the first embodiment>
This embodiment has the following features.

Feature (1): In real-world environments where accurate supervised information is unavailable, a machine learning system using weakly supervised learning can automatically identify and extract implemented rules that are likely to contain errors.

Feature (2): Feature (1) above enables active learning in weakly supervised learning to be bidirectional, with addition, modification, and deletion, enabling the subject and machine learning model to work together to improve the performance of the machine learning model through trial and error. The subject can be, for example, a human, AI, or machine, and is the entity that performs the learning of the machine learning model.

<Active Learning in Weakly Supervised Learning of the First Embodiment>
In active learning under weak supervision, the following three types of output probability distributions are obtained during learning:

Probability distribution based on the raw output of the labeling function before learning; Output probability distribution after weighting the labeling function after learning; Output probability distribution based on the classifier after learning. Note that in this embodiment, because the labeling function is repeatedly modified or deleted and the machine learning model is repeatedly learned, the machine learning model learned in the previous iteration of a certain iteration is referred to as "post-learning." If there is no previous iteration, that is, in the first iteration, a machine learning model trained using another learning method (e.g., the learning method of Non-Patent Document 1) may be used, or a machine learning model with random parameters may be used. A model that has completed the final learning is referred to as "trained." The "probability distribution based on the raw output of the labeling function before learning" and the "output probability distribution after weighting the labeling function after learning" are collectively referred to as the "output probability distribution of the labeling function."
A labeling function is a function that uses rules based on the subject's knowledge and assigns labels to input data. In an ideal environment where the implemented labeling function is free of errors, inconsistencies, and overlaps, the three probability distributions mentioned above will be roughly consistent during model training. However, it is not realistic to continually create such rules, and the probability of errors differing will vary depending on the subject and the difficulty of the task. Therefore, of the three output probability distributions mentioned above, two are selected: "a probability distribution based on the raw output of the labeling function before training" or "an output probability distribution after weighting the labeling function after training," and "an output probability distribution based on the learning device after training." Evaluation is performed using a measure of the difference between these two probability distributions.

When adding data using Bayesian active learning in conventional supervised learning, the BALD (Bayesian Active Learning by Disagreement) score for each data is calculated based on the mutual information between the parameters of the discriminative model and the output probability distribution, and data is labeled in order of the highest score.

In this embodiment, when extracting labeling functions to be modified or deleted, a score indicating the necessity is first calculated for each input data. Specifically, the measure representing the difference is taken as the score for each input data. Furthermore, this score for each input data is used to calculate the score for each labeling function to be evaluated.

Based on the score calculated for each labeling function, the labeling function that should be most modified or deleted is estimated, extracted, and presented to the subject.

After the subject makes a decision (delete, modify, or do nothing) and completes the work, the machine learning model is trained again using the updated set of labeling functions, just as after adding labeling functions through active learning (for example, the learning method in non-patent document 1).

<Learning device 100 according to the first embodiment>
FIG. 1 is a functional block diagram of a learning device 100 according to the first embodiment, and FIGS. 2 and 3 show the processing flow thereof.

The learning device 100 includes a labeling function processing unit 110, a first probability distribution calculation unit 120, an identification unit 140, an evaluation unit 150, a presentation information generation unit 160, a labeling function correction unit 170, and a parameter update unit 180.

The learning device 100 takes an input dataset D = (d(1), d(2), ..., d(M)) as input, performs active learning in weakly supervised learning, presents a labeling function to be modified or deleted to a subject, accepts the modification or deletion of the labeling function from the subject, and modifies the labeling function. The input dataset D includes M pieces of input data d(m), where m = 1, 2, ..., M.

The learning device 100 receives an input dataset _Dq = ( _dq (1), _dq (2), ..., _dq (Q)) as input, and again performs learning of the machine learning model using a group of labeling functions after modifying or deleting the labeling functions. The input dataset _Dq includes Q pieces of input data d(q), where q = 1, 2, ..., Q. The input datasets D and _Dq may be the same dataset or different datasets. Neither of them is an input dataset with a correct answer assigned.

The labeling function is modified or deleted and the machine learning model is trained repeatedly, and the classifier that meets specified conditions is output as the trained classifier.

The learning device 100 is a special device configured by loading a special program into a publicly available or dedicated computer having, for example, a central processing unit (CPU) and a main memory (RAM). The learning device 100 executes each process under the control of the central processing unit. Data input to the learning device 100 and data obtained from each process are stored in the main memory, and the data stored in the main memory is read by the central processing unit as needed and used for other processes. At least a portion of each processing unit in the learning device 100 may be configured with hardware such as an integrated circuit. Each memory unit in the learning device 100 may be configured with, for example, a main memory such as RAM (Random Access Memory), or middleware such as a relational database or key-value store. However, each memory unit does not necessarily have to be internal to the learning device 100; it may be configured with an auxiliary memory device consisting of a hard disk, optical disk, or semiconductor memory element such as flash memory, and be configured external to the learning device 100.

Each part is explained below.

<Labeling function processing unit 110>
The labeling function processing unit 110 includes a labeling function group. The labeling function group includes L labeling functions, where L is any integer equal to or greater than 1. A labeling function is a function that assigns a label to input data according to a rule. Assigning a label is called a vote, and not being able to assign a label is called an abstention. The output of a labeling function is also called a voting result, and the voting result also includes a value for the case of abstention.

The labeling function processing unit 110 receives the input data set D = (d(1), d(2), ..., d(M)), calculates the voting result v(m) = (v(m,1), v(m,2), ..., v(m,L)) for each labeling function for each input data d(m) (S110), and outputs v(1), v(2), ..., v(M). The voting result of the i-th labeling function LF(i) for the input data d(m) is defined as v(m,i), where i = 1, 2, ..., L.

<First Probability Distribution Calculation Unit 120>
The first probability distribution calculation unit 120 receives M voting results v(m) as input, and calculates (S120) and outputs a probability distribution p(y _b ) based on the raw output (voting results) of the labeling function before learning. The probability distribution corresponding to the voting result v(m) is defined as p(m,y _b ), where p(y _b ) = (p(1,y _b ),p(2,y _b ), ...,p(M,y _b )). When N types of labels are assigned, the probability distribution p(m,y _b ) = (p(m,y _b ,1),p(m,y _b ,2), ...,p(m,y _b ,N)). The probability distribution p(m,y _b ) based on the raw output of the labeling function is obtained by obtaining the number of votes for each label from the voting result v(m) for the input data d(m), and dividing the number of votes for each label by the total number of votes obtained. For example, if input data d(m) is assigned a label of "0", "1", or "2", abstention is assigned "-1", L=5, and the voting result v(m) is (0,0,1,-1,2), then from the four voting results excluding abstentions, the number of votes for label "0", 2, is divided by the total number of votes received, 4, to determine the probability of label "0" as 0.5. Similarly, the probability of "1" is set to 0.25 and the probability of "2" is set to 0.25, and the probability distribution p(m, y _b )=(0.5,0.25,0.25) is calculated.

<Identification unit 140>
The classifier 140 includes a trained classifier, and receives M voting results v(m) and an input data set D=(d(1), d(2), . . . , d(M)).

The identification unit 140 includes a second probability distribution calculation unit 145 and a third probability distribution calculation unit 147. The identification unit 140 corresponds to a neural network model including a soft-attention structure, with the classifier of this neural network model corresponding to the second probability distribution calculation unit 145 and the attention part corresponding to the third probability distribution calculation unit 147. In this example, the neural network model including the soft-attention structure is a model equipped with two linear transformation layers in the classifier and two linear transformation layers in the attention part as well. The neural network model including the soft-attention structure performs training based on the labeling functions of the labeling function processing unit 110, and only input data voted for by any of the labeling functions is used for training as weakly labeled data. An input dataset voted for by any of the labeling functions is denoted as _Dk = ( _dk (1), _dk (2),..., _dk (P)). P is the number of pieces of input data voted for by any of the labeling functions.

The classification unit 140 uses the M voting results v(m) to extract input data that has been voted for by any of the labeling functions, and uses the extracted input data set _D = ( _d (1), _d (2),..., _d (P)) as weakly labeled data for learning. In other words, it filters out input data for which all voting results v(m) are abstentions from the input data set D = (d(1),d(2),...,d(M)).

The second probability distribution calculation unit 145 uses the input data set D _k =(d _k (1), d _k (2), ..., d _k (P)) as the input of the trained classifier, calculates the output probability distribution p(y _c ) (the output probability distribution p(y _c ) based on the trained classifier) (S145), and outputs it.
The third probability distribution calculation unit 147 takes M voting results v(m) as input, and uses the voting results v(m) that have been voted for from any of the labeling functions to calculate and output the output probability distribution p(y _a ) after weighting the learned labeling function using an attention structure (S147), and outputs it.

<Evaluation Unit 150>
The evaluation unit 150 receives the output probability distribution p(y _a ), the probability distribution p(y _b ), the output probability distribution p(y _c ), and M voting results v(m) as input, and calculates a score s _k (p) that indicates the need for correction or deletion for each piece of input data d _k (p) (S150-1). A measure of the difference between two probability distributions is used as the score s _k (p) that indicates the need for correction or deletion. For example, measures that can be used to measure the difference include KL divergence, JS divergence, mutual information, and conditional entropy. One of the two probability distributions is an output probability distribution (p(y _a ) or p(y _b )) based on a labeling function group, and the other is an output probability distribution p(y _c ) based on a trained classifier. For example, the KL divergence between the output probability distribution p(y _c ) and the probability distribution p(y _b ) is used as the score.

Next, the evaluation unit 150 calculates a score s(i) indicating the necessity of correction or deletion for each labeling function being evaluated from the score indicating the necessity of correction or deletion for each piece of input data d _k (p) (S150-2), and outputs the score. Here, p = 1, 2, ..., P. This score s(i) uses the statistical value of the score s _k (p) for input data to which the labeling function assigned a label without abstaining. The evaluation unit 150 determines whether the input data was assigned a label by the labeling function without abstaining based on the M voting results v(m). For example, the mean, median, maximum value, minimum value, sum, variance, standard deviation, coefficient of variation, etc. can be used as the statistical quantity. For example, the sum is used as the score indicating the necessity of correction or deletion for each labeling function.

In this embodiment, the greater the difference between the two probability distributions, the more likely it is that the labeling function should be corrected or deleted. Therefore, the need for correction or deletion of the labeling function is determined based on the magnitude of the difference between the two probability distributions for the input data. Specifically, for input data to which the labeling function has assigned a label without rejection, in other words, for input data to which the labeling function contributes to the difference between the two probability distributions, the statistical value of the score s _k (p) indicating the need for correction or deletion in the input data d _k (p) is set as the score s(i) indicating the need for correction or deletion in the labeling function. A high need for correction or deletion in the labeling function indicates a high possibility that the labeling function contains an error.

<Presentation Information Generator 160, Presentation Unit 165, and Input Unit 166>
The presentation information generation unit 160 receives as input a score s(i) indicating the need for modification or deletion of each labeling function, generates information (presentation information) for presenting to the subject the labeling function whose score s(i) indicates the highest need for modification or deletion (S160), and outputs the information to the presentation unit 165. The labeling function whose score s(i) indicates the highest need for modification or deletion can be said to be the labeling function that should be modified or deleted from the group of labeling functions. For example, if a measure (e.g., KL divergence) is used as a measure of difference, the greater the difference between two probability distributions, the larger the value of the measure. Therefore, the greater the difference between the two probability distributions, the larger the statistical value. Therefore, the labeling function with the largest score s(i) is presented to the subject. On the other hand, if a measure (e.g., KL divergence) is used as a measure of difference, the greater the difference between the two probability distributions, the smaller the statistical value. Therefore, the labeling function with the smallest score s(i) is presented to the subject.

The presentation unit 165 is a device, such as a display or touch panel, for presenting presentation information to the subject. Various presentation methods are possible. Examples of presentation methods are described below, and the presentation information is information for realizing each presentation method via the presentation unit 165.

(Presentation method 1)
For example, an editing screen based on the work screen when the subject created the labeling function that most likely needs to be modified or deleted is presented to the subject. With this configuration, the subject is asked whether or not the labeling function needs to be modified or deleted. The subject may modify, delete, or do nothing with the presented labeling function. The subject modifies or deletes the labeling function from the editing screen presented on the presentation unit 165 via the input unit 166. With this configuration, modifications can be made using the same programming language and method as when the labeling function was created. For example, a button for saving the modifications and a button for deleting or disabling the labeling function itself may be added to the editing screen.

The input unit 166 is a device for accepting input by operating a main body such as a keyboard, mouse, or touch panel.

(Presentation method 2)
In conjunction with the labeling function editing screen of presentation method 1, one or more pieces of input data with the highest scores for each input data are displayed among the input data voted for by the target labeling function. This input data can be said to be the input data that serves as the basis for extracting labeling functions that should be modified or deleted from the group of labeling functions. By displaying data with a high probability of incorrect label voting together with the labeling function, it is expected that the intellectual work cost for modifying and deleting labeling functions can be reduced.

Furthermore, when displaying multiple input data together, it is possible to simply select the data with the highest scores in descending order, or to select data by semantic grouping by combining techniques such as clustering. The latter method can efficiently encourage the subject to modify the labeling function while ensuring semantic grouping. This is a method that has been shown to be effective in reducing the cost of human intellectual work in batch-based active learning in weakly supervised learning.

(Presentation method 3)
Regarding the display of input data in Presentation Method 2, data is displayed that visualizes which parts of the input data should be focused on. This is expected to further reduce the cost of intellectual work. For example, by combining explainable AI technology such as attention, it is possible to present which parts of each input data have been determined to be incorrect, allowing the subject to make corrections and deletions more efficiently. For example, it is possible to present which parts have been determined to be incorrect by highlighting them, etc.

<Labeling Function Correction Unit 170>
The labeling function correction unit 170 receives input from the subject via the input unit 166, and deletes or corrects the labeling function in accordance with the input content (S170).

<Parameter Update Unit 180>
After the subject has completed the decision-making and task, the machine learning model is trained again using the updated labeling function set, just as after adding the labeling function through active learning (see Figure 3).

The learning device 100 receives an input data set _Dq = ( _dq (1), _dq (2), ..., _dq (Q)), performs steps S110, S145, and S147, and obtains an output probability distribution p(yq _,c ) based on the trained classifier corresponding to the input data set _Dq and an output probability distribution p(yq _,a ) after weighting the trained labeling function.

The parameter update unit 180 updates the parameters of the neural network model including the soft-attention structure of the identification unit 140 so that the difference between the output probability distributions p(y _q,c ) and p(y _q,a ) becomes smaller (S180).

The learning device 100 repeats S145, S147, and S180 until a predetermined condition is met (until the conditional branch of S181 is YES), and when the predetermined condition is met (when the conditional branch of S181 is YES), the neural network model including a soft-attention structure with parameters is taken as the machine learning model after training. The predetermined condition is a condition for determining whether the learning of the neural network model has converged, and could be, for example, the number of repetitions exceeding a predetermined number, or the difference between parameters before and after the update being equal to or less than a predetermined value.

Furthermore, the modification or deletion of the labeling functions and the training of the machine learning model are repeated until a predetermined condition is met (until the conditional branch of S182 becomes YES), and the neural network model including a soft-attention structure with parameters when the predetermined condition is met (when the conditional branch of S182 becomes YES) is defined as the trained machine learning model. The predetermined condition is a condition for determining whether the group of labeling functions includes a labeling function that is having a negative impact on the machine learning system, and could be, for example, the number of repetitions exceeding a predetermined number, or the score s(i) corresponding to the labeling function that indicates the greatest need for modification or deletion being equal to or less than a predetermined value. Note that the conditional branch of S182 may not be provided and the subject may determine whether the group of labeling functions is appropriate and terminate the processing of the learning device 100.

<Effects>
With the above configuration, rules that are adversely affecting a machine learning system in weakly supervised learning can be estimated and presented to the subject. By combining this with conventional technology, rules can be freely and efficiently added, modified, and deleted in weakly supervised learning, enabling trial and error when building a machine learning model. In a user study involving multiple document classification tasks, the method of this embodiment was confirmed to be able to identify the rule that needed the most modification with a probability of 60% or more, and the top three with a probability of 80% or more.

<Other Modifications>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capabilities of the devices that execute the processes or as needed. Other modifications are possible within the scope of the present invention.

<Program and recording medium>
The various processes described above can be implemented by loading a program that executes each step of the above method into the recording unit 2020 of the computer 2000 shown in Figure 4, and operating the control unit 2010, input unit 2030, output unit 2040, display unit 2050, etc.

The program describing this processing content can be recorded on a computer-readable recording medium. Examples of computer-readable recording media include magnetic recording devices, optical disks, magneto-optical recording media, and semiconductor memory.

This program may be distributed, for example, by selling, transferring, or lending portable recording media such as DVDs or CD-ROMs on which the program is recorded. Furthermore, this program may be distributed by storing it in a storage device of a server computer and transferring it from the server computer to other computers via a network.

A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or transferred from a server computer in its own storage device. Then, when executing a process, the computer reads the program stored on its own recording medium and executes the process in accordance with the read program. Alternatively, the computer may read the program directly from a portable recording medium and execute the process in accordance with the program. Furthermore, each time a program is transferred from a server computer to the computer, the computer may execute the process in accordance with the received program. Alternatively, the server computer may not transfer the program to the computer, but may instead execute the process through a so-called ASP (Application Service Provider) service, which realizes the processing function simply by issuing execution instructions and obtaining the results. In this embodiment, the program includes information used for processing by a computer that is equivalent to a program (such as data that does not directly instruct the computer but has properties that define computer processing).

In addition, in this form, the device is configured by executing a specified program on a computer, but at least some of the processing content may also be realized in hardware.

Claims (5)

The system is provided with a group of labeling functions that assign labels to input data sets and a classifier that performs learning based on the labeling functions, and presents labeling functions to be corrected or deleted from the group of labeling functions based on the output of the group of labeling functions and the output of the classifier ;
The labeling function to be presented is determined by a measure that measures the difference between an output probability distribution based on a group of labeling functions before learning and an output probability distribution based on a classifier after learning.
Learning device. 2. The learning device of claim 1,
The labeling function to be modified or deleted is presented together with the input data that supports it.
Learning device. a labeling function processing unit that uses a labeling function group including one or more labeling functions to obtain a voting result for each labeling function for input data included in the input dataset;
a second probability distribution calculation unit that uses the input data as an input to a trained classifier and calculates an output probability distribution based on the trained classifier;
a probability distribution calculation unit that calculates an output probability distribution of a labeling function using the voting result;
an evaluation unit that calculates a score s _k (p) representing the need for correction or deletion in the input data using an output probability distribution based on the trained classifier and an output probability distribution of the labeling function, and calculates a score s(i) representing the need for correction or deletion in the labeling function from the score s _k (p);
a presentation information generation unit that generates information for presenting to a subject a labeling function whose score s(i) indicates the highest need for modification or deletion;
a labeling function correction unit that receives input from the subject to whom the labeling function is presented and deletes or corrects the labeling function according to the input content,
Learning device. Using a learning device including a group of labeling functions that assign labels to input data sets and a classifier that performs learning based on the labeling functions, a labeling function to be corrected or deleted from the group of labeling functions is presented based on the output of the group of labeling functions and the output of the classifier;
The labeling function to be presented is determined by a measure that measures the difference between an output probability distribution based on a group of labeling functions before learning and an output probability distribution based on a classifier after learning.
How to learn. A program for causing a computer to function as the learning device according to any one of claims 1 to 3 .