JP7635641B2 - Teacher data creation device, teacher data creation method, and teacher data creation program - Google Patents

Description

The present invention relates to a teacher data creation device, a teacher data creation method, and a teacher data creation program.

In recent years, there has been progress in the development of automated text classification for a variety of applications, including spam filters for emails and filters for harmful text on web pages. In particular, there has been progress in the development of algorithms for text classification using machine learning.

In relation to the above technology, the following prior art is disclosed in Patent Document 1 below: A first feature vector of reference data relating to a specific field is calculated. From the first feature vector, search conditions, such as a combination of search terms, used to collect data relating to the specific field are generated. Data relating to the specific field is collected from a document database based on the search conditions. A second feature vector of the collected data is calculated. The similarity between the second feature vector and the first feature vector is calculated. Then, from the collected data, data whose similarity falls within a predetermined range is extracted as training data to be used in machine learning.

Patent Document 2 also discloses the following prior art. A plurality of teacher images taught to one of a plurality of teaching classes are stored together with the teaching class. A set of teacher images selected from the plurality of teaching classes is set as a training image group, and a predetermined number of training image groups, which are mutually different sets, are selected. Each teacher image is classified into one of the plurality of teaching classes using a classifier constructed by learning using each training image group. Then, when the class with the highest classification rate of each class classified by each classifier using the predetermined number of training image groups in each teacher image is different from the teaching class of each teacher image, the teacher image is identified as a specific teacher image and displayed together with the classification rate of each class. In this way, the user can correct the teaching class of the specific teacher image based on the classification rate of each displayed class, thereby correcting the erroneous teaching class taught by the teacher image.

JP 2018-124617 A JP 2017-162232 A

However, the prior art described in Patent Document 1 has a problem in that it is not possible to create training data in which each sentence is consistently labeled. In addition, the prior art described in Patent Document 2 has a problem in that the accuracy of the training classes taught to the training images depends on the accuracy of the classifier. There is also a problem in that a relatively large number of training images, etc., required for training each classifier, are required in advance.

The present invention has been made to solve such problems. In other words, the objective is to provide a teacher data creation device, a teacher data creation method, and a teacher data creation program that are capable of creating teacher data with consistent labels, regardless of the amount of labeled teacher data.

The above-mentioned problems of the present invention are solved by the following means.

(1) A teacher data creation device for creating teacher data used to train a classifier that classifies data, the teacher data creation device having an output control unit that outputs second data similar to input first data and classification information assigned to the second data, a reception unit that receives the classification information to be assigned to the first data, and a teacher data creation unit that assigns the received classification information to the first data to create the teacher data.

(2) The teacher data creation device described in (1) above, wherein the output control unit further outputs the similarity of the second data to the first data together with the second data and the classification information.

(3) The teacher data creation device described in (1) or (2) above, wherein the first data and the second data are text data.

(4) The teacher data creation device described in any of (1) to (3) above, wherein the output control unit has a conversion unit that converts the first data into a feature vector, and an output unit that outputs the second data whose feature vector is similar to the feature vector into which the first data is converted, and the classification information assigned to the second data.

(5) The teacher data creation device described in (4) above, in which the conversion unit converts the first data into the feature vector using a trained model generated by unsupervised learning using the data.

(6) The teacher data creation device according to any one of (1) to (5) above, wherein the second data is data to which the classification information has been previously assigned by the teacher data creation unit, and which constitutes the teacher data together with the classification information.

(7) The teacher data creation device according to any one of (1) to (6) above, wherein the output control unit further outputs the classification information previously received by the reception unit together with guidance information for receiving the classification information in the reception unit.

(8) The teacher data creation device according to any one of (1) to (7) above, wherein the output control unit outputs the classification information previously received by the reception unit in a selectable manner, and the reception unit receives the classification information selected from the classification information selectably output by the output control unit as the classification information to be assigned to the first data.

(9) The teacher data creation device described in (8) above, in which the output control unit further outputs a selection object for enabling reception of additional classification information other than the classification information previously received by the reception unit, and an input field for inputting the additional classification information, and the reception unit receives the classification information inputted into the input field by selecting the selection object as the classification information to be assigned to the first data.

(10) A method executed by a teacher data creation device, the teacher data creation method for creating teacher data to be used for training a classifier that classifies data, the teacher data creation method having the steps of (a) outputting second data similar to input first data and classification information assigned to the second data, (b) accepting the classification information to be assigned to the first data, and (c) assigning the accepted classification information to the first data to create the teacher data.

(11) The method for creating training data described in (10) above, in which step (a) further outputs the similarity of the second data to the first data together with the second data and the classification information.

(12) The teacher data creation method described in (10) or (11) above, in which the first data and the second data are text data.

(13) A method for creating teacher data described in any of (10) to (12) above, wherein the step (a) includes a step (d) of converting the first data into a feature vector, and a step (e) of outputting the second data, the feature vector of which is similar to the feature vector into which the first data is converted, and the classification information assigned to the second data.

(14) In the step (d), the first data is converted into the feature vector by a trained model generated by unsupervised learning using the data. The training data creation method described in (13) above.

(15) The teacher data creation method according to any one of (10) to (14) above, in which the second data is data to which the classification information was previously assigned in step (c) and which, together with the classification information, constitutes the teacher data.

(16) The teaching data creation method according to any one of (10) to (15) above, wherein in step (a), the classification information previously accepted in step (b) is further output together with guidance information for accepting the classification information in step (b).

(17) The teacher data creation method according to any one of (10) to (16), in which in the step (a), the classification information previously accepted in the step (b) is output in a selectable manner, and in the step (b), the classification information selected from the classification information selectably output in the step (a) is accepted as the classification information to be assigned to the first data.

(18) The teacher data creation method described in (17) above, further comprising: in step (a), outputting a selection object for enabling the acceptance of additional classification information other than the classification information previously accepted in step (b) and an input field for inputting the additional classification information; and in step (b), accepting the classification information entered in the input field by selecting the selection object as the classification information to be assigned to the first data.

(19) A teacher data creation program for causing a computer to execute the teacher data creation method described in any one of (10) to (18) above.

The system outputs second data similar to the input first data and classification information assigned to the second data, accepts classification information to be assigned to the first data, and assigns the accepted classification information to the first data to create the teacher data. This makes it possible to create teacher data with consistent labels assigned regardless of the quantity of teacher data to which labels have been assigned.

FIG. 1 is a diagram showing the configuration of a teacher data creation system. FIG. 2 is a block diagram of a teacher data creation device. FIG. 2 is a functional block diagram of a control unit of the teacher data creation device. 13 is a diagram showing an output screen output to an operation display unit by an output control unit. FIG. FIG. 2 is a block diagram showing the functions of a control unit when a classifier is trained using training data. FIG. 2 is a block diagram showing the functions of a control unit during classification using a trained classifier trained using training data. 4 is a flowchart showing the operation of the teacher data creation system.

Below, a teacher data creation device, a teacher data creation method, and a teacher data creation program according to an embodiment of the present invention will be described with reference to the drawings. Note that in the drawings, identical elements are given the same reference numerals, and duplicate explanations will be omitted. Also, the dimensional ratios in the drawings have been exaggerated for the convenience of explanation, and may differ from the actual ratios.

Figure 1 is a diagram showing the configuration of the teacher data creation system 10. Figure 2 is a block diagram of the teacher data creation device 100. The teacher data creation device 100 may be composed of multiple devices. For example, a part or all of the memory unit 120 may be composed of a cloud server that is virtually constructed by multiple servers located on a network such as the Internet.

The teacher data creation system 10 may include a teacher data creation device 100 and an input device 200.

The input device 200 is a device for inputting data (hereinafter also referred to as "input data") to which a correct class label (hereinafter simply referred to as "label") is assigned when creating training data (hereinafter simply referred to as "training data") used in machine learning, and is installed as necessary. For simplicity of explanation, the input data will be described as text data. Training data is created by assigning a label to the input data. In other words, training data is composed of a combination of input data and a label. For convenience of explanation, in order to distinguish training data to be created from existing training data that has already been created, the existing training data will be described below as a combination of "training text data" (corresponding to the input data) and a label. The existing training data may be training data that was previously created by assigning a label accepted by the operation display unit 140 to the input data.

The training data is used to train (machine learning) a classifier that classifies text data (data). The classifier may be, for example, a neural network model. Specifically, the classification of text data by a classifier corresponds to the output of a class into which text data should be classified when text data is input to the classifier.

The input device 200 may be omitted if input data can be obtained only by the teacher data creation device 100.

The input data may be text data (hereinafter also referred to as "text data"). In addition to text data, the input data may include voice, sound, images (including images other than text images), etc. For simplicity of explanation, the following description will be given assuming that the input data is text data.

The input device 200 is a device capable of acquiring input data such as text data. The input device 200 may be configured, for example, by a device that converts text content contained in HTML (HyperText Markup Language) data into text data (text data) or extracts text data from the content. The input device 200 may be configured by a scanner that reads out text contained in a paper medium as an image, and a character conversion unit that converts the image into text data. The input device 200 may be configured by a microphone and a voice recognition device for converting voice into text data.

As shown in FIG. 2, the teacher data creation device 100 includes a control unit 110, a memory unit 120, a communication unit 130, and an operation display unit 140. These components are connected to each other via a bus 150. The teacher data creation device 100 is configured, for example, by a computer terminal.

The control unit 110 is composed of a CPU (Central Processing Unit) and memories such as RAM (Random Access Memory) and ROM (Read Only Memory), and controls each part of the teacher data creation device 100 and performs calculation processing according to a program. The functions of the control unit 110 will be described in detail later.

The storage unit 120 is composed of a hard disk drive (HDD), a solid state drive (SSD), etc., and stores various programs and data.

The communication unit 130 is an interface circuit (e.g., a LAN card) for communicating with external devices via a network.

The operation display unit 140 may be configured, for example, by a touch panel. The operation display unit 140 accepts various inputs from the user. The operation display unit 140 displays various information. As described below, the operation display unit 140 displays teacher sentence data similar to the input data and labels assigned to the teacher sentence data. The operation display unit 140 constitutes a reception unit. The labels constitute classification information.

The functions of the control unit 110 are explained below.

Figure 3 is a functional block diagram of the control unit 110. The control unit 110 functions as an acquisition unit 111, a conversion unit 112, a similarity calculation unit 113, an output control unit 114, and a teacher data creation unit 115. For ease of explanation, FIG. 3 also shows the memory unit 120 and the operation display unit 140. The output control unit 114 constitutes the output unit.

The acquisition unit 111 acquires the input data by receiving it from the input device 200. The acquisition unit 111 may also acquire the input data by receiving it from an external device via the communication unit 130. When the input data is input at the operation display unit 140, the acquisition unit 111 may acquire the input data via the operation display unit 140. When the input data is stored in the memory unit 120, the acquisition unit 111 may acquire the input data by reading it from the memory unit 120.

The conversion unit 112 converts the input data into a feature vector. The conversion unit 112 may convert the input data into a feature vector as follows. Below, an example will be described in which the input data is Japanese text data.

The conversion unit 112 performs word segmentation to separate the sentences that are input data into words (nouns, verbs, adjectives, adverbs, particles, etc.). For example, by word segmentation, the input data "I am a cat" is separated into the words "I," "is," "cat," "de," and "aru." For word segmentation, general-purpose technologies such as MeCab and Sudachi can be used.

Next, the conversion unit 112 performs distributed representation, which converts each word into a vector. Through distributed representation, each word is converted into a vector of, for example, 100 dimensions. Through distributed representation, multiple vectors corresponding to each word are calculated. For distributed representation, general-purpose technologies such as Word2Vec and fastText can be used. The conversion unit 112 can perform distributed representation using a trained model such as a neural network generated by unsupervised learning using texts that are publicly available in online interactive encyclopedias such as Wikipedia and SNS such as Twitter.

Next, the conversion unit 112 calculates one feature vector from the distributed representation of the multiple vectors. For example, the feature vector can be calculated as the average, sum, or maximum value of the multiple vectors.

The conversion unit 112 may be configured as a single module that integrates modules that perform word segmentation, distributed representation, and feature vector calculation. For example, the conversion unit 112 may be configured as a trained model generated by unsupervised learning using text data (data). The trained model may convert input data into a feature vector. The trained model may be a trained model such as a neural network generated by unsupervised learning using publicly available text.

The similarity calculation unit 113 calculates the similarity (hereinafter, simply referred to as "similarity") of the feature vector of each teacher sentence data included in the existing teacher data to the feature vector (feature vector of the input data) calculated by the conversion unit 112. As described above, the existing teacher data is a combination of teacher sentence data and a label assigned to the teacher sentence data. The similarity can be calculated, for example, as the Euclidean distance or cosine distance between the feature vector calculated by the conversion unit 112 and each teacher sentence data included in the existing teacher data. The existing teacher data can be stored, for example, in the storage unit 120 in association with the feature vector of the teacher sentence data included in the teacher data. The similarity calculation unit 113 can use the existing teacher data and the feature vector associated with the existing teacher data by reading them from the storage unit 120.

The output control unit 114 extracts, from the existing teacher data, existing teacher data whose similarity calculated by the similarity calculation unit 113 is equal to or greater than a predetermined threshold value as specific teacher data. The predetermined threshold value may be appropriately set by experiment or the like from the viewpoint of appropriately labeling the input data. The output control unit 114 may output the similarity between the input data, the specific teacher data, and the teacher sentence data included in the specific teacher data by displaying it on the operation display unit 140. Note that the output control unit 114 does not have to output the similarity and/or the input data. The output control unit 114 may output the similarity between the input data, the specific teacher data, and the teacher sentence data included in the specific teacher data by transmitting it to an external device or the like.

The output control unit 114 may further output the labels included in the existing teacher data together with guidance information (hereinafter simply referred to as "guidance information") for guiding the label to be assigned to the input data. The output control unit 114 may display the labels included in the existing teacher data selectable as labels to be assigned to the input data. In other words, the output control unit 114 may display labels previously accepted as labels to be assigned to the input data in a selectable manner on the operation display unit 140. The output control unit 114 may further output a selection object 146 (see FIG. 4) for allowing labels other than the labels included in the existing teacher data to be accepted as additional labels (additional classification information), and an input field 147 (see FIG. 4) for inputting the additional label.

Figure 4 shows the output screen 141 output to the operation display unit by the output control unit 114.

In the example of the output screen 141 shown in FIG. 4, the input data is displayed (output) in the input sentence column 142 as "Eat spaghetti for lunch." In the output screen 141, the specific teacher data (teacher sentence data and label) and the similarity are displayed in the similar sentence column 143. In the similar sentence column, the teacher sentence data included in the specific teacher data are displayed in the "sentence" row, the label included in the specific teacher data are displayed in the "label" row, and the similarity is displayed in the "similarity" row. For example, the similarity of the teacher sentence data "Eat natto and rice for breakfast" to the input sentence "Eat spaghetti for lunch" is relatively high at 90%, and it can be seen that "Eat spaghetti for lunch" and "Eat natto and rice for breakfast" are judged to be particularly similar. Note that when teacher data is generated for the first time, when existing teacher data has not been generated and specific teacher data does not exist, the specific teacher data and the similarity are not displayed.

The output screen 141 displays the labels included in the existing teacher data together with the guidance information 144. The displayed labels may be all of the labels included in the existing teacher data. In the example of FIG. 4, "meal", "excretion", "sleep", and "exercise" are displayed as the labels included in the existing teacher data. The guidance information may be displayed on the label that is assigned to the teacher sentence data with the highest similarity. The guidance information may be displayed on the label that is the most numerous among the labels included in the specific teacher data. The labels included in the existing teacher data are displayed in the label assignment column 145. The labels included in the existing teacher data are displayed together with selectable check boxes, so that they can be selected. The user can select one of the labels by checking the check box with reference to the guidance information 144. The selected label is accepted as the label to be assigned to the input data.

The output screen 141 further displays a selection object 146 for allowing labels other than those contained in the existing training data to be accepted as additional labels, and an input field 147 for inputting the additional label. The selection object 146 is composed of the word "add" and a selectable check box, and checking the check box allows the user to input an additional label into the input field 147. The additional label entered into the input field 147 is accepted as a label to be assigned to the input data (hereinafter also referred to as an "assigned label").

The output screen 141 may further display a "previous sentence" button 148 and a "next sentence" button 149. When the "previous sentence" button 148 is selected, the input data (teacher sentence data) to which a label was most recently assigned is displayed in the "input sentence" column 142. This makes it possible to change the label assigned to the input data to which a label was most recently assigned. When the "next sentence" button 149 is selected, the input data next acquired by the acquisition unit 111 is displayed in the "input sentence" column 142. Teacher sentence data similar to the input data and the labels assigned to the teacher sentence data are displayed together with the degree of similarity. In addition, labels and the like included in existing teacher data are displayed.

The explanation will be continued by referring again to FIG. 3. The teacher data creation unit 115 creates teacher data (new teacher data) by adding labels input by the user on the output screen 141 displayed on the operation display unit 140 to the input data.

We will explain how to train a classifier using the created training data, and how to classify text data using the trained classifier. For simplicity's sake, we will use an example in which the classifier 117 is a neural network model.

Figure 5 is a block diagram showing the functions of the control unit 110 when the classifier 117 learns using training data. The functions of the control unit 110 shown in Figure 5 can be realized by the control unit 110 executing a program.

The control unit 110 functions as a classifier 117 and a loss calculation unit 118. The classifier 117 is trained using teacher data. More specifically, the classifier 117 is trained using a combination of teacher text data and labels (teacher classes) assigned to the teacher text data, which constitute the teacher data. The loss calculation unit 118 calculates the difference between the classification result (class) classified (estimated) by the classifier 117 when the teacher text data is input to the classifier 117 and the label assigned to the teacher text data as the class loss. The classification result may be the likelihood for each class. The class loss may be calculated by subtracting the likelihood of the class of the label assigned to the teacher text data in the classification result by the classifier 117 from 1. The classifier 117 is trained by backpropagation so that the calculated class loss becomes small.

Figure 6 is a block diagram showing the functions of the control unit 110 during classification (estimation) by a trained classifier 117 that has been trained using training data. The functions of the control unit 110 shown in Figure 6 can be realized by the control unit 110 executing a program.

The control unit 110 can function as a trained classifier 117. The classifier 117 classifies the text data by outputting a classification result (class) when text data to be classified is input. The classification result can be the likelihood for each class. As a result, the text data is classified into the class with the highest likelihood in the classification result.

Figure 7 is a flowchart showing the operation of the teacher data creation system 10. This flowchart can be executed by the control unit 110 of the teacher data creation device 100 in accordance with a program.

The control unit 110 determines whether input data has been acquired (S101). If the control unit 110 determines that input data has not been acquired (S101: NO), it executes step S101 again.

When the control unit 110 determines that input data has been acquired (S101: YES), it converts the input data into a feature vector (S102).

The control unit 110 reads out the existing teacher data and the feature vectors of the teacher sentence data contained in the existing teacher data from the memory unit 120 (S103).

The control unit 110 calculates the feature vector of the input data and calculates the similarity of the feature vector of each teaching sentence data to the feature vector of the input data (S104).

The control unit 110 extracts existing teacher data that includes teacher sentence data whose similarity is equal to or greater than a predetermined threshold as specific teacher data (S105).

The control unit 110 outputs the specific teacher data and the similarity (S106).

The control unit 110 outputs each label included in the existing teacher data by, for example, displaying it on the operation display unit 140 together with guidance information in a selectable manner (S107). The user can assign labels that are consistent with the specific teacher data by referring to the displayed specific teacher data and similarity, as well as each label and guidance information included in the existing teacher data.

The control unit 110 determines whether an assigned label to be assigned to the input data has been obtained by, for example, selecting one of the labels contained in the existing teacher data displayed on the operation display unit 140 (S108). If the control unit 110 determines that an assigned label to be assigned to the input data has not been obtained (S108: NO), it executes step S108 again.

If the control unit 110 determines that an assigned label to be assigned to the input data has been obtained (S108: YES), it assigns the assigned label to the input data to create teacher data (new teacher data) (S109).

The embodiment provides the following advantages:

The system outputs second data similar to the input first data and classification information assigned to the second data, accepts classification information to be assigned to the first data, and assigns the accepted classification information to the first data to create the teacher data. This makes it possible to create teacher data with consistent labels assigned regardless of the amount of teacher data to which labels have been assigned. Furthermore, the more data classified as teacher data there is, the more second data similar to the first data there is, so the consistency of the teacher data can be improved. Furthermore, because second data similar to the first data is output, human error in labeling can be suppressed.

Furthermore, the similarity of the second data to the first data is output together with the second data and the classification information. This makes it possible to improve the accuracy of the training data.

Furthermore, the first data and the second data are text data. This makes it possible to create training data that is more effectively and consistently labeled.

The system further converts the first data into a feature vector, and outputs second data whose feature vector is similar to the feature vector into which the first data was converted, and classification information added to the second data. This makes it easier to create highly accurate and consistent training data.

Furthermore, the first data is converted into a feature vector using a trained model generated by unsupervised learning. This makes it easier and less costly to create consistent training data.

Furthermore, the second data is data that, together with the classification information, constitutes teacher data due to the classification information having been previously assigned to the second data by the teacher data creation unit. This makes it possible to create consistent teacher data more easily and effectively.

Furthermore, previously accepted classification information is output together with guidance information for accepting more classification information. This makes it possible to assign classification information to the input first data more easily and quickly.

Furthermore, previously accepted classification information is output in a selectable manner, and selected classification information from the selectable output classification information is accepted as classification information to be assigned to the first data. This makes it easier to assign labels and reduces human error in label assignment.

Furthermore, a selection object for enabling reception of additional classification information other than the classification information previously received, and an input field for inputting the additional classification information are further output, and the classification information inputted into the input field after the selection object is selected is accepted as the classification information to be assigned to the first data. This allows for more flexible label assignment.

The teacher data creation device, teacher data creation method, and teacher data creation program described above are intended to describe the main configurations in order to explain the features of the above-mentioned embodiments, but are not limited to the above-mentioned configurations and can be modified in various ways within the scope of the claims. Furthermore, configurations that are typically used for creating teacher data, etc. are not excluded.

For example, some steps in the above-described flowchart may be omitted and other steps may be added. Also, some of the steps may be executed simultaneously, or one step may be divided into multiple steps and executed.

The means and methods for performing various processes in the above-mentioned system can be realized by either a dedicated hardware circuit or a programmed computer. The above-mentioned programs may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the programs recorded on the computer-readable recording medium are usually transferred to and stored in a storage unit such as a hard disk. The above-mentioned programs may be provided as standalone application software, or may be incorporated as a function into the software of a device such as the teacher data creation device.

10. Teacher data creation system,
100 Teacher data creation device,
110 control unit,
111 Acquisition unit,
112 conversion unit,
113 Similarity calculation unit,
114 Output control unit,
115 Teacher data creation unit,
117 Classifier,
118 loss calculation unit,
120 memory unit,
130 Communication department,
140 Operation display unit,
200 Input device,
201 images,
202 Object of analysis.

Claims (19)

A training data creation device for creating training data used for training a classifier that classifies data, comprising:
an output control unit that outputs second data similar to the input first data and classification information assigned to the second data;
a reception unit that receives the classification information to be assigned to the first data;
a teacher data creation unit that creates the teacher data by adding the received classification information to the first data;
A teacher data creation device having the above structure. The teacher data creation device according to claim 1, wherein the output control unit further outputs the similarity of the second data to the first data together with the second data and the classification information. The teacher data creation device according to claim 1 or 2, wherein the first data and the second data are text data. The output control unit is
A conversion unit that converts the first data into a feature vector;
The teacher data creation device according to any one of claims 1 to 3, further comprising: an output unit that outputs the second data, the feature vector of which is similar to the feature vector into which the first data is converted, and the classification information assigned to the second data. The teacher data creation device according to claim 4, wherein the conversion unit converts the first data into the feature vector using a trained model generated by unsupervised learning using the data. The teacher data creation device according to any one of claims 1 to 5, wherein the second data is data to which the classification information has been previously assigned by the teacher data creation unit, and which constitutes the teacher data together with the classification information. The teacher data creation device according to any one of claims 1 to 6, wherein the output control unit further outputs the classification information previously accepted by the acceptance unit together with guidance information for accepting the classification information in the acceptance unit. The output control unit outputs the classification information previously accepted by the acceptance unit in a selectable manner;
The teacher data creation device according to any one of claims 1 to 7, wherein the reception unit receives the selected classification information from the classification information selectably output by the output control unit as the classification information to be assigned to the first data. the output control unit further outputs a selection object for enabling reception of additional classification information other than the classification information previously received by the reception unit, and an input field for inputting the additional classification information;
The teacher data creation device according to claim 8 , wherein the reception unit receives the classification information inputted into the input field when the selected object is selected as the classification information to be assigned to the first data. A method executed by a teacher data creation device,
A method for creating training data for use in training a classifier that classifies data, comprising the steps of:
(a) outputting second data similar to the input first data and classification information assigned to the second data;
(b) receiving the classification information to be assigned to the first data;
(c) adding the received classification information to the first data to create the teacher data;
A method for creating teacher data comprising the steps of: The teaching data creation method according to claim 10, wherein step (a) further outputs the similarity of the second data to the first data together with the second data and the classification information. The teacher data creation method according to claim 10 or 11, wherein the first data and the second data are text data. The step (a) comprises:
(d) converting the first data into a feature vector;
The teacher data creation method according to any one of claims 10 to 12, further comprising a step (e) of outputting the second data whose feature vector is similar to the feature vector into which the first data is converted, and the classification information assigned to the second data. The teaching data creation method according to claim 13, wherein in step (d), the first data is converted into the feature vector by a trained model generated by unsupervised learning using the data. The teacher data creation method according to any one of claims 10 to 14, wherein the second data is data to which the classification information was previously assigned in step (c) and which constitutes the teacher data together with the classification information. The teaching data creation method according to any one of claims 10 to 15, wherein in step (a), the classification information previously accepted in step (b) is further output together with guidance information for accepting the classification information in step (b). In the step (a), the classification information previously received in the step (b) is output in a selectable manner;
The teacher data creation method according to any one of claims 10 to 16, wherein the step (b) accepts the selected classification information from the classification information output in a selectable manner in the step (a) as the classification information to be assigned to the first data. In the step (a), a selection object for enabling reception of additional classification information other than the classification information previously received in the step (b) and an input field for inputting the additional classification information are further output;
18. The teacher data creation method according to claim 17, wherein in the step (b), the classification information entered in the input field when the selected object is selected is accepted as the classification information to be assigned to the first data. A teacher data creation program for causing a computer to execute the teacher data creation method according to any one of claims 10 to 18.