JP7637994B2 - PROGRAM, INFORMATION PROCESSING METHOD, INFORMATION PROCESSING DEVICE, AND METHOD FOR GENERATING LEARNING MODEL - Google Patents

Description

The present invention relates to a program, an information processing method, an information processing device, and a method for generating a learning model.

Creating exam questions for entrance exams, qualification exams, and other such tests requires knowledge and effort. As a result, technology has been proposed to assist in the creation of exam questions.

For example, Patent Document 1 discloses an automatic question creation device that can automatically create complex grammar fill-in-the-blank questions without much effort.

JP 2008-129449 A

However, the technology described in Patent Document 1 focuses on the automatic creation of fill-in-the-blank questions, and does not consider the generation of questions in formats other than fill-in-the-blank questions. Since there are a wide variety of formats for test questions, there is a need for technology that supports the creation of a wide variety of questions.

The purpose of this disclosure is to provide a program etc. that can improve the efficiency of creation work related to a variety of problems.

A program according to one aspect of the present disclosure causes a computer to execute a process of accepting conditions for creating test questions from a user, generating questions and answers based on the accepted conditions, and outputting the generated questions and answers to the user.

This disclosure can improve the efficiency of creation work related to a variety of problems.

FIG. 1 is a schematic diagram of an information processing system. 1 is a block diagram showing an example of the configuration of an information processing device; FIG. 2 is a block diagram showing an example of the configuration of a user terminal. FIG. 2 is an explanatory diagram showing an overview of a first example of a learning model. FIG. 11 is an explanatory diagram showing an overview of a second example of a learning model. FIG. 13 is an explanatory diagram showing an overview of a third example of a learning model. FIG. 13 is an explanatory diagram showing an overview of a fourth example of a learning model. FIG. 2 is an explanatory diagram showing an example of a test question. 13 is a flowchart illustrating an example of a learning model generation processing procedure. 11 is a flowchart showing an example of a procedure for generating test questions executed by the information processing system. FIG. 13 is a schematic diagram showing an example of a reception screen. FIG. 13 is a schematic diagram illustrating an example of a template. 13 is a flowchart showing an example of a detailed procedure for generating test questions. 13 is a flowchart showing another example of a detailed procedure for generating test questions. 13 is a flowchart showing another example of a detailed procedure for generating test questions. 13 is a flowchart showing another example of a detailed procedure for generating test questions. FIG. 13 is a schematic diagram showing an example of a confirmation screen. 13 is a flowchart illustrating an example of a processing procedure for a review request executed by the information processing apparatus according to the second embodiment. 13 is a flowchart illustrating an example of a procedure for a correction receiving process executed by the information processing apparatus. FIG. 13 is a schematic diagram showing an example of a screen including a correction history. 13 is a flowchart illustrating an example of a procedure of a generation process of analysis information executed by the information processing apparatus according to the third embodiment. FIG. 13 is a schematic diagram showing an example of a screen including analysis information.

This disclosure will be specifically described with reference to drawings showing embodiments thereof.

First Embodiment
1 is a schematic diagram of an information processing system 100. The information processing system 100 includes an information processing device 1 and a user terminal 2. The information processing device 1 and the user terminal 2 are communicatively connected via a network N such as the Internet. The number of user terminals 2 may be two or more.

The information processing device 1 is a device capable of various information processing and sending and receiving information, such as a server computer, a personal computer, a quantum computer, etc. The user terminal 2 is an information terminal device such as a personal computer, a smartphone, a tablet terminal, etc.

The information processing device 1 generates test questions in response to a question creation request received from the user terminal 2, and provides the generated test questions to the user terminal 2. The information processing device 1 realizes a question creation management service to assist users in creating test questions. A user who enjoys the question creation service can use the user terminal 2 to send and receive information to and from the information processing device 1, and obtain information provided by the information processing device 1.

In this specification, test questions include questions for various types of tests, such as entrance exams and qualification exams, and study questions for various types of tests. Test questions include a question statement that indicates the content of the question, and an answer that indicates the correct answer or model answer to the question statement. For example, when the answer format is multiple choice, the question statement may include a question body that indicates the main text of the question, and options. Test questions may further include an explanation for the answer.

FIG. 2 is a block diagram showing an example of the configuration of the information processing device 1. The information processing device 1 includes a control unit 11, a storage unit 12, and a communication unit 13. The information processing device 1 may be configured to perform distributed processing using multiple computers, may be realized by multiple virtual machines provided in a single server, or may be realized using a cloud server.

The control unit 11 includes a processor using one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), etc. The control unit 11 uses built-in memory such as ROMs (Read Only Memory) or RAMs (Random Access Memory), clocks, counters, etc. to control each component and execute processing.

The storage unit 12 includes a non-volatile memory such as a hard disk, a flash memory, or an SSD (Solid State Drive). The storage unit 12 may be an external storage device connected to the information processing device 1. The storage unit 12 stores various programs and data referenced by the control unit 11. The storage unit 12 of this embodiment stores a program 1P for causing a computer to execute a process related to the generation of test questions, and a learning model 121 required for executing this program 1P. The learning model 121 is a machine learning model that has learned predetermined training data. The learning model 121 is expected to be used as a program module that constitutes part of artificial intelligence software. The data stored in the storage unit 12 may include, for example, generated test questions, existing test questions, templates described later, keyword information, advice tables, etc.

A program (program product) including program 1P may be provided by a non-transitory recording medium 1A on which the program is recorded in a readable manner. The storage unit 12 stores the program read from the recording medium 1A by a reading device (not shown). The recording medium 1A is, for example, a magnetic disk, an optical disk, or a semiconductor memory. The program may also be downloaded from an external server connected to a communications network and stored in the storage unit 12. Program 1P may be a single computer program or may be composed of multiple computer programs, and may be executed on a single computer or multiple computers interconnected by a communications network.

The communication unit 13 includes a communication module for communicating with an external device via the network N. The control unit 11 transmits and receives data to and from the user terminal 2 via the communication unit 13.

The configuration of the information processing device 1 is not limited to the above example, and may include, for example, an operation unit for accepting user operations, a display unit for displaying various information, etc.

Figure 3 is a block diagram showing an example configuration of the user terminal 2. The user terminal 2 includes a control unit 21, a storage unit 22, a communication unit 23, a display unit 24, and an operation unit 25.

The control unit 21 includes a processor using one or more CPUs, GPUs, etc. The control unit 21 uses built-in memory such as ROM or RAM, a clock, a counter, etc. to control each component and execute processing. The storage unit 22 includes non-volatile memory such as a hard disk, flash memory, SSD, etc. The storage unit 22 stores various programs and data referenced by the control unit 21. The communication unit 23 includes a communication module for performing processing related to communication. The control unit 21 transmits and receives data to and from the information processing device 1 via the communication unit 23.

The display unit 24 includes a display device such as a liquid crystal display or an organic electroluminescence (EL) display. The display unit 24 displays various information in accordance with instructions from the control unit 21. The operation unit 25 is an interface that accepts user operations. The operation unit 25 includes, for example, a keyboard, a touch panel device with a built-in display, a speaker, and a microphone. The operation unit 25 accepts operation input from the user and sends a control signal according to the operation content to the control unit 21.

The information processing device 1 accepts test question creation conditions from the user via the user terminal 2, and automatically generates test questions according to the creation conditions using the learning model 121. As described above, the test questions include at least a question statement and an answer.

Creation conditions include, for example, question creation conditions and answer creation conditions. Question creation conditions include information about the question (question) format, and are classified into multiple conditions, for example, fill-in-the-blanks and prompts. Answer creation conditions include information about the answer format, and are classified into multiple conditions, for example, multiple choice and written. Users can select creation conditions according to their purpose in creating questions from among the pre-set question creation conditions and answer creation conditions.

Figure 4 is an explanatory diagram showing an overview of a first example of the learning model 121. The learning model 121 shown in Figure 4 takes question text and answers related to existing test questions (hereinafter also simply referred to as existing questions) as input, and outputs question text and answers related to new test questions (hereinafter also simply referred to as new questions). The learning model 121 is, for example, a natural language processing model using a neural network. Note that a new test question means a new test question whose content is different from at least the existing questions that are input elements, and does not need to be new content for all existing questions.

In this embodiment, the learning model 121 generates questions and answers by using natural language processing technology using, for example, a Transformer model. The learning model 121 may be a pre-training model using a Transformer, such as BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer)-3, GPT-2, etc.

The learning model 121 has an input layer that accepts input of existing questions, an intermediate layer (hidden layer) that extracts features of the existing questions, and an output layer that outputs new questions. For example, the existing questions are input to the input layer as tokens generated by analyzing the text of a sentence including the question and the answer using morphological analysis or the like. The intermediate layer has multiple nodes that extract features of each input value, and passes the features extracted using various parameters to the output layer. The output layer is configured to output new questions based on the features output from the intermediate layer. In the Transformer model, the intermediate layer is provided with an Attention layer, and the position of a word in the entire sentence can be extracted. Note that natural language processing models such as the Transformer and BERT are well-known technologies, so further detailed explanations are omitted.

The learning model 121 stored in the memory unit 12 of the information processing device 1 is a trained model that has undergone machine learning processing in advance. The learning processing is a process of optimizing various parameters in the neural network using a large amount of training data given in advance. The parameters in question are, for example, weights (coupling coefficients) between neurons.

The information processing device 1 may fine-tune the pre-trained learning model 121 as a model for generating question statements and answers. The information processing device 1 may, for example, collect a large amount of existing questions and new questions to generate training data for fine-tuning. The training data for fine-tuning is, for example, a data set that associates question statements and answers related to existing questions with question statements and answers related to new questions. The training data may be a data set that associates question statements and answers related to a first existing question with question statements and answers related to a second existing question that is different from the first existing question.

The information processing device 1 inputs sentences linking existing questions and new questions contained in the training data into the input layer of the pre-learning neural network model, and obtains new questions output from the output layer after arithmetic processing in the intermediate layer. The information processing device 1 compares the new questions output from the output layer with the new questions contained in the training data, and optimizes various parameters so that the new questions output from the output layer approach the correct answer. Through the above-mentioned processing, a learning model 121 can be constructed that has been trained to be able to appropriately output new questions in response to existing questions.

The configuration of the learning model 121 is not limited to the above example. The learning model 121 only needs to be capable of generating new questions that are different from existing questions. The learning model 121 may use other learning algorithms, such as a recurrent neural network (RNN) or a long short-term memory (LSTM).

The information processing device 1 prestores in the storage unit 12 question group data in which creation conditions are associated with a large collection of existing test questions. For example, when the information processing device 1 receives a question format and a descriptive format as creation conditions, the information processing device 1 extracts existing questions that correspond to the question format and the descriptive format from the question group data. The information processing device 1 provides the extracted existing questions as input data to the learning model 121, and obtains new questions to be output. This allows the information processing device 1 to generate new question sentences and answers that correspond to the creation conditions.

The learning model 121 is not limited to generating new questions based on existing questions. FIG. 5 is an explanatory diagram showing an overview of a second example of the learning model 121. The learning model 121 shown in FIG. 5 is a model that takes a reference sentence and a question for the reference sentence as input, and outputs a question sentence and an answer. The output layer of the learning model 121 is configured to output the question sentence and the answer. The configurations of the other learning models 121 exemplified below are all similar to the learning model 121 shown in FIG. 4, so detailed explanations will be omitted.

The reference sentence may include, for example, a sentence (text data) extracted from a textbook, reference book, etc. The question content includes a sentence indicating the content to be asked based on the reference sentence.

The information processing device 1 trains (fine-tunes) the learning model 121 using training data in which reference sentences and question contents are associated with problem sentences and answers. This constructs the learning model 121 that has been trained to be able to appropriately output problem sentences and answers for the reference sentences and question contents.

Figure 6 is an explanatory diagram showing an overview of a third example of the learning model 121. The learning model 121 shown in Figure 6 is a model that receives a question sentence indicating a question as input and outputs an answer to the question sentence. The output layer of the learning model 121 is configured to output an answer to the question sentence.

The question sentence that is input to the learning model 121 may be generated based on, for example, the above-mentioned reference sentence, or a longer sentence extracted from a textbook, reference book, etc. The information processing device 1 extracts sentences that fall within the scope of the question from a textbook, reference book, etc., and generates the question sentence based on the extracted sentence. The question sentence may be generated using, for example, a natural language processing model that generates a question sentence based on a sentence. The question sentence may be generated by an external device or manually.

The information processing device 1 trains the learning model 121 using training data in which the generated questions and answers are associated with each other. This constructs the learning model 121 that has been trained to be able to appropriately output answers to questions.

Figure 7 is an explanatory diagram showing an overview of a fourth example of the learning model 121. The learning model 121 shown in Figure 7 is a model that takes as input a sentence extracted from a textbook, reference book, etc., and outputs a summary of the sentence. The output layer of the learning model 121 is configured to output a summary.

The information processing device 1 learns the learning model 121 using training data in which sentences that combine sentences and summaries extracted from textbooks, reference books, etc., are associated with the summaries. This constructs the learning model 121 that has been trained to be able to appropriately output summaries for sentences extracted from textbooks, etc.

Based on the summary generated using the learning model 121, the information processing device 1 can generate a fill-in-the-blank question in which the words in the summary are masked according to a predetermined rule, for example, and an answer corresponding to the blank to be filled in. Alternatively, based on the summary generated using the learning model 121, a descriptive question asking about a predetermined word and an answer may be generated. The word or predetermined word to be filled in may be determined by, for example, using keyword information stored in advance in the storage unit 12 to identify a word or word that matches a keyword stored in the keyword information from among the words included in the summary.

In the above-mentioned learning process, it is preferable that fine tuning is performed for each user's problem creation purpose using training data that corresponds to the user's problem creation purpose. The training data may be, for example, data from a textbook or reference book that corresponds to the user's problem creation purpose, or existing problems that correspond to the user's problem creation purpose. This makes it possible to appropriately generate problems that correspond to the user's problem creation purpose using a learning model 121 customized for each user.

When the answer format is multiple choice, the information processing device 1 generates multiple options based on the answers generated using the learning model 121. Each option includes a correct answer or an incorrect answer similar to the answer. The method of generating incorrect answers is not limited, but for example, the information processing device 1 may generate incorrect answers using a machine learning model that takes an answer as input and outputs an incorrect answer to the answer. The information processing device 1 may generate incorrect answers by vectorizing the words and phrases and identifying words and phrases whose similarity to the vector of the answer word and phrase is equal to or greater than a predetermined value.

The information processing device 1 may also generate an incorrect answer by acquiring data with the second highest priority (probability) after the answer identified as the output data with the highest priority (probability) based on the output data of the learning model 121, or data with the second highest priority (probability) after that. Similarly, the information processing device 1 can generate correct and incorrect sentences based on the likelihood of words in a sentence. For example, if the answer to a question that asks for a correct answer is "The Transformer uses positional encoding," the information processing device 1 generates an incorrect answer "The Transformer uses positional vectors" by replacing "positional encoding" with "positional vectors," which is the next candidate after "positional encoding." The information processing device 1 also generates an incorrect answer "The Transformer erases positional encoding" by replacing the verb "use" with "erase," which is the next candidate after "use."

The information processing device 1 may also identify an incorrect answer according to the answer using a table that classifies and stores similar words. If the answer includes a numerical value, the information processing device 1 may generate an incorrect answer by converting the numerical value of the correct answer according to a predetermined rule. The information processing device 1 may generate an incorrect answer by converting between a positive expression and a negative expression. The information processing device 1 generates a predetermined number of options in which the answer is replaced with an incorrect answer.

Figure 8 is an explanatory diagram showing an example of a test question. Figure 8 shows an example of a test question generated when the creation conditions are fill-in-the-blank and multiple choice. The test question includes a question statement and an answer generated using the learning model 121. The question statement also includes a standard phrase indicating a question according to the question creation conditions, and options. The automatically generated test question may further include an explanation according to the answer.

Figure 9 is a flowchart showing an example of a process procedure for generating a learning model 121. The processes in each of the following flowcharts may be executed by the control unit 11 in accordance with a program 1P stored in the memory unit 12 of the information processing device 1, or may be realized by a dedicated hardware circuit (e.g., an FPGA or ASIC) provided in the control unit 11, or may be realized by a combination of these. In the following, steps are abbreviated as S. Figure 9 explains an example of generating the learning model 121 of the third example described above.

The control unit 11 of the information processing device 1 acquires training data (S11). The control unit 11 acquires a question generated based on a sentence extracted from a textbook or reference book according to the user's purpose for creating questions, and an answer to the question. The control unit 11 acquires training data in which a label indicating the answer, which is the objective variable, is assigned to a sentence that connects the question and the answer, which are explanatory variables.

Based on the training data, the control unit 11 generates a learning model 121 that outputs an answer when a question sentence is input (S12). Specifically, the control unit 11 inputs a sentence that combines the question sentence and the answer included in the training data as input data to the learning model 121, and obtains the answer output from the learning model 121. The control unit 11 adjusts various parameters, for example, using the backpropagation method, so as to optimize (minimize or maximize) the error between the output answer and the answer included in the training data, i.e., the answer that is the correct answer. Before learning or fine tuning begins, the definition information describing the learning model 121 is set to initial settings or values that have been previously learned. When learning is completed by the error and the number of learning times satisfying a predetermined standard, optimized parameters are obtained.

When the learning is completed, the control unit 11 stores definition information related to the learning model 121 in the memory unit 12 as the learned learning model 121, and ends the series of processes. When fine-tuning of the learning model 121 is performed using training data according to the user's purpose for creating questions, the control unit 11 may store the generated learning model 121 in association with information related to the training data used and identification information of the user to which the learning model 121 should be applied. The control unit 11 may generate multiple types of learning models 121 and store them in the memory unit 12.

Figure 10 is a flowchart showing an example of a test question generation process performed by the information processing system 100.

The control unit 21 of the user terminal 2 accepts the test question creation conditions input by the user operating the operation unit 25 (S21). In S21, the control unit 21 displays on the display unit 24 a screen for accepting the creation conditions provided by the information processing device 1 in response to a login process using, for example, account information, and accepts the creation conditions specified by the user using the acceptance screen.

Figure 11 is a schematic diagram showing an example of a reception screen. The reception screen includes a reception field 241 that receives the specification of creation conditions. In the example shown in Figure 11, the reception field 241 is configured to be able to accept question creation conditions, answer creation conditions, and question creation patterns. The information processing device 1 displays each of the previously set question creation conditions and answer creation conditions, and question creation patterns in the reception field 241 in a selectable manner. Question creation patterns include, for example, automatic and manual generation of questions.

The reception field 241 can also accept detailed conditions such as the number of answers when the question creation condition is fill-in-the-blank, or the number of options when the answer creation condition is multiple choice. The reception field 241 may further be configured to accept the use of existing questions or the generation of new questions when the creation pattern is automatically generated. The user inputs the desired creation condition selection using the operation unit 25. The control unit 21 accepts the designation of the creation conditions via the operation unit 25.

The creation conditions may be accepted using a template. FIG. 12 is a schematic diagram showing an example of a template. For example, when a check box indicating "select from template" is selected on the acceptance screen of FIG. 11, the information processing device 1 displays a screen including the template as shown in FIG. 12 via the user terminal 2.

A number of templates are prepared according to combinations of pre-set question creation conditions and answer creation conditions. FIG. 12 shows an example of a template that corresponds to the creation conditions of a fill-in-the-blank format with three answers and a multiple-choice format with four options. The user specifies the creation conditions by selecting a template according to the creation conditions. As shown in FIG. 12, the layout pattern of correct and incorrect answers in each option can also be preset using the template.

Returning to FIG. 10, the explanation continues. The control unit 21 transmits the accepted creation conditions to the information processing device 1 (S22).

The control unit 11 of the information processing device 1 receives the creation conditions (S23). The control unit 11 may receive existing questions or reference sentences and questions, etc., corresponding to the creation conditions and in accordance with the user's purpose for creating questions.

The control unit 11 generates test questions including questions and answers using the learning model 121 based on the received creation conditions (S24). In detail, the control unit 11 inputs predetermined input data according to the creation conditions to the learning model 121 and obtains output data output from the learning model 121. The control unit 11 generates questions and answers according to the predetermined creation conditions by executing predetermined processing based on the output results from the learning model 121 as necessary.

Figure 13 is a flowchart showing an example of a detailed procedure for generating test questions. The processing procedure shown in the flowcharts of Figures 13 to 16 corresponds to the details of S24 in the flowchart of Figure 10.

The control unit 11 refers to the question group data and acquires existing questions that correspond to the received creation conditions (S201). The control unit 11 may receive existing questions that correspond to the user's question creation purpose, transmitted from the user terminal 2. The control unit 11 inputs the acquired existing questions into the learning model 121 of the first example (S202), and acquires new questions that are output from the learning model 121 (S203). The test questions are generated by the above-mentioned process. The control unit 11 returns the process to step S25 in the flowchart of FIG. 10.

When multiple learning models 121 are stored in the memory unit 12, the control unit 11 may identify a learning model 121 according to at least one of the logged-in user and the creation conditions, and generate questions using the identified learning model 121.

The control unit 11 may also perform post-processing based on the output results from the learning model 121 according to the creation conditions, and generate question text and answers according to the specified creation conditions. For example, the control unit 11 complements the question text by generating a specified number of options in which one or more answers are replaced with incorrect answers based on the answers output from the learning model 121. In this case, the control unit 11 may generate multiple patterns of combinations of question text and options (answers).

FIG. 14 is a flowchart showing another example of a detailed procedure for generating test questions.
The control unit 11 acquires a reference sentence and a question content according to the user's purpose for creating a question (S211) via, for example, the user terminal 2. The control unit 11 inputs the acquired reference sentence and question content to the learning model 121 of the second example (S212), and acquires a question sentence and an answer output from the learning model 121 (S213).

FIG. 15 is a flowchart showing another example of a detailed procedure for generating test questions.
The control unit 11 acquires a sentence corresponding to the subject for which the test question is to be created (S221) via, for example, the user terminal 2. The sentence corresponding to the subject for which the test question is to be created is, for example, a reference sentence corresponding to the user's purpose for creating the question, or a sentence extracted from a textbook, a reference book, etc. The control unit 11 generates a question sentence by converting the acquired sentence into a question format using, for example, a predetermined natural language processing model (S222).

The control unit 11 inputs the generated question text into the learning model 121 of the third example (S223) and obtains the answer output from the learning model 121 (S224). Through the above process, a question text that serves as a problem text and an answer to the question text are generated.

FIG. 16 is a flowchart showing another example of a detailed procedure for generating test questions.
The control unit 11 acquires sentences corresponding to the subject for which the test questions are to be created (S231), for example, via the user terminal 2. The sentences corresponding to the subject for which the test questions are to be created are, for example, a series of sentences extracted from a textbook, a reference book, or the like according to the user's purpose for creating questions.

The control unit 11 inputs the acquired sentence into the learning model 121 of the fourth example (S232) and acquires a summary sentence output from the learning model 121 (S233). The control unit 11 generates a question sentence and an answer based on the acquired summary sentence (S234). For example, the control unit 11 generates a fill-in-the-blank question sentence in which some words in the summary sentence are masked according to a predetermined rule, and an answer in which words corresponding to the masked parts are extracted.

Returning to FIG. 10, the explanation will be continued. The control unit 11 transmits the generated test questions to the user terminal 2 corresponding to the logged-in user (S25).

The control unit 21 of the user terminal 2 receives the test questions (S26) and displays a confirmation screen including the received test questions on the display unit 24 (S27). The user can confirm the test questions through the confirmation screen and decide which test questions to use as planned questions.

Figure 17 is a schematic diagram showing an example of a confirmation screen. In the example shown in Figure 17, the confirmation screen includes test questions including the generated question text and answers, a reset button 242 for accepting a request to change the options, and a save button 243 for accepting a request to save the generated test questions.

The information processing device 1 displays the question text and answers including options through the confirmation screen. When the information processing device 1 receives a tap operation of the reset button 242 through the user terminal 2, it displays new options with changed content of the options on the confirmation screen. The changed content of the options is, for example, the content of the incorrect answers included in the options, the combination or arrangement of correct answers and incorrect answers in the options, etc. The information processing device 1 uses the reset button 242 to accept the selection of any combination of question text and options desired by the user from among multiple combinations of question text and options. The information processing device 1 may also accept a change to the question text itself through the confirmation screen. In this case, the information processing device 1 generates a new combination of question text and answers using, for example, different input data, and displays it on the confirmation screen.

When the control unit 21 receives a tap operation of the save button 243 by the user while the selected option pattern is displayed, the control unit 21 receives a save request for the displayed combination of question sentence and answer. The selection of the test question is confirmed by receiving the save request.

Returning to FIG. 10, the explanation will be continued. The control unit 21 accepts the save request, i.e., the selection of the confirmed test question (S28), and transmits the accepted selection information to the information processing device 1 (S29).

The control unit 11 of the information processing device 1 receives the selection information (S30). The control unit 11 stores the selected combination of question and answer in the memory unit 12 as one of the test questions to be asked (S31). The control unit 11 transmits the confirmed test question to the user terminal 2 in a specified data format (S32).

The control unit 21 of the user terminal 2 receives the test questions (S33) and ends the series of processes. The information processing system 100 may repeatedly execute the processes from S21 to S31 until a predetermined number of questions is reached.

In the above process, the control unit 11 is not limited to generating questions at the timing when a question creation request is received from the user, but may generate a large number of test questions in advance using the learning model 121 according to the user's question creation purpose and store them in the storage unit 12. When the control unit 11 receives creation conditions from the user, it extracts test questions that correspond to the received creation conditions from among the generated test questions.

The test questions may be configured so that they can be generated partially or entirely manually by the user. For example, the user uses the template shown in FIG. 12 to generate the question text and specify answer phrases for fill-in-the-blank test questions. The user uses the operation unit 25 to input any sentence that corresponds to the question text in a designated field of the template, and to specify the fill-in-the-blank portion, i.e., to select the answer phrase. The information processing device 1 acquires the test question input by the user through the user terminal 2.

Based on the acquired input content, the information processing device 1 generates a question text in which a part of the question text is masked. The information processing device 1 may generate multiple incorrect answers based on the words that are the answers, and generate multiple options by combining the answers and the incorrect answers.

Test questions may also be generated by modifying parts of existing test questions. The information processing device 1 may generate new test questions by, for example, reading existing question statements and answers that correspond to the creation conditions from the question group data, and changing the order of the options included in the existing question statements or the arrangement of correct and incorrect answers within the options according to predetermined rules. This allows new test questions to be generated efficiently by utilizing existing test questions.

According to this embodiment, the learning model 121 can be used to automatically generate test questions according to various creation conditions received from the user, which can improve the efficiency of the user's question creation work and enable questions to be created according to various needs. The learning model 121 can accurately generate test questions that are in line with the characteristics of each test by learning using data according to the user's question creation purpose. The user can customize the test questions as desired by specifying the creation conditions, improving convenience.

Second Embodiment
In the second embodiment, a configuration for accepting reviews and corrections to the generated test questions will be described. In the following embodiments, differences from the first embodiment will be mainly described, and the same reference numerals will be used to designate the same components as the first embodiment, and detailed descriptions thereof will be omitted.

Figure 18 is a flowchart showing an example of a processing procedure for a review request executed by the information processing device 1 of the second embodiment.

After generating the test questions, the control unit 11 of the information processing device 1 acquires reviewer identification information for multiple reviewers and the review order for each reviewer, for example, via the user terminal 2 of the user who created the questions (S41). The reviewer identification information is information for identifying the reviewer who is requested to review the test questions, and includes, for example, the reviewer name, reviewer ID, etc.

The user terminal 2 may, for example, display a review setting screen provided by the information processing device 1 after the questions are created on the display unit 24, and use the review setting screen to accept reviewer identification information and review order designation from the user. The user terminal 2 transmits the accepted reviewer identification information and review order to the information processing device 1.

The control unit 11 outputs a review request to the user terminal 2 of the reviewer identified by the acquired reviewer identification information according to the acquired review order (S42). The reviewer who received the review request uses the user terminal 2 to check the review screen provided by the information processing device 1, for example, and reviews the generated test questions. The user terminal 2 transmits the review results received from the reviewer to the information processing device 1.

The control unit 11 acquires the review results sent from the user terminal 2 (S43). Based on the acquired review results, the control unit 11 stores in the memory unit 12 a review history including, for example, the reviewer identification information of the reviewer who completed the review and the review date and time, in association with the test question to be reviewed (S44).

The control unit 11 determines whether or not to end the review request (S45). If it is determined that the review request is not to end because the review request has not been output to all of the specified reviewers (S45: NO), the control unit 11 returns the process to S42 and outputs the review request to the next reviewer according to the review order. If it is determined that the review request is to end because the review request has been output to all of the specified reviewers (S45: YES), the control unit 11 ends the series of processes.

During the above-mentioned review stage or at an appropriate timing independent of the review, the information processing device 1 can accept corrections to the generated test questions from each reviewer or question creator. Figure 19 is a flowchart showing an example of a correction acceptance processing procedure executed by the information processing device 1.

The control unit 11 of the information processing device 1 accepts corrections to the test question through the user terminal 2 (S51). The control unit 11 stores the correction history, including the content of the accepted correction, the person who made the correction, the date and time of the correction, etc., in the memory unit 12 in association with the test question to be corrected (S52).

After accepting the corrections, the control unit 11 accepts, for example, a request to display a test question from the user terminal 2, and outputs the test question to the user terminal 2 by associating the correction history with the test question (S53).

Figure 20 is a schematic diagram showing an example of a screen including a correction history. After generating the test questions, the information processing device 1 can generate the screen shown in Figure 20 in response to a display request sent from the user terminal 2 at any timing, and output it to the user terminal 2.

In the example shown in FIG. 20, the screen including the correction history includes a test question display section 244 and a correction history display section 245 for each test question. The information processing device 1 displays the generated test questions on the test question display section 244. The information processing device 1 also reads out the correction history stored in association with the test questions from the storage section 12, and displays it in chronological order for each correction unit on the correction history display section 245.

As shown in the upper part of FIG. 20, the information processing device 1 may display detailed review information on the screen, including the review progress, details of the reviewer, review results, etc., based on the review history stored in the memory unit 12.

When the user wishes to make corrections to a test question, the user inputs the corrections using the operation unit 25. When the information processing device 1 accepts the corrections via the user terminal 2, it displays the corrected test question on the test question display unit 244 and also displays the new correction history on the correction history display unit 245.

By selecting one of the correction histories in the correction history display section 245, the user can revert (re-correct) to the state before the selected correction history. When the information processing device 1 receives an instruction to re-correct to the state before a specific correction history via the user terminal 2, it executes re-correction that undoes all corrections made up to that specific correction history. By re-correcting, the content of the test question can be reverted to a specific version. Based on the information displayed on the screen, the user can clearly understand the correction history and can easily and efficiently execute new corrections.

The corrections may be configured to be executable for each question in a series of questions that includes multiple questions. The correction history is not limited to being displayed for each test question, and the correction history for a series of questions may be displayed together.

According to this embodiment, the generated test questions can be efficiently reviewed among multiple reviewers. By displaying the review and correction history in association with the test questions, information related to the test question creation work can be easily and reliably grasped. The test question information processing system can provide optimal support for question management in addition to question creation.

Third Embodiment
In the third embodiment, a configuration for providing analytical information for generated test questions will be described. Fig. 21 is a flowchart showing an example of an analytical information generation process executed by the information processing device 1 of the third embodiment.

The control unit 11 of the information processing device 1 acquires the test results for the generated test questions (S61). The control unit 11 generates an analysis screen including analysis information for the test questions based on the acquired test results (S62). The analysis information includes, for example, the correct answer rate for the test questions and advice according to the correct answer rate. The control unit 11 outputs the generated analysis screen to the user terminal 2 (S63), and the series of processes ends.

FIG. 22 is a schematic diagram showing an example of a screen including analysis information. The user terminal 2 displays the analysis screen shown in FIG. 22 on the display unit 24 based on the information received from the information processing device 1.

The analysis screen includes, for example, a correct answer rate display section 246 that displays the correct answer rate of the test questions, and an advice display section 247 that displays advice. The information processing device 1 displays the correct answer rate of each test question calculated based on the test results on the correct answer rate display section 246 in the form of, for example, a graph. The information processing device 1 also identifies the cause of the deviation in the correct answer rate by comparing the calculated correct answer rate with a target correct answer rate set in advance, and generates advice to bring the correct answer rate closer to the target correct answer rate. The cause of the deviation may be identified based on, for example, the difficulty of the question text, the length of the question text, the number of options, etc. The cause of the deviation may be identified based on the tendency of the user's test questions taking into account past test questions related to the same user. For example, the information processing device 1 stores an advice table in advance in the storage section 12 that associates the cause of the deviation with advice, and generates advice according to the identified cause of the deviation by referring to the advice table, and displays it on the advice display section 247.

According to this embodiment, by providing analytical information on the generated test questions, it is possible to favorably support the generation of high-quality test questions that better meet the needs of users.

The embodiments disclosed herein are illustrative in all respects and should not be considered restrictive. The technical features described in each embodiment may be combined with one another, and the scope of the present invention is intended to include all modifications within the scope of the claims and equivalents to the scope of the claims. The sequences shown in each embodiment are not limited, and each processing step may be executed in a different order, and multiple processes may be executed in parallel, to the extent that there is no contradiction. The entity that performs each process is not limited, and the process of each device may be executed by another device, to the extent that there is no contradiction.

The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations regardless of the citation format. Furthermore, the claims use a format in which a claim cites two or more other claims (multi-claim format), but this is not limited to this. They may also be written in a format in which multiple claims cite at least one other claim (multi-multi-claim).

REFERENCE SIGNS LIST 100 Information processing system 1 Information processing device 2 User terminal 11, 21 Control unit 12, 22 Storage unit 13, 23 Communication unit 24 Display unit 25 Operation unit 1P Program 121 Learning model 1A Recording medium

Claims (15)

Accepts test question creation conditions from the user,
A learning model outputs new questions and answers using existing questions and answers, and generates questions and answers by inputting existing questions and answers whose creation conditions correspond to the creation conditions received;
A program for causing a computer to execute a process of outputting the generated question and answer to a user. The program according to claim 1 , wherein the question creation conditions include a fill-in-the-blank or question-based question creation condition, and an answer creation condition includes a multiple-choice or written answer creation condition. The program according to claim 2 , further comprising: a generating section configured to generate an answer and an incorrect answer similar to the answer when a multiple-choice question is received as the answer generating condition. Generate a plurality of candidate combinations of the question sentence and answer;
The program according to claim 1 or 2, further comprising: accepting a selection of any one of the plurality of generated candidate combinations of question sentences and answers. Acquire an existing question sentence or a modified question sentence obtained by modifying a part of the existing question sentence;
3. The program according to claim 1, further comprising: outputting the acquired existing question sentence or the changed question sentence together with the generated question sentence. Obtaining a plurality of reviewers and a review order for the generated question sentence and answer;
3. The program according to claim 1, further comprising: outputting a review request to each reviewer in accordance with the acquired review order. 3. The program according to claim 1, further comprising: when a correction to the generated question or answer is accepted, storing a correction history in association with the question or answer. Accepts test question creation conditions from the user,
A learning model outputs new questions and answers using existing questions and answers, and generates questions and answers by inputting existing questions and answers whose creation conditions correspond to the creation conditions received;
An information processing method in which a computer executes a process of outputting the generated question sentence and answer to a user. Accepts test question creation conditions from the user,
A learning model outputs new questions and answers using existing questions and answers, and generates questions and answers by inputting existing questions and answers whose creation conditions correspond to the creation conditions received;
An information processing device comprising: a control unit that executes a process of outputting the generated question sentence and answer to a user. Acquire the test question creation conditions and the sentences used for creating the test questions received from the user;
Acquire a question sentence by converting the acquired sentence into a question sentence format according to the creation conditions;
generating an answer by inputting the acquired question text into a learning model that uses the question text to output an answer to the question text;
Outputting the question and the answer to the user
A program that causes a computer to execute a process. Acquire the test question creation conditions and the sentences used for creating the test questions received from the user;
Acquire a question sentence by converting the acquired sentence into a question sentence format according to the creation conditions;
generating an answer by inputting the acquired question text into a learning model that uses the question text to output an answer to the question text;
Outputting the question and the answer to the user
An information processing method in which processing is performed by a computer. Acquire the test question creation conditions and the sentences used for creating the test questions received from the user;
Acquire a question sentence by converting the acquired sentence into a question sentence format according to the creation conditions;
generating an answer by inputting the acquired question text into a learning model that uses the question text to output an answer to the question text;
Outputting the question and the answer to the user
Equipped with a control unit that executes processing
Information processing device. Acquire the test question creation conditions and the sentences used for creating the test questions received from the user;
generating a summary by inputting the received sentence into a learning model that uses the sentence to output a summary of the sentence;
Using the generated summary, a question sentence and an answer are generated according to the generation conditions in accordance with a predetermined rule;
The generated question and answer are output to the user.
A program that causes a computer to execute a process. Acquire the test question creation conditions and the sentences used for creating the test questions received from the user;
inputting the received sentence into a learning model that uses the sentence to output a summary of the sentence, and outputting the summary;
Using the output summary, a question sentence and an answer are generated according to the creation conditions in accordance with a predetermined rule;
The generated question and answer are output to the user.
An information processing method in which processing is performed by a computer. Acquire the test question creation conditions and the sentences used for creating the test questions received from the user;
inputting the received sentence into a learning model that uses the sentence to output a summary of the sentence, and outputting the summary;
Using the output summary, a question sentence and an answer are generated according to the creation conditions in accordance with a predetermined rule;
The generated question and answer are output to the user.
Equipped with a control unit that executes processing
Information processing device.

Images