JP6913706B2 - Exam question prediction system and exam question prediction method - Google Patents

Description

The present invention relates to a system for predicting test questions, and in particular, a test question prediction system and a test question prediction system for predicting test questions based on classifying question sentences of past questions into categories and predicting question categories. Regarding the method.

[Conventional technology]
Conventionally, there is a method in which an examination question is created using a computer and the examination question is provided via a network.
However, no method has been established to classify the question sentences of past exam questions and accurately predict this year's exam questions.

[Related technology]
As a related prior art, there is Japanese Patent Application Laid-Open No. 2007-248605 "Examination Question Creation Method, Examination Question Creation System and Examination Question Creation Program" (Patent Document 1).
Patent Document 1 shows that it is possible to create a test question that satisfies the evaluation criteria according to the type of test.

JP-A-2007-248605

The conventional test question creation method does not predict the next test question by efficiently classifying the sentences of the past test questions using artificial intelligence and analyzing the question category from the classified contents. There was a problem that the problem could not be predicted efficiently and accurately.

In particular, when the amount of past exam questions becomes enormous, the work of classifying exam questions into question categories increases, and it becomes difficult to efficiently and accurately analyze the exam questions into question categories and predict the next exam questions. rice field.
Patent Document 1 also does not create an examination question by predicting the next question category from the past question contents.

The present invention has been made in view of the above circumstances, classifies question sentences that have been asked in the past into categories, predicts the next question category from the classification results, and uses the prediction results to solve past problems. It is an object of the present invention to provide a test question prediction system and a test question prediction method that can easily create a prediction question for test preparation based on the test question.

The present invention for solving the problems of the above-mentioned conventional example is a test question prediction system including a processing device for predicting a test question, in which the processing device extracts a word from a question sentence of a past test question. For the extracted words, give numerical values to each element of the vector to generate vector data, perform vectorization, classify the exam questions into categories based on the vector data, and classify the exam questions into categories for each past year. The number of questions is arranged in the past time series, and the number of questions for each category of the next exam question is predicted by machine learning using the difference series that gradually generates multiple sequences that shift the arrangement to the past one year. It is characterized by that.

In the present invention, in the above-mentioned test question prediction system, a processing device trains a machine learning learning model to obtain desired vector data by using a question sentence of teacher data, and uses the trained learning model to perform a vector. It is characterized by performing conversion.

In the present invention, in the above-mentioned test question prediction system, a processing device trains a machine learning learning model to be classified into a desired category using vector data of teacher data, and uses the trained learning model to perform a vector. It is characterized by classifying exam questions into categories from the converted vector data.

The present invention is characterized in that, in the test question prediction system, the processing device uses a recurrent neural network for machine learning.

The present invention is characterized in that, in the test question prediction system, the processing device extracts questions from past test questions according to the predicted number of questions for each category and creates the next predicted question.

The present invention provides a test question prediction method Ru is predictive of test questions to the processor, to extract words from the question sentence past exam, the word obtained by the extraction by applying a number to each element of the vector row Align vectorization of generating vector data, to classify the test problems based on the vector data in the category, so the question number in the past year each test problems classified into categories are arranged in time series in the past, the arrangement the characterized Rukoto is predicted by the machine learning question count for each category of the next exam using stepwise multiple generated differenced series arrangement for shifting a year past.

According to the present invention, the processing apparatus extracts a word from a question sentence of a past test question, assigns a numerical value to each element of the vector to the extracted word, performs vectorization to generate vector data, and vector data. the test problems classified into categories based on the question number in the past year each test problems that are categorized into place in a time series in the past, stepwise multiple sequences to shift the arrangement to one year past Since it is a test question prediction system that predicts the number of questions for each category of the next test question by machine learning using the generated difference series, the question sentences of the past test questions are classified into categories, and the next test question is based on the classification result. It has the effect of appropriately predicting the number of questions in each test question category.

It is a schematic diagram of this system. It is a schematic diagram of this mounting engine. It is a schematic diagram of a sentence feature amount extraction engine. It is a schematic diagram of a sentence classification engine. It is a schematic diagram which shows the self-Attention use example of RNN. It is a schematic diagram of a question tendency prediction engine. It is a schematic diagram of a prediction and a problem making process. It is a schematic diagram of the model of the neural network in this apparatus. It is a figure which shows the number of times of a question of a category for each year. It is a figure which shows how to make the difference data.

Embodiments of the present invention will be described with reference to the drawings.
[Outline of Embodiment]
The test question prediction system (this system) according to the embodiment of the present invention is a prediction question creation processing device (this device) that extracts words from question sentences of past test questions, and vector elements for the extracted words. Perform vectorization to generate vector data by giving a numerical value for each, classify the exam questions into categories based on the vector data, and the next exam question from the number of questions of the exam questions classified into the category for each past year It predicts the number of questions for each category, classifies the question sentences of past exam questions into categories, and can appropriately predict the number of questions for each category of the next exam questions from the classification results.

Furthermore, in this system, the processing device extracts questions from the past exam questions according to the predicted number of questions for each category and creates the next predicted question, and for the category in which the number of questions is properly predicted. Prediction questions can be created with high accuracy, and candidates can receive prediction questions with high prediction accuracy for exam preparation.

[This system: Fig. 1]
This system will be described with reference to FIG. FIG. 1 is a schematic view of this system.
As shown in FIG. 1, this system basically includes a prediction problem creation processing device 1, a prediction problem providing server 2, an Internet 3, and an examinee computer (PC) 4.
Each device is connected via the Internet 3, and a plurality of examinee PCs 4 are originally connected.

[Prediction problem creation processing device 1]
The prediction question creation processing device 1 classifies past question sentences into categories, analyzes the question categories, predicts the next exam question, and creates the exam question.
The prediction problem creation processing device 1 includes a control unit 11, a storage unit 12, and an interface unit 13. A display unit 14 and an input unit 15 are connected to the interface unit 13, and further connected to the Internet 3. ..

The control unit 11 reads the processing program stored in the storage unit 12 and executes the processing described later.
The storage unit 12 stores the processing program and also stores the past test questions.
The display unit 14 makes the display necessary for creating the prediction problem.
The input unit 15 makes inputs necessary for creating a prediction problem.

[Prediction problem providing server 2]
The prediction problem providing server 2 is a computer having a control unit and a storage unit and connected to the Internet 3. The prediction problem created by the prediction problem creation processing device 1 is input and stored, and the prediction problem is sent to the student PC 4. To deliver. In the claims, it is referred to as a "prediction problem providing device".
In FIG. 1, the prediction problem creation processing device 1 and the prediction problem providing server 2 are connected via the Internet 3, but they may be connected via an in-house network.
Further, the prediction problem creation processing device 1 and the prediction problem providing server 2 may be integrated into one device.

[Examinee PC4]
The examinee PC 4 is a computer that can access the prediction problem providing server 2 via the Internet 3 and receive the provided prediction problem. In the claims, the term "examinee's device" is used, and the device may be a terminal device such as a tablet terminal or a smartphone as well as a computer.

Further, the examinee PC 4 may transmit the answer data to the prediction question providing server 2 when answering the provided prediction question. In that case, the prediction question providing server 2 scores the answer data of the examinee, gives advice for passing, and encourages the student to take a lecture according to his / her ability.

[This mounting engine: Fig. 2]
Next, the mounting engine (main mounting engine) realized by the program (software) operating in the prediction problem creation processing device 1 will be described with reference to FIG. FIG. 2 is a schematic view of the present mounting engine.
As shown in FIG. 2, the mounting engine 10 includes a test sentence classification engine 100 and a questioning tendency prediction engine 200.
The test sentence classification engine 100 includes a sentence feature amount extraction engine and a sentence classification engine.

The sentence feature extraction engine of the test sentence classification engine 100 has acquired a trained model of AI (Artificial Intelligence) deep learning that reads a large amount of question sentences and vectorizes the sentences. A word is extracted from the input problem, and vector data for the word is output.

The sentence classification engine of the test sentence classification engine 100 classifies the categories of the test questions (question sentences) using another learned model based on the vector data.
The question tendency prediction engine 200 predicts the number of questions for each category next time from the transition of the number of questions for each category in the past, the transition of the latest questions, and the number of the latest questions.
Hereinafter, each engine will be specifically described.

[Sentence feature extraction engine: Fig. 3]
Next, the text feature extraction engine will be described with reference to FIG. FIG. 3 is a schematic diagram of a text feature extraction engine.
As shown in FIG. 3, the sentence feature extraction engine reads test questions, for example, sentence data (text data) such as bar examination options into the trained model 110, extracts keyword words from sentences, and is easy to classify. Convert to vector data.

In the sentence feature amount extraction engine, the key elements (directions) of the category classification are set in advance as vectors v1, v2, v3, ....
The vector data is a numerical value of how much the words extracted from the text data of the examination questions (for example, "today", "fine", "application", etc.) are related to these elements.
That is, the vector data has a direction indicating the element and a size indicating the strength of the relationship with the direction, and is information (feature vector) representing the feature amount of each word.
Here, morphological analysis is used to extract words from sentence data. Morphological analysis extracts important words by decomposing part of speech and deleting unnecessary words.

The trained model 110 inputs a large amount of problem sentences of teacher data, and learns so that the size of each element of the vectors v1, v2, v3 ... For the extracted specific word becomes an optimum numerical value.
As the teacher data to be trained by the trained model 110, firstly, only the question sentence and the classification label data are used, secondly, the question sentence, the choices and the correct answer data of the question are used, and thirdly, the text data of the book is used.

Specifically, vector v1 is an element of "peace" related to category a, vector v2 is an element of "imprisonment" related to category b, and vector v3 is an element of "corporation" related to category c.
Then, if the trained model 110 of the sentence feature extraction engine extracts the words "today", "imprisonment", and "corporation" from the input question sentence, the words are dropped into vector data so that the categorization can be facilitated. Perform conversion processing.

In the example of FIG. 3, the word "today" is given a numerical value of "0.8" for v1, "0" for v2, and "0.1" for v3, and the word "fine" is given a numerical value of "0" for v1. , V2 is "0.5", v3 is "0.3", and the word "application" is v1 is "0.05", v2 is "0", v3 is "0.72". Each numerical value is given.
If the problem sentences are different, even if the same word "today" is extracted, the numerical values of the vectors are not always the same.
In addition, since the text of the question to be input is different each time, the extracted words are also different each time.

Vectors are an important factor set to categorize sentences in exam questions, and a high numerical value for a vector element for a word in a sentence is highly relevant to the category corresponding to that vector. Is shown.

Then, the relationship between the entire word and the vector in the input sentence, that is, the entire vertical numerical value of the vector indicates the relationship between the entire word and the vector.
As a result, the feature amount for each vector of the input sentence (question sentence or option) is extracted instead of the individual word.

In the trained model 110 of the sentence feature extraction engine, a task of predicting words appearing before and after when one word is input is learned to acquire an intermediate layer, and the learned intermediate layer is used as a word feature vector. It may be calculated by.
That is, for the input word, the numerical value of the vector may be obtained by using the prediction probability of the word appearing before or after the word by using the middle layer.

[Sentence classification engine: Fig. 4]
Next, the sentence classification engine will be described with reference to FIG. FIG. 4 is a schematic diagram of a sentence classification engine.
As shown in FIG. 4, the sentence classification engine first inputs a large amount of vector data output from the trained model 110 of FIG. 3 into the trained model 120 as teacher data, and trains the trained model 120 to be classified into a desired category. ..

Then, the vector data obtained by performing the conversion process on the problem sentence that is not the teacher data by the sentence feature amount extraction engine of FIG. 3 is input to the trained model 120 of the sentence classification engine of FIG. 4 and classified into categories. ..
In Fig. 4, since the bar examination is assumed, the classified categories are "Civil Code Vol. 8 Relatives Chapter 2 Family", "Chapter" is the middle category, "Section" is the small category, and up to the middle category. It may be a classification of up to a small category.

[Example of using RNN: Fig. 5]
In the sentence classification engine, a recurrent neural network (RNN) is used, and the context of the series data is learned by calculating in order from the front.
Here, in the "Bidirectional" model that predicts future information by propagating information not only from the past to the future but also from the future to the past, the importance (weight) of the unit itself for each input. ) Is used properly, and an example of using "Self-Attention" is introduced in Fig. 5. FIG. 5 is a schematic view showing an example of using Self-Attention of RNN.

As shown in FIG. 5, in a plurality of units (memory) 121 to 126, which unit is important for the learned word is calculated, and the same information as the input is used at that time.
In FIG. 5, it is shown that the units 121, 123, 125 are heavier than the units 122, 124, 126, and the unit 126 is heavier than the units 122, 124.
The example of FIG. 5 is a learning model used in a sentence classification engine, and is excellent in language processing because it can watch only words with high relevance (for example, "imprisonment" and "year").

[Question trend prediction engine: Fig. 6]
Next, the questioning tendency prediction engine implemented in the prediction problem creation processing device 1 will be described with reference to FIG. FIG. 6 is a schematic diagram of a questioning tendency prediction engine.
The test sentence classification engine 100 inputs the number of questions (numerical data) classified for each category in the past year into the trained model 210, and predicts the number of questions (numerical data) for each category in the next year.
As the trained model 210, models such as multiple regression and LSTM (Long Short-Term Memory network) are used.
The details of questioning tendency prediction and question creation will be described below.

[Forecasting and problem creation processing: Fig. 7]
The process of creating a prediction problem (this process) in this apparatus will be described with reference to FIG. 7. FIG. 7 is a schematic diagram of the process of prediction and problem creation.
As shown in FIG. 7, this process reads the exam questions for a specific period in the past (a period that goes back continuously from the previous exam), and for those exam questions, a predetermined subcategory (simply "" It has a small category prediction step that predicts the small category of the next exam based on the data classified into "category") and a question creation step that creates a prediction problem using the small category prediction result. ing.

The small categories are those in which the fields are classified in advance according to the content of the problem, and the work of classifying into the small categories is performed manually.
The data classified into small categories is the number of questions (number of questions) for each category. For example, the number of applications for category 1 is 2 times, and the number of questions for category 2 is 3 times.

In the example of FIG. 7, a category is predicted from the already classified test questions to create a prediction question, and the processing described in FIGS. 2 to 5 classifies the questions from the question sentences that are not classified into categories. It is a thing.
Therefore, the problem sentences that are not classified are classified into categories in FIGS. 2 to 5, and the category prediction is performed by the processing by the prediction program in FIGS. 6 and 6 to create a prediction problem.

The sub-category prediction step and the problem creation step of this process will be specifically described.
[Small category prediction step]
The control unit 11 of the present device reads a processing program from the storage unit 12 and executes a small category prediction step, and externally inputs the past test questions (past questions) of a specific period in the CSV (Comma Separated Values) format. Enter in, total the number of questions for each subcategory for each exam (the processing up to this point may be performed by the exam text classification engine 100), and execute the prediction program based on the number of questions for each subcategory. Output the small category prediction result. This sub-category prediction result is the predicted number of questions for each sub-category of the next exam question.

Here, the AI program uses a recurrent neural network (RNN) that can handle time-series data as a prediction program, and the unit of the intermediate layer (hidden layer) is replaced with an LSTM block. RNNs are suitable for learning time series information and past data.
If the exam questions consist of multiple subjects, the number of questions for each subject is fixed, so the classification is based on rules.

[Problem creation step]
The control unit 11 of the present device reads the processing program from the storage unit 12 and executes the problem creation step, and obtains the small category prediction result (predicted number of questions for each small category) obtained in the small category prediction step. Input, execute the question creation program, randomly extract questions from past questions, and output the question prediction results.
That is, according to the number of predicted questions for each subcategory, questions are randomly extracted from the past questions corresponding to the subcategory to create a predicted question as a whole.

In this question creation step, past questions are randomly extracted, but the questions that were asked last time or two times before the previous question may be weighted and selected at the time of extraction so that they are difficult to be selected.
In addition, for past questions, instead of the past questions themselves, questions may be created with slightly modified contents and extracted including them.

[Neural network model: Fig. 8]
Next, the model of the neural network used in this apparatus will be described with reference to FIG. FIG. 8 is a schematic diagram of a neural network model in this device.
In the neural network in this apparatus, as shown in FIG. 8, input data x is input to the input layer 111, which is input to the fully connected layer 114 via the LSTM layers 112 and 113 of the intermediate layer, and the fully connected is performed. Then, the output data y is output to the output layer 115 as a predicted value.

The LSTM layers 112 and 113 of the intermediate layer hold the time series information in the memory unit for a certain period of time window, and learn about the information in the time window.
The LSTM layer relaxes the limitation that only short-term memory of RNN can be realized, and some memory units are input from the input layer and some are fed back the output from the intermediate layer. It enables long-term memory of information.
By using the model of FIG. 8 in this apparatus, it is possible to improve the accuracy of the prediction result of the small category.

[Number of questions in each category for each year: Fig. 9]
Next, the number of questions in each category (small category) will be described with reference to FIG. FIG. 9 is a diagram showing the number of questions in the category for each year.
As shown in FIG. 9, the number of questions for each year is classified into, for example, 93 categories, and the number of questions for each category from 2001 (H1) to 2017 (H29) is can get.

Regarding the category number I, the number of questions in the category is _{represented by n ij} , i represents the category number, and j represents the year.
In the method of creating the difference data described below, the row data of FIG. 9 of H1 to H29 is used for each category (each category number). For example, in category number 2, _{the row data of n 21} , n ₂₂ , n ₂₃ , ... Is used as a unit.

[How to create difference data: Fig. 10]
Next, how to create the difference data for each small category will be described with reference to FIG. FIG. 10 is a diagram showing how to create the difference data.
Simply inputting the number of questions for each small category of past questions into the AI program is not enough to obtain accurate small category prediction results. Therefore, create the difference data explained below and create the difference. The numerical value of the matrix obtained from the data is input to the AI program to improve the prediction accuracy.

As shown in FIG. 10, (1) to (29) are the original time series data for the years H1 to H29. _{For example, the data in the rows n 21} , n ₂₂ , n ₂₃ , ... For category number 2 shown in FIG. 9 corresponds to time series data.
The numerical value represented by parentheses is the number of questions in the actual test, and the numerical value represented by square brackets is the predicted value.

Then, an array in which the time-series data is shifted to the left by one year is placed below it, and this is repeated to generate a multi-stage array, for example, a 13-stage array difference data is generated.
Further, the numerical value [14] of the number of questions in this category of H14 is predicted from the values of the data (1) to (13) in the vertical column of H1.

Further, the predicted value [15] of H15 is obtained from the values of the vertical column data of H1 and the data (2) to (14) of the vertical column of H2. In this method, the predicted value is obtained by using the value of the matrix of 13 rows × 2 columns as input data.
Similarly, the predicted value [26] of H26 is obtained by using the value of the matrix of 13 rows × 14 columns of H1 to H13 as input data.

In this device, time series data is used for 14 years, and in order to obtain the predicted value of H30, the value of the matrix of 13 rows × 14 columns of H4 to H17 is used as input data, and the AI of the small category prediction step. Input to the program to obtain the predicted value [30].

Since there are 93 subcategories, 93 subdivision data as shown in FIG. 10 are arranged and an array (93, 14, 13) is input to the AI program, and all the subcategories are subcategories. You will get the prediction result.
The small category prediction result obtained in this way has high accuracy, and the prediction accuracy can be improved even in the problem creation in the problem creation step.

[Processing of prediction problem providing server 2]
Next, the processing in the prediction problem providing server 2 of this system will be described.
The prediction problem providing server 2 uploads the prediction problem data created by the prediction problem creation processing device 1 from the prediction problem creation processing device 1, stores it in the storage unit, and accesses the prediction problem data from the examinee PC4. offer.

The prediction question providing server 2 may receive answer data for the prediction question from the examinee PC4, perform scoring, and notify the examinee PC4 of the scoring result.
In addition to simply notifying the scoring results, it also provides analysis results of the answers of all the answerers of the prediction question, points out the weaknesses of individual examinees, and provides effective learning methods for passing as advice. You may try to do it. The advice also includes introducing online lectures that can be used as a reference for studying and inviting them to the lectures.

[Effect of Embodiment]
According to this system and the test question prediction method, the device 1 extracts words from the question sentences of past test questions, assigns numerical values to each of the vector elements for the extracted words, and generates vector data. Based on the vector data, the exam questions are classified into categories, and the number of questions for each category of the next exam question is predicted from the number of questions in the exam questions classified into categories for each past year. , It has the effect of classifying the question sentences of past exam questions into categories and appropriately predicting the number of questions for each category of the next exam questions from the classification results.

Furthermore, in this system, the processing device extracts questions from past exam questions according to the predicted number of questions for each category and creates the next predicted question, so the category in which the number of questions is properly predicted. It is possible to create a prediction question with high accuracy, and the examinee can receive a prediction question with high prediction accuracy as a test preparation.

The present invention is an examination question prediction system capable of predicting the next question category from the contents of questions asked in the past and easily creating a prediction question for exam preparation based on the past question using the prediction result. It is suitable for the test question prediction method.

1 ... Prediction question creation processing device, 2 ... Prediction question providing server, 3 ... Internet, 4 ... Candidate PC, 10 ... Mounting engine, 11 ... Control unit, 12 ... Storage unit, 13 ... Interface unit, 14 ... Display unit, 15 ... Input section, 111 ... Input layer, 100 Test sentence classification engine, 112, 113 ... Intermediate layer, 114 ... Fully connected layer, 115 ... Output layer

Claims (6)

A test question prediction system equipped with a processing device that predicts test questions.
The processing device extracts words from the question sentences of past test questions, assigns numerical values to each of the vector elements to the extracted words, performs vectorization to generate vector data, and tests based on the vector data. The questions are classified into categories, the number of questions in the exam questions classified into categories for each past year is arranged in the past time series, and multiple sequences that shift the arrangement to the past one year are generated step by step. An exam question prediction system characterized by predicting the number of questions for each category of the next exam question by machine learning using a series. The processing device is characterized in that a learning model of machine learning is trained so as to obtain desired vector data by using a problem sentence of teacher data, and vectorization is performed using the learned learning model. Described exam question prediction system. The processing device trains the machine learning learning model to be classified into a desired category using the vector data of the teacher data, and classifies the test questions from the vector data vectorized using the trained learning model into categories. The test question prediction system according to claim 1 or 2, wherein the test questions are classified. The test question prediction system according to any one of claims 1 to 3, wherein the processing device uses a recurrent neural network for machine learning. The test question prediction system according to any one of claims 1 to 4 , wherein the processing device extracts questions from past test questions according to the predicted number of questions for each category and creates the next predicted question. A vector that is a test question prediction method in which a processing device predicts test questions, extracts words from question sentences of past test questions, assigns numerical values to each of the vector elements for the extracted words, and generates vector data. Based on the vector data, the exam questions are classified into categories, the number of questions in the exam questions classified into the categories is arranged in the past time series for each past year, and the arrangement is made one year in the past. A test question prediction method characterized in that the number of questions for each category of the next test question is predicted by machine learning using a difference series in which a plurality of sequences to be shifted are generated stepwise.

Images