JP7299580B2 - Inappropriate answer detection method, its system, and its program - Google Patents

Description

An embodiment of the present invention relates to an inappropriate answer detection method for detecting an inappropriate answer.

It is generally known to reduce bad responses in surveys. For example, it is disclosed to efficiently collect more good answers by reducing bad answers (see Patent Document 1).

JP 2016-95656 A

However, since the contents of each survey are different, there is no known method for detecting inappropriate answers that can be applied to all surveys.
SUMMARY OF THE INVENTION It is an object of the present invention to provide an inappropriate answer detection method, a system therefor, and a program therefor that can be applied to various surveys and that improve accuracy in detecting inappropriate answers.

An inappropriate answer detection method according to the aspect of the present invention acquires a plurality of response data of a survey including a plurality of Likert scale items, and detects an inappropriate response among the plurality of response data. Obtaining the required response time, determining the number of times the same response category is continuously selected for the plurality of Likert scale items of the target response data, and obtaining the response categories for the plurality of Likert scale items of the target response data. and calculating a plurality of Mahalanobis distances of the plurality of Likert scale items of the target response data based on the above, and calculating the response time required for each response of the plurality of response data. to obtain the number of items in the survey, and based on the response time, the number of times, the numerical value indicating the tendency, the plurality of Mahalanobis distances, the representative value, and the number of items, the subject detecting that an answer in the answer data is inappropriate, wherein the numerical value indicating the tendency is selected by shifting the answer categories one by one for consecutive Likert scale items of the target answer data. number of times .

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide an inappropriate answer detection method, a system therefor, and a program therefor, which are applicable to various surveys and improve the detection accuracy of inappropriate answers.

1 is a configuration diagram showing the configuration of an investigation system according to a first embodiment of the present invention; FIG. FIG. 4 is a flowchart showing the procedure of an inappropriate answer detection method applied to the survey system according to this embodiment; FIG. 4 is a schematic diagram showing various numerical values regarding the total divergence value of respondents to each question of the questionnaire according to the present embodiment; FIG. 4 is a conceptual diagram showing a method of constructing an inappropriate answer prediction model according to this embodiment by machine learning; FIG. 11 is a flowchart showing the procedure for creating explanatory variables used in the inappropriate answer detection method according to the second embodiment;

(First embodiment)
FIG. 1 is a configuration diagram showing the configuration of an investigation system 1 according to the first embodiment of the present invention. In addition, the same reference numerals are given to the same parts in the drawings, and redundant explanations are omitted as appropriate.

The survey system 1 is a system that conducts surveys and collects responses via the network NT. The survey system 1 has an inappropriate response detection function for detecting inappropriate responses in response data collected for questionnaires. Note that the inappropriate answer detection function may be applied to any system, and does not have to have a function to conduct an investigation. Also, a system having only an inappropriate answer detection function may be configured.

The respondent terminal 2 is a terminal used by the respondent to answer the questionnaire conducted by the survey system 1 . The respondent terminal 2 is mainly composed of a computer. For example, the respondent terminal 2 may be any type of computer such as a desktop, laptop or tablet, or may be a mobile phone such as a smart phone.

For example, the network NT is the Internet, a fixed telephone network, a mobile telephone network, or the like, and these may be combined or other communication networks may be used. Moreover, the survey system 1 may be connected to any communication path as long as it can communicate with the respondent terminal 2 that responds to the survey.

The survey system 1 includes a questionnaire generator 11, a questionnaire answerer 12, an answer data totalizer 13, an appropriateness determination unit 14, an input unit 15, an output unit 16, a survey data storage unit ST1, and an answer data storage unit ST2.

The investigation system 1 is mainly composed of a computer. For example, the computer is a web server. The number of computers is not limited to one, and may be configured in a network. The survey system 1 may directly input questionnaire response data to the response data storage ST2 without providing the questionnaire generator 11, the questionnaire answerer 12, and the survey data storage ST1.

The questionnaire generation unit 11 generates a questionnaire form Fm in which questions for conducting a survey are described based on the survey data stored in the survey data storage unit ST1. The questionnaire generation unit 11 discloses the generated questionnaire form Fm in a state accessible from the outside. For example, the questionnaire generator 11 displays the questionnaire form Fm on the website.

The questionnaire response unit 12 performs a process of receiving response data from the respondent terminal 2 . The questionnaire response unit 12 receives data input from the respondent terminal 2 via the network NT via the input unit 15 . The questionnaire response unit 12 generates response data based on the received data and records it in the response data storage unit ST2. For example, the respondent answers in the input fields of the questionnaire form Fm displayed on the website, and the questionnaire answering section 12 receives the data input in the input fields. The questionnaire answering section 12 generates answer data based on the received data.

The response data totaling section 13 totalizes a plurality of response data stored in the response data storage section ST2. The response data totaling section 13 transmits the totaled data to the propriety determining section 14 . A plurality of answer data to be aggregated are all answer data for one survey, but some answer data may not be included. For example, when the survey is ongoing, the aggregation may be performed using only the response data stored in the response data storage unit ST2 at the time of aggregation. Alternatively, the answer data Di to be judged inappropriate may be removed, and the remaining answer data may be aggregated.

The suitability determination unit 14 determines whether or not the selected answer data Di is inappropriate based on predetermined determination conditions and the data aggregated by the answer data aggregation unit 13 . The suitability determination unit 14 outputs the suitability of the determined response data Di to the output unit 16 .

The input unit 15 is a network interface for connecting with the network NT in order to receive data from the respondent terminal 2, for example. The input unit 15 may be an input device for directly inputting questionnaire response data to the survey system 1 . For example, such an input device may be a keyboard, mouse, data input display, or various media readers.

The output unit 16 presents the suitability of the answer data Di determined by the suitability determination unit 14 . For example, the output unit 16 performs display on a display, printing by a printer, or writing to various media.

Next, the items of the questionnaire conducted in the survey will be explained. The questionnaire items are question items for obtaining explanatory variables used for determining whether or not the response data Di is inappropriate.

The questionnaire includes a plurality of Likert scale items (hereinafter referred to as "Likert scale items"). A Likert scale item is an item of a format in which a question is answered by selecting one from a plurality of options classified in stages. The number of options is often three or more, but two may be acceptable. For example, the Likert scale items are “1. Satisfied”, “2. Somewhat satisfied”, “3. This is a question item in an answer format that prompts the customer to select one from five choices of ". dissatisfied."

In addition, the questionnaire may include items regarding attribute variables and free description response items. Items about attribute variables are question items that ask respondents to answer gender, age, marital status (for example, marital status), or the like. The free description answer item is a question item in which a respondent freely describes an answer.

FIG. 2 is a flowchart showing the procedure of the inappropriate answer detection method applied to the survey system 1 according to this embodiment.

The propriety determination unit 14 obtains the response time required for the respondent of the response data Di to respond to the questionnaire (step S101). For example, when a questionnaire is executed by a computer program (such as a questionnaire on a website), the response time may be the time from the start of the questionnaire to the end of the questionnaire. For example, the operation to start or end is pressing a button displayed on the screen.

The propriety determination unit 14 obtains the number of times the same answer category on the Likert scale is continuously selected in the answer data Di (step S102). Specifically, if the answer to a question item is 3 out of 5 options from 1 to 5, if the answer to the question item before this question item is also 3, it is counted once. do. In this way, the number of times counted up to the answer to the final Likert scale item is the number of times obtained here.

It should be noted that the method of counting the number of times explained here is an example, and in a program or the like, depending on the conditions, etc., the number of times may not be limited to one, but may be two or three or more. The same applies to how to count subsequent times.

The propriety determination unit 14 obtains a variable indicating the tendency of the answer category of the Likert scale item of the answer data Di (step S103). Here, the variables that indicate the tendencies of the answer categories are broadly divided into three as explained below, but as long as at least one variable is obtained, any number of variables or all variables may be obtained. Depending on the variables to be obtained, some of the calculations may be performed by the answer data totalizing section 13, and the propriety judging section 14 may obtain the target variables using the calculation results of the answer data totalizing section 13. FIG. It should be noted that any variable may be obtained as long as it indicates the tendency of the response category of the Likert scale item, without being limited to the variables described below.

The first variable is the number of times an answer category was selected with a shift of one for each consecutive Likert scale item of the answer data Di. That is, this variable is the number of times the response categories for consecutive Likert scale items were selected in a zigzag manner. Specifically, when the answer to a certain question item is No. 3 out of 5 options, if the answer to the previous question item is No. 2 or 4, it is counted as once. In this way, the number of times counted up to the final response to the Likert scale item becomes the variable obtained here.

The second variable indicates whether or not the ratio of responses to b or less response categories to a Likert scale item having a response categories is equal to or greater than a predetermined value. Here, a is 2 or more and b is less than a.

With a predetermined value of 100%, this variable indicates whether or not only b or fewer response categories are answered. As a specific example, in the response data Di, only three response categories No. 2, No. 3 or No. 4 among five response categories No. 1 to No. 5 were answered for all Likert scale items. A case will be described.

If b=3, then this condition is met, so this variable is set to "1" to indicate that this condition is met. If b=2, this condition is not met, so this variable is set to "0" to indicate that this condition is not met.

Next, the predetermined value is 90%, and in response data Di, for 9 Likert scale items out of 10 Likert scale items, the 2nd out of 5 response categories from 1st to 5th , No. 3 or No. 4, and for the remaining one Likert scale item, the response category No. 1 will be explained.

If b=3, then this condition is met, so this variable is set to "1" to indicate that this condition is met. If b=2, this condition is not met, so this variable is set to "0" to indicate that this condition is not met.

The numerical values set in the variables may be reversed between "0" and "1", or numerical values other than these (for example, positive and negative numbers) may be used. Also, a plurality of variables may be provided corresponding to a plurality of conditions. For example, a total of four variables may be provided for a=3 and b=2 and a=4 and b=3 for the predetermined values of 100% and 90%, respectively.

The third variable is a total divergence value obtained by summing the divergence between the average of the response categories of each Likert scale item aggregated from a plurality of response data and the response category of each Likert scale item of the response data Di. For example, the average of the answer categories of each Likert scale item is calculated by the answer data aggregation unit 13, and other calculations are performed by the adequacy determination unit 14. FIG.

FIG. 3 is a schematic diagram showing various numerical values regarding the total divergence values of the respondents P1, P2, P3, P4, and P5 for each question Q1 to Q3 of the questionnaire.

The questionnaire includes three questions Q1, Q2, and Q3 as Likert scale items. For each of the questions Q1-Q3, there are five answer categories that are alternatives. Each answer category is digitized from 1 to 5 corresponding to the content that changes in stages.

First, the average of each question Q1 to Q3 is calculated. For question 1, the numerical values of the answer categories of each of the respondents P1 to P5 are 2, 3, 3, 4, and 4, respectively. Therefore, the average for question Q1 is 3.2. Similarly, the averages of questions Q1 and Q2 are 4 and 2.4, respectively.

Next, the divergence of the answer categories for each question Q1-Q3 of each respondent P1-P5 is calculated. The divergence is the absolute value of the difference between the average of questions Q1-Q3 and the numerical values of the answer categories of respondents P1-P5. The numerical value of the answer category for question Q1 of respondent P1 is 2, and the average of question Q1 is 3.2. The absolute value of 2-3.2 is 1.2, which is the deviation of answer category for question Q1 of respondent P1. In this way, the divergence of the answer categories for each question Q1-Q3 of each respondent P1-P5 is calculated.

The deviations of the answer categories for the questions Q1 to Q3 of the respondent P1 are 1.2, 1, and 0.4, respectively. The total divergence value of respondent P1 is the total divergence of the answer categories for each question Q1 to Q3, which is 2.6. In this way, the total divergence value of each respondent P1 to P5 is calculated.

The suitability determination unit 14 calculates the Mahalanobis distance for the Likert scale item of the answer data Di (step S104). Note that the suitability determination unit 14 may calculate the p-value of the Mahalanobis distance for the Likert scale item. Alternatively, the Mahalanobis distance may be calculated using a pair of Likert scale items having the highest correlation in the questionnaire, or the p-value of this Mahalanobis distance may be calculated. A pair of correlated Likert scale items is two Likert scale items whose question contents are related to each other.

Here, the Mahalanobis distance will be explained. The Mahalanobis distance is a statistic representing the distance between the observed data vector and the sample mean vector. The Mahalanobis distance is obtained by the following equation.

As shown in FIG. 3, when there are three Likert scale items (questions Q1 to Q3) in total, the three items are used to calculate the Mahalanobis distance of the answer data Di. xi in the above equation is a vector (2, 3, 2) where i is the respondent P1. The average vector becomes (3.2, 4, 2.4). The covariance matrix Σ is a covariance matrix (3×3) calculated from the 5×3 data matrix on the left in FIG. This calculation gives one Mahalanobis distance for each respondent P1-P5. The p-value of the Mahalanobis distance is the p-value of the test result for the magnitude of the Mahalanobis distance thus calculated.

The response data counting unit 13 obtains the representative value of the response time of all the response data in the survey from which the response data Di was obtained (step S105). The representative value is, for example, the median value, but may be the average value, the mode value, or any other statistic. Here, the representative value is explained as the median value. For example, assume that five people responded to a survey, and the response times were 10 minutes, 15 minutes, 12 minutes, 14 minutes, and 19 minutes. At this time, the median response time in this survey is 14 minutes. Therefore, the median response time for all response data in this survey is set to 14. Thus, the median response time is set to a different value for each survey. The answer data counting unit 13 transmits the obtained median answer time to the appropriateness determination unit 14 .

The suitability determination unit 14 acquires the number of question items of the questionnaire for which the answer data Di was acquired (step S106). For example, if the number of all question items written in a questionnaire for conducting a certain survey is 100 items, the number of question items of all answer data for this survey is set to 100 items. In this way, the number of question items is set to a different value for each survey.

The propriety judging section 14 judges whether or not the target answer data i is inappropriate by using various information including the numerical values and variables obtained above as explanatory variables (step S107). Explanatory variables may include attribute variables and information obtained from free-form response items. For example, the propriety judging section 14 determines a judging function for judging whether or not the answer data i is inappropriate, corresponding to the contents of the investigation. The propriety judging section 14 judges whether or not the answer data i is inappropriate by using the determined judging function with the explanatory variable as an argument.

Here, for explanatory variables, it is desirable to use values standardized within each survey data in order to obtain relative values within each survey data. For example, when using “response time” as an explanatory variable, the “response time” of the first survey uses the standardized value within the data of the first survey as the explanatory variable, and the “response time” of the second survey as the explanatory variable. The values standardized within the data of the survey in 2 are used as explanatory variables. Even if the first survey and the second survey had the same non-standardized response time, if the average response time was longer in the first survey, the standardized response time would be the second survey. survey will be larger. Similarly, for other variables, it is desirable to use standardized values as explanatory variables if standardization is possible.

The judgment function may be set in advance in the suitability judgment unit 14, or may be automatically created by the suitability judgment unit 14 for each investigation. Also, the explanatory variables, conditional expressions, etc. used in the judgment function may be arbitrarily changed for each investigation. Furthermore, the judgment function may be an artificially created program or the like, a program for executing a prediction model that has been learned by machine learning using a computer, or the like, or may be created through the intervention of both humans and computers. A program or the like may be used.

A method of constructing an inappropriate answer prediction model by machine learning will be described with reference to FIG. The inappropriate response prediction model is used as a determination function for determining whether or not the target response data i is inappropriate in the appropriateness determination unit 14 of the survey system 1 .

Conduct multiple studies to collect training data for machine learning. In this survey, the questionnaire includes instruction items in addition to the items for obtaining the explanatory variables described above. The instruction item is a question for confirming whether or not the respondent to the questionnaire will give an appropriate answer. For example, the content of the instruction item is "Please select option No. 4." On the other hand, if the respondent selects an option other than No. 4, the respondent can be recognized as an inappropriate respondent. This makes it possible to know who is an inappropriate respondent among the respondents of each survey.

Here, k surveys are conducted, and the sample size (number of respondents) of each survey is n. Note that the sample size may vary from survey to survey.

By performing k investigations, k investigation data SR1, SR2, . . . , SRk are obtained. Each survey data SR1 to SRk includes answer data for n persons. From each response data, explanatory variables such as gender, response time and Mahalanobis distance are created.

Next, for each survey, data matrices DM1, DM2, . The size of each data matrix DM1 to DMk is n×v, where v is the number of variables. Since there are k investigations, k n×v data matrices DM1 to DMk are created.

The data matrices DM1 to DMk created in this manner are stored in the training data storage unit ST3. The data of the data matrices DM1 to DMk stored in the training data storage unit ST3 are treated as training data for performing machine learning.

The machine learning unit 21 performs machine learning for constructing an inappropriate answer prediction model for predicting whether or not answer data is an inappropriate answer, based on the training data stored in the training data storage unit ST3. The machine learning unit 21 performs supervised learning using the variable created from the instruction item as data with a correct answer label.

The machine learning algorithm performed by the machine learning unit 21 is, for example, a random forest, but any machine learning may be performed. For example, the machine learning algorithm may be neural network, deep learning, support vector machine (SVM), decision tree, or the like, or any combination thereof.

In this manner, the machine learning unit 21 generates an inappropriate answer prediction model by performing machine learning. The generated inappropriate answer prediction model may be applied as it is as a judgment function of the propriety judging unit 14, may be used as a function of initial values on the premise that it will be updated by reinforcement learning, may be modified accordingly.

Note that the machine learning unit 21 and the training data storage unit ST3 may be part of the configuration of the survey system 1 or may be implemented in a computer that operates independently of the survey system 1 .

According to this embodiment, based on the response data of a survey including a plurality of Likert scale items, a numerical value indicating the tendency of the response data is obtained, and based on various information obtained from the response data including this numerical value, the response data By detecting that is inappropriate, the detection accuracy of inappropriate answers can be improved.

In addition, this embodiment can detect inappropriate responses by including multiple Likert scale items in a questionnaire for conducting a survey, so it is a highly versatile detection method that can be applied to various survey contents. .

Furthermore, this embodiment was realized by applying a prediction model built by machine learning, and when it was verified with response data obtained from various devices such as personal computers, smartphones, and tablets, inappropriate responses were found on all devices. It was demonstrated to be effective for detection. Therefore, this embodiment is a highly versatile detection method that does not depend on the device that responds.

(Second embodiment)
FIG. 5 is a flowchart showing the procedure for creating explanatory variables used in the inappropriate answer detection method according to the second embodiment.

The inappropriate answer detection method according to this embodiment is used when a questionnaire for conducting a survey includes at least one free description answer item in addition to a plurality of Likert scale items. A free description answer item is an item in which a respondent freely describes an answer to a question.

In the inappropriate answer detection method according to the present embodiment, in addition to explanatory variables used in the inappropriate answer detection method according to the first embodiment shown in FIG. Information obtained from responses to free-description items was used as an explanatory variable. Other points are the same as the inappropriate answer detection method according to the first embodiment.

The propriety determination unit 14 obtains the number of characters of the free description reply in the reply data Di (step S201). For example, the adequacy determination unit 14 counts the number of characters in all free description responses, and calculates the sum of the number of characters in each free description response. The required number of characters may be the number of characters of the free description response to the predetermined free description response item, or may be the total number of characters of each free description response to the predetermined free description response item.

The propriety determination unit 14 obtains the positive score of the free description response in the response data Di (step S202). A positive score is a score set for each positive word. For example, the positive score for an open-ended answer is the sum of the positive scores for all positive words in all open-ended answers, but It can be the total.

Here, the positive words and the positive scores of the words are stored in the dictionary set in the suitability determination unit 14 . The dictionary may be uniquely created for the survey system 1, may be a known dictionary, or may be a so-called polarity dictionary in which negative words and negative scores of the words are also stored.

The propriety determination unit 14 obtains the number of positive characters in the free description reply in the reply data Di (step S203). Specifically, the suitability determination unit 14 extracts all positive words included in all free description responses, and calculates the total number of characters of the extracted positive words.

The propriety determination unit 14 obtains the number of negative characters of all free description responses in the response data Di (step S204). Specifically, the suitability determination unit 14 extracts all negative words included in all free description responses, and calculates the total number of characters of the extracted negative words.

The propriety determination unit 14 obtains the positive score of the most positive word in the free description responses in the response data Di (step S205). Specifically, the propriety determination unit 14 extracts all positive words included in all free description responses, and obtains positive scores for all of the extracted positive words. Obtain the highest positive score among the obtained positive scores. This highest positive score becomes the positive score of the most positive word.

The propriety determination unit 14 obtains the negative score of the most negative word in the free description response in the response data Di (step S206). Specifically, the propriety determination unit 14 extracts all negative words included in all free description responses, and obtains negative scores for all of the extracted negative words. Obtain the highest negative score among the obtained negative scores. This highest negative score becomes the negative score of the most negative word.

The propriety determination unit 14 determines whether or not a free description response is given in the response data Di (step S207). If there are no answers (blanks) to all free-description reply items, it is determined that no free-description reply has been made. It should be noted that a judgment condition may be provided in advance for judging whether or not a free-description reply has been made. For example, if the answer is less than a predetermined number of characters and does not make sense as an answer, it may be determined that the free description answer has not been given. Note that if a certain percentage (for example, 90%) or more of all free-description response items are blank, the adequacy determination unit 14 may determine that no free-description response has been made.

The propriety judging section 14 sets "1" in the reply data Di when it judges that the free description reply is given, and sets "0" when it judges that it is not. As for the numerical values to be set, "0" and "1" may be set inversely, or numerical values other than these (for example, positive numbers and negative numbers) may be used.

The propriety determination unit 14 determines whether or not the answer data Di includes a specific sentence that does not contain any substantial content in the free description answer (step S208). For example, such a specific sentence may be "none", "none", "not in particular", or "none in particular", and may be partly or entirely in hiragana or katakana. By setting such a specific sentence in advance, the suitability determination unit 14 determines whether or not the free description response is answered with the specific sentence.

In addition, when the propriety determination unit 14 responds with a specific sentence to at least a certain percentage of items (for example, 80% of all items) with respect to a plurality of free description response items, at least one item has a specific sentence. It is also possible to determine whether or not a response is made with specific sentences by setting a condition such as when a response is made with sentences or when all items are answered with specific sentences.

The propriety judging section 14 sets "1" in the answer data Di when it judges that the answer is given in a specific sentence that does not contain substantial content, and sets it to "0" when it judges otherwise. set. As for the numerical values to be set, "0" and "1" may be set inversely, or numerical values other than these (for example, positive numbers and negative numbers) may be used.

The suitability determination unit 14 adds the numerical values obtained in steps S201 to 208 described above as explanatory variables, and, in the same manner as in step S107 of the inappropriate answer detection method according to the first embodiment shown in FIG. is inappropriate. Note that the propriety determination unit 14 may add at least one of the numerical values obtained in steps S201 to S208 described above as an explanatory variable.

According to this embodiment, in a survey that includes Likert scale items and free-description response items, by using explanatory variables obtained from responses to free-description response items, detection accuracy can be improved more than in the first embodiment. .

In addition, the present invention is not limited to the above-described embodiments, and components may be deleted, added, or changed. Also, a new embodiment may be created by combining or exchanging constituent elements of a plurality of embodiments. Even if such an embodiment is directly different from the above-described embodiments, those having the same meaning as the present invention have been described as embodiments of the present invention, and the description thereof has been omitted.

DESCRIPTION OF SYMBOLS 1... Survey system 2... Respondent terminal 11... Questionnaire generation part 12... Questionnaire response part 13... Answer data totalization part 14... Suitability judgment part 15... Input part 16... Output part Fm... Questionnaire form , ST1 -- Survey data storage unit, ST2 -- Response data storage unit.

Claims (19)

Retrieve multiple response data for surveys containing multiple Likert scale items,
Acquiring the response time required to respond to target response data from which inappropriate responses are to be detected among the plurality of response data;
Obtaining the number of times the same answer category was continuously selected for the plurality of Likert scale items of the target answer data;
obtaining a numerical value indicating a tendency of the target response data based on the response categories for the plurality of Likert scale items of the target response data;
calculating a plurality of Mahalanobis distances for the plurality of Likert scale items of the target response data;
calculating a representative value of the response time required to respond to each of the plurality of response data;
obtain the number of items in the survey;
Detecting that the answer of the target answer data is inappropriate based on the answer time, the number of times, the numerical value indicating the tendency, the plurality of Mahalanobis distances, the representative value, and the number of items. ,
The numerical value indicating the tendency is the number of times the response category is selected by shifting one by one for the continuous Likert scale items of the target response data.
An inappropriate answer detection method characterized by : Retrieve multiple response data for surveys containing multiple Likert scale items,
Acquiring the response time required to respond to target response data from which inappropriate responses are to be detected among the plurality of response data;
Obtaining the number of times the same answer category was continuously selected for the plurality of Likert scale items of the target answer data;
obtaining a numerical value indicating a tendency of the target response data based on the response categories for the plurality of Likert scale items of the target response data;
calculating a plurality of Mahalanobis distances for the plurality of Likert scale items of the target response data;
calculating a representative value of the response time required to respond to each of the plurality of response data;
obtain the number of items in the survey;
Detecting that the answer of the target answer data is inappropriate based on the answer time, the number of times, the numerical value indicating the tendency, the plurality of Mahalanobis distances, the representative value, and the number of items. ,
The numerical value indicating the tendency indicates whether or not the ratio of responses to only b number of response categories to a Likert scale item having a number of response categories in the target response data is equal to or greater than a predetermined value, and a is 2 or more and b is less than a
An inappropriate answer detection method characterized by : Retrieve multiple response data for surveys containing multiple Likert scale items,
Acquiring the response time required to respond to target response data from which inappropriate responses are to be detected among the plurality of response data;
Obtaining the number of times the same answer category was continuously selected for the plurality of Likert scale items of the target answer data;
obtaining a numerical value indicating a tendency of the target response data based on the response categories for the plurality of Likert scale items of the target response data;
calculating a plurality of Mahalanobis distances for the plurality of Likert scale items of the target response data;
calculating a representative value of the response time required to respond to each of the plurality of response data;
obtain the number of items in the survey;
Detecting that the answer of the target answer data is inappropriate based on the answer time, the number of times, the numerical value indicating the tendency, the plurality of Mahalanobis distances, the representative value, and the number of items. ,
The numerical value indicating the tendency is a total deviation value obtained by summing the deviations between the average of the response categories of each Likert scale item of the plurality of response data and the response category of each Likert scale item of the target response data.
An inappropriate answer detection method characterized by : The survey includes an open-ended response item,
4. The inappropriateness of the answer of the target answer data is detected based on the number of characters of the free description answer to the free description answer item of the target answer data. The inappropriate answer detection method according to item 1 . The survey includes an open-ended response item,
2. Inappropriateness of the answer of the target answer data is detected based on a positive score of a positive word included in the free description answer to the free description answer item of the target answer data. 4. The inappropriate answer detection method according to any one of claims 3 to 4 . 6. The inappropriate answer detection method according to claim 5 , wherein said positive score is the highest positive score among said free description answers. The survey includes an open-ended response item,
Inappropriateness of the answer of the target answer data is detected based on the number of positive characters, which is the number of characters of a positive word included in the free description answer to the free description answer item of the target answer data. The inappropriate answer detection method according to any one of claims 1 to 3 . The survey includes an open-ended response item,
2. Inappropriateness of an answer of said target answer data is detected based on a negative score of a negative word included in a free description answer to said free description answer item of said target answer data. 4. The inappropriate answer detection method according to any one of claims 3 to 4 . 9. The inappropriate answer detection method according to claim 8 , wherein said negative score is the highest negative score among said free description answers. The survey includes an open-ended response item,
Inappropriateness of the answer of the target answer data is detected based on the number of negative characters, which is the number of characters of negative words included in the free description answer to the free description answer item of the target answer data. The inappropriate answer detection method according to any one of claims 1 to 3 . The survey includes an open-ended response item,
Claims 1 to 3, characterized in that it is detected that an answer of said target answer data is inappropriate based on whether or not a free description answer has been given to said free description answer item of said target answer data. Item 4. The inappropriate answer detection method according to any one of items 3 . The survey includes an open-ended response item,
Detecting that the answer of the target answer data is inappropriate based on whether or not the answer to the free description answer item of the target answer data is answered with a specific sentence that does not contain substantial content. 4. The inappropriate answer detection method according to any one of claims 1 to 3 . 4. The inappropriateness according to any one of claims 1 to 3, comprising detecting inappropriateness of the answer of the target answer data by using a model trained by machine learning. Answer detection method. response data storage means for storing multiple response data of a survey including multiple Likert scale items;
an answer time acquisition means for acquiring an answer time required to answer target answer data for detecting an inappropriate answer among the plurality of answer data stored in the answer data storage means;
a frequency calculation means for calculating the number of consecutive selections of the same answer category for the plurality of Likert scale items of the target answer data;
response tendency calculation means for calculating a numerical value indicating a tendency of the target response data based on the response categories of the target response data for the plurality of Likert scale items;
Mahalanobis distance calculation means for calculating a plurality of Mahalanobis distances for the plurality of Likert scale items of the target answer data;
a representative value calculation means for calculating a representative value of the response time required for each response of the plurality of response data;
an item number acquiring means for acquiring the number of items of the survey;
the answer time acquired by the answer time acquisition means, the number of times calculated by the number of times calculation means, the numerical value indicating the tendency calculated by the answer trend calculation means, the plurality calculated by the Mahalanobis distance calculation means , the representative value calculated by the representative value calculating means, and the number of items acquired by the number of items acquiring means, it is detected that the answer of the target answer data is inappropriate. a detection means,
The numerical value indicating the tendency is the number of times the response category is selected by shifting one by one for the continuous Likert scale items of the target response data.
An inappropriate answer detection system characterized by : response data storage means for storing multiple response data of a survey including multiple Likert scale items;
an answer time acquisition means for acquiring an answer time required to answer target answer data for detecting an inappropriate answer among the plurality of answer data stored in the answer data storage means;
a frequency calculation means for calculating the number of consecutive selections of the same answer category for the plurality of Likert scale items of the target answer data;
response tendency calculation means for calculating a numerical value indicating a tendency of the target response data based on the response categories of the target response data for the plurality of Likert scale items;
Mahalanobis distance calculation means for calculating a plurality of Mahalanobis distances for the plurality of Likert scale items of the target answer data;
a representative value calculation means for calculating a representative value of the response time required for each response of the plurality of response data;
an item number acquiring means for acquiring the number of items of the survey;
the answer time acquired by the answer time acquisition means, the number of times calculated by the number of times calculation means, the numerical value indicating the tendency calculated by the answer trend calculation means, the plurality calculated by the Mahalanobis distance calculation means , the representative value calculated by the representative value calculating means, and the number of items acquired by the number of items acquiring means, it is detected that the answer of the target answer data is inappropriate. a detection means,
The numerical value indicating the tendency indicates whether or not the ratio of responses to only b number of response categories to a Likert scale item having a number of response categories in the target response data is equal to or greater than a predetermined value, and a is 2 or more and b is less than a
An inappropriate answer detection system characterized by : response data storage means for storing multiple response data of a survey including multiple Likert scale items;
an answer time acquisition means for acquiring an answer time required to answer target answer data for detecting an inappropriate answer among the plurality of answer data stored in the answer data storage means;
a frequency calculation means for calculating the number of consecutive selections of the same answer category for the plurality of Likert scale items of the target answer data;
response tendency calculation means for calculating a numerical value indicating a tendency of the target response data based on the response categories of the target response data for the plurality of Likert scale items;
Mahalanobis distance calculation means for calculating a plurality of Mahalanobis distances for the plurality of Likert scale items of the target answer data;
a representative value calculation means for calculating a representative value of the response time required for each response of the plurality of response data;
an item number acquiring means for acquiring the number of items of the survey;
the answer time acquired by the answer time acquisition means, the number of times calculated by the number of times calculation means, the numerical value indicating the tendency calculated by the answer trend calculation means, the plurality calculated by the Mahalanobis distance calculation means , the representative value calculated by the representative value calculating means, and the number of items acquired by the number of items acquiring means, it is detected that the answer of the target answer data is inappropriate. a detection means,
The numerical value indicating the tendency is a total deviation value obtained by summing the deviations between the average of the response categories of each Likert scale item of the plurality of response data and the response category of each Likert scale item of the target response data.
An inappropriate answer detection system characterized by : A computer obtains multiple response data for a survey containing multiple Likert scale items,
the computer acquires a response time required to respond to target response data for which an inappropriate response is to be detected from among the plurality of response data;
obtaining the number of times the computer has continuously selected the same answer category for the plurality of Likert scale items of the target answer data;
the computer obtains a numerical value indicating a tendency of the target response data based on the response categories for the plurality of Likert scale items of the target response data;
The computer calculates a plurality of Mahalanobis distances for the plurality of Likert scale items of the target response data,
The computer calculates a representative value of the response time required to respond to each of the plurality of response data;
the computer obtains the number of items in the survey;
The computer detects that the answer of the target answer data is inappropriate based on the answer time, the number of times, the numerical value indicating the tendency, the plurality of Mahalanobis distances, the representative value, and the number of items. function to include
The numerical value indicating the tendency is the number of times the response category is selected by shifting one by one for the continuous Likert scale items of the target response data.
Inappropriate answer detection program characterized by . A computer obtains multiple response data for a survey containing multiple Likert scale items,
the computer acquires a response time required to respond to target response data for which an inappropriate response is to be detected from among the plurality of response data;
obtaining the number of times the computer has continuously selected the same answer category for the plurality of Likert scale items of the target answer data;
the computer obtains a numerical value indicating a tendency of the target response data based on the response categories for the plurality of Likert scale items of the target response data;
The computer calculates a plurality of Mahalanobis distances for the plurality of Likert scale items of the target response data,
The computer calculates a representative value of the response time required to respond to each of the plurality of response data;
the computer obtains the number of items in the survey;
The computer detects that the answer of the target answer data is inappropriate based on the answer time, the number of times, the numerical value indicating the tendency, the plurality of Mahalanobis distances, the representative value, and the number of items. function to include
The numerical value indicating the tendency indicates whether or not the ratio of responses to only b number of response categories to a Likert scale item having a number of response categories in the target response data is equal to or greater than a predetermined value, and a is 2 or more and b is less than a
Inappropriate answer detection program characterized by . A computer obtains multiple response data for a survey containing multiple Likert scale items,
the computer acquires a response time required to respond to target response data for which an inappropriate response is to be detected from among the plurality of response data;
obtaining the number of times the computer has continuously selected the same answer category for the plurality of Likert scale items of the target answer data;
the computer obtains a numerical value indicating a tendency of the target response data based on the response categories for the plurality of Likert scale items of the target response data;
The computer calculates a plurality of Mahalanobis distances for the plurality of Likert scale items of the target response data,
The computer calculates a representative value of the response time required to respond to each of the plurality of response data;
the computer obtains the number of items in the survey;
The computer detects that the answer of the target answer data is inappropriate based on the answer time, the number of times, the numerical value indicating the tendency, the plurality of Mahalanobis distances, the representative value, and the number of items. function to include
The numerical value indicating the tendency is a total deviation value obtained by summing the deviations between the average of the response categories of each Likert scale item of the plurality of response data and the response category of each Likert scale item of the target response data.
Inappropriate answer detection program characterized by .

Images