JP6607885B2 - Information processing apparatus and information processing method - Google Patents

Description

  The present invention relates to an information processing apparatus and an information processing method for generating a learned model by performing learning based on a plurality of data.

  Conventionally, utilization of artificial intelligence (AI) has been attempted, and the research field has also been diversified (see Patent Document 1). Artificial intelligence has been tried, including autonomous driving, marketing targeting, financial credit, and personnel. Most of the current artificial intelligence (AI) is often provided in a black box type for users who have actual domain knowledge (for example, a person in charge of a company). It's difficult to understand directly.

  When a product using artificial intelligence is provided, a user may have a question about the conclusion of artificial intelligence or want to know the process of making a conclusion. In this case, the AI developer often analyzes the generated model from the program and then explains or presents it to the user in an organized manner. Note that users often lack knowledge and skills regarding artificial intelligence, and it is generally difficult for users themselves to analyze models.

  In addition, the above-mentioned analysis by the AI developer cannot be realized unless the AI developer and the user are close to each other, and the analysis is not necessarily possible in all cases. It's not realistic. In addition, it may take a long time for the AI developer to appropriately communicate with the user for analysis.

  In addition, when a model generated by artificial intelligence is disclosed to the user, the user may feel uncomfortable with the components of the model and want to improve the model. On the other hand, in order to eliminate this sense of incongruity, it is common to directly modify data or a program if it has been the case before, and it takes very much skill, time (man-hours), and cost.

JP-A-10-207504

  The present invention provides an information processing apparatus and an information processing method that allow a user to obtain a model with less discomfort for the user or a conclusion with less discomfort by the user's own operation.

An information processing apparatus according to the present invention includes:
A learning unit that performs learning based on a plurality of data and generates a learned model;
An output unit for outputting the learned model;
A correction unit that corrects the learned model based on correction information input to the learned model output from the output unit, and generates a corrected model;
May be provided.

An information processing apparatus according to the present invention includes:
The correction model further includes an evaluation unit that performs evaluation using past performance data,
The output unit may output an evaluation result by the evaluation unit.

In the information processing apparatus according to the present invention,
The evaluation unit may evaluate the corrected model in real time when the correction information is input.

An information processing apparatus according to the present invention includes:
A data dividing unit for dividing the plurality of data;
A part of the data divided by the data dividing unit is used for learning in the learning unit,
You may use the remainder of the data divided | segmented by the said data division part for the evaluation in the said evaluation part.

In the information processing apparatus according to the present invention,
The learning unit generates the learning model using a plurality of adopted variables and a coefficient for each adopted variable,
The correction unit may be capable of correcting the magnitude of the coefficient.

In the information processing apparatus according to the present invention,
The learning unit generates the learning model using a plurality of adopted variables and a coefficient for each adopted variable,
It may be possible to correct whether or not the employment variable is used by the correction unit.

An information processing apparatus according to the present invention includes:
A standardization unit for standardizing a plurality of the data;
The learning unit may calculate the coefficient using standardized data.

An information processing apparatus according to the present invention includes:
When the learned model or the corrected model is generated, an abnormality detection unit that detects that information that should not be used may be used.

An information processing method according to the present invention includes:
Performing machine learning on a plurality of data to generate a learned model;
Outputting the learned model; and
A step of inputting correction information for the learned model output by the output unit;
Correcting the learned model based on the input correction information to generate a corrected model;
May be provided.

  According to one aspect of the present invention, a learned model generated by the learning unit is output, and correction information for the output learned model is input to generate a corrected model. For this reason, a model with little discomfort for the user or a conclusion with less discomfort can be obtained by the user's own operation.

FIG. 1 is a block diagram for explaining an outline of a system including an information processing apparatus according to an embodiment of the present invention. FIG. 2 is a diagram showing an aspect of generating a learned model in the information processing apparatus that can be used in the embodiment of the present invention. FIG. 3 is a diagram for illustrating a relationship between information input to the prediction unit that can be used in the embodiment of the present invention and information output from the prediction unit. FIG. 4 is a diagram for explaining an information processing process in the information processing apparatus that can be used in the embodiment of the present invention. FIG. 5 is a diagram showing a display mode on a display screen that can be used in the embodiment of the present invention.

Embodiment << Configuration >>
In the present embodiment, the following description will be made using an aspect in which artificial intelligence is used in an application for applicants who wish to join a certain company, government office, etc., but the present invention is not limited to this, It can be used in various fields such as targeting in the marketing field and credit in the financial field.

  In the present embodiment, the information processing apparatus 1 is described using an aspect in which the information processing apparatus 1 is installed in an external organization different from the recruitment support organization. For example, the information processing apparatus may be installed in an employment support organization, and in this case, the employment support organization may own or manage the information processing device. In addition, the user device 100 such as a personal computer, a tablet, or a smartphone owned or managed by the user may constitute the information processing device. The information processing apparatus according to the present embodiment may be composed of a single device or a plurality of devices. Further, when an information processing apparatus is configured from a plurality of devices, each device does not need to be provided in the same space such as the same room, and may be provided in different rooms, different buildings, different areas, and the like. Further, when an information processing apparatus is configured from a plurality of apparatuses, a part of the information processing apparatus may be owned and / or managed by an employment support organization, and the rest may be owned and / or managed by an external organization. In this embodiment, “and / or” may be used for explanation, but the word “or” also includes the meaning of “and”. That is, the term A or B is a concept including A alone, B alone, and A and B.

  As illustrated in FIG. 1, the information processing apparatus 1 includes a device storage unit 40 that stores various types of information, a device control unit 30 that performs various types of control, and a device output unit 45 that outputs various types of information. May be. The device control unit 30 may include a learning unit 10, an evaluation unit 20, an abnormality detection unit 21, and the like which will be described later. A device output unit 45 may be connected to the device control unit 30, and various information may be output from the device output unit 45 in the form of electronic data, paper media, or the like. Information output by the device output unit 45 may be output to the user device 100 and the engine device 50 installed in the employment support organization.

  The past data may include applicant information for past applicants, first evaluation information (described later) for the past applicants, and selection results for the past applicants. The past data is for a plurality of applicants. As the past data, the selection result of last year may be used, but the selection result of the past several years may be used as the past data without being limited to this. The selection results may include the results of each selection test such as a document selection test, a primary selection test, a secondary selection test,..., An nth selection test (“n” is an integer), a final selection test, etc. .

  The past data may include, for example, data on a target company that the applicant wishes to join. The past data may include data in a company other than the target company (for example, a company in the same industry) in addition to the data in the target company or instead of the data in the target company.

  The engine device 50 provided in the recruitment support engine includes an engine storage unit 70 that stores various information, an engine control unit 60 that performs various controls, and an engine output unit 75 that outputs various information. Good. Information can be exchanged between the engine device 50 and the information processing device 1 via the Internet or the like (see FIG. 1), and the device control unit 30 performs information processing using information stored in the engine storage unit 70. Alternatively, the information output from the device output unit 45 may be received by the engine control unit 60 and output by the engine output unit 75. The institution storage unit 70 stores a login ID that each applicant has in the employment support organization, a password associated with the login ID, applicant information (described later) associated with the login ID, and the like. May be. Applicants may include mid-career applicants as well as new graduate applicants such as students. Further, the engine storage unit 70 may store past data including past performance data and the like, and predicted data including application data for the current fiscal year for which no conclusion has been made. The output unit of the present embodiment includes a device output unit 45 and an engine output unit 75, and electronic data output by the output unit is a personal computer, tablet, smartphone, etc. owned or managed by the user May be displayed on the display screen 110 of the user apparatus 100. In the present embodiment, description will be given below using a mode in which the device output unit 45 performs output and display on the display screen 110. However, the present invention is not limited to this, and the engine output unit 75 performs output. The display on the display screen 110 may be performed.

  The information processing apparatus 1 according to the present embodiment has an artificial intelligence function, performs learning based on a plurality of data and generates a learned model, and a device output unit 45 that outputs a learned model. And a correction unit 15 that corrects the learned model based on the correction information input to the learned model output by the device output unit 45 to generate a corrected model.

  In addition to the display screen 110 described above, the user device 100 includes a user operation unit 120 that constitutes an input unit that can input correction information, past data including past performance data, etc. You may have the user memory | storage part 130 by which the anticipation data containing application data etc. are memorize | stored.

  The information processing apparatus 1 may include an evaluation unit 20 that evaluates the corrected model using past data. Further, the device output unit 45 may output an evaluation result by the evaluation unit 20. The user who has received the evaluation result by the evaluation unit 20 may further input correction information from the user operation unit 120, and in this case, a new corrected model may be generated by the correction unit 15. Such correction may be repeatedly performed while viewing the evaluation result in the evaluation unit 20.

  The information processing apparatus 1 may include a data dividing unit 5 that divides a plurality of data. When the data dividing unit 5 is provided as described above, a part of the data divided by the data dividing unit 5 is used for learning in the learning unit 10 and the remaining part of the data divided by the data dividing unit 5 is used as an evaluation unit. It may be used for evaluation at 20. As an example, 50% to 90% of a plurality of data, more specifically 60% to 80% (for example, 70%) is used as training data used for learning in the learning unit 10, and a plurality of data Of these, 50% to 10%, more specifically, 40% to 20% (for example, 30%) may be used as evaluation data used for evaluation in the evaluation unit 20 (see FIG. 4). This is advantageous in that the accuracy of the learned model generated by the learning unit 10 can be increased by increasing the parameter of the training data.

  The evaluation unit 20 may evaluate the high accuracy when the learned model and the corrected model are used. The evaluation data is applied to the learned model generated by the learning unit 10, and the evaluation data is applied to one or more corrected models generated by the correction unit 15. It may be evaluated whether it has accuracy. As an example, AUC (Area Under the Curve) may be used as a measure of accuracy (see FIG. 5).

  The apparatus output unit 45 may output the accuracy of the selected model so that the user can recognize the accuracy. Also, a plurality of threshold values may be provided, and the accuracy may be classified and output such as extremely high, high, normal, low, and extremely low. In the embodiment shown in FIG. 5, AUC is used as a measure of accuracy, and the numerical value is 0.855, which indicates that the numerical value is “very high”.

  The learning unit 10 may generate a learning model using a plurality of adopted variables and a coefficient for each adopted variable. Employment variables are based on information such as language ability, presence / absence of internship, interpersonal contact ability, group leadership, classification of humanities / science, connection between reasons for applying and expertise, and abundant descriptions other than academic work in the entry sheet. It may be determined.

The learning unit 10 can employ various models as a machine learning technique. For example, a logistic regression model, a random forest model, a tree model, or the like can be employed. When the logistic regression model is adopted, the acceptance rate P for each applicant is P = exp (t) / (1 + exp (t)) (where t = a ₀ × x ₀ + a ₁ × x ₁ + a ₂ × x ₂ +... a _n × x _n + b). The learning section _{10, a 0, a 1,} ···, a coefficient of a _n, may be estimated by using the training data maximum likelihood. _{Incidentally, x 0, x 1, ···} , x n is the unique value set in each of the _{applicant, a 0, a 1, ···} , a common value coefficients of applicant a _n Become.

  When output by the apparatus output unit 45, for example, “weight” may be used as a concept corresponding to a coefficient, and “item” may be used as a concept corresponding to an adopted variable (see FIG. 5). In the present embodiment, description is made using an aspect in which the item and the adopted variable are exactly the same information (variable), but the present invention is not limited to this, and the item and the adopted variable may be different information. As described above, when the item and the adopted variable are different, when the value of the coefficient related to the item is modified, the coefficients for the plurality of adopted variables are modified. On the other hand, in the present embodiment, description will be made on the assumption that the coefficient and the weight are different from each other. However, the present invention is not limited to this, and the coefficient and the weight may be the same information (size). When adopting an aspect in which the coefficient and the weight are different concepts, the coefficient may be changed by a different value by changing the weight by “1”.

  In addition, when the item and the adopted variable are the same, when the weight value related to one item is corrected, only the coefficient related to one adopted variable in the calculation formula (for example, the above-described logistics regression model) is corrected. Therefore, it is advantageous in that the calculation formula can be easily corrected.

  As described above, in the present embodiment, it is assumed that the item and the adopted variable are exactly the same information (variable). Therefore, in the present embodiment, in the following, the term and the adopted variable will be mainly used as a generic term. This will be described using “items”.

  Information regarding the coefficient may be correctable by the user operation unit 120. As an example, as shown in FIG. 5, the weight may be increased or decreased or changed by the user operation unit 120. The magnitude of the weight may be schematically output as, for example, “adjustment amount” to the display screen 110 or the like, and the adjustment amount may be changed by the user operation unit 120.

  The user operation unit 120 may be able to correct the use of the item. As an example, as shown in FIG. 5, whether or not an item has been adopted can be selected with a check box, and an item that is not checked with a check box may not be used when creating a model. It should be noted that the weight for items not used at the time of model creation is “0”.

As shown in FIG. 1, a standardization unit 4 that standardizes data may be provided. In the standardization part 4, the magnitude | size of the coefficient with respect to each item is averaged. For _{example, a 0, a 1, ···} , may be combined orders between variable weights a _n. In order to make it easy to understand intuitively, the past data including the training data and the evaluation data and the prediction data may be standardized so as to have an average of 0 and a variance of 1. The learning unit 10 may calculate a coefficient for the item using the standardized data. For example, when “TOEIC (registered trademark)” is indicated by a numerical value of 990 points and “Decisiveness” is indicated by a numerical value of 10 points, these numerical values are converted into evaluation values and standardized. You may use for the calculation of the coefficient with respect to an element. When such a standardization unit 4 is employed, the relationship between the change amount of the weight for each item and the change amount of the coefficient may correspond. In other words, when the weight is changed by “1”, the coefficient may be changed by the same value (for example, “0.15”) regardless of which item is associated. By adopting such an aspect, it is beneficial in that a corrected model can be easily generated, and when viewed from the user, the change of the weight “1” for each item can be grasped with a similar impact. It is also useful in that it can be done.

  As shown in FIG. 1, a prediction unit 25 that performs prediction using data that has not been concluded instead of past actual data may be provided. Speaking of employment information, for example, for the data for new graduates this year, the prediction unit 25 uses the learned model or the corrected model to predict the prediction result, and the device output unit 45 outputs the prediction result. May be. In this case, an expected result is output to each of a plurality of applicants.

  When the learned model or the corrected model is generated, an abnormality detection unit 21 that detects that information that should not be used is used may be provided. Items that should be used in past data stored in the user storage unit 130 or the engine storage unit 70 may be performed by the user, and information that should not be used by the user may be used as training data. is there. In this case, a specific item may be adopted with an abnormally large weight. The abnormality detection unit 21 may detect the possibility that information that should not be used originally is used by detecting an item having an abnormal value. This detection result may be output by the device output unit 45 so that the user can recognize it.

  As an example, the only item that can be used to predict the result of process A (for example, document selection) is the data of the previous process in time series before process A. However, if the input data is not defined when using the training data or if the input data is defined by mistake, the data of the process (for example, interview information) later in the time series than the process A is used and the learned model Or a modified model may be generated. In such a case, a model is generated using information that cannot be originally known, and the accuracy of the model is lowered. In the example given above, an applicant who has an interview score means an applicant who has passed the document selection, and the coefficient or weight for the item becomes extremely high. In this regard, according to the abnormality detection unit 21, it is possible to detect that there is a high possibility that information that should not be used is used by detecting an obvious abnormal value. In addition, when an obvious abnormal value is detected, the information may be output by the device output unit 45 so as to be emphasized. As an example, an item that is likely to cause an abnormal value may be highlighted on the display screen 110. Alternatively, a threshold value may be set, and the apparatus control unit 30 may determine that the value is an abnormal value when a coefficient or weight is greater than or equal to the threshold value. In the form shown in FIG. 5, the message “Please check the following warnings” is displayed. “The learning result shows that the weight of the variable in red is very high. Check if it is a variable that changes or is only obtained after selection, and if necessary, uncheck it and change it so that it is not used. Is displayed on the display screen 110.

  Next, the configuration used when generating the learned model will be described with a more specific example.

  As shown in FIG. 1, the learning unit 10 includes a first learning unit 11 that generates first evaluation information for each predetermined factor based on a document created by the applicant, applicant information about the applicant, and first You may have the 2nd learning part 12 which produces | generates an applicant's 2nd evaluation information based on the 1st evaluation information from the learning part 11. FIG.

  As training data and evaluation data used when generating a model, performance data older than a certain period (for example, 5 years) may be deleted. In this way, when a learned model is generated only based on actual data within a certain period (for example, within five years), it is useful in that a highly accurate learned model can be generated using relatively new actual data. It is.

  There are various “factors” used in the first evaluation information, for example, various factors such as communication ability, enthusiasm, logic, expertise, leadership, and solidity ( (See “First Evaluation Information” in FIG. 3). The contents of this factor may be changed as appropriate, and the number of factors may be changed as appropriate. The content of the factor may be changed by, for example, an input from the device operation unit 35 provided in the information processing device 1.

  The document created by the applicant may be, for example, an entry sheet created by the applicant. This entry sheet may be created on the WEB and stored in the engine storage unit 70 such as a server. The second evaluation information may be information (selection priority information) indicating a possibility that selection for the applicant is prioritized (see FIG. 3). This selection priority information may include information on the possibility that the applicant passes a selection test such as a document selection test, a primary selection test, a secondary selection test,..., An n-th selection test, a final selection test.

  The first learning unit 11 may generate first evaluation information for each factor using a word used in a document such as an entry sheet and a predetermined factor.

  The first learning unit 11 may generate first evaluation information for each factor using the frequency of use of the word used in the document such as an entry sheet and the similarity between the word and the factor.

  The first learning unit 11 has an artificial intelligence function, and creates a feature vector corresponding to each factor from the proximity between words used in a large amount of text data (for example, between words used on the WEB). May be. And the 1st learning part 11 may calculate the similarity degree of the word currently used in documents, such as an entry sheet, and each factor using the said feature vector. The first learning unit 11 may create an AI dictionary that stores the proximity between each factor and a word and / or stores a feature vector corresponding to each factor. The closeness of the word is determined by inferring from the way the word is used (for example, the context of the word) on the WEB site (including the SNS site), for example, by the artificial intelligence function of the first learning unit 11. (Learned) may be updated as needed or as appropriate. Information on word proximity may be stored in the device storage unit 40 and / or the engine storage unit 70.

  The first learning unit 11 may mine the closeness between words used on the WEB. The first learning unit 11 picks up a plurality of words related to each factor from Big Text Data (creates a classification vocabulary table) and closes between each factor and the picked-up word. Then, a feature vector associated with each factor may be created. As an example, when the factor of “enthusiasm” is adopted, words such as vitality, aspiration, enthusiasm, motivation, hard work, etc. are picked up via the Internet as words close to “enthusiasm” (a classification lexicon is created. ), And a feature vector related to “enthusiasm” may be created based on the closeness between “enthusiasm” and words such as vitality, aspiration, enthusiasm, motivation, and hard.

  And the 1st learning part 11 may produce | generate the similarity degree of the word used in documents, such as an entry sheet, and each factor using the feature vector created in this way. At this time, the importance of the word in the document is generated from the frequency of use of the word used in the document such as the entry sheet, and the first evaluation information for each factor is generated based on the result. Good. When determining the importance level, consider whether the word is a commonly used word. For words such as “particle” and “I”, the importance level is set low (for example, “0”) and is not often used. The degree of importance may be increased for words.

  When determining the importance, documents such as entry sheets of other applicants who wish to be adopted by the target company may be considered. For example, words that are frequently used in documents such as entry sheets of other applicants that are desired to be adopted by the target company may be less important, and words that are not frequently used may be made more important. . Adopting such an aspect is advantageous in that a relative comparison can be made among applicants.

  The past data described above may include applicant information for past applicants, first evaluation information for the past applicants, and employment withdrawal information or early retirement information for the past applicants. Recruitment decline information is information that the applicant has passed the selection test but declined due to the applicant's application. The early retirement information is information that the applicant has entered the target company, but has retired from the target company within a predetermined period such as one to three years. This “predetermined period” may be changed as appropriate from the apparatus operation unit 35.

  Applicant information includes basic information including the applicant's educational background, applicant's behavior information, examination information including the results of the exams the applicant has received, attribute information including information on the qualifications held by the applicant, and applicants Any one or more of text information including the desired motive may be included (see FIG. 2). The applicant information may include all of these basic information, behavior information, inspection information, attribute information, and text information.

  The basic information may include information on majors and seminars in addition to the applicant's educational background. The action information may include information regarding the number of times of briefing session participation (of the target company), the number of times of log-in, and information on the applicants. The inspection information may include a WEB test result, a proper inspection result, and the like. The attribute information may include qualifications, skills, etc. possessed by the applicant. The text information may include information related to off-campus activities in addition to the motivation.

  The second learning unit 12 performs “learning” so as to generate the second evaluation information with high accuracy for the subject applicant. Here, “learning” means that items (adopted variables) that match past performance data and their coefficients are determined by automatic trial and error. That is, in the second learning unit 12, information such as applicant information, first evaluation information, and second evaluation information based on past data is aggregated, and the difference between the past performance data is the largest while changing parameters. Iterative learning is performed so that it becomes smaller, and the applied items and coefficients are determined.

  A learned model is generated using the items and coefficients determined in this way, but what is generated when the user corrects the learned model is the corrected model. The past model should match the most objective facts based on past results, but if the training data happens to be biased or if you want to adopt a trend that differs from the past, etc. It will be beneficial to adopt a model. In addition, by adopting such an aspect, it is beneficial to be able to generate a modified model to be used in the future while considering past performance data.

  When generating a learned model, it is also possible to generate a learned model for a person with a high personnel evaluation after hiring in consideration of personnel evaluation after hiring. When such a learned model is generated, it is beneficial in that recruitment activities can be performed in anticipation of success after joining the company.

  The training data and the evaluation data are basically data in the same period and are generated by randomly dividing the population, for example, 7: 3, but are not limited thereto. In some cases, it is possible to use data of different periods.

  When prediction is performed by the prediction unit 25, by selecting a target selection test from the device operation unit 35, second evaluation information in the selection test is output from the device output unit 45 or the engine output unit 75. Also good. Further, the second evaluation information in the selection test may be displayed on the display screen 110 by selecting the target selection test from the user operation unit 120. In the mode shown in FIG. 3, as an example, the selection priority information that is the second evaluation information is indicated by the number of stars, and the higher the number of stars, the higher the possibility of passing a predetermined selection test. .

  In the present embodiment, a program for providing the information processing apparatus 1 of the present embodiment, that is, a program for manufacturing (generating) the information processing apparatus 1 of the present embodiment by being installed, and the program A storage medium on which is installed is also provided.

"Method"
Next, an example of the information processing method of this embodiment will be described with reference to FIG. In addition, since it overlaps with the above, only a brief description will be given, but all aspects described in the above “configuration” and all aspects described in the following “action / effect” can be applied in the “method”. .

  First, past data is standardized by the standardization unit 4 (standardization process).

  Next, the standardized data is divided into training data and evaluation data by a data dividing unit 5 at a certain ratio (for example, 7: 3).

  Next, a learned model is generated by performing machine learning on a plurality of data included in the training data (learned model generation step).

  When the learned model is generated in this way, the learned model is output from the device output unit 45, and the learned model is output from the display screen 110 of the user device 100. At this time, the accuracy (for example, AUC) when the evaluation unit 20 evaluates the evaluation data using the learned model may be output (see FIG. 5).

  Next, when the user operates the user operation unit 120 of the user device 100, correction information for the learned model is input.

  When the correction information is input in this manner, the corrected model is corrected based on the correction information input by the correction unit 15 to generate a corrected model (modified model generation step). For example, the information regarding the coefficient (weight) may be corrected by adjusting the adjustment amount by the user operation unit 120, or the information regarding the item (element) may be determined by determining whether the item is adopted by the user operation unit 120. It may be corrected.

  Next, the evaluation unit 20 evaluates the evaluation data using the corrected model, and calculates the accuracy (for example, AUC) for the evaluation data.

  When the accuracy for the corrected model is calculated, the result is output by the device output unit 45, and the accuracy is displayed on the display screen 110 of the user device 100.

  The user may input the correction information by looking at the content displayed on the display screen 110, or may confirm the corrected model without further input of the correction information.

  For each of the learned model and the corrected model, the abnormality detection unit 21 may determine whether information that should not be used is used. When it is detected by the abnormality detection unit 21 that information that should not be used is used, that fact may be output by the device output unit 45 and displayed on the display screen 110 (see FIG. 5). ).

  When such a display is made, the user reviews the learned model or the corrected model, and deletes or defines an item used in the learned model or the corrected model by the user operation unit 120. It may be modified.

  The prediction unit 25 may make a prediction for data for which no conclusion has been made, such as employment information for this year. In this case, the predicted result for each of the plurality of applicants may be predicted by using the learned model or the corrected model. The prediction result may be output by the device output unit 45 and displayed on the display screen 110 (see FIG. 3).

《Action ・ Effect》
Next, operations and effects according to the present embodiment having the above-described configuration, which have not been described yet, will be mainly described.

  In the present embodiment, when the learned model generated by the learning unit 10 is output and the correction information for the output learned model is input, the modified model is generated. Therefore, it is possible to obtain a model with less discomfort for the user or a conclusion with less discomfort by the user's own operation. In theory, the trained model is the most accurate model. However, a model that does not necessarily match the user's understanding, subjective evaluation, or existing knowledge may be generated. By adopting this aspect, this aspect is advantageous in that a corrected model that is a model that is consistent with the user's understanding and subjective evaluation can be generated.

  In addition, when the corrected model is used in this way, it is also useful when, for example, it is desired to adopt a tendency different from the past. In this case, it is also advantageous in that it is possible to make predictions for individual applicants by increasing the weight or coefficient for the item to be regarded as important and adopting a corrected model close to the desired model.

  When the evaluation unit 20 evaluates the high accuracy when the learned model is used, it is advantageous in that it can recognize how much accuracy the learned model has. Further, when the evaluation unit 20 evaluates the high accuracy when the corrected model is used, it is beneficial in that it can recognize how much accuracy the corrected model has. By making it possible to evaluate the accuracy in this way, the evaluation result by the evaluation unit 20 can be obtained for the learned model and each corrected model. For this reason, an index for determining whether or not to use the corrected model can be shown in an actual forecast (for example, a forecast for recruitment information of the current year), which is advantageous in that the user can easily make a judgment.

  When an aspect in which the evaluation in the evaluation unit 20 is performed immediately (in real time) is adopted, it is advantageous in that the weights and items can be corrected while viewing the evaluation result of the evaluation unit 20. That is, when correction information is input from the user operation unit 120 or the like, when the evaluation unit 20 adopts an aspect in which the evaluation unit 20 evaluates the corrected model immediately (in real time), it is used while looking at the evaluation result in the evaluation unit 20 It is beneficial in that a corrected model can be generated efficiently while being based on the user's senses.

  In the case of adopting a mode in which a plurality of data is divided by the data dividing unit 5 to generate training data and evaluation data (see FIG. 4), the data originally in the same aggregate is randomly divided to obtain training data. And can be used as evaluation data. This is advantageous in that a learned model can be generated using objective data without arbitraryness. In addition, when it is desired to analyze a certain tendency, for example, when it is desired to make the training data and the evaluation data different from each other, for example, the data dividing unit 5 determines the training data and the evaluation data according to the conditions input from the user operation unit 120. You may make it divide | segment into evaluation data.

  When an aspect in which the information on the weight or coefficient can be corrected by the user operation unit 120 (see FIG. 5), the coefficient size for the item that the user wants to emphasize or the coefficient size for the item that the user wants to ignore is large. It is only necessary to make adjustments by reducing the size, and it is advantageous in that a corrected model desired by the user can be generated by a relatively simple operation while making relatively fine settings.

  Even when the user operation unit 120 adopts a mode in which the use / non-use of the item (adopted variable) can be corrected (see FIG. 5), the user only has to select the use / nonuse of the item. This is advantageous in that a modified model desired by the user can be generated by a relatively simple operation.

  By adopting a mode in which the data is standardized by the standardization unit 4, it is possible to average the magnitude of the coefficient for each item. For this reason, it is beneficial in that the elements to be considered can be considered without any bias and machine learning can be performed efficiently. It is also beneficial in that the impact when changing the weight can be made uniform.

  When the abnormality detection unit 21 is employed, it is possible to prevent generation of a corrected model by using information that should not be used. For this reason, for example, it is advantageous in that it is possible to prevent a corrected model using the wrong item from being generated by selecting the use of the wrong item by the user.

  The description of the above-described embodiment and the disclosure of the drawings are merely examples for explaining the invention described in the claims, and are described in the claims by the description of the above-described embodiments or the disclosure of the drawings. The invention made is not limited. It should be noted that the description of the scope of claims at the beginning of the application is only the scope of the rights required at the time of filing, and can be changed as appropriate.

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 4 Standardization part 5 Data division part 10 Learning part 11 First learning part 12 Second learning part 15 Correction part 20 Evaluation part 21 Abnormality detection part 30 Device control part 35 Device operation part 40 Device storage part 45 Device output part (Output part)
110 Display screen 120 User operation unit

Claims (8)

A learning unit that performs learning based on the training data and generates a learned model;
An output unit for outputting the learned model;
A correction unit that corrects the learned model based on correction information input to the learned model output from the output unit, and generates a corrected model;
An evaluation unit that evaluates the learned model and the corrected model using evaluation data different from the training data among past performance data;
With
Compared to the accuracy for the training data using the learned model, the accuracy for the training data using the determined modified model is reduced,
The output unit outputs the accuracy when the learned model is evaluated with evaluation data as an evaluation result for the learned model,
When the correction information is input, the evaluation unit evaluates the corrected model using evaluation data,
The output unit outputs the accuracy when the modified model is evaluated with evaluation data as an evaluation result for the modified model,
Enabling correction by the correction unit repeatedly while looking at the accuracy when the corrected model is evaluated in the evaluation unit,
An information processing apparatus , wherein the correction information can be corrected so as not to use an adopted variable used in the learned model by an input from a user operation unit . The output unit outputs an evaluation result by the evaluation unit,
The information processing apparatus according to claim 1 , wherein the evaluation unit evaluates the corrected model in real time when the correction information is input. A data division unit for dividing a plurality of data;
Using a part of the data divided by the data dividing unit as the training data,
The information processing apparatus according to claim 1 or 2, characterized by using the remainder of the divided data in the data dividing unit for evaluation in the evaluation unit. The learning unit generates the learning model using a plurality of adopted variables and a coefficient for each adopted variable,
Wherein the correction unit, the information processing apparatus according to any one of claims 1 to 3, characterized in that the magnitude of the coefficient is modifiable. The learning unit generates the learning model using a plurality of adopted variables and a coefficient for each adopted variable,
It becomes selectable Adoption of the adoption variable check box, according to the selection in the check box according to claim 1, characterized in that the presence or absence of use of the employed variables by the correcting unit is modifiable 5. The information processing apparatus according to any one of 4 . It further includes a standardization unit that standardizes multiple data,
The learning unit uses the standardized data processing apparatus according to any one of claims 4 or 5, characterized in that to calculate the coefficients. The abnormality detection part which detects that the information which should not be utilized is produced | generated when the said learned model or the said corrected model is produced | generated further, The any one of Claim 1 thru | or 6 characterized by the above-mentioned. The information processing apparatus according to claim 1. Performing machine learning on the training data to generate a learned model;
Outputting the learned model by an output unit;
A step of evaluating the learned model using evaluation data different from the training data among past performance data by the evaluation unit;
Outputting the accuracy when the learned model is evaluated with evaluation data as an evaluation result for the learned model by the output unit;
A step of inputting correction information for the learned model output by the output unit;
Correcting the learned model based on the input correction information to generate a corrected model;
When the correction information is input, the evaluation unit uses the evaluation data to evaluate the corrected model;
Outputting the accuracy when the modified model is evaluated with evaluation data by the output unit as an evaluation result for the modified model;
With
Compared to the accuracy for the training data using the learned model, the accuracy for the training data using the determined modified model is reduced,
Enabling correction by the correction unit repeatedly while looking at the accuracy when the corrected model is evaluated in the evaluation unit,
The information processing method characterized by being able to correct so that the adoption variable currently used by the said learned model may not be used by the input from a user operation part as said correction information .