JP7012696B2 - Information processing equipment and information processing method - Google Patents

Description

The present invention relates to an information processing apparatus and an information processing method for generating a trained model by learning based on a plurality of data.

Conventionally, attempts have been made to utilize artificial intelligence (AI), and research fields are also diverse (see Patent Document 1). Attempts are being made to utilize artificial intelligence, including autonomous driving, targeting in the marketing field, credit in the financial field, and personnel fields. Most of the current artificial intelligence (AI) is often provided in a black box type to users who have actual domain knowledge (for example, the person in charge of a company), and AI users say "How is AI?" It is difficult to directly understand "Did you make the decision?"

When providing a product that utilizes artificial intelligence, the user may have doubts about the conclusion of artificial intelligence or may want to know the process of drawing a conclusion. In this case, the AI developer often analyzes the generated model programmatically and then organizes and explains or presents it to the user. It should be noted that users often lack knowledge and skills regarding artificial intelligence, and it is generally difficult for users themselves to analyze models.

In addition, the above analysis by AI developers can only be realized if the AI developer and the user are close to each other, and it is not always possible to analyze in all cases, and it is a service for consumers, not for companies. Then it is not realistic. In addition, it may take a lot of time for AI developers to properly communicate with users and perform analysis.

In addition, when the model generated by artificial intelligence is disclosed to the user, the user may feel uncomfortable with the components of the model and want to improve it. On the other hand, in order to eliminate this discomfort, it is common to directly modify the data or the program in the past, which requires a great deal of skill, time (man-hours), and cost.

Japanese Patent Application Laid-Open No. 10-207504

The present invention provides an information processing apparatus and an information processing method capable of obtaining a model with less discomfort for the user or a conclusion with less discomfort by the user's own operation.

The information processing apparatus according to the present invention is
A learning unit that trains based on multiple data to generate a trained model,
An output unit that outputs the trained model and
A modification unit that modifies the trained model based on the modification information input to the trained model output by the output unit to generate a modified model, and a modification unit.
May be provided.

The information processing apparatus according to the present invention is
The modified model is further equipped with an evaluation unit that evaluates using past actual data.
The output unit may output the evaluation result by the evaluation unit.

In the information processing apparatus according to the present invention
The evaluation unit may evaluate the modified model in real time when the modification information is input.

The information processing apparatus according to the present invention is
Further provided with a data dividing unit for dividing a plurality of the above data,
A part of the data divided by the data division unit is used for learning in the learning unit.
The rest of the data divided by the data division unit may be used for evaluation by the evaluation unit.

In the information processing apparatus according to the present invention
The learning unit generates the learning model using a plurality of adopted variables and coefficients for each adopted variable.
The correction unit may make it possible to correct the magnitude of the coefficient.

In the information processing apparatus according to the present invention
The learning unit generates the learning model using a plurality of adopted variables and coefficients for each adopted variable.
The correction unit may make it possible to correct whether or not the adopted variable is used.

The information processing apparatus according to the present invention is
Further equipped with a standardization unit for standardizing a plurality of the above data,
The learning unit may calculate the coefficient using standardized data.

The information processing apparatus according to the present invention is
Anomaly detection units may be further provided to detect that information that should not be used is used when the trained model or the modified model is generated.

The information processing method according to the present invention
The process of performing machine learning on multiple data to generate a trained model,
The process of outputting the trained model and
The process of inputting the correction information for the trained model output by the output unit, and
A process of modifying the trained model based on the input modification information to generate a modified model, and
May be provided.

According to one aspect of the present invention, the trained model generated by the learning unit is output, and the modified information for the output trained model is input to generate the modified model. Therefore, it is possible to obtain a model with less discomfort for the user or a conclusion with less discomfort by the user's own operation.

FIG. 1 is a block diagram for explaining an outline of a system including an information processing apparatus according to an embodiment of the present invention. FIG. 2 is a diagram showing an aspect of generating a trained model in an information processing apparatus that can be used in the embodiment of the present invention. FIG. 3 is a diagram for showing the relationship between the information input to the prediction unit and the information output from the prediction unit that can be used in the embodiment of the present invention. FIG. 4 is a diagram for explaining an information processing process in an information processing apparatus that can be used in the embodiment of the present invention. FIG. 5 is a diagram showing a display mode on a display screen that can be used in the embodiment of the present invention.

Embodiment << Configuration >>
In the present embodiment, the following description will be made using a mode in which artificial intelligence is used for recruitment for applicants who wish to join a certain company, government office, etc., but the invention is not limited to this, and the invention according to the present application is described. It can be used in various fields such as targeting in the marketing field and credit in the financial field.

In the present embodiment, the information processing apparatus 1 will be described by using a mode in which the information processing apparatus 1 is installed in an external organization different from the recruitment support organization, but the present invention is not limited to this. For example, the information processing device may be installed in the recruitment support organization, and in this case, the recruitment support organization may own or manage the information processing device. Further, a user device 100 such as a personal computer, a tablet, or a smartphone owned or managed by the user may constitute an information processing device. The information processing device of the present embodiment may be composed of one device or a plurality of devices. Further, when the information processing device is composed of a plurality of devices, each device does not have to be provided in the same space such as the same room, but may be provided in a different room, a different building, a different area, or the like. When the information processing device is composed of a plurality of devices, a part of the information processing device may be owned and / or managed by a recruitment support organization, and the rest may be owned and / or managed by an external organization. In this embodiment, "and / or" may be used for explanation, but the word "or" also includes the meaning of "and". That is, the word A or B is a concept including A alone, B alone, and A and B.

As shown in FIG. 1, the information processing device 1 has a device storage unit 40 that stores various information, a device control unit 30 that performs various controls, and a device output unit 45 that outputs various information. You may. The device control unit 30 may include a learning unit 10, an evaluation unit 20, an abnormality detection unit 21, and the like, which will be described later. The device output unit 45 may be connected to the device control unit 30, and various information may be output from the device output unit 45 in the form of electronic data, paper media, or the like. The information output by the device output unit 45 may be output to the user device 100 and the engine device 50 installed in the recruitment support organization.

The past data may include applicant information for past applicants, first evaluation information for the past applicants (described later), and selection results for the past applicants. The past data is for a plurality of applicants. The selection result of last year may be used as the past data, but the selection result is not limited to this, and the selection result of the past several years may be used as the past data. The selection result may include the results of each selection test such as a document selection test, a primary selection test, a secondary selection test, ..., An nth selection test (“n” is an integer), a final selection test, and the like. ..

The past data may include, for example, data in the target company that the applicant wishes to join. Further, the past data may include data in a company other than the target company (for example, a company in the same industry) in addition to the data in the target company or in place of the data in the target company.

Even if the engine device 50 provided in the recruitment support organization has an engine storage unit 70 that stores various information, an engine control unit 60 that performs various controls, and an engine output unit 75 that outputs various information. good. Information can be exchanged between the engine device 50 and the information processing device 1 via the Internet or the like (see FIG. 1), and the device control unit 30 performs information processing using the information stored in the engine storage unit 70. Alternatively, the information output from the device output unit 45 may be received by the engine control unit 60 and output by the engine output unit 75. The institutional storage unit 70 stores the login ID that each applicant has in the recruitment support organization, the password associated with the login ID, the applicant information associated with the login ID (described later), and the like. You may. Applicants may include mid-career applicants as well as new graduate applicants such as students. Further, the institutional storage unit 70 may store past data including past actual data and the like, and forecast data including, for example, application data for the current fiscal year for which no conclusion has been reached. The output unit of this embodiment includes a device output unit 45 and an engine output unit 75, and the electronic data output by the output unit is a personal computer, tablet, smartphone, etc. owned or managed by the user. It may be displayed on the display screen 110 or the like of the user apparatus 100 of the above. Hereinafter, the present embodiment will be described with reference to an embodiment in which the device output unit 45 outputs and displays on the display screen 110, but the present invention is not limited to this, and the engine output unit 75 outputs. The display may be performed on the display screen 110.

The information processing device 1 of the present embodiment has an artificial intelligence function, a learning unit 10 that performs learning based on a plurality of data to generate a trained model, and a device output unit 45 that outputs a trained model. , A modification unit 15 that modifies the trained model based on the modification information input to the trained model output by the device output unit 45 and generates a modified model may be provided.

In the user device 100, in addition to the display screen 110 described above, a user operation unit 120 constituting an input unit capable of inputting correction information, past data including past actual data and the like, and no conclusion have been reached, for example, this year. It may have a user storage unit 130 in which expected data including application data and the like are stored.

The information processing apparatus 1 may have an evaluation unit 20 that evaluates the modified model using past data. Further, the device output unit 45 may output the evaluation result by the evaluation unit 20. The user who received the evaluation result by the evaluation unit 20 may further input the correction information from the user operation unit 120, and in this case, the correction unit 15 may generate a new modified model. Such correction may be repeated while observing the evaluation result in the evaluation unit 20.

The information processing apparatus 1 may have a data dividing unit 5 that divides a plurality of data. When the data division unit 5 is provided in this way, a part of the data divided by the data division unit 5 is used for learning by the learning unit 10, and the rest of the data divided by the data division unit 5 is used as an evaluation unit. It may be used for the evaluation at 20. As an example, 50% to 90%, more specifically 60% to 80% (for example, 70%) of the plurality of data is used as training data used in the learning in the learning unit 10, and the data of the plurality of data is used. Of these, 50% to 10%, more specifically 40% to 20% (for example, 30%) may be used as evaluation data used for evaluation by the evaluation unit 20 (see FIG. 4). It is advantageous in that the accuracy of the trained model generated by the learning unit 10 can be improved by increasing the parameter of the training data.

The evaluation unit 20 may evaluate the high accuracy when the trained model and the modified model are used. The evaluation data is applied to the trained models generated by the learning unit 10, and the evaluation data is applied to one or more modified models generated by the modifying unit 15, and how much these models are. It may be evaluated whether it has accuracy. As an example, AUC (Area Under the Curve) may be used as a measure of accuracy (see FIG. 5).

The device output unit 45 may output the accuracy for the selected model so that the user can recognize the accuracy. Further, a plurality of threshold values may be provided, and the accuracy may be classified into extremely high, high, normal, low, extremely low, and the like for output. In the embodiment shown in FIG. 5, AUC is used as a measure of accuracy, and the value is 0.855, which is shown to be an “extremely high” value.

The learning unit 10 may generate a learning model using a plurality of adopted variables and coefficients for each adopted variable. Recruitment variables are based on information such as language ability, participation in internships, interpersonal contact ability, group leadership, humanities / science division, connection between reasons for application and specialty, and abundance of non-academic content on the entry sheet. It may be decided.

The learning unit 10 can adopt various models as the machine learning technique, for example, a logistics regression model, a random forest model, a tree model, and the like. When the logistics regression model is adopted, the pass rate P for each applicant is P =
It will be calculated by exp (t) / (1 + exp (t)) (however, t = a ₀ × x ₀ + a ₁ × x ₁ + a ₂ × x ₂ + ... an × x _n + b ₎ . In the learning unit 10, the _coefficients of a ₀ , a ₁ , ..., And an may be estimated by the maximum likelihood method using the training data. By the way, x ₀ , x ₁ , ..., X _n are unique values set for each applicant, and the _coefficients of a ₀ , a ₁ , ..., an are common to all applicants. Become.

When output by the device output unit 45, for example, "weight" may be used as a concept corresponding to a coefficient, and for example, "item" may be used as a concept corresponding to an adopted variable (see FIG. 5). In the present embodiment, the item and the adopted variable will be described using exactly the same information (variable), but the present invention is not limited to this, and the item and the adopted variable may be different information. When the item and the adopted variable are different in this way, if the value of the coefficient related to the item is corrected, the coefficient for a plurality of adopted variables is corrected. On the other hand, in the present embodiment, the coefficients and the weights will be described on the premise that they are different, but the present invention is not limited to this, and the coefficients and the weights may have exactly the same information (magnitude). When adopting an embodiment in which the coefficient and the weight are different concepts, the coefficient may be changed by a different value by changing the weight by "1".

If the item and the adopted variable are the same, when the weight value for one item is modified, only the coefficient for one adopted variable in the calculation formula (for example, the above-mentioned logistics regression model) is modified. Therefore, it is useful in that the calculation formula can be easily modified.

As described above, in the present embodiment, it is premised that the item and the adopted variable have exactly the same information (variable). Therefore, in the present embodiment, the item and the adopted variable are mainly referred to as a generic term in the following. This will be explained using "items".

Information about the coefficient may be modifiable by the user operation unit 120. As an example, as shown in FIG. 5, the size of the weight may be increased or decreased and may be changed by the user operation unit 120. The magnitude of the weight may be schematically output as, for example, an "adjustment amount" on the display screen 110 or the like, and the adjustment amount may be changed by the user operation unit 120.

The user operation unit 120 may be able to modify whether or not the item is used. As an example, as shown in FIG. 5, it becomes possible to select whether or not to adopt an item with a check box, and an item not checked with the check box may not be used at the time of model creation. The weight for items that are not used when creating the model is "0".

As shown in FIG. 1, a standardization unit 4 for standardizing data may be provided. In the standardization unit 4, the magnitude of the coefficient for each item is averaged. For example, a ₀ , a ₁ , ...
_-The weights of an may be ordered between variables. For the sake of intuitive understanding, past data including training data and evaluation data and prediction data may be standardized so as to have an average of 0 and a variance of 1. The learning unit 10 may calculate the coefficient for the item using the standardized data. For example, if "TOEIC (registered trademark)" is indicated by a numerical value with a maximum of 990 points and "decision power" is indicated by a numerical value with a maximum of 10 points, these values are converted into evaluation values and standardized, and the item ( It may be used to calculate the coefficient for the element). When such a standardization unit 4 is adopted, the amount of change in weight and the amount of change in coefficient for each item may have a corresponding relationship. That is, when the weight is changed by "1", the coefficient may be changed by the same value (for example, "0.15") regardless of which item it is related to. By adopting such an aspect, it is advantageous in that a modified model can be easily generated, and it is possible to capture the change of the weight "1" for each item with the same impact from the user's point of view. It is also beneficial in that it can be done.

As shown in FIG. 1, a prediction unit 25 may be provided to make a prediction using data for which a conclusion has not been reached, instead of actual data in the past. Regarding employment information, for example, for data for new graduates this year, the prediction unit 25 predicts the prediction result by using the trained model or the modified model, and the device output unit 45 outputs the prediction result. You may. In this case, the expected result will be output to each of the plurality of applicants.

Anomaly detection units 21 may be provided to detect that information that should not be used is being used when the trained model or the modified model is generated. The user may perform items that should be used in the past data stored in the user storage unit 130 or the institutional storage unit 70, and the user may use information that should not be originally used as training data. be. In this case, it is possible that a particular item is adopted with an unusually large weight. The abnormality detection unit 21 may detect the possibility that information that should not be originally used is used by detecting an item having an abnormal value. This detection result may be output by the device output unit 45 so that the user can recognize it.

As an example, the only item that can be used to predict the result of process A (for example, document screening) is originally the data of the process before process A in the time series. However, if the input data is not defined when using the training data, or if the input data is defined incorrectly, the data of the process (for example, interview information) later in the time series than the process A is used and the trained model is used. Alternatively, a modified model may be generated. In such a case, the model is generated using information that is not originally known, and the accuracy of the model is reduced. In the example given earlier, an applicant who has an interview score means an applicant who has passed the document screening, and the coefficient or weight for the item is extremely high. In this regard, according to the abnormality detection unit 21, by detecting an obvious abnormal value, it is possible to detect that there is a high possibility that information that should not be used is being used. When a clear abnormal value is detected, the information may be emphasized and output by the device output unit 45. As an example, an item that is likely to cause an abnormal value may be highlighted on the display screen 110. Further, a threshold value may be set, and the device control unit 30 may determine that the value is an abnormal value when a coefficient or a weight is set to a value equal to or higher than the threshold value. In the mode shown in Fig. 5, the message "Please check the following warning" is displayed, and "As a result of learning, the weight of the variable in red is very high. Depending on the selection result. Check if it is a variable that changes or is a variable that can be obtained only after selection, and if necessary, uncheck it and change it to "Do not use". Is displayed on the display screen 110.

Next, the configuration used when generating the trained model will be described with more specific examples.

As shown in FIG. 1, the learning unit 10 has a first learning unit 11 that generates first evaluation information for each predetermined factor based on a document created by the applicant, and an applicant information and a first regarding the applicant. It may have a second learning unit 12 that generates a second evaluation information of the applicant based on the first evaluation information from the learning unit 11.

As training data and evaluation data used when generating a model, actual data older than a certain period (for example, 5 years) may be deleted. In this way, when a trained model is generated based only on actual data within a certain period (for example, within 5 years), it is advantageous in that a highly accurate trained model can be generated using relatively new actual data. Is.

Various factors can be mentioned as the "factor" used in the first evaluation information, for example, various factors such as communication ability, enthusiasm, logic, expertise, leadership, and solidity (can be mentioned). See "First Evaluation Information" in FIG. 3). The content of this factor may be changed as appropriate, and the number of factors may be changed as appropriate. The content of the factor may be changed by input from the device operation unit 35 provided in the information processing device 1, for example.

The document created by the applicant may be, for example, an entry sheet created by the applicant. This entry sheet may be created on the WEB and stored in an institutional storage unit 70 such as a server. The second evaluation information may be information indicating the possibility that the selection for the applicant is prioritized (selection priority information) (see FIG. 3). This selection priority information may include information regarding the possibility that the applicant will pass a selection test such as a document selection test, a primary selection test, a secondary selection test, ..., Nth selection test, and a final selection test.

The first learning unit 11 may generate first evaluation information for each factor by using a word used in a document such as an entry sheet and a predetermined factor.

The first learning unit 11 may generate the first evaluation information for each factor by using the frequency of use of the word used in the document such as the entry sheet and the similarity between the word and the factor.

The first learning unit 11 has an artificial intelligence function, and creates a feature vector corresponding to each factor from the closeness between words used in a large amount of text data (for example, between words used on the WEB). You may. Then, the first learning unit 11 may calculate the degree of similarity between the word used in the document such as the entry sheet and each factor by using the feature vector. Further, the first learning unit 11 may create an AI dictionary that stores the closeness of each factor and the word and / or stores the feature vector corresponding to each factor. The closeness of a word is determined by the artificial intelligence function of the first learning unit 11, for example, by estimating from the usage of the word (the context of the word, etc.) on a site on the WEB (including an SNS site, etc.). It may be (learned) and updated at any time or as appropriate. Information about word proximity may be stored in the device storage unit 40 and / or the institutional storage unit 70.

The first learning unit 11 may mine the closeness between words used on the WEB. The first learning unit 11 picks up a plurality of words related to each factor from the Big Text Data (creates a classification vocabulary table), and at the same time, the closeness between each factor and the picked up words. Then, a feature vector related to each factor may be created. As an example, when the factor "enthusiasm" is adopted, words such as vitality, aspiration, zeal, motivation, and hard work are picked up via the Internet as words close to "enthusiasm" (create a classification vocabulary table). ), "Enthusiasm" may be used to create a feature vector related to "enthusiasm" from the closeness between these words such as vitality, aspiration, zeal, motivation, and hard work.

Then, the first learning unit 11 may generate the similarity between the words used in the document such as the entry sheet and each factor by using the feature vector created in this way. At this time, even if the importance of the word in the document is generated from the frequency of use of the word used in the document such as the entry sheet, and the first evaluation information for each factor is generated based on the result. good. When determining the importance, consider whether the word is a commonly used word, and reduce the importance of words such as "particle" and "me" (for example, "0"), and use them less often. Words may be more important.

When determining the importance, documents such as entry sheets of other applicants who wish to be adopted by the target company may also be considered. For example, words that are frequently used in documents such as entry sheets of other applicants who wish to be adopted by the target company may be made less important, and words that are not used often may be made more important. .. Adopting such an aspect is useful in that relative comparisons between applicants can be made.

The above-mentioned past data may include applicant information for past applicants, first evaluation information for the past applicants, and recruitment refusal information or early retirement information for the past applicants. Recruitment decline information is information that the applicant has passed the selection test but declined the recruitment at the request of the applicant himself / herself. Early retirement information is information that an applicant has joined the target company but has retired from the target company within a predetermined period such as within 1 to 3 years. This "predetermined period" may be appropriately changed from the device operation unit 35.

Applicant information includes basic information including the applicant's educational background, applicant behavior information, inspection information including the results of the tests taken by the applicant, attribute information including information on the qualifications held by the applicant, and the applicant. It may contain any one or more of the textual information including the aspirational motive of (see FIG. 2). The applicant information may include all of these basic information, behavior information, inspection information, attribute information and text information.

Basic information may include information on majors, seminars, etc., in addition to the academic background of the applicant. The action information may include the number of times of participation in the briefing session (of the target company), the number of logins, information on the joint application destination, and the like. The inspection information may include the result of the WEB test, the result of the appropriate inspection, and the like. The attribute information may include the qualifications, skills, etc. possessed by the applicant. The textual information may include information on off-campus activities in addition to the motivation for aspiration.

The second learning unit 12 performs "learning" so as to generate highly accurate second evaluation information for the target applicant. Here, "learning" means that items (adopted variables) that match past actual data and their coefficients are determined by automatic trial and error. That is, in the second learning unit 12, information such as applicant information, first evaluation information, second evaluation information, etc. based on past data is aggregated, and the difference from the past actual data is the largest while changing the parameters. Repeated learning is performed so that it becomes smaller, and the items and coefficients to be applied are determined.

A trained model will be generated using the items and coefficients determined in this way, and what is generated by the user modifying this trained model is the modified model. Based on past performance, the trained model should match the most objective facts, but if the training data happens to be biased or if you want to adopt a tendency different from the past, after correction It will be beneficial to adopt the model. Further, by adopting such an embodiment, it is useful to be able to generate a modified model to be used in the future while considering past actual data.

When generating a trained model, a trained model may be generated for a person with a high personnel evaluation after hiring, taking into consideration the personnel evaluation after hiring. When such a trained model is generated, it is useful to be able to carry out recruitment activities in anticipation of success after joining the company.

The training data and the evaluation data are basically data in the same period, and are generated by randomly dividing the population, for example, at 7: 3, but the data is not limited to this. In some cases, it is possible to use data from different periods.

When making a prediction by the prediction unit 25, by selecting the target selection test from the device operation unit 35, the second evaluation information in the selection test is output from the device output unit 45 or the engine output unit 75. May be good. Further, by selecting the target selection test from the user operation unit 120, the second evaluation information in the selection test may be displayed on the display screen 110. In the embodiment shown in FIG. 3, as an example, the selection priority information which is the second evaluation information is shown by the number of stars, and it is shown that the larger the number of stars, the higher the possibility of passing the predetermined selection test. ..

In the present embodiment, a program for providing the information processing device 1 of the present embodiment, that is, a program for which the information processing device 1 of the present embodiment is manufactured (generated) by installing the program, and the program thereof. A storage medium with the installed is also provided.

"Method"
Next, an example of the information processing method of the present embodiment will be described with reference to FIG. In addition, since it overlaps with the above, only a brief explanation will be given, but all the embodiments described in the above "configuration" and all the embodiments described in the following "action / effect" can be applied in the "method". ..

First, the past data is standardized by the standardization unit 4 (standardization process).

Next, the standardized data is divided into training data and evaluation data at a fixed ratio (for example, 7: 3) by the data division unit 5.

Next, machine learning is performed on a plurality of data included in the training data to generate a trained model (trained model generation process).

When the trained model is generated in this way, the trained model is output by the device output unit 45, and the trained model is output by the display screen 110 of the user device 100. At this time, the evaluation unit 20 may also output the accuracy (for example, AUC) when the evaluation data is evaluated using the trained model (see FIG. 5).

Next, the user operates the user operation unit 120 of the user apparatus 100 to input correction information for the trained model.

When the correction information is input in this way, the trained model is modified based on the modification information input by the modification unit 15 to generate a modified model (modified model generation step). For example, the information on the coefficient (weight) may be corrected by adjusting the adjustment amount by the user operation unit 120, or the information on the item (element) may be corrected by determining whether or not the item is adopted by the user operation unit 120. You may modify it.

Next, the evaluation unit 20 evaluates the evaluation data using the modified model, and calculates the accuracy (for example, AUC) of the evaluation data.

When the accuracy for the modified model is calculated, the result is output by the device output unit 45, and the accuracy is displayed on the display screen 110 of the user device 100.

The user may input the correction information by looking at the content displayed on the display screen 110, or may determine the modified model without further inputting the correction information.

For each of the trained model and the modified model, the abnormality detection unit 21 may determine whether or not information that should not be used is used. When it is detected by the abnormality detection unit 21 that information that should not be used is being used, that fact may be output by the device output unit 45 and displayed on the display screen 110 (see FIG. 5). ).

When such a display is made, the user reviews the trained model or the modified model, and the user operation unit 120 deletes or defines the items used in the trained model or the modified model. You may modify it.

The forecasting unit 25 may make predictions for data for which no conclusion has been reached, such as recruitment information for this year. In this case, the expected results for each of the plurality of applicants may be predicted by using the trained model or the modified model. The predicted result may be output by the device output unit 45 and displayed on the display screen 110 (see FIG. 3).

《Action / Effect》
Next, the actions / effects of the present embodiment having the above-described configuration, which have not been described yet, will be mainly described.

In the present embodiment, when the mode in which the trained model generated by the learning unit 10 is output and the modified information for the output trained model is input to generate the modified model is adopted. , A model with less discomfort for the user or a conclusion with less discomfort can be obtained by the user's own operation. In theory, the trained model is the most accurate model. However, a model that does not always match the user's understanding, subjective evaluation, and existing knowledge may be generated. In this respect, by adopting this aspect, it is advantageous in that a modified model, which is a model in which the user's understanding and subjective evaluation are consistent, can be generated.

In addition, when using the modified model in this way, it is also useful, for example, when it is desired to adopt a tendency different from the past. In this case, it is also advantageous to increase the weight or coefficient for the item to be emphasized so that the modified model close to the desired model can be adopted and the prediction for each applicant can be made.

When the evaluation unit 20 evaluates the high accuracy when the trained model is used, it is useful in that it can recognize how accurate the trained model has. Further, when the evaluation unit 20 evaluates the high accuracy when the modified model is used, it is useful in that the accuracy of the modified model can be recognized. By making it possible to evaluate the accuracy in this way, it becomes possible to obtain the evaluation result by the evaluation unit 20 for the trained model and each modified model. Therefore, it is useful that the actual forecast (for example, the forecast for the recruitment information of this year) can show an index for deciding whether or not to use the modified model, and the user can easily judge.

When the mode in which the evaluation by the evaluation unit 20 is performed immediately (real time) is adopted, it is advantageous in that the weights and items can be corrected while viewing the evaluation result of the evaluation unit 20. That is, when the evaluation unit 20 immediately (real-time) evaluates the modified model when the correction information is input from the user operation unit 120 or the like, it is used while looking at the evaluation result in the evaluation unit 20. It is useful in that the modified model can be efficiently generated based on the human sense.

When the aspect of generating training data and evaluation data by dividing a plurality of data by the data dividing unit 5 is adopted (see FIG. 4), the data originally in the same aggregate is randomly divided and the training data is used. And it can be used as evaluation data. Therefore, it is useful in that a trained model can be generated using objective data without any arbitrariness. If, for some reason, the training data and the evaluation data are to be different populations, such as when it is desired to analyze a certain tendency, the data division unit 5 sets the training data according to the conditions input from the user operation unit 120, for example. It may be divided into evaluation data.

When the aspect in which the information on the weight or the coefficient can be modified by the user operation unit 120 is adopted (see FIG. 5), the magnitude of the coefficient for the item that the user wants to emphasize is increased or the coefficient is large for the item that the user wants to disregard. It is advantageous in that it is only necessary to make adjustments by reducing the coefficient, and it is possible to generate the modified model desired by the user with relatively simple operations while making relatively detailed settings.

Even when the user operation unit 120 adopts a mode in which whether or not the item (adopted variable) is used can be corrected (see FIG. 5), the user only has to select whether or not to use the item. It is useful in that the modified model desired by the person can be generated with a relatively simple operation.

By adopting a mode in which data is standardized by the standardization unit 4, the magnitude of the coefficient for each item can be averaged. Therefore, it is advantageous in that the factors to be considered can be considered without bias and machine learning can be performed efficiently. It is also beneficial in that the impact when changing the weight can be made uniform.

When the abnormality detection unit 21 is adopted, it is possible to prevent the modified model from being generated by using information that should not be used. Therefore, for example, it is advantageous in that the user selects to use the erroneous item, so that the modified model using the erroneous item can be prevented from being generated.

The description of the embodiment and the disclosure of the drawings described above are merely examples for explaining the invention described in the claims, and are described in the claims by the description of the above-described embodiments or disclosure of the drawings. The inventions made are not limited. In addition, it should be noted that the description of the scope of claims at the time of filing is only the scope of claims at the time of filing and can be changed as appropriate.

1 Information processing device 4 Standardization unit 5 Data division unit 10 Learning unit 11 First learning unit 12 Second learning unit 15 Correction unit 20 Evaluation unit 21 Anomaly detection unit 30 Device control unit 35 Device operation section 40 Device storage section 45 Device output section (Output section)
110 Display screen 120 User operation unit

Claims (9)

A learning unit that trains based on training data and generates a trained model,
A modification unit that modifies the trained model based on the modification information input to the trained model to generate a modified model,
An evaluation unit that evaluates the modified model using evaluation data different from the training data,
An output unit that outputs the evaluation result for the modified model by the evaluation unit, and an output unit.
Equipped with
An information processing device that can be modified so that at least one of the items used in the trained model is not used as the modification information by input from the user operation unit. The information processing apparatus according to claim 1, wherein the weights for the items used in the trained model can be adjusted. The information processing apparatus according to claim 1 or 2, wherein the definition of the item used in the modified model can be modified by input from the user operation unit. The information processing apparatus according to any one of claims 1 to 3, wherein the learning unit uses an item related to selection of an applicant as an item used in the trained model. The information processing apparatus according to any one of claims 1 to 4, further comprising an abnormality detection unit that detects that information that should not be used is being used. The information processing apparatus according to any one of claims 1 to 5, wherein the correction unit can repeatedly make corrections while observing the accuracy when the modified model is evaluated by the evaluation unit. The learning unit
The first learning department that generates the first evaluation information based on the document created by the applicant,
The second learning department that generates the second evaluation information of the applicant based on the applicant information about the applicant and the first evaluation information.
The information processing apparatus according to any one of claims 1 to 6. The process of learning based on training data and generating a trained model by the learning department,
A process of modifying the trained model based on the modification information input to the trained model by the modification unit to generate a modified model.
The process of evaluating the modified model by the evaluation unit using evaluation data different from the training data, and
The process of outputting the evaluation result for the modified model by the evaluation unit by the output unit, and
Equipped with
An information processing method that can be modified so that at least one of the items used in the trained model is not used as the modification information by input from the user operation unit. A program to be installed in an information processing device
The information processing device in which the program is installed
A learning function that trains based on training data to generate a trained model,
A modification function that modifies the trained model to generate a modified model based on the modification information input to the trained model, and
An evaluation function that evaluates the modified model using evaluation data different from the training data, and
An output function that outputs the evaluation result for the modified model by the evaluation function and
Equipped with
A program that can be modified so as not to use at least one of the items used in the trained model by input from the user operation unit as the modification information.

Images