JP6909686B2 - Generation device, generation method, and generation program - Google Patents

Description

The present invention relates to a generator, a generator, and a generator.

In recent years, with the rapid spread of the Internet, sales via the Internet through electronic commerce services and the like have become popular. For example, techniques are provided that provide bidding advice on the purchase of similar products from multiple different types of online auctions.

Japanese Unexamined Patent Publication No. 2003-281410

However, in the above-mentioned prior art, it may be difficult to appropriately generate a model corresponding to an index related to the user's behavior. For example, when the log information about a certain index is small, the amount of training data used for learning is insufficient, and it is difficult to appropriately generate a model corresponding to the index.

The present application has been made in view of the above, and an object of the present application is to provide a generation device, a generation method, and a generation program capable of appropriately generating a model corresponding to an index related to a user's behavior.

The generation device according to the present application inputs the content information acquired by the acquisition unit and the acquisition unit that acquires the content information related to the content into a model that outputs a score corresponding to an index related to the user's behavior regarding the content. It is characterized by including a generation unit that generates evaluation information about the model based on an output value output from the model and prediction information predicted from information different from the model.

According to one aspect of the embodiment, there is an effect that a model corresponding to an index related to the user's behavior can be appropriately generated.

FIG. 1 is a diagram showing an example of a generation process according to an embodiment. FIG. 2 is a diagram showing an example of distribution processing according to the embodiment. FIG. 3 is a diagram showing a configuration example of the generation system according to the embodiment. FIG. 4 is a diagram showing a configuration example of the generator according to the embodiment. FIG. 5 is a diagram showing an example of the first learning data storage unit according to the embodiment. FIG. 6 is a diagram showing an example of a prediction information storage unit according to the embodiment. FIG. 7 is a diagram showing an example of the second learning data storage unit according to the embodiment. FIG. 8 is a diagram showing an example of a model information storage unit according to the embodiment. FIG. 9 is a diagram showing an example of a user information storage unit according to the embodiment. FIG. 10 is a diagram showing an example of a log information storage unit according to an embodiment. FIG. 11 is a diagram showing an example of a product information storage unit according to the embodiment. FIG. 12 is a flowchart showing an example of the generation process according to the embodiment. FIG. 13 is a flowchart showing an example of the distribution process according to the embodiment. FIG. 14 is a diagram showing a change in the magnification based on the required time according to the embodiment. FIG. 15 is a hardware configuration diagram showing an example of a computer that realizes the function of the generator.

Hereinafter, the generator, the generation method, and the mode for carrying out the generation program (hereinafter referred to as “the embodiment”) according to the present application will be described in detail with reference to the drawings. It should be noted that this embodiment does not limit the generator, the generation method, and the generation program according to the present application. Further, in each of the following embodiments, the same parts are designated by the same reference numerals, and duplicate description is omitted.

(Embodiment)
[1. Generation process]
First, an example of the generation process according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an example of a generation process according to an embodiment. In FIG. 1, the output value output from the model when the generation device 100 inputs the score corresponding to the index related to the user's behavior regarding the content into the model, and the prediction information predicted from the information different from the model. Based on this, a case where evaluation information about the model is generated is shown. In the example of FIG. 1, as an index related to the user's behavior regarding the content, a case where the index indicates whether the product corresponding to the delivered content sells quickly is shown as an example. It is assumed that the product corresponding to the distributed content is sold early when the product is purchased by the user within a predetermined period (for example, within 24 hours) from the start of distribution of the content. The index for which the evaluation information is generated is not limited to "sells quickly" and may be various indexes. For example, the index targeted by the model may be various indexes such as "Click Through Rate (CTR)", "Conversion Rate (CVR)", and "Sell high".

[Configuration of generation system]
First, prior to the description of FIG. 1, the generation system 1 shown in FIG. 3 will be described. As shown in FIG. 3, the generation system 1 includes a terminal device 10 and a generation device 100. The terminal device 10 and the generation device 100 are connected to each other via a predetermined network N so as to be communicable by wire or wirelessly. FIG. 3 is a diagram showing a configuration example of the generation system according to the embodiment. The generation system 1 shown in FIG. 3 may include a plurality of terminal devices 10 and a plurality of generation devices 100.

The terminal device 10 is an information processing device used by the user. The terminal device 10 is realized by, for example, a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like. In the example shown in FIG. 1, a case where the terminal device 10 is a smartphone used by a user is shown. In the following, the terminal device 10 may be referred to as a user. That is, in the following, the user can be read as the terminal device 10. For example, FIG. 2 shows a case where the terminal device 10 is a smartphone used by a user identified by the user ID “U101” (hereinafter, may be referred to as “user U101”). As described above, when "user U * (* is an arbitrary numerical value)" is described, it means that the user is a user identified by the user ID "U *". For example, when "User U101" is described, the user is a user identified by the user ID "U101".

The generation device 100 is an information processing device that uses content information about the content to generate a model that outputs a score indicating an evaluation based on an index corresponding to the user's behavior regarding the content. Further, the generation device 100 generates prediction information from information different from the model. In the example of FIG. 1, the generation device 100 generates prediction information for predicting a score from statistical information related to the content. Further, the generation device 100 generates evaluation information about the model based on the output value (score) output from the model by inputting the content information and the prediction information.

Further, in the generation system 1, the generation device 100 provides an electronic commerce service in which products provided by a plurality of distributors can be purchased. That is, the generation system 1 provides an electronic commerce service that allows the purchase of products provided by a plurality of sellers.

Further, the generation device 100 may provide various services such as an electronic commerce service and a search service. Further, when an external information processing device provides a search service or an electronic commerce service, the generation device 100 does not have to provide the search service or the electronic commerce service. In this case, the generation device 100 acquires various information from an external information processing device or the like that provides a search service or an electronic commerce service, and based on the acquired various information, generates processing or provides information to the external information processing device. You may go.

In the example of FIG. 1, the generation device 100 distributes the content to a large number of users such as users U1 to U5 by using the model M1 corresponding to the index “Click Through Rate (CTR)”. It shall be. That is, in the example of FIG. 1, it is assumed that the generation device 100 has already acquired sufficient information for generating the model M1. For example, the generation device 100 generates a model using the user's behavior regarding the index "CTR" as correct answer information. In the following, the target information to which the correct answer information is associated is also referred to as "learning data".

For example, the generation device 100 collects information for generating correct answer information. In the example of FIG. 1, the generation device 100 distributes contents such as product GD101 (A product) to product GD105 (E product) to the user, and acquires information (behavior information) regarding the user's behavior with respect to the distributed content.

As described above, when "product GD * (* is an arbitrary numerical value)" is described, it means that the product is a product identified by the product ID "GD *". For example, when "Product GD1" is described, the product is a product identified by the product ID "GD1". In the following, the product information of "product GD * (* is an arbitrary numerical value)" may be referred to as "product GD * (* is an arbitrary numerical value)". For example, in the following, the product GD1 can be read as the product information (content) of the product GD1. Although the first learning data storage unit 121 in FIG. 1 shows five users U1 to U5, the generation device 100 is not limited to the users U1 to U5, and a large number of users (for example, one million users) are shown. And 10 million users, etc.) to acquire behavioral information for the distributed content. In the example of FIG. 1, the generation device 100 adds data DT101 to DT105 and the like corresponding to each of the products GD101 to GD105 and the like to which the content is distributed to the first learning data storage unit 121.

As described above, when "data DT * (* is an arbitrary numerical value)" is described, it means that the data is the data identified by the data ID "DT *". For example, when "data DT1" is described, the data is data identified by the data ID "DT1".

The “data ID” shown in the first learning data storage unit 121 in FIG. 1 indicates identification information for identifying data. The “target product” shown in the first learning data storage unit 121 in FIG. 1 indicates a target product in the data identified by the data ID.

The “correct answer information” shown in the first learning data storage unit 121 in FIG. 1 indicates the correct answer information corresponding to the data identified by the data ID. The “click” in the “correct answer information” shown in the first learning data storage unit 121 in FIG. 1 indicates whether or not the content of the corresponding target product has been clicked by the distribution destination user. For example, when the "click" is "1", it indicates that the content of the corresponding target product has been clicked by the delivery destination user. For example, when the "click" is "0", it indicates that the content of the corresponding target product was not clicked by the delivery destination user.

In the "prediction information" shown in the first learning data storage unit 121 in FIG. 1, "sell quickly" means that the target product may be purchased early by the user who is the delivery destination of the content of the corresponding target product. Is indicated by a score, but the details will be described later. The "content information" shown in the first learning data storage unit 121 in FIG. 1 indicates information related to the content corresponding to the data identified by the data ID. “Content information” indicates information regarding the distribution of content corresponding to the data identified by the data ID.

The "content data" in the "content information" shown in the first learning data storage unit 121 in FIG. 1 indicates the content data provided to the user in the distribution corresponding to the data identified by the data ID. In FIG. 5, an example in which conceptual information such as "CDT101" is stored in "content data" is shown, but in reality, various information about the content, a file path name indicating the storage location, and the like are stored. NS. For example, the "content data" includes various information such as a product name, a category, an image, a title, a description, and other text information related to the corresponding product. For example, the "content data" may be static information that does not change depending on the delivery destination user or the delivery date and time among the information related to the content. For example, the "content data" may include a price at which a contract is established in a flea market format or a price at which a contract is established (prompt decision price) even within the auction period in the auction format. Hereinafter, the flea market format may be described as "flea market format", and the auction format in which the prompt decision price is set may be described as "prompt decision auction format".

The "log data" in the "content information" shown in the first learning data storage unit 121 in FIG. 1 indicates the history information in the content distribution of the target product corresponding to the data identified by the data ID. The "log data" includes a user who is a delivery destination of the content delivery of the target product corresponding to the data identified by the data ID and various information at the time of delivery. In FIG. 5, an example in which conceptual information such as "LDT101" is stored in "log data" is shown, but in reality, various information related to content distribution or a file path name indicating the storage location is stored. Will be done. For example, the "log data" may include information about a user who is a delivery destination of the content delivery of the target product corresponding to the data identified by the data ID.

Further, for example, the "log data" may include dynamic information that dynamically changes depending on the user's behavior (bid) or the like regarding the target product, and may include dynamic information at the time of content distribution. For example, the "log data" may include various information such as the current price, the number of bids, the number of bidders, and the remaining time at the time of content distribution. The above is an example, and the log data may include various information, not limited to the above. The "content data" and the "log data" may include information that overlaps with each other as long as the information is related to the content.

For example, in the example shown in FIG. 1, the data (data DT101) identified by the data ID "DT101" indicates that the target product is the product GD101 (A product). Further, the data DT101 indicates that "click" is "1" in the correct answer information. Further, the data DT101 indicates that the "content data" is "CDT101" and the "log data" is "LDT101 (U1 or the like)" in the content information.

That is, the data DT101 indicates that the data is related to the distribution of the content of the product GD101 (product A) to the user U1. The data DT101 indicates that the user U1 clicks on the content of the delivered product GD101.

Although not shown in the first learning data storage unit 121 in FIG. 1, the correct answer information includes an item of the index "sell quickly". For example, in the first learning data storage unit 121 in FIG. 1, for a very small number of contents (for example, contents of less than 1% of the total), the correct answer information is acquired for the index "sell fast". do. For example, it is assumed that the first learning data storage unit 121 in FIG. 1 has acquired correct answer information regarding the index "sell quickly" for the content related to the products sold in the flea market format or the prompt decision auction format.

For example, "quick selling" of the correct answer information in the first learning data storage unit 121 in FIG. 1 indicates whether or not the user who is the delivery destination of the content of the corresponding target product has purchased the target product. For example, when "sell fast" is "1", it indicates that the user has purchased the content within a predetermined period after the distribution of the content of the corresponding target product is started. For example, when "sell fast" is "1", it indicates that the user has purchased the content within 24 hours after the distribution of the content of the corresponding target product is started. Further, the time indicated when "sell fast" is "1" indicates the time elapsed from the start of distribution of the content of the corresponding target product to the purchase by the user. Further, for example, when "sell fast" is "0", it indicates that the user did not purchase the content within a predetermined period after the distribution of the content of the corresponding target product was started.

For example, in the example of FIG. 1, the first learning data storage unit 121 shows that the user U501 purchased the product GD501 one hour after the start of distribution of the content of the frima format product GD501, although the number is small. Training data shall be included. In the example of FIG. 1, the generation device 100 generates the model M2 from the data in the first learning data storage unit 121 by using the information available for the generation of the model M2.

Further, in the example of FIG. 1, it is assumed that the generation device 100 has already generated each model based on the training data. For example, the generation device 100 generates a model by performing learning using the data DT101 to DT105 in the first learning data storage unit 121 as learning data (teacher data). For example, the generation device 100 generates the model M1 by performing learning using the data DT101 to DT105 in the first learning data storage unit 121 as learning data (teacher data).

Further, for example, the generation device 100 learns the data DT101 to DT105 in the first learning data storage unit 121 using the training data (teacher data) that can be used to generate the model M2. Generates model M2. For example, the generation device 100 uses the data DT101 to DT105 in the first learning data storage unit 121 as the learning data (teacher data) associated with the correct answer information corresponding to the index "sell fast". , Model M2 is generated by learning. As described above, in the example of FIG. 1, the generation device 100 extracts data that can be used for generating the model M2 from the data in the first learning data storage unit 121, and uses the extracted data to generate the model M2. Generate. Therefore, in the example of FIG. 1, the generation device 100 generates the model M2 by using the data of the data amount smaller than the data amount available for the generation of the model M1.

For example, the generation device 100 generates a model using the feature amount related to the content. For example, the generation device 100 generates a model using a feature amount related to the content of the content. For example, the generation device 100 generates a model using the feature amount related to the product corresponding to the content. For example, the generation device 100 generates a model using feature quantities related to product categories, sales modes, and the like corresponding to the content. For example, the generation device 100 generates a model using the features related to the user to whom the content is distributed. In the example of FIG. 1, the generation device 100 learns the characteristics of the product or the like corresponding to the content included in the content information, and generates a model for predicting whether or not the product corresponding to the content is purchased by the user. The features included in the content information learned by the generation device 100 may be input to the generation device 100 by a person such as an administrator of the generation device 100, or may be automatically learned (extracted) by the generation device 100. May be good. The generation device 100 may generate the model M2 by using the same feature amount (feature) as the model M1.

For example, the generation device 100 generates the model M2 using data or the like indicating that the user U501 has purchased the product GD501. For example, in the generation device 100, for the data in which the index "fast-selling", which is the correct answer information, is "1", the content information corresponding to the data (hereinafter, also referred to as "fast-selling data") is input to the model M2. In this case, the learning process is performed so that the score output by the model M2 approaches "1". For example, the generation device 100 performs learning processing so that the score output by the model M2 approaches "1" when the content data, log data, or the like of the fast-selling data is input to the model M2. The generator 100 may change the weighting of the index "sell fast" based on the time from the start of sales to the purchase, and the details of this point will be described later.

Further, for example, in the generation device 100, with respect to the data in which the index "fast-selling" which is the correct answer information is "0", the content information corresponding to the data (hereinafter, also referred to as "non-fast-selling data") is stored in the model M2. Is input, the learning process is performed so that the score output by the model M2 approaches "0". For example, the generation device 100 performs learning processing so that the score output by the model M2 approaches "1" when the content data, log data, or the like of non-fast-selling data is input to the model M2.

The model learning method is not limited to the above-mentioned method, and any known technique can be applied. It should be noted that the generation of each model may be performed by appropriately using various conventional techniques related to machine learning. For example, the model may be generated by using a technique related to machine learning of supervised learning such as SVM (Support Vector Machine). Further, for example, model generation may be performed using a technique related to machine learning of unsupervised learning. For example, model generation may be performed using a technique of deep learning. For example, the model may be generated by appropriately using various deep learning techniques such as DNN (Deep Neural Network), RNN (Recurrent Neural Network), and CNN (Convolutional Neural Network). The description regarding the generation of the model is an example, and the generation of the model may be performed by a learning method appropriately selected according to the information that can be acquired and the like. That is, if the generation device 100 can learn the model M2 so as to output the score corresponding to the correct answer information when the content information included in the training data is input, any method can be used to generate the model M2. May be done.

By the above processing, in the example of FIG. 1, the generation device 100 generates a model (model M2) identified by the model ID “M2” as shown in the model information storage unit 124. As described above, when "model M * (* is an arbitrary numerical value)" is described, it means that the model is a model identified by the model ID "M *". For example, when described as "model M2", the model is a model identified by the model ID "M2". Further, as shown in the model information storage unit 124 in FIG. 1, the model M2 is a prediction target "sells quickly" indicating an index to be predicted, that is, a product that is a target of content delivered to the user is quickly delivered to the user. It is a model used for predicting whether or not it will be purchased, and the specific model data thereof is "model data MDT2". For example, the generation device 100 inputs the content information to the model M2, and causes the model M2 to output a score indicating whether or not the product corresponding to the input content information can be sold quickly. Then, the generation device 100 predicts the evaluation of the content corresponding to the index corresponding to the user's behavior regarding the content based on the score output by the model M2. Specifically, the generation device 100 predicts the evaluation of the content corresponding to the index "sell fast", which indicates whether the product corresponding to the content sells quickly, based on the score output by the model M2.

Here, as described above, in the content distribution using the model M1 corresponding to the different index "CTR", the generation device 100 has sufficient correct answer information to generate the model M2 corresponding to the index "sell fast". Can not be obtained in many cases. Therefore, the generator 100 needs to evaluate whether or not the model M2 can accurately predict the index "sell fast". Therefore, in the example of FIG. 1, the generation device 100 generates evaluation information in order to evaluate whether or not the model M2 can accurately predict the index “fast selling”.

First, the generation device 100 acquires the prediction information corresponding to the target index “fast selling” (step S11). In the example of FIG. 1, the generation device 100 receives the prediction information corresponding to the index “fast selling” from the prediction information storage unit 122.

The “target ID” shown in the prediction information storage unit 122 in FIG. 1 indicates identification information for identifying the target. The “target” shown in the prediction information storage unit 122 in FIG. 1 indicates a target identified by the corresponding target ID. The "element" shown in the prediction information storage unit 122 in FIG. 1 includes items such as "category" and "keyword". Further, the "content" shown in the prediction information storage unit 122 in FIG. 1 includes items such as "trend" and "score". For example, "trend" indicates the statistical tendency of each element. For example, "score" indicates the statistical tendency of each element as a score.

The “category” shown in the prediction information storage unit 122 in FIG. 1 indicates a statistical tendency for each category regarding the corresponding object. The "category" includes items such as "A category", "B category", and "C category". Further, for example, the category such as "A category" may be "fashion", "food" or the like. Further, for example, the category such as "A category" may be "shoes", "drinking water" or the like. For example, the "category" may include categories of various hierarchies.

Further, the “keyword” shown in the prediction information storage unit 122 in FIG. 1 indicates a statistical tendency for each keyword regarding the corresponding target. The "keyword" includes items such as "keyword A" and "keyword B". For example, the keyword such as "keyword A" is not limited to a specific product name or the like, but may be an abbreviation or a common name thereof, and the keyword such as "keyword A" is "restock" or "popular product". , "Limited to the actual product", "Same-day delivery", etc. may be used.

For example, in the example shown in FIG. 1, the target (target TG1) identified by the target ID "TG1" is the target "sells quickly", and the product targeted for the content delivered to the user is purchased quickly by the user. Indicates that.

Further, the target TG1 indicates that the tendency of the A category is "medium (average A time)" and the score is "0.2". For example, in the target TG1, when the category of the product is the A category, the product is purchased in an average A time, the tendency to sell quickly is moderate, and the score indicating the evaluation is "0.2". Show that.

For example, the closer the score is to "1", the more likely it is that a product belonging to that category will sell faster. For example, the score of each category may be closer to "1" as the average time until the products of the category are purchased is shorter than the average time until the products of all categories are purchased. The above is an example, and the score of each category may be determined by appropriately using various information. For example, the generator 100 may determine the score of each category using various statistical information such as AUC (Area Under the Curve).

Further, the target TG1 indicates that the tendency of the B category is "high (average B time)" and the score is "0.5". For example, the target TG1 indicates that when the category of the product is the B category, the product is purchased in an average B time, tends to sell quickly, and the score indicating the evaluation is "0.5". ..

Further, the target TG1 indicates that the tendency of the keyword A is "low (average D time)" and the score is "0.05". For example, in the target TG1, when the keyword A is included in the content information of the product, the product is purchased in an average D time, the tendency to sell quickly is low, and the score indicating the evaluation is "0.05". Show that.

For example, the closer the score is to "1", the higher the possibility that the product whose keyword is included in the content information will sell faster. For example, the score of each keyword is closer to "1" as the average time until the product whose keyword is included in the content information is shorter than the average time until the product of all keywords is purchased. You may become. The above is an example, and the score of each keyword may be determined by appropriately using various information.

Further, the target TG1 indicates that the tendency of the keyword B is "high (average E time)" and the score is "0.6". For example, in the target TG1, when the keyword B is included in the content information of the product, the product is purchased in an average E time, tends to sell quickly, and the score indicating the evaluation is "0.6". Show that.

In the example of FIG. 1, it is assumed that the generation device 100 has already acquired the prediction information as shown in the prediction information storage unit 122. For example, the generation device 100 may generate the prediction information as shown in the prediction information storage unit 122 based on the purchase information of the product in the past content distribution, or may acquire the prediction information from an external device.

In addition, the generation device 100 generates prediction information corresponding to the index “sell fast” (step S12). In the example of FIG. 1, as shown in the first learning data storage unit 121, the generation device 100 generates prediction information corresponding to the index “fast selling” for the data DT101 to DT105 and the like. For example, the generation device 100 generates forecast information corresponding to the index "sell fast" for the data DT101 to DT105, etc., based on the product category corresponding to each data, the keywords included in the content data, and the like. For example, the generation device 100 generates prediction information corresponding to the index "fast-selling" based on the ratio of the number of fast-selling distributions for each category, the similarity of words (keywords) common to fast-selling products, and the like. do.

For example, the generation device 100 calculates as a score corresponding to the index "fast selling" of the data DT 101 based on the score of the category to which the product GD 101 of the data DT 101 belongs and the score of each keyword included in the content data CDT 101. For example, the generation device 100 may calculate the average of the score of the category to which the product GD101 of the data DT101 belongs and the score of each keyword included in the content data CDT101 as the score corresponding to the index "sell fast" of the data DT101. good. The above is an example, and the generation device 100 may appropriately use various information to calculate as prediction information (score) corresponding to the index “fast selling” of the data DT101.

In the example of FIG. 1, the generation device 100 calculates the prediction information of the index “fast selling” as the score “0.5” for the data DT101 (commodity A). Further, the generation device 100 calculates the prediction information of the index "sell fast" as the score "0.75" for the data DT102 (product B).

Then, the generation device 100 selects a model to be evaluated (step S13). In the example of FIG. 1, the generator 100 selects the model M2 corresponding to the index “fast selling” to be evaluated.

Then, the generation device 100 causes the model M2 to output the score by inputting the content information into the model M2. In the example of FIG. 1, the generation device 100 inputs the content data, log data, and the like of the products GD101 to GD105 stored in the first learning data storage unit 121 into the model M2.

In the example of FIG. 1, the generation device 100 calculates a score indicating the possibility that the product GD101 (A product) can be quickly sold to the user U1 to be the delivery destination by the processing as shown in the processing group PS12-1. The generation device 100 inputs the content information DT101 into the model M2 (step S14-1). Specifically, the generation device 100 inputs data DT101 including information about the content data CDT101 of the product A and the user U1 into the model M2. The model M2 to which the data DT1 is input outputs a score (step S15-1). In the example of FIG. 1, the model M2 to which the data DT101 is input outputs a score “0.55” as shown in the score SC1-1.

In addition, the generation device 100 calculates a score indicating the possibility that the product GD102 (B product) can be quickly sold to the user U2, which is the delivery destination, by the processing as shown in the processing group PS12-2. The generation device 100 inputs the content information DT 102 into the model M2 (step S14-2). Specifically, the generation device 100 inputs the content data CDT102 of the B product, the data DT102 including the information about the user U2, and the like into the model M2. The model M2 to which the data DT102 is input outputs a score (step S15-2). In the example of FIG. 1, the model M2 to which the data DT2 is input outputs a score “0.5” as shown in the score SC1-2.

In the example of FIG. 1, the generation device 100 generates a score list including scores corresponding to each product GD101 to GD105 and the like by the processing as shown in the processing groups PS12-1 and PS12-2 (step S16). In the example of FIG. 1, the generation device 100 predicts the score corresponding to each product GD101 to GD105 (A to E products) as shown in the score list SL1.

Then, the generation device 100 generates evaluation information based on the output value of the model M2 and the prediction information (step S17). In the example of FIG. 1, the generator 100 has a score corresponding to each product GD101 to GD105 (A to E products) as shown in the score list SL1 and an index "sells quickly" shown in the first learning data storage unit 121. Evaluation information is generated based on the prediction information corresponding to.

For example, the generation device 100 predicts that the score corresponding to each product GD101 to GD105 (A to E products) as shown in the score list SL1 and the index "sell quickly" shown in the first learning data storage unit 121. Evaluation information is generated based on comparison with information (score).

For example, the generation device 100 generates evaluation information according to the difference between the score (output score) of each product output by the model M2 and the score (prediction score) which is the prediction information. For example, the generation device 100 generates evaluation information according to the comparison between the average (error average) of the difference between the output score of each product and the predicted score of each product and a predetermined threshold value.

For example, the generation device 100 generates evaluation information that sets the accuracy of the model to "low" when the error average of the model to be evaluated is less than the first threshold value. Further, for example, when the error average of the model to be evaluated is equal to or more than the first threshold value and is less than the second threshold value larger than the first threshold value, the generation device 100 provides evaluation information that sets the accuracy of the model to "medium". Generate. Further, for example, when the error average of the model to be evaluated is equal to or greater than the second threshold value, the generation device 100 generates evaluation information in which the accuracy of the model is “high”.

In the example of FIG. 1, the generation device 100 generates the evaluation information RP11. For example, the generation device 100 calculates the error average of the model M2 to be evaluated as the error ER1 which is less than the first threshold value. Then, the generation device 100 generates the evaluation information RP11 in which the accuracy of the model M2 to be evaluated is "low". As described above, in the example of FIG. 1, the generation device 100 evaluates the model targeting the index “fast selling” in which the data available for generating the model is insufficient by using other information. It is possible to evaluate the accuracy of the model even when there is not enough data to generate. Further, when the accuracy of the model is low, the generator 100 can be used even if the data is not sufficient by updating the model based on the evaluation information so as to improve the accuracy of the model. It is possible to generate a model and gradually improve the accuracy of the model. For example, the generation device 100 improves the accuracy of the model by updating the model so that the score output by the model approaches the value (score) of the prediction information, even if the data is not sufficient. Is possible.

As described above, the generation device 100 outputs the output value output from the model by inputting the score corresponding to the index related to the user's behavior regarding the content into the model, and the prediction information predicted from the information different from the model. Based on, evaluation information about the model can be generated. Therefore, the generation device 100 can evaluate the accuracy of the model by generating the evaluation information for evaluating the model as described above. Therefore, the generation device 100 can appropriately generate a model corresponding to the index related to the user's behavior. Further, the generation device 100 evaluates the model by using the information of the past distributed history, and responds without performing the process for acquiring the information for evaluating the model by random distribution or the like. It is possible to evaluate a model with insufficient data. Specifically, the generation device 100 evaluates a model using information showing a statistical tendency, and does not perform a process (live test) for actually collecting a history using the model. The model can be evaluated based on the information (offline data) that has already been acquired. That is, even if the logic (model) cannot be evaluated by the log, the generation device 100 can evaluate the model by the already acquired information (offline data). In the example of FIG. 1, the generator 100 can evaluate the model M2, which is difficult to evaluate with the log based on the CTR and targets the fast-selling, even when the number of logs corresponding to the fast-selling is small.

[2. Delivery process]
An example of the distribution process according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram showing an example of distribution processing according to the embodiment. FIG. 2 shows a case where the generation device 100 distributes the content based on the predicted score when the distribution request from the user is acquired. Further, in the example of FIG. 2, the generation device 100 updates the model M2 by using the information about the content distributed by using the model M2.

First, the generation device 100 acquires a content distribution request (step S21). In the example of FIG. 2, the generation device 100 acquires a content distribution request from the terminal device 10 used by the user U101.

The generation device 100 that has acquired the content distribution request predicts the score by inputting the content information into the model. For example, the generation device 100 inputs content data such as the products GD1 to GD5 and GD101 to GD105 stored in the product information storage unit 127 and information about the user U101 to be the distribution destination into the model M2.

In the example of FIG. 2, the generation device 100 calculates a score indicating the possibility that the product GD1 (V product) can be quickly sold to the user U101 as the delivery destination by the processing as shown in the processing group PS22-1. The generation device 100 inputs the content information DT1 to the model M2 (step S22-1). Specifically, the generation device 100 inputs the data DT1 including the content data CDT1 of the V product, the information about the user U101, and the like into the model M2. The model M2 to which the data DT1 is input outputs a score (step S23-1). In the example of FIG. 2, the model M2 into which the data DT1 is input outputs a score “0.56” as shown in the score SC11-1.

In addition, the generation device 100 calculates a score indicating the possibility that the product GD2 (W product) can be quickly sold to the user U101, which is the distribution destination, by the processing as shown in the processing group PS22-2. The generation device 100 inputs the content information DT2 into the model M2 (step S22-2). Specifically, the generation device 100 inputs data DT2 including information about the content data CDT2 of the W product and the user U101 into the model M2. The model M2 to which the data DT2 is input outputs a score (step S23-2). In the example of FIG. 2, the model M2 to which the data DT2 is input outputs the score “0.58” as shown in the score SC11-2.

In the example of FIG. 2, the generation device 100 predicts the score corresponding to each product GD1 to GD5, GD101 to GD105, etc. by the processing as shown in the processing groups PS22-1, PS22-2, etc. (step S24). In the example of FIG. 2, the generation device 100 predicts the score corresponding to each product GD1 to GD5, GD101 to GD105, and the like, as shown in the score list SL11. Further, the generation device 100 predicts the ranking of each product GD1 to GD5, GD101 to GD105, etc., based on the scores corresponding to the respective products GD1 to GD5, GD101 to GD105, and the like. In the example of FIG. 2, the generator 100 predicts the ranking of each product GD1 to GD5, GD101 to GD105, and the like in descending order of score.

After that, the generation device 100 distributes the content based on the prediction result (step S25). For example, the generation device 100 distributes the contents in order from the one with the highest score. In the example of FIG. 2, the generation device 100 is a terminal device 10 in which the user U101 uses the contents in the order of product GD5 (Z product), product GD102 (B product), product GD2 (W product), and product GD1 (V product). Deliver to. For example, the generation device 100 delivers to the terminal device 10 the contents listed in the order of product GD5 (Z product), product GD102 (B product), product GD2 (W product), and product GD1 (V product).

After that, the generation device 100 acquires the purchase information of the product corresponding to the content delivered from the user U101 (step S26). In the example of FIG. 2, the generation device 100 acquires the action information indicating that the user U101 has purchased the Z product from the terminal device 10. The generation device 100 may acquire behavioral information indicating that the user U101 has purchased the Z product from an external device other than the terminal device 10. Further, in the example of FIG. 2, it is assumed that the time when the user U101 purchases the Z product is the time when 3 hours have passed from the start of distribution of the content of the Z product.

Then, the generation device 100 adds training data based on the information acquired in step S26 (step S27). For example, the generation device 100 adds a combination of information about the distributed content and the user's behavior with respect to the distributed content as learning data. Specifically, the generation device 100 adds the data DT201 or the like corresponding to the product GD5 to which the content is distributed to the second learning data storage unit 123. For the sake of explanation, the second learning data storage unit 123 and the first learning data storage unit 121 are shown separately, but the second learning data storage unit 123 and the first learning data storage unit 121 are the same storage unit. It may be.

The “data ID” shown in the second learning data storage unit 123 in FIG. 2 indicates identification information for identifying the data. The “target product” shown in the second learning data storage unit 123 in FIG. 2 indicates a target product in the data identified by the data ID.

"Sell fast" in the "correct answer information" shown in the second learning data storage unit 123 in FIG. 2 indicates whether or not the user to whom the content of the corresponding target product is distributed has purchased the target product. .. For example, when "sell fast" is "1", it indicates that the user has purchased the content within a predetermined period after the distribution of the content of the corresponding target product is started. For example, when "sell fast" is "1", it indicates that the user has purchased the content within 24 hours after the distribution of the content of the corresponding target product is started. Further, the time indicated when "sell fast" is "1" indicates the time elapsed from the start of distribution of the content of the corresponding target product to the purchase by the user. Further, for example, when "sell fast" is "0", it indicates that the user did not purchase the content within a predetermined period after the distribution of the content of the corresponding target product was started.

The “content information” shown in the second learning data storage unit 123 in FIG. 2 indicates information related to the content corresponding to the data identified by the data ID. “Content information” indicates information regarding the distribution of content corresponding to the data identified by the data ID.

Since the "content information" shown in the second learning data storage unit 123 in FIG. 2 is the same as the "content data" in the first learning data storage unit 121 in FIG. 1, the description thereof will be omitted.

For example, in the example shown in FIG. 2, the data (data DT201) identified by the data ID "DT201" indicates that the target product is the product GD5 (Z product). Further, the data DT201 indicates that among the correct answer information, "sell quickly" is "1 (3 hours)". Further, the data DT201 indicates that the "content data" is "CDT5" and the "log data" is "LDT5 (U101 or the like)" in the content information.

That is, the data DT201 indicates that the data is related to the distribution of the content of the product GD5 (Z product) to the user U101. The data DT201 indicates that the user U101 has purchased the delivered product GD5. Further, the data DT201 indicates that the purchase of the product GD5 by the user U101 is 3 hours after the start of distribution of the content of the product GD5.

Then, the generation device 100 updates the model based on the training data (step S28). For example, the generation device 100 updates the model M2 using the data DT201 or the like in the second learning data storage unit 123 as learning data (teacher data). For example, the generator 100 outputs a score output by the model M2 when the content information corresponding to the data DT 201 is input to the model M2 for the data DT 201 whose index "sell fast" which is the correct answer information is "1". Updates the model M2 so that is closer to "1".

As described above, the generation device 100 predicts (generates) a score indicating the degree to which the product corresponding to the content is purchased by the user early by using the model. For example, in the example of FIG. 2, the generator 100 inputs the data DT1 including the content data CDT1 of the V product and the information about the user U101 into the model M2, so that the V product is quickly purchased by the user U101 in the model M2. The score indicating the degree of the model is output. Further, in the example of FIG. 2, the generation device 100 causes the model M2 to output a score indicating the degree to which each product is purchased early to the user U101 in the same manner for each of the other products. Then, in the example of FIG. 2, the generation device 100 delivers the content to the terminal device 10 of the user U101 so that the content of the product having a high score is prioritized and visually recognized by the user. As a result, the generation device 100 can preferentially provide the content for which the product is purchased to the user.

Further, the generation device 100 updates the model M2 by using the information about the content distributed by using the model M2. As a result, the generation device 100 can further improve the prediction accuracy of the model M2. As described above, in the example of FIG. 2, the generator 100 uses the added data even after the generation of the model targeting the index “fast selling” in which the data available for generating the model is not sufficient. By updating the model, the accuracy of the model can be improved. That is, the generation device 100 evaluates the model using information other than the information used for model generation, and updates the generated model according to the addition of data, so that even if the data is not sufficient, the generation device 100 evaluates the model. It is possible to generate an available model and gradually improve the accuracy of that model.

(2-1. Target of weight prediction based on time to sell)
In the example of FIG. 1, for the sake of simplicity, a case where the score is the same regardless of the elapsed time when the product is sold early is shown, but the generator 100 weights the score according to the time until the product is sold. You may set the score. This point will be described with reference to FIG. FIG. 14 is a diagram showing a change in the magnification based on the required time according to the embodiment.

As shown in FIG. 14, the generation device 100 may increase the weight as the elapsed time from the start of sales is shorter, even if the product is purchased by the user within a predetermined period from the start of sales. For example, the generation device 100 may generate a model so that the shorter the elapsed time from the start of sales, the higher the score, even if the product is purchased by the user within a predetermined period from the start of sales.

The second magnification α2 is a magnification in which the difference between the weight at the start of sales and the weight at the threshold value is smaller than that of the first magnification α1. That is, the second magnification α2 is a magnification in which the rate of decrease in weight with the passage of time is smaller than that of the first magnification α1. Further, the third magnification α3 is a magnification in which the difference between the weight at the start of sales and the weight at the threshold value is smaller than that of the second magnification α2. That is, the third magnification α3 is a magnification in which the rate of decrease in weight with the passage of time is smaller than that of the second magnification α2. The generation device 100 may generate a model or the like by using a plurality of magnifications having different weights in this way.

For example, the generation device 100 may generate a model using a magnification having the highest prediction accuracy among various magnifications such as the first magnification α1, the second magnification α2, and the third magnification α3. For example, the generation device 100 generates a plurality of models based on each of various magnifications such as the first magnification α1, the second magnification α2, and the third magnification α3, and among the generated plurality of models, the highest prediction accuracy. Content distribution and the like may be performed using a model with a high value.

[3. Generator configuration]
Next, the configuration of the generation device 100 according to the embodiment will be described with reference to FIG. FIG. 4 is a diagram showing a configuration example of the generator according to the embodiment. As shown in FIG. 4, the generation device 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The generation device 100 has an input unit (for example, a keyboard, a mouse, etc.) that receives various operations from the administrator of the generation device 100, and a display unit (for example, a liquid crystal display, etc.) for displaying various information. You may.

(Communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. Then, the communication unit 110 is connected to the network N by wire or wirelessly, and transmits / receives information to / from the terminal device 10.

(Memory unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. As shown in FIG. 4, the storage unit 120 according to the embodiment includes a first learning data storage unit 121, a prediction information storage unit 122, a second learning data storage unit 123, a model information storage unit 124, and a user. It has an information storage unit 125, a log information storage unit 126, and a product information storage unit 127.

(1st learning data storage unit 121)
The first learning data storage unit 121 according to the embodiment stores various information related to the learning data. FIG. 5 is a diagram showing an example of the first learning data storage unit according to the embodiment. For example, the first learning data storage unit 121 stores teacher data used for generating a model. The first learning data storage unit 121 shown in FIG. 5 includes items such as "data ID", "target product", "correct answer information", "prediction information", and "content information". "Correct answer information" includes items such as "click". "Forecast information" includes items such as "sell quickly". Further, the "content information" includes items such as "content data" and "log data".

The "data ID" indicates identification information for identifying the data. For example, the data identified by the data ID "DT101" corresponds to the data DT101 shown in the example of FIG. “Target product” indicates a target product in the data identified by the data ID.

"Correct answer information" indicates the correct answer information corresponding to the data identified by the data ID. The "click" in the "correct answer information" indicates whether or not the content of the corresponding target product has been clicked by the delivery destination user. For example, when the "click" is "1", it indicates that the content of the corresponding target product has been clicked by the delivery destination user. For example, when the "click" is "0", it indicates that the content of the corresponding target product was not clicked by the delivery destination user.

"Sell fast" in "Forecast information" indicates the possibility that the target product will be purchased early by the user who is the delivery destination of the content of the corresponding target product. For example, "sell faster" indicates that the higher the score, the more likely it is that the user will buy it sooner. For example, the closer "sell quickly" to "1", the more likely it is that the user will purchase the content within a predetermined period after the distribution of the content of the corresponding target product is started. For example, the closer "sell fast" to "1", the more likely it is that the user will purchase the content within 24 hours after the content of the corresponding target product is distributed. Further, for example, the closer "sells quickly" to "0", the less likely it is that the user will purchase the content within a predetermined period after the distribution of the content of the corresponding target product is started.

"Content information" indicates information about the content corresponding to the data identified by the data ID. “Content information” indicates information regarding the distribution of content corresponding to the data identified by the data ID.

The "content data" in the "content information" indicates the content data provided to the user in the distribution corresponding to the data identified by the data ID. In FIG. 5, an example in which conceptual information such as "CDT101" is stored in "content data" is shown, but in reality, various information about the content, a file path name indicating the storage location, and the like are stored. NS. For example, the "content data" includes various information such as a product name, a category, an image, a title, a description, and other text information related to the corresponding product. For example, the "content data" may be static information that does not change depending on the delivery destination user or the delivery date and time among the information related to the content. For example, the "content data" may include a price at which a contract is established in a flea market format or a price at which a contract is established (prompt decision price) even within the auction period in the auction format.

The "log data" in the "content information" indicates the history information in the content distribution of the target product corresponding to the data identified by the data ID. The "log data" includes a user who is a delivery destination of the content delivery of the target product corresponding to the data identified by the data ID and various information at the time of delivery. In FIG. 5, an example in which conceptual information such as "LDT101" is stored in "log data" is shown, but in reality, various information related to content distribution or a file path name indicating the storage location is stored. Will be done. For example, the "log data" may include information about a user who is a delivery destination of the content delivery of the target product corresponding to the data identified by the data ID.

Further, for example, the "log data" may include dynamic information that dynamically changes depending on the user's behavior (bid) or the like regarding the target product, and may include dynamic information at the time of content distribution. For example, the "log data" may include various information such as the current price, the number of bids, the number of bidders, and the remaining time at the time of content distribution. The above is an example, and the log data may include various information, not limited to the above. The "content data" and the "log data" may include information that overlaps with each other as long as the information is related to the content.

For example, in the example shown in FIG. 5, the data (data DT101) identified by the data ID "DT101" indicates that the target product is the product GD101 (A product). Further, the data DT101 indicates that the "click" of the correct answer information is "1". Further, the data DT101 indicates that the "fast selling" of the forecast information is "0.5". Further, the data DT101 indicates that the "content data" is "CDT101" and the "log data" is "LDT101 (U1 or the like)" in the content information.

That is, the data DT101 indicates that the data is related to the distribution of the content of the product GD101 (product A) to the user U1. The data DT101 indicates that the user U1 clicks on the content of the delivered product GD101. In addition, the data DT101 indicates that the user U1 has a moderate possibility (for example, 50%) of purchasing the delivered product GD101 early.

The first learning data storage unit 121 is not limited to the above, and may store various information depending on the purpose. For example, the first learning data storage unit 121 may store information regarding the date and time when the learning data was added. Further, for example, the first learning data storage unit 121 may store information indicating by what kind of determination processing each learning data is added. For example, the first learning data storage unit 121 may store information indicating what kind of information is used to generate the prediction information.

(Information storage unit for prediction 122)
The prediction information storage unit 122 according to the embodiment stores various information used for generating prediction information. FIG. 6 is a diagram showing an example of a prediction information storage unit according to the embodiment. For example, the prediction information storage unit 122 stores information indicating a statistical tendency to generate prediction information. The prediction information storage unit 122 shown in FIG. 6 includes items such as "target ID", "target", "category", and "keyword".

The "target ID" indicates identification information for identifying the target. For example, the "target ID" indicates identification information for identifying a target for which prediction information is generated. “Target” indicates a target identified by the corresponding target ID. The "element" includes items such as "category" and "keyword". In addition, the "content" includes items such as "trend" and "score". For example, "trend" indicates the statistical tendency of each element. For example, "score" indicates the statistical tendency of each element as a score.

"Category" indicates the statistical tendency of each category with respect to the corresponding object. The "category" includes items such as "A category", "B category", and "C category". In the example shown in FIG. 6, the “category” is illustrated with an abstract reference numeral such as “A category”, but may be information indicating a specific category name or the like. Further, for example, the category such as "A category" may be "fashion", "food" or the like. Further, for example, the category such as "A category" may be "shoes", "drinking water" or the like. For example, the "category" may include categories of various hierarchies.

In addition, "keyword" indicates a statistical tendency for each keyword regarding the corresponding target. The "keyword" includes items such as "keyword A" and "keyword B". In the example shown in FIG. 6, the "keyword" is illustrated with an abstract code such as "keyword A", but may be information such as a character string such as a specific product name. For example, the keyword such as "keyword A" is not limited to a specific product name or the like, but may be an abbreviation or a common name thereof, and the keyword such as "keyword A" is "restock" or "popular product". , "Limited to the actual product", "Same-day delivery", etc. may be used.

For example, in the example shown in FIG. 6, the target (target TG1) identified by the target ID “TG1” is the target “sells quickly”, and the product targeted for the content delivered to the user is purchased quickly by the user. Indicates that.

Further, the target TG1 indicates that the tendency of the A category is "medium (average A time)" and the score is "0.2". For example, in the target TG1, when the category of the product is the A category, the product is purchased in an average A time, the tendency to sell quickly is moderate, and the score indicating the evaluation is "0.2". Show that.

Further, the target TG1 indicates that the tendency of the B category is "high (average B time)" and the score is "0.5". For example, the target TG1 indicates that when the category of the product is the B category, the product is purchased in an average B time, tends to sell quickly, and the score indicating the evaluation is "0.5". ..

Further, the target TG1 indicates that the tendency of the keyword A is "low (average D time)" and the score is "0.05". For example, in the target TG1, when the keyword A is included in the content information of the product, the product is purchased in an average D time, the tendency to sell quickly is low, and the score indicating the evaluation is "0.05". Show that.

Further, the target TG1 indicates that the tendency of the keyword B is "high (average E time)" and the score is "0.6". For example, in the target TG1, when the keyword B is included in the content information of the product, the product is purchased in an average E time, tends to sell quickly, and the score indicating the evaluation is "0.6". Show that.

The prediction information storage unit 122 is not limited to the above, and may store various model information depending on the purpose.

(2nd learning data storage unit 123)
The second learning data storage unit 123 according to the embodiment stores various information related to the learning data. FIG. 7 is a diagram showing an example of the second learning data storage unit according to the embodiment. For example, the second learning data storage unit 123 stores the teacher data used for generating the model. The second learning data storage unit 123 shown in FIG. 7 includes items such as "data ID", "target product", "correct answer information", and "content information". "Correct answer information" includes items such as "sell quickly". Further, the "content information" includes items such as "content data" and "log data".

The "data ID" indicates identification information for identifying the data. For example, the data identified by the data ID "DT201" corresponds to the data DT201 shown in the example of FIG. “Target product” indicates a target product in the data identified by the data ID.

"Correct answer information" indicates the correct answer information corresponding to the data identified by the data ID. "Sell quickly" in the "correct answer information" indicates whether or not the user to whom the content of the corresponding target product is distributed has purchased the target product. For example, when "sell fast" is "1", it indicates that the user has purchased the content within a predetermined period after the distribution of the content of the corresponding target product is started. For example, when "sell fast" is "1", it indicates that the user has purchased the content within 24 hours after the distribution of the content of the corresponding target product is started. Further, the time indicated when "sell fast" is "1" indicates the time elapsed from the start of distribution of the content of the corresponding target product to the purchase by the user. Further, for example, when "sell fast" is "0", it indicates that the user did not purchase the content within a predetermined period after the distribution of the content of the corresponding target product was started.

"Content information" indicates information about the content corresponding to the data identified by the data ID. “Content information” indicates information regarding the distribution of content corresponding to the data identified by the data ID.

The "content data" in the "content information" indicates the content data provided to the user in the distribution corresponding to the data identified by the data ID. In FIG. 7, an example in which conceptual information such as "CDT5" is stored in "content data" is shown, but in reality, various information about the content or a file path name indicating the storage location is stored. NS. For example, the "content data" includes various information such as a product name, a category, an image, a title, a description, and other text information related to the corresponding product. For example, the "content data" may be static information that does not change depending on the delivery destination user or the delivery date and time among the information related to the content. For example, the "content data" may include a price at which a contract is established in a flea market format or a price at which a contract is established (prompt decision price) even within the auction period in the auction format.

The "log data" in the "content information" indicates the history information in the content distribution of the target product corresponding to the data identified by the data ID. The "log data" includes a user who is a delivery destination of the content delivery of the target product corresponding to the data identified by the data ID and various information at the time of delivery. In FIG. 7, an example in which conceptual information such as "LDT5" is stored in "log data" is shown, but in reality, various information related to content distribution or a file path name indicating the storage location is stored. Will be done. For example, the "log data" may include information about a user who is a delivery destination of the content delivery of the target product corresponding to the data identified by the data ID.

Further, for example, the "log data" may include dynamic information that dynamically changes depending on the user's behavior (bid) or the like regarding the target product, and may include dynamic information at the time of content distribution. For example, the "log data" may include various information such as the current price, the number of bids, the number of bidders, and the remaining time at the time of content distribution. The above is an example, and the log data may include various information, not limited to the above. The "content data" and the "log data" may include information that overlaps with each other as long as the information is related to the content.

For example, in the example shown in FIG. 7, the data (data DT201) identified by the data ID "DT201" indicates that the target product is the product GD5 (Z product). Further, the data DT201 indicates that the "fast selling" of the correct answer information is "1 (3 hours)". Further, the data DT201 indicates that the "content data" is "CDT5" and the "log data" is "LDT5 (U101 or the like)" in the content information.

That is, the data DT201 indicates that the data is related to the distribution of the content of the product GD5 (Z product) to the user U101. The data DT201 indicates that the user U101 has purchased the delivered product GD5. Further, the data DT201 indicates that the purchase of the product GD5 by the user U101 is 3 hours after the start of distribution of the content of the product GD5.

The second learning data storage unit 123 is not limited to the above, and may store various information depending on the purpose. For example, the second learning data storage unit 123 may store information regarding the date and time when the learning data was added. Further, for example, the second learning data storage unit 123 may store information indicating by what kind of determination processing each learning data is added. For example, the second learning data storage unit 123 may store information indicating whether or not each learning data is determined by the selection of the administrator.

(Model information storage unit 124)
The model information storage unit 124 according to the embodiment stores information about the model. For example, the model information storage unit 124 stores model information (model data) generated by the generation process. FIG. 8 is a diagram showing an example of a model information storage unit according to the embodiment. The model information storage unit 124 shown in FIG. 8 includes items such as “model ID”, “prediction target”, and “model data”. Although only the models M1 and M2 are shown in FIG. 8, a large amount of model information may be stored according to each prediction target (prediction target) such as M3 and M4.

The "model ID" indicates identification information for identifying the model. For example, the model identified by the model ID "M2" corresponds to the model M2 shown in the example of FIG. “Prediction target” indicates the prediction target of the corresponding model. Further, the "model data" indicates the data of the corresponding corresponding model associated with the model data. For example, "model data" includes information including nodes in each layer, functions adopted by each node, connection relationships of nodes, and connection coefficients set for connections between nodes.

For example, in the example shown in FIG. 8, the model (model M1) identified by the model ID "M1" predicts whether the prediction target is "CTR" and the content delivered to the user is clicked by the user. Indicates that it will be used. Further, it is shown that the model data of the model M1 is the model data MDT1.

For example, in the example shown in FIG. 8, in the model (model M2) identified by the model ID "M2", the prediction target is "sell quickly", and the product targeted for the content delivered to the user is quickly delivered to the user. Indicates that it is used to predict whether it will be purchased. Further, it is shown that the model data of the model M2 is the model data MDT2.

For example, the model M2 (model data MDT2) is a first layer that is any of an input layer, an output layer, and an input layer to an output layer into which content information related to the content is input, and belongs to a layer other than the output layer. Each element belonging to each layer other than the output layer is included with respect to the content information input to the input layer, including the element and the second element whose value is calculated based on the first element and the weight of the first element. As the first element, in order to make the computer function so as to output the value of the score corresponding to the index related to the user's behavior regarding the content from the output layer by performing the calculation based on the first element and the weight of the first element. It is a model.

Here, it is assumed that the models M1, M2, etc. are realized by the regression model represented _{by "y = a 1} * x ₁ + a ₂ * x ₂ + ... + a _i * x _i". In this case, for example, the first element included in the model M2 corresponds to input data (xi) such as x1 and x2. Further, the weight of the first element corresponds to the coefficient ai corresponding to xi. Here, the regression model can be regarded as a simple perceptron having an input layer and an output layer. When each model is regarded as a simple perceptron, the first element corresponds to any node of the input layer, and the second element can be regarded as the node of the output layer.

Further, it is assumed that the models M1, M2 and the like are realized by a neural network having one or a plurality of intermediate layers such as DNN. In this case, for example, the first element included in the model M2 corresponds to either the node of the input layer or the intermediate layer. Further, the second element corresponds to a node in the next stage, which is a node to which a value is transmitted from a node corresponding to the first element. Further, the weight of the first element corresponds to a connection coefficient which is a weight considered for the value transmitted from the node corresponding to the first element to the node corresponding to the second element.

The model information storage unit 124 is not limited to the above, and may store various model information depending on the purpose.

(User information storage unit 125)
The user information storage unit 125 according to the embodiment stores various information about the user. For example, the user information storage unit 125 stores information about a plurality of workers (users) who perform tasks such as crowdsourcing. FIG. 9 is a diagram showing an example of a user information storage unit according to the embodiment. The user information storage unit 125 shown in FIG. 9 includes items such as "user ID", "age", "gender", "home", "work location", and "interest".

The "user ID" indicates identification information for identifying a user. For example, the user identified by the user ID "U1" corresponds to the user U1 shown in the example of FIG. Further, "age" indicates the age of the user identified by the user ID. The "age" may be the specific age of the user identified by the user ID, such as 35 years old. Further, "gender" indicates the gender of the user identified by the user ID.

Further, "home" indicates the location information of the user's home identified by the user ID. In the example shown in FIG. 9, "home" is illustrated with an abstract code such as "LC11", but may be information indicating latitude or longitude. Further, for example, "home" may be an area name or an address.

Further, the "work location" indicates the location information of the user's work location identified by the user ID. In the example shown in FIG. 9, the "work location" is shown by an abstract code such as "LC12", but may be information indicating latitude or longitude. Further, for example, the "work location" may be a region name or an address.

Further, "interest" indicates the interest of the user identified by the user ID. That is, "interest" indicates an object that the user identified by the user ID is highly interested in. In the example shown in FIG. 9, one "interest" is shown for each user, but there may be a plurality of "interests".

For example, in the example shown in FIG. 9, the age of the user identified by the user ID "U1" is "20's", and the gender is "male". Further, for example, the user identified by the user ID "U1" indicates that his / her home is "LC11". Further, for example, the user identified by the user ID "U1" indicates that the work location is "LC12". Further, for example, the user identified by the user ID "U1" indicates that he / she is interested in "sports".

The user information storage unit 125 is not limited to the above, and may store various information depending on the purpose. For example, the user information storage unit 125 may store information regarding the demographic attribute and information regarding the psychographic attribute of the user. For example, the user information storage unit 125 may store information such as a name, family structure, income, interests, and lifestyle.

(Log information storage unit 126)
The log information storage unit 126 according to the embodiment stores information related to various histories such as user actions. FIG. 10 is a diagram showing an example of a log information storage unit according to an embodiment. For example, the log information storage unit 126 stores various behavioral information including the user's behavioral information with respect to the content delivered to the terminal device 10 of each user. The log information storage unit 126 shown in FIG. 10 includes items such as "user ID", "history ID", "type", "content", and "date and time".

The "user ID" indicates identification information for identifying a user. For example, the user identified by the user ID "U1" corresponds to the user U1 shown in the example of FIG. Further, the "history ID" indicates information that identifies the user's behavior. Further, "type" indicates information regarding the type of action of the corresponding user. In addition, "content" indicates the content targeted in the corresponding user's behavior. The "date and time" indicates the date and time when the corresponding user's action was performed. Although the "date and time" is abstractly illustrated as "dt11" or the like, a specific date and time such as "August 6, 2017 19:31:52" may be stored.

For example, in the example shown in FIG. 10, the user (user U1) identified by the user ID "U1" has a history identified by the history ID "LG11" (history LG11) or a history identified by the history ID "LG12". It indicates that the action corresponding to (History LG12) or the like has been performed. Specifically, in the example shown in FIG. 10, it is shown that the user U1 has made a content distribution request (corresponding to the history LG 11) at the date and time dt11. For example, the history LG 11 indicates that the content such as the A product was delivered to the terminal device 10 used by the user U1 at the date and time dt11. In FIG. 10, the history LG 11 shows that the content such as the A product has been delivered by using the model M1 that predicts the CTR of each content. That is, the history LG 11 indicates that the content has been delivered based on the clickability of each content as a criterion for evaluation. Further, in the example shown in FIG. 10, it is shown that the user U1 clicks the content of the product A at the date and time dt12 after the date and time dt11. Further, in the example shown in FIG. 10, it is shown that the user U1 purchased the A product at the date and time dt13 after the date and time dt12.

For example, in the example shown in FIG. 10, the user (user U2) identified by the user ID "U2" has a history identified by the history ID "LG21" (history LG21) or a history identified by the history ID "LG22". It indicates that the action corresponding to (History LG22) or the like has been performed. Specifically, in the example shown in FIG. 10, it is shown that the user U2 has made a content distribution request (corresponding to the history LG21) at the date and time dt21. For example, the history LG 21 indicates that the content such as the B product was delivered to the terminal device 10 used by the user U2 at the date and time dt21. In FIG. 10, the history LG 21 shows that the content such as the B product has been delivered by using the model M1 that predicts the CTR of each content. That is, the history LG 21 indicates that the content has been distributed based on the clickability of each content as a criterion for evaluation. Further, in the example shown in FIG. 10, it is shown that the user U2 clicks the content of the B product at the date and time dt22 after the date and time dt21. Further, in the example shown in FIG. 10, it is shown that the user U2 did not purchase the B product.

The log information storage unit 126 is not limited to the above, and may store various information depending on the purpose. Further, although FIG. 10 shows a case where the action information is stored in the log information storage unit 126 for each user ID, the action information is not limited to each user ID and may be stored, for example, in chronological order.

(Product Information Storage Unit 127)
The product information storage unit 127 according to the embodiment stores information about the product. For example, the product information storage unit 127 stores detailed information of the product exhibited in the product. FIG. 11 shows an example of information about a product stored in the product information storage unit. As shown in FIG. 11, the product information storage unit 127 includes items such as "product ID", "product name", "sales mode", and "content data" as information about the product.

The "product ID" indicates identification information for identifying the product. Further, the "product name" indicates the name of the product identified by the product ID. In addition, the "sales mode" indicates a sales mode of the corresponding product. For example, the "sales mode" indicates a listing format of a product such as an auction format or a flea market format. For example, when the "sales mode" is "flea market", it indicates that the sales form of the product is the flea market format. For example, when the "sales mode" is "auction", it indicates that the sales form of the product is the auction format. For example, when the "sales mode" is "auction (prompt decision)", the sales form of the product is the auction format, and the price at which the contract is concluded (prompt decision price) is set even within the auction period. Show that there is.

“Content data” indicates content data corresponding to a product identified by a product ID. In FIG. 11, an example in which conceptual information such as “CDT1” is stored in “content data” is shown, but in reality, various information about the content, a file path name indicating the storage location, and the like are stored. NS. For example, the "content data" includes various information such as categories, images, titles, descriptive texts, and other text information related to the corresponding products. For example, the "content data" in FIG. 11 may be static information that does not change depending on the delivery destination user or the delivery date and time among the information related to the content. For example, the "content data" may include a price at which a contract is established in a flea market format or a price at which a contract is established (prompt decision price) even within the auction period in the auction format. In the example of FIG. 11, the item of "product name" is shown to illustrate the product name, but the information of "product name" may be included in the "content data".

For example, in the example shown in FIG. 11, the product (product GD1) identified by the product ID "GD1" indicates that the product name is "V product". Further, the product GD1 is in the sales mode "flea market", and the content data is "CDT1".

The product information storage unit 127 is not limited to the above, and may store various information depending on the purpose.

(Control unit 130)
Returning to the description of FIG. 4, the control unit 130 is a controller, and is stored in a storage device inside the generation device 100 by, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). It is realized by executing various programs (corresponding to an example of a generation program) using the RAM as a work area. Further, the control unit 130 is a controller, and is realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 130 is any one of an input layer, an output layer, and an input layer to an output layer in which content information related to the content is input by information processing according to the model M2 or the like stored in the model information storage unit 124. Content input to the input layer, including a first element that belongs to a layer other than the output layer, and a second element whose value is calculated based on the weights of the first element and the first element. The value of the score corresponding to the index related to the user's behavior regarding the content by performing the calculation based on the weight of the first element and the first element with each element belonging to each layer other than the output layer as the first element for the information. Is output from the output layer.

As shown in FIG. 4, the control unit 130 includes an acquisition unit 131, a generation unit 132, a prediction unit 133, and a distribution unit 134, and realizes or executes an information processing function or operation described below. .. The internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 4, and may be another configuration as long as it is a configuration for performing information processing described later. The control unit 130 includes a second element whose value is calculated based on the first element and the weight of the first element by information processing according to the model M2 (model data MDT2) stored in the storage unit 120. With respect to the content information input to the input layer including The value of the score corresponding to the index related to the behavior of is output from the output layer.

(Acquisition unit 131)
The acquisition unit 131 acquires various types of information. For example, the acquisition unit 131 includes a first learning data storage unit 121, a prediction information storage unit 122, a second learning data storage unit 123, a model information storage unit 124, a user information storage unit 125, and a log information storage unit. Various information is acquired from the unit 126, the product information storage unit 127, and the like. Further, the acquisition unit 131 may acquire various information from an external information processing device. Further, the acquisition unit 131 may acquire various information from the terminal device 10 or the like.

For example, the acquisition unit 131 acquires content information regarding the content. For example, the acquisition unit 131 acquires content information including log information regarding distribution of the content to the terminal device used by the user. For example, the acquisition unit 131 acquires the content information corresponding to the target exhibited in a manner in which the transaction is established in response to the consent to the amount set for the target related to the content. For example, the acquisition unit 131 acquires the content information corresponding to the target exhibited in a manner in which the transaction is established in response to the bid for the target within the period set for the target related to the content.

In the example of FIG. 1, the acquisition unit 131 collects information for generating correct answer information. For example, the acquisition unit 131 distributes contents such as product GD101 (A product) to product GD105 (E product) to the user, and acquires information (behavior information) regarding the user's behavior with respect to the distributed content.

In the example of FIG. 1, the acquisition unit 131 acquires the prediction information corresponding to the target index “fast selling”. For example, the acquisition unit 131 acquires the prediction information corresponding to the index “fast selling” from the prediction information storage unit 122.

For example, the acquisition unit 131 acquires a content distribution request. In the example of FIG. 2, the acquisition unit 131 acquires the content distribution request from the terminal device 10 used by the user U101.

(Generator 132)
The generation unit 132 generates various information. For example, the generation unit 132 uses the learning data stored in the first learning data storage unit 121 to generate a model as shown in the model information storage unit 124. For example, the generation unit 132 uses the content information acquired by the acquisition unit 131 to generate a model that outputs a score indicating an evaluation based on an index corresponding to the user's behavior regarding the content. In addition, the generation unit 132 generates a model using the user's behavior information regarding the content as correct answer information.

Further, the generation unit 132 inputs the content information acquired by the acquisition unit 131 into the model that outputs the score corresponding to the index related to the user's behavior regarding the content, and the output value output from the model and the model Generate evaluation information about the model based on the prediction information predicted from different information.

For example, the generation unit 132 generates evaluation information of the model based on the prediction information predicted from the elements related to the index. For example, the generation unit 132 evaluates the model based on the output value output from the model by inputting the content information into the model that outputs the score indicating the evaluation based on the index of whether or not the target is purchased by the user. Generate information. For example, the generation unit 132 relates to the model based on the output value output from the model by inputting the score indicating the evaluation based on the index regarding the purchase presence / absence of the target within a predetermined period from the start of distribution of the content into the model. Generate evaluation information.

For example, the generation unit 132 generates the evaluation information of the model based on the prediction information predicted from the elements related to the category and the keyword corresponding to the object related to the content. For example, the generation unit 132 generates evaluation information of the model based on the prediction information predicted from the statistical information related to the content. For example, the generation unit 132 generates a model based on the content information and the correct answer information based on the prediction information. For example, the generation unit 132 generates a model based on the content information in which the predetermined action corresponding to the index is performed by the predetermined user and the correct answer information indicating that the predetermined user has performed the predetermined action. ..

For example, the generation unit 132 generates a model so that the shorter the time from the start of distribution of the content to the purchase of the target, the higher the score is output. For example, the generation unit 132 generates a model by increasing the weight of the behavioral information in which the time from the start of distribution of the content to the purchase of the target is shorter.

For example, the generation unit 132 generates models M1, M2, etc., and stores the generated models M1, M2, etc. in the model information storage unit 124. The generation unit 132 may generate models M1, M2, etc. using any learning algorithm. For example, the generation unit 132 generates models M1, M2, and the like using learning algorithms such as a neural network, a support vector machine (SVM), clustering, and reinforcement learning. As an example, when the generation unit 132 generates models M1, M2, etc. using a neural network, the models M1, M2, etc. include an input layer containing one or more neurons and an intermediate layer containing one or more neurons. It has an output layer containing the above neurons.

The generation unit 132 generates a model and stores the generated model in the model information storage unit 124. Specifically, the generation unit 132 is a first element that is any of an input layer, an output layer, and an input layer to an output layer into which content information related to the content is input, and belongs to a layer other than the output layer. A second element whose value is calculated based on the first element and the weight of the first element, and each element belonging to each layer other than the output layer with respect to the content information input to the input layer. By performing an operation based on the first element and the weight of the first element as one element, a model that outputs a score value corresponding to an index related to the user's behavior regarding the content is generated from the output layer.

In the example of FIG. 1, the generation unit 132 generates a model based on the training data. For example, the generation unit 132 generates a model using the user's behavior regarding the index “CTR” as correct answer information.

In the example of FIG. 1, the generation unit 132 generates a model by performing learning using the data DT101 to DT105 in the first learning data storage unit 121 as learning data (teacher data). For example, the generation unit 132 generates the model M1 by performing learning using the data DT101 to DT105 in the first learning data storage unit 121 as learning data (teacher data).

Further, for example, the generation unit 132 learns the data DT101 to DT105 in the first learning data storage unit 121 using the training data (teacher data) that can be used to generate the model M2. Generates model M2. For example, the generation unit 132 uses the data DT101 to DT105 in the first learning data storage unit 121 as the learning data (teacher data) to which the correct answer information corresponding to the index "sells quickly" is associated. , Model M2 is generated by learning.

For example, the generation unit 132 generates a model using the feature amount related to the content. For example, the generation unit 132 generates a model using the feature amount related to the content. For example, the generation unit 132 generates a model using the feature amount related to the product corresponding to the content. For example, the generation unit 132 generates a model using feature quantities related to product categories, sales modes, and the like corresponding to the content. For example, the generation unit 132 generates a model using the features related to the user to whom the content is distributed. In the example of FIG. 1, the generation unit 132 learns the characteristics of the product or the like corresponding to the content included in the content information, and generates a model for predicting whether or not the product corresponding to the content is purchased by the user. The features included in the content information learned by the generation unit 132 may be input to the generation unit 132 by a person such as an administrator of the generation unit 132, or may be automatically learned (extracted) by the generation unit 132. May be good.

For example, the generation unit 132 generates the model M2 using data or the like indicating that the user U501 has purchased the product GD501. For example, the generation unit 132 has the model M2 when the content information corresponding to the data (fast-selling data) is input to the model M2 for the data in which the index "fast-selling" which is the correct answer information is "1". The learning process is performed so that the score output by is close to "1". For example, the generation unit 132 performs learning processing so that the score output by the model M2 approaches "1" when the content data, log data, or the like of the fast-selling data is input to the model M2. The generation unit 132 may change the weighting of the index “sell fast” based on the time from the start of sales to the purchase, and the details of this point will be described later.

Further, for example, in the generation unit 132, when the content information corresponding to the data (non-fast-selling data) is input to the model M2 for the data in which the index "fast-selling" which is the correct answer information is "0". , The learning process is performed so that the score output by the model M2 approaches "0". For example, the generation unit 132 performs learning processing so that the score output by the model M2 approaches "1" when the content data, log data, or the like of non-fast-selling data is input to the model M2.

In the example of FIG. 1, the generation unit 132 generates the prediction information corresponding to the index “sell fast”. For example, as shown in the first learning data storage unit 121, the generation unit 132 generates prediction information corresponding to the index “fast selling” for the data DT101 to DT105 and the like. For example, the generation unit 132 generates forecast information corresponding to the index "sell fast" for the data DT101 to DT105, etc., based on the product category corresponding to each data, the keywords included in the content data, and the like.

In the example of FIG. 1, the generation unit 132 generates evaluation information based on the output value of the model M2 and the prediction information. For example, in the generation unit 132, the generation device 100 has a score corresponding to each product GD101 to GD105 (A to E products) as shown in the score list SL1, and an index "sells quickly" shown in the first learning data storage unit 121. The evaluation information is generated based on the prediction information corresponding to "". For example, the generation unit 132 predicts that the score corresponds to each product GD101 to GD105 (A to E products) as shown in the score list SL1 and the index "sells quickly" shown in the first learning data storage unit 121. Evaluation information is generated based on comparison with information (score). For example, the generation unit 132 generates evaluation information according to the difference between the score (output score) of each product output by the model M2 and the score (prediction score) which is the prediction information. For example, the generation unit 132 generates evaluation information according to the comparison between the average (error average) of the difference between the output score of each product and the predicted score of each product and a predetermined threshold value.

For example, the generation unit 132 generates evaluation information that sets the accuracy of the model to "low" when the error average of the model to be evaluated is less than the first threshold value. For example, the generation unit 132 generates evaluation information in which the accuracy of the model is "medium" when the error average of the model to be evaluated is equal to or more than the first threshold value and less than the second threshold value larger than the first threshold value. .. For example, when the error average of the model to be evaluated is equal to or greater than the second threshold value, the generation unit 132 generates evaluation information in which the accuracy of the model is “high”.

In the example of FIG. 1, the generation unit 132 generates the evaluation information RP11. For example, the generation unit 132 calculates the error average of the model M2 to be evaluated as the error ER1 which is less than the first threshold value. Then, the generation device 100 generates the evaluation information RP11 in which the accuracy of the model M2 to be evaluated is "low".

In the example of FIG. 2, the generation unit 132 updates the model based on the training data. For example, the generation unit 132 updates the model M2 using the data DT201 or the like in the second learning data storage unit 123 as learning data (teacher data). For example, the generation unit 132 outputs the score output by the model M2 when the content information corresponding to the data DT 201 is input to the model M2 for the data DT 201 whose index "sell fast" which is the correct answer information is "1". Updates the model M2 so that is closer to "1".

(Prediction unit 133)
The prediction unit 133 predicts various types of information. The prediction unit 133 includes a first learning data storage unit 121, a prediction information storage unit 122, a second learning data storage unit 123, a model information storage unit 124, a user information storage unit 125, and a log information storage unit 126. And, various information is predicted by using the information stored in the product information storage unit 127 and the like. For example, the prediction unit 133 predicts various information based on various information acquired by the acquisition unit 131.

For example, the prediction unit 133 uses a model to predict the score of the content. For example, the prediction unit 133 predicts the score of the content by inputting the content information about the content into the model. For example, the prediction unit 133 predicts the evaluation of the content corresponding to the index corresponding to the user's behavior regarding the content based on the score output by the model M2. For example, the prediction unit 133 predicts the evaluation of the content corresponding to the index "sell fast", which indicates whether the product corresponding to the content sells quickly, based on the score output by the model M2.

In the example of FIG. 1, the prediction unit 133 generates a score list including scores corresponding to the respective products GD101 to GD105 by the processing as shown in the processing groups PS12-1 and PS12-2. As shown in the score list SL1, the prediction unit 133 predicts the score corresponding to each product GD101 to GD105 (A to E products) and the like. Based on the score output by the prediction unit 133 model M2, the evaluation of the content corresponding to the index corresponding to the user's behavior regarding the content is predicted. Specifically, based on the score output by the prediction unit 133 model M2, the evaluation of the content corresponding to the index "fast selling" indicating whether the product corresponding to the content sells quickly is predicted.

In the example of FIG. 2, the prediction unit 133 predicts the score corresponding to each product GD1 to GD5, GD101 to GD105, etc. by the processing as shown in the processing groups PS22-1, PS22-2, and the like. For example, the prediction unit 133 predicts the score corresponding to each product GD1 to GD5, GD101 to GD105, and the like, as shown in the score list SL11. For example, the prediction unit 133 predicts the ranking of each product GD1 to GD5, GD101 to GD105, etc., based on the scores corresponding to each product GD1 to GD5, GD101 to GD105, and the like. For example, the prediction unit 133 predicts the ranking of each product GD1 to GD5, GD101 to GD105, and the like in descending order of score.

For example, the prediction unit 133 calculates the score using a model having an arbitrary structure such as the regression model and the neural network described above. Specifically, the model M2 is a value (that is, a plurality of indexes) that quantifies the prediction of a predetermined target when the content information regarding the content (that is, each element used for calculating the score described above) is input. The coefficient is set to output a score that suggests whether the action corresponding to each of the above is likely to be performed by the user. The prediction unit 133 uses such a model M2 to calculate a score for each listing target.

In the above example, the model M2 is a model that outputs a quantified value based on a plurality of indexes corresponding to the user's behavior regarding the content when the content information regarding the content is input. .. However, the model (model X) according to the embodiment may be a model generated based on the result obtained by repeating the input / output of data to the model M2. For example, the model X may be a model (model Y) trained to input content information related to the content and output the score output by the model M2. Alternatively, the model M2 may be a model trained to input content information related to the content and output the output value of the model Y. Further, when the prediction unit 133 performs prediction processing using GAN (Generative Adversarial Networks), the model M2 may be a model forming a part of GAN.

(Distribution unit 134)
The distribution unit 134 provides various types of information. For example, the distribution unit 134 provides various information to the terminal device 10. The distribution unit 134 provides the service based on the score predicted by the prediction unit 133. For example, the distribution unit 134 provides a content distribution service based on the score predicted by the prediction unit 133. Further, for example, the distribution unit 134 may provide information about the model generated by the generation unit 132 to an external information processing device. Further, for example, the distribution unit 134 may provide the information output by the model to an external information processing device.

The distribution unit 134 distributes the content based on the score output by the model. For example, the distribution unit 134 distributes the content based on the score output by the model among the content group including the content related to the target traded in the auction format and the content related to the target traded in the flea market format.

In the example of FIG. 1, the distribution unit 134 distributes the content to the terminal devices 10-1 to 10-5 used by the users U1 to U5. In the example of FIG. 2, the distribution unit 134 distributes the content based on the prediction result by the prediction unit 133. For example, the distribution unit 134 distributes the contents in order from the one with the highest score. For example, the distribution unit 134 distributes the contents to the terminal device 10 used by the user U101 in the order of product GD5 (Z product), product GD102 (B product), product GD2 (W product), and product GD1 (V product). For example, the distribution unit 134 distributes the contents listed in the order of product GD5 (Z product), product GD102 (B product), product GD2 (W product), and product GD1 (V product) to the terminal device 10.

[4. Generation process flow]
Next, the procedure of the generation process by the generation system 1 according to the embodiment will be described with reference to FIG. FIG. 12 is a flowchart showing an example of the generation process according to the embodiment.

As shown in FIG. 12, the generation device 100 acquires the first training data (step S101). For example, the generation device 100 acquires learning data from the first learning data storage unit 121.

After that, the generation device 100 generates a model based on the first training data (step S102). In the example of FIG. 1, the generation device 100 generates the model M2 using the first learning data stored in the first learning data storage unit 121.

After that, the generation device 100 generates prediction information from information other than the model (step S103). In the example of FIG. 1, the generation device 100 generates prediction information from the statistical information stored in the prediction information storage unit 122.

After that, the generation device 100 generates evaluation information about the model based on the output value output from the model and the prediction information (step S104). In the example of FIG. 1, the generation device 100 generates the evaluation information RP11 regarding the model M2.

[5. Delivery processing flow]
Next, the procedure of the distribution process by the generation system 1 according to the embodiment will be described with reference to FIG. FIG. 13 is a flowchart showing an example of the distribution process according to the embodiment.

As shown in FIG. 13, the generation device 100 acquires the delivery request (step S201). In the example of FIG. 2, the generation device 100 acquires a distribution request from the terminal device 10 used by the user U101.

In addition, the generation device 100 generates a score for each product using the model (step S202). In the example of FIG. 2, the generation device 100 uses the model M2 to predict the scores of contents (product information) such as products GD1 to GD5 and GD101 to GD105.

In addition, the generation device 100 provides content related to the product based on the generated score (step S203). In the example of FIG. 2, the generator 100 distributes contents in order from the highest score, such as product GD5 (Z product), product GD102 (B product), product GD2 (W product), and product GD1 (V product). ..

[6. effect〕
As described above, the generation device 100 according to the embodiment has an acquisition unit 131 and a generation unit 132. The acquisition unit 131 acquires content information related to the content. Further, the generation unit 132 inputs the content information acquired by the acquisition unit 131 into the model that outputs the score corresponding to the index related to the user's behavior regarding the content, and the output value output from the model and the model Generate evaluation information about the model based on the prediction information predicted from different information.

As described above, the generation device 100 according to the embodiment is different from the output value output from the model by inputting the content information into the model that outputs the score corresponding to the index related to the user's behavior regarding the content. By generating the evaluation information about the model based on the prediction information predicted from the information, it is possible to appropriately generate the model corresponding to the index related to the user's behavior.

Further, in the generation device 100 according to the embodiment, the generation unit 132 generates evaluation information of the model based on the prediction information predicted from the elements related to the index.

As described above, the generator 100 according to the embodiment appropriately generates the model corresponding to the index related to the user's behavior by generating the evaluation information of the model based on the prediction information predicted from the elements related to the index. It can be made generateable.

Further, in the generation device 100 according to the embodiment, the acquisition unit 131 acquires the content information including the log information regarding the distribution of the content to the terminal device 10 used by the user.

As described above, the generation device 100 according to the embodiment appropriately obtains a model corresponding to the index related to the user's behavior by acquiring the content information including the log information regarding the distribution of the content to the terminal device 10 used by the user. Can be generated.

Further, in the generation device 100 according to the embodiment, the generation unit 132 outputs the content information from the model by inputting the content information into the model that outputs the score indicating the evaluation based on the index regarding whether or not the user has purchased the target. Generate evaluation information about the model based on the output value.

As described above, the generation device 100 according to the embodiment converts the content information into the output value output from the model by inputting the content information into the model that outputs the score indicating the evaluation based on the index of whether or not the target is purchased by the user. By generating the evaluation information about the model based on the model, it is possible to appropriately generate the model corresponding to the index related to the user's behavior.

Further, in the generation device 100 according to the embodiment, the generation unit 132 is output from the model by inputting to a model that outputs a score indicating an evaluation based on an index regarding whether or not the target is purchased within a predetermined period from the start of distribution of the content. Generates evaluation information about the model based on the output value.

As described above, the generation device 100 according to the embodiment is an output value output from the model by inputting the score indicating the evaluation based on the index regarding the purchase presence / absence of the target within a predetermined period from the start of distribution of the content into the model. By generating the evaluation information about the model based on the above, it is possible to appropriately generate the model corresponding to the index related to the user's behavior.

Further, in the generation device 100 according to the embodiment, the generation unit 132 generates evaluation information of the model based on the prediction information predicted from the elements related to the category and the keyword corresponding to the object related to the content.

As described above, the generation device 100 according to the embodiment can be used as an index related to the user's behavior by generating the evaluation information of the model based on the prediction information predicted from the elements related to the category and the keyword corresponding to the object related to the content. The corresponding model can be appropriately generated.

Further, in the generation device 100 according to the embodiment, the generation unit 132 generates evaluation information of the model based on the prediction information predicted from the statistical information related to the content.

As described above, the generation device 100 according to the embodiment generates the evaluation information of the model based on the prediction information predicted from the statistical information related to the content, thereby generating the model corresponding to the index related to the user's behavior. It can be appropriately generated.

Further, in the generation device 100 according to the embodiment, the generation unit 132 generates a model based on the content information and the correct answer information based on the prediction information.

In this way, the generation device 100 according to the embodiment can appropriately generate a model corresponding to an index related to the user's behavior by generating a model based on the content information and the correct answer information based on the prediction information. can do.

Further, in the generation device 100 according to the embodiment, the generation unit 132 includes content information in which a predetermined action corresponding to the index is performed by a predetermined user and correct answer information indicating that the predetermined user has performed the predetermined action. Generate a model based on.

As described above, the generation device 100 according to the embodiment is based on the content information in which the predetermined action corresponding to the index is performed by the predetermined user and the correct answer information indicating that the predetermined user has performed the predetermined action. By generating the model, it is possible to appropriately generate a model corresponding to the index related to the user's behavior.

Further, the generation device 100 according to the embodiment has a distribution unit 134. The distribution unit 134 distributes the content based on the score output by the model.

As described above, the generation device 100 according to the embodiment can provide information according to the predictive evaluation of the index related to the user's behavior regarding the content by distributing the content based on the score output by the model.

[7. Hardware configuration]
The generation device 100 according to the above-described embodiment is realized by, for example, a computer 1000 having a configuration as shown in FIG. FIG. 15 is a hardware configuration diagram showing an example of a computer that realizes the function of the generator. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F). ) Has 1700.

The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400, and controls each part. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started, a program depending on the hardware of the computer 1000, and the like.

The HDD 1400 stores a program executed by the CPU 1100, data used by such a program, and the like. The communication interface 1500 receives data from another device via the network N and sends it to the CPU 1100, and transmits the data generated by the CPU 1100 to the other device via the network N.

The CPU 1100 controls an output device such as a display or a printer, and an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. Further, the CPU 1100 outputs the generated data to the output device via the input / output interface 1600.

The media interface 1700 reads a program or data stored in the recording medium 1800 and provides the program or data to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. And so on.

For example, when the computer 1000 functions as the generator 100 according to the embodiment, the CPU 1100 of the computer 1000 is controlled by executing a program or data (for example, model M2 (model data MDT2)) loaded on the RAM 1200. The function of the unit 130 is realized. The CPU 1100 of the computer 1000 reads and executes these programs or data (for example, model M2 (model data MDT2)) from the recording medium 1800, but as another example, these programs from another device via the network N. May be obtained.

Although some of the embodiments and modifications of the present application have been described in detail with reference to the drawings, these are examples and vary based on the knowledge of those skilled in the art, including the embodiments described in the disclosure line of the invention. It is possible to carry out the present invention in another form in which the above is modified or improved.

[8. others〕
Further, among the processes described in the above-described embodiments and modifications, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed. It is also possible to automatically perform all or part of the processed processing by a known method. In addition, the processing procedure, specific name, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of the device is functionally or physically distributed / physically in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

Further, the above-described embodiments and modifications can be appropriately combined as long as the processing contents do not contradict each other.

Further, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

1 Generation system 100 Generation device 121 1st learning data storage unit 122 Prediction information storage unit 123 2nd learning data storage unit 124 Model information storage unit 125 User information storage unit 126 Log information storage unit 127 Product information storage unit 130 Control unit 131 Acquisition unit 132 Generation unit 133 Prediction unit 134 Distribution unit 10 Terminal device N network

Claims (12)

The acquisition department that acquires content information related to the content,
The content information acquired by the acquisition unit, and a first score is the output value output from the model by inputting the model that outputs a score corresponding to the index relating to user behavior related to the content, from the model a score expected from different from any information of the first score and the model output, based on a comparison of the second score is the score of comparable form an output value output from the model , A generator that generates evaluation information about the model,
A generator characterized by comprising. The generator
The generator according to claim 1, wherein the evaluation information of the model is generated based on the second score predicted from the elements related to the index. The acquisition unit
The generation device according to claim 1 or 2, wherein the content information including log information relating to the distribution of the content to the terminal device used by the user is acquired. The generator
Evaluation information about the model based on the first score output from the model by inputting the content information into a model that outputs a score indicating an evaluation based on an index regarding whether or not the user has purchased the target for the content. The generator according to any one of claims 1 to 3, wherein the generator is produced. The generator
Evaluation information about the model based on the first score output from the model by inputting the score indicating the evaluation based on the index regarding the purchase presence / absence of the target within a predetermined period from the start of distribution of the content into the model. The generator according to claim 4, wherein the generator is generated. The generator
The generation according to any one of claims 1 to 5, wherein the evaluation information of the model is generated based on the second score predicted from the elements related to the category and the keyword corresponding to the object related to the content. Device. The generator
The generator according to any one of claims 1 to 6, wherein the evaluation information of the model is generated based on the second score predicted from the statistical information related to the content. The generator
The generator according to any one of claims 1 to 7, wherein the model is generated based on the content information and the correct answer information based on the second score. The generator
The model is generated based on content information in which a predetermined action corresponding to the index is performed by a predetermined user and correct answer information indicating that the predetermined user has performed the predetermined action. The generator according to any one of claims 1 to 8. A distribution unit that distributes content based on the first score output by the model.
The generator according to any one of claims 1 to 9, further comprising. It ’s a computer-executed generation method.
The acquisition process to acquire content information related to content,
The content information acquired by the acquisition step, a first score is the output value output from the model by inputting the model that outputs a score corresponding to the index relating to user behavior related to the content, from the model a score expected from different from any information of the first score and the model output, based on a comparison of the second score is the score of comparable form an output value output from the model , A generation process that generates evaluation information about the model,
A generation method characterized by including. Acquisition procedure to acquire content information about content, and
The content information acquired by the acquisition procedure, the first score is the output value output from the model by inputting the model that outputs a score corresponding to the index relating to user behavior related to the content, from the model a score expected from different from any information of the first score and the model output, based on a comparison of the second score is the score of comparable form an output value output from the model , A generation procedure for generating evaluation information about the model,
A generator characterized by having a computer execute.

Images