JP7571971B2 - Program, learning model generation method, and information processing method - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act Published on May 22, 2020 on the website of Confit's 2020 National Conference of the Japanese Society for Artificial Intelligence (34th) (https://confit.atlas.jp/guide/event/jsai2020/subject/1M3-GS-13-02/advanced) Published on June 9, 2020 at the 2020 National Conference of the Japanese Society for Artificial Intelligence (34th) (https://www.ai-gakkai.or.jp/jsai2020/) Published on June 20, 2020 on the PLUG AI website (https://hp.package-ai.jp/)

The present invention relates to a program, a method for generating a learning model, and an information processing method.

Conventionally, when evaluating designs, an evaluation device has been disclosed that extracts a first characteristic amount of a first design to be evaluated, calculates the similarity between the first design and each of a plurality of existing second designs based on the extracted first characteristic amount, and predicts customer impressions of the first design based on the calculated similarity amount and previously acquired information indicating customer impressions of each of the second designs (see, for example, Patent Document 1).

JP 2018-195078 A

However, the invention described in Patent Document 1 may not be able to properly evaluate package design.

One aspect is to provide programs etc. that enable appropriate evaluation of package design.

In one aspect, the program causes a computer to execute a process of acquiring a package image including a product package design, accepting a selection of gender or age, inputting the acquired package image into a first learning model that has been trained to output information relating to the favorability of the package design corresponding to the accepted gender or age when the package image is input, and outputting information relating to the favorability of the package design corresponding to the accepted gender or age.

On one hand, it makes it possible to properly evaluate package design.

FIG. 1 is an explanatory diagram showing an overview of an evaluation system for package design. FIG. 2 is a block diagram showing an example of the configuration of a server. FIG. 2 is an explanatory diagram illustrating an example of a record layout of a training data DB. FIG. 2 is a block diagram showing a configuration example of a terminal. FIG. 13 is an explanatory diagram for explaining a favorability prediction model. FIG. 13 is an explanatory diagram for explaining an output process of information regarding the degree of favorability for all genders and all age groups. 13 is a flowchart showing a processing procedure for outputting information regarding the degree of favorability for all genders and all age groups. FIG. 13 is an explanatory diagram showing an example of a display screen of a weighted average value of favorability. 13 is a flowchart showing a processing procedure for outputting information related to a degree of favorability. FIG. 11 is an explanatory diagram showing an example of a record layout of a training data DB for learning using a score. FIG. 11 is an explanatory diagram illustrating a favorability prediction model that is trained using scores. 13 is a flowchart showing a processing procedure for outputting a package image and a heat map in association with each other. FIG. 13 is an explanatory diagram illustrating an example of a heat map display screen. FIG. 13 is a block diagram showing an example of the configuration of a server according to a third embodiment. FIG. 13 is an explanatory diagram showing an example of a record layout of a training data DB according to the third embodiment. 13 is a flowchart showing a processing procedure for identifying an image word. FIG. 13 is an explanatory diagram for explaining an appearance ratio prediction model. FIG. 13 is an explanatory diagram showing an example of a display screen of the appearance rate of image words. 13 is a flowchart showing a processing procedure for outputting the appearance rate of an image word. FIG. 13 is a block diagram showing an example of the configuration of a server according to a fourth embodiment. FIG. 13 is an explanatory diagram showing an example of a record layout of a training data DB according to the fourth embodiment. FIG. 11 is an explanatory diagram illustrating a standard deviation prediction model. FIG. 13 is an explanatory diagram showing an example of a display screen of the standard deviation of favorability levels. 13 is a flowchart showing a processing procedure for outputting a standard deviation of favorability levels. FIG. 13 is an explanatory diagram showing an example of a display screen for various prediction results.

The present invention will be described in detail below with reference to drawings showing an embodiment of the invention.

(Embodiment 1)
The first embodiment relates to a form in which information on the favorability of a product package design is output using artificial intelligence (AI) based on a package image including the package design of the product. Package design refers to planning, verifying, and devising (designing) graphics or shapes for products that require packaging or containers, such as food, beverages, and daily necessities, taking into consideration the product characteristics, concept, usability, etc.

Figure 1 is an explanatory diagram showing an overview of an evaluation system for package design. The system of this embodiment includes an information processing device 1 and an information processing terminal 2, and each device transmits and receives information via a network N such as the Internet.

The information processing device 1 is an information processing device that processes, stores, and transmits/receives various types of information. The information processing device 1 is, for example, a server device, a personal computer, or a general-purpose tablet PC (personal computer). In this embodiment, the information processing device 1 is a server device, and for the sake of simplicity, will be referred to as server 1 below.

The information processing terminal 2 is a terminal device that accepts and transmits the selection of package images, including the product package design, and receives and displays information related to the favorability of the package design. The information processing terminal 2 is, for example, an information processing device such as a smartphone, a mobile phone, a tablet, or a personal computer terminal. For the sake of simplicity, the information processing terminal 2 will be read as terminal 2 below.

The server 1 according to this embodiment acquires a package image including a product package design from the terminal 2. The server 1 accepts the user's selection of gender or age. When a package image is input, the server 1 inputs the acquired package image into a favorability prediction model (first learning model) that has been trained to output information regarding the favorability of the package design corresponding to the received gender or age, and outputs information regarding the favorability of the package design corresponding to the received gender or age. The favorability prediction model will be described later. The server 1 transmits the output information regarding the favorability of the package design to the terminal 2.

Figure 2 is a block diagram showing an example of the configuration of the server 1. The server 1 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, a display unit 15, a reading unit 16, and a large-capacity storage unit 17. Each component is connected by a bus B.

The control unit 11 includes a processor such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), and GPU (Graphics Processing Unit), and performs various information processing, control processing, and the like related to the server 1 by reading and executing the control program 1P stored in the storage unit 12. The control program 1P can be deployed so that it is executed on a single computer or at one site, or distributed across multiple sites and on multiple computers interconnected by a communications network. Although the control unit 11 is described in FIG. 2 as being a single processor, it may also be a multi-processor.

The storage unit 12 includes memory elements such as RAM (Random Access Memory) and ROM (Read Only Memory), and stores the control program 1P or data required for the control unit 11 to execute processing. The storage unit 12 also temporarily stores data required for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing processing related to communication, and transmits and receives information to and from the terminal 2 via the network N.

The input unit 14 is an input device such as a mouse, keyboard, touch panel, or button, and outputs received operation information to the control unit 11. The display unit 15 is a liquid crystal display or an organic EL (electroluminescence) display, etc., and displays various information according to instructions from the control unit 11.

The reading unit 16 reads the portable storage medium 1a, which may be a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 16 and store it in the mass storage unit 17. The control unit 11 may also download the control program 1P from another computer via a network N or the like and store it in the mass storage unit 17. The control unit 11 may also read the control program 1P from the semiconductor memory 1b.

The mass storage unit 17 includes a recording medium such as a hard disk drive (HDD) or a solid state drive (SSD). The mass storage unit 17 includes a favorability prediction model 171 and a training data database (DB) 172. The favorability prediction model 171 is a predictor that predicts information about the favorability of a package design included in a package image based on the package image, and is a trained model generated by machine learning. The training data DB 172 stores training data for constructing (generating) the favorability prediction model 171. In this embodiment, the training data is stored in the training data DB 172, but the method of storing the training data is not particularly limited. For example, the training data may be stored in a trained calculation file.

In this embodiment, the memory unit 12 and the large-capacity memory unit 17 may be configured as an integrated memory device. The large-capacity memory unit 17 may also be configured from multiple memory devices. Furthermore, the large-capacity memory unit 17 may be an external memory device connected to the server 1.

Server 1 may be executed on a single computer, or may be executed in a distributed manner on multiple computers, or may be executed in a distributed manner on virtual machines.

FIG. 3 is an explanatory diagram showing an example of a record layout of the training data DB 172. The training data DB 172 includes an image ID column, an image column, a gender column, an age column, and a favorability column. The image ID column stores a unique image ID to identify each image. The image column stores package image data. The gender column stores the gender of the user who evaluated the favorability of the package design. The age column stores the age of the user who evaluated the favorability of the package design. The favorability column stores the favorability of the package design. The favorability is an evaluation of the impression of the product's package design from the perspective of likes and dislikes. In this embodiment, the favorability is classified from a maximum of 5 (like) to a minimum of 1 (dislike or not like). Note that the favorability values described above are merely examples and are not limited to these.

Figure 4 is a block diagram showing an example of the configuration of the terminal 2. The terminal 2 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, a display unit 25, and an image capture unit 26. Each component is connected by a bus B.

The control unit 21 includes an arithmetic processing device such as a CPU or MPU, and performs various information processing, control processing, etc. related to the terminal 2 by reading and executing the control program 2P stored in the memory unit 22. Note that while the control unit 21 is described as a single processor in FIG. 4, it may be a multiprocessor. The memory unit 22 includes memory elements such as RAM and ROM, and stores the control program 2P or data, etc. required for the control unit 21 to execute processing. The memory unit 22 also temporarily stores data, etc. required for the control unit 21 to execute arithmetic processing.

The communication unit 23 is a communication module for performing communication-related processing, and transmits and receives information to and from the server 1 via the network N. The input unit 24 may be a keyboard, a mouse, or a touch panel integrated with the display unit 25. The display unit 25 is a liquid crystal display or an organic EL display, etc., and displays various information according to instructions from the control unit 21.

The photographing unit 26 is, for example, a photographing device such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera. The photographing unit 26 may be composed of multiple photographing devices. The photographing unit 26 may not be built into the terminal 2, but may be externally connected directly to the terminal 2 and configured to be capable of photographing.

Next, the process of outputting information regarding the favorability of a package design will be described. Terminal 2 acquires a package image including the package design of the product. The package image may be an image captured by the photographing unit 26 of terminal 2, or an image stored in advance in the memory unit 22 or a photo album. Terminal 2 accepts the user's selection of gender or age. Specifically, terminal 2 accepts the selection of gender, age, or a combination of gender and age. Terminal 2 transmits the acquired package image and the accepted gender or age to server 1.

The server 1 receives the package image and the gender or age group transmitted from the terminal 2. When a package image is input, the server 1 inputs the received package image into a favorability prediction model 171 that has been trained to output information regarding the favorability of the package design corresponding to the received gender or age group, and outputs information regarding the favorability of the package design corresponding to the received gender or age group.

The information relating to the level of favorability includes a score classifying the level of favorability from a maximum of 5 (like) to a minimum of 1 (dislike), or a weighted average of the level of favorability scores from a maximum of 5 to a minimum of 1. The output process of the information relating to the level of favorability using the level of favorability prediction model 171 will be described later. The server 1 transmits the output information relating to the level of favorability of the package design to the terminal 2. The terminal 2 receives the information relating to the level of favorability transmitted from the server 1, and displays the received information relating to the level of favorability on the screen.

Next, the output process of information related to favorability using the favorability prediction model 171 will be described. FIG. 5 is an explanatory diagram of the favorability prediction model 171. The favorability prediction model 171 is used as a program module that is part of artificial intelligence software. The favorability prediction model 171 is a predictor of information related to favorability that has constructed a neural network that receives as input a package image including a package design and outputs information related to the favorability of the package design.

The server 1 constructs (generates) the favorability prediction model 171 by performing deep learning to learn the feature quantities of the parts of the package design contained in the package image that affect the favorability. For example, the favorability prediction model 171 is a CNN (Convolution Neural Network) that has an input layer that accepts input of the package image, an output layer that outputs information related to the favorability of the package design contained in the package image, and an intermediate layer that has been trained by backpropagation.

The input layer has a number of neurons that accept input of pixel values of each pixel included in the package image, and passes the input pixel values to the intermediate layer. The intermediate layer has a number of neurons that extract features of the package image, and passes the extracted image features to the output layer. Specifically, the intermediate layer is configured with an alternating convolution layer that convolves the pixel values of each pixel input from the input layer, and a pooling layer that maps the pixel values convolved in the convolution layer, compressing the pixel information of the package image while ultimately extracting the features of the package image. The intermediate layer then predicts information related to the favorability of the package design included in the package image, using a fully connected layer whose parameters have been learned by backpropagation. The prediction result is output to the output layer, which has neurons.

In addition, the package image may be input to the input layer after passing through an alternating convolution layer and a pooling layer to extract features.

In this embodiment, the network structure of ResNet50 (Residual Network50) may be applied as the favorability prediction model 171. ResNet50 is a CNN model with 50 layers. Note that, instead of the above-mentioned ResNet50, for example, VGG16, InceptionV3, DenseNet, InceptionResNetV2, etc. may be used.

In this embodiment, the favorability prediction model 171 is described as a CNN, but the favorability prediction model 171 is not limited to a CNN and may be a trained model constructed with any learning algorithm, such as a neural network other than a CNN, an SVM (Support Vector Machine), a Bayesian network, or a regression tree.

The server 1 acquires from the training data DB 172 a number of combinations of training data in which package images including package designs are associated with the favorability of the package designs. The training data is data in which package images are labeled with the favorability of the package designs included in the package images. Note that, in the present embodiment, an example has been described in which the favorability levels are classified into five levels, from a maximum of 5 (like) to a minimum of 1 (dislike), but this is not limiting. The favorability levels may be set according to actual needs.

In the present embodiment, an example has been described in which pre-stored training data is acquired from training data DB 172, but the present invention is not limited to this. For example, server 1 may generate training data for learning the favorability level by labeling the favorability level of the package design contained in each of a plurality of package images. Specifically, server 1 generates training data in which a favorability level (e.g., 1, 2, 3, 4, or 5) is associated with each package image.

The server 1 performs learning using the acquired training data. Specifically, the server 1 inputs a package image, which is training data, into the input layer, and acquires information about the favorability of the package design contained in the package image from the output layer after arithmetic processing in the intermediate layer. The information about the favorability includes a probability (likelihood) value indicating the likelihood of the favorability, or a weighted average value of the favorability. Figure 5 explains an example of the weighted average value of the favorability. As shown in the figure, the prediction result output from the output layer is a weighted average value of the favorability (from favorability level 5 to favorability level 1).

The server 1 compares the prediction results output from the output layer with the information labeled for the package images in the training data, i.e., the correct answer value, and optimizes the parameters used in the calculation process in the intermediate layer so that the predicted value from the output layer approaches the correct answer value. The parameters in question are, for example, the weights (coupling coefficients) between neurons and the coefficients of the activation functions used in each neuron. There are no particular limitations on the method of optimizing the parameters, but for example, the server 1 optimizes various parameters using the backpropagation method.

The server 1 performs the above process for each package image included in the training data to generate a favorability prediction model 171. As a result, for example, the server 1 can use the training data to train the favorability prediction model 171, thereby constructing a model that can output information related to the favorability of the package design included in the package image.

When the server 1 acquires a package image, it inputs the acquired package image to the favorability prediction model 171. The server 1 performs a calculation process in the middle layer of the favorability prediction model 171 to extract features of each package image. The server 1 inputs the extracted features to the output layer of the favorability prediction model 171, and acquires information regarding the favorability of the package design contained in the package image as an output. As shown in the figure, the weighted average value of the favorability is output from the output layer. The weighted average value of the favorability is "3.84". Note that the calculated weighted average value of the favorability may be corrected by rounding up, rounding down, or the like.

In the above example, the weighted average value of the favorability is output as the prediction result, but this is not limited to this. For example, the probability value itself indicating each favorability may be output as the prediction result. For example, the output layer outputs prediction results in which the probability values of "favorability 5", "favorability 4", "favorability 3", "favorability 2", and "favorability 1" for the package image are "0.05", "0.85", "0.03", "0.03", and "0.04", respectively. In addition, the prediction result may be output using a predetermined threshold. For example, when the server 1 determines that the probability value of "favorability 4" (0.85) is equal to or greater than a predetermined threshold (e.g., 0.80), it outputs a prediction result in which the favorability of the package design is "favorability 4". In addition, the favorability corresponding to the highest probability value from among the probability values of each favorability predicted by the favorability prediction model 171 may be output as the prediction result without using the above threshold.

Note that the prediction process for information related to favorability is not limited to the prediction process using machine learning described above. For example, information related to favorability may be predicted using local feature extraction methods such as A-KAZE (Accelerated KAZE), SIFT (Scale Invariant Feature Transform), SURF (Speed-Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), and HOG (Histograms of Oriented Gradients).

Next, we will explain the process of outputting information regarding the degree of favorability for all genders and all age groups based on the information regarding the degree of favorability output from multiple favorability prediction models 171 prepared for each gender or age group.

Figure 6 is an explanatory diagram explaining the output process of information regarding the favorability level for all genders and all age groups. The server 1 acquires a package image from the terminal 2. The server 1 acquires each of a plurality of favorability prediction models 171 prepared corresponding to each gender and age group from the mass storage unit 17.

The server 1 inputs the acquired package images into each acquired favorability prediction model 171, and outputs the weighted average favorability of the package designs corresponding to each gender and age group. The server 1 calculates the average value of the weighted average favorability based on the output weighted average favorability of the package designs corresponding to each gender and age group. The server 1 outputs the calculated average value of the weighted average favorability as the favorability for all genders and all age groups.

As shown in the figure, in the favorability prediction models 171 corresponding to men in their 20s, women in their 20s, men in their 30s, women in their 30s, men in their 40s, women in their 40s, men in their 50s, and women in their 50s, the server 1 inputs package images into each favorability prediction model 171 and outputs weighted average favorability values of package designs corresponding to men in their 20s, women in their 20s, men in their 30s, women in their 30s, men in their 40s, women in their 40s, men in their 50s, and women in their 50s. The server 1 calculates an average value of the weighted average favorability values based on the output weighted average favorability values for each gender and age group. The server 1 outputs the calculated average value of the weighted average favorability values as the favorability values for all genders and age groups. Note that while an example of the weighted average favorability values has been described, the same can be applied to the favorability values themselves.

Note that in FIG. 6, an example of multiple favorability prediction models 171 prepared corresponding to each gender and age group is described, but the same can be applied to multiple favorability prediction models 171 prepared for each gender, or multiple favorability prediction models 171 prepared for each age group.

Figure 7 is a flowchart showing the processing steps when outputting information regarding the degree of favorability for all genders and all age groups. The control unit 11 of the server 1 acquires a package image from the terminal 2 by the communication unit 13 (step S101). The control unit 11 acquires multiple genders and age groups accepted through the terminal 2 via the communication unit 13 (step S102). For example, the control unit 11 accepts inputs of male in his 20s, female in his 20s, male in his 30s, female in his 30s, male in his 40s, female in his 40s, male in his 50s, and female in her 50s.

The control unit 11 acquires a favorability prediction model 171 for each gender and age group from the mass storage unit 17 according to the input gender and age group (step S103). The control unit 11 inputs the acquired package image into the favorability prediction model 171 corresponding to each gender and age group, and outputs a weighted average value of the favorability of the package design included in the package image for each gender and age group (step S104). The control unit 11 calculates the average value of the weighted average value of the favorability based on the output weighted average value of the favorability corresponding to each gender and age group (step S105), and ends the process.

Figure 8 is an explanatory diagram showing an example of a display screen for weighted average favorability. The screen includes a package image display field 11a, a favorability display field 11b, and a selection order display field 11c. The package image display field 11a is a display field that displays the package images and file names of the package images to be evaluated. The favorability display field 11b is a display field that displays the weighted average favorability for the whole group (all genders and all ages), gender, age, or gender and age (for example, women in their 20s). The selection order display field 11c is a display field that displays the order in which the package images were selected.

The terminal 2 accepts the selection of multiple package images and stores the order of selection of the accepted package images in the memory unit 22. The terminal 2 accepts the setting of evaluation conditions for evaluating the package images. The evaluation conditions are conditions for evaluating the package images. The evaluation conditions include evaluation by overall (all genders and all ages), evaluation by gender, evaluation by age, evaluation by gender and age (for example, women in their 20s), etc. Note that one or a combination of two or more of the overall, gender, age, or gender and age may be set as the evaluation condition.

The terminal 2 transmits the received multiple package images and the set evaluation conditions to the server 1. The server 1 receives the multiple package images and evaluation conditions transmitted from the terminal 2. The server 1 obtains a weighted average value of the favorability of the corresponding package design according to the received evaluation conditions.

For example, if male is set as the evaluation condition, the server 1 inputs the multiple acquired package images into the favorability prediction model 171 corresponding to male, and outputs the weighted average value of the favorability of the package design included in each package image corresponding to male. Alternatively, if gender and age are set as the evaluation condition, the server 1 inputs the multiple acquired package images into the favorability prediction model 171 corresponding to the set gender and age, and outputs the weighted average value of the favorability of the package design included in each package image corresponding to the set gender and age.

Furthermore, when the entirety is set as the evaluation condition, the server 1 inputs each acquired package image into the favorability prediction model 171 corresponding to each gender and age group, and outputs the weighted average favorability of the package design included in each package image corresponding to each gender and age group. The server 1 calculates the average value of the weighted average favorability based on the weighted average favorability of each acquired package image. The server 1 outputs the average value of the calculated weighted average favorability as the weighted average favorability of the entire package image.

The server 1 transmits the weighted average value of the favorability obtained according to the evaluation conditions to the terminal 2 in association with each package image. The terminal 2 displays the weighted average value of the favorability transmitted from the server 1 on the screen in association with each package image.

Specifically, the terminal 2 displays the package images and the file names of the package images in the package image display field 11a. The terminal 2 displays the weighted average value of the favorability corresponding to the package images and a vertical bar indicating the weighted average value in the favorability display field 11b according to the set evaluation conditions. The terminal 2 acquires the order of selection of the package images from the memory unit 22, and displays the acquired order of selection in the selection order display field 11c.

As shown, package images and the file names of the package images (e.g., 1.jpg, 2.jpg) are displayed in package image display field 11a. Each package image is associated with a weighted average favorability score corresponding to each package image for overall, male, female, female in their 20s, and male in their 20s, and a vertical bar indicating the weighted average favorability score is displayed in favorability display field 11b. The order in which the package images were selected (1, 2) is displayed in selection order display field 11c.

Figure 9 is a flowchart showing the processing steps when outputting information related to the degree of favorability. The control unit 21 of the terminal 2 acquires a package image including a package design using the memory unit 22 or the photographing unit 26 (step S211). The control unit 21 may acquire a single package image or may acquire multiple package images (e.g., 10 images). The control unit 21 accepts, via the input unit 24, the setting of evaluation conditions set based on one or a combination of two or more of the following, in order to evaluate the package image (step S212).

The control unit 21 transmits the acquired package image and the accepted evaluation conditions to the server 1 via the communication unit 23 (step S213). The control unit 11 of the server 1 receives the package image and evaluation conditions transmitted from the terminal 2 via the communication unit 13 (step S111). The control unit 11 identifies a learned favorability prediction model 171 according to the received evaluation conditions (step S112). The control unit 11 inputs the received package image into the identified favorability prediction model 171, and outputs information related to the favorability of the package design included in the package image according to the set evaluation conditions (step S113). Note that in the case of multiple package images, the control unit 11 inputs each of the received package images into the favorability prediction model 171, and outputs information related to the favorability of the package design included in each of the package images according to the set evaluation conditions.

The control unit 11 transmits the output information relating to the degree of favorability of the package design to the terminal 2 via the communication unit 13 (step S114). The control unit 21 of the terminal 2 receives the information relating to the degree of favorability transmitted from the server 1 via the communication unit 23 (step S214). The control unit 21 displays the received information relating to the degree of favorability via the display unit 25 in association with the set evaluation conditions (step S215), and ends the process.

According to this embodiment, it is possible to output information regarding the favorability of a package design using the favorability prediction model 171 based on a package image including the package design.

According to this embodiment, by outputting the degree of preference for package designs by gender or age group, it is possible to understand the degree of preference that varies depending on the user's gender and age group.

<Modification 1>
A process of training the favorability prediction model 171 using a score set for each product category will be described. Product categories include, for example, beer, chuhai/cocktails, carbonated drinks, tea/black tea/coffee, energy drinks, seasonings, instant noodles/bag noodles, soups, desserts, room temperature foods or frozen foods, etc. Other product categories may include detergents, finishing agents, water bottles, cosmetics, etc.

Figure 10 is an explanatory diagram showing an example of a record layout of training data DB172 for learning using scores. Note that the same reference numerals are used for the contents that overlap with those in Figure 3, and the description will be omitted. Training data DB172 includes a category column and a score column. The category column stores the product category. The score column stores the evaluation index set for each product category.

First, the process of creating the score will be explained. The server 1 accepts input of the brand name and company name for searching for products for each product category. The brand name and company name are, for example, "Brand A AAA Co., Ltd." Note that the brand name and company name are not limited to "brand + company name" and may be, for example, "product name + company name" or "brand name + category name." The server 1 obtains the number of search hits for the accepted brand name and company name via a search engine. The search engine may be, for example, GOOGLE (registered trademark) or YAHOO (registered trademark), etc.

The server 1 calculates the logarithm of the number of search hits for the acquired brand name and company name, and outputs the calculated logarithm of the number of search hits as a score. The server 1 calculates the average score for multiple brands within the product category. The server 1 stores the calculated average score in the training data DB 172 in association with the product category.

Next, a process of training the favorability prediction model 171 using the scores will be described.
Fig. 11 is an explanatory diagram for explaining the favorability prediction model 171 that is trained using scores. Description of the contents that overlap with Fig. 5 will be omitted. The favorability prediction model 171 of Modification 1 is applied with the network structure of ResNet50. The server 1 acquires from the training data DB 172 a plurality of combinations of training data in which package images, scores corresponding to the package images, and favorability of the package design included in the package images are associated with each other.

The server 1 performs learning using the acquired training data. Specifically, the server 1 inputs the package images, which are the training data, to the network 16a of ResNet 50 excluding the fully connected layer (FC) 16c. The network 16a extracts the features of the package image and passes the extracted features of the package image to the fully connected layer 16c, whose parameters have been learned by backpropagation. The server 1 inputs the score corresponding to the package image to the fully connected layer 16b. The fully connected layer 16c combines the features of the package image with the score input to the fully connected layer 16b, and predicts information related to the favorability of the package design based on the combined information. The prediction result is output to an output layer having neurons.

The server 1 compares the prediction results output from the output layer with the information labeled for the package image and score in the training data, i.e., the correct answer value, and optimizes the parameters used in the calculation process in the intermediate layer so that the predicted value from the output layer approaches the correct answer value. In this way, the server 1 uses the training data to train the favorability prediction model 171, and updates the trained favorability prediction model 171 in the large-capacity storage unit 17.

When the server 1 acquires a package image and a score corresponding to the package image, the server 1 inputs the acquired package image and score into the favorability prediction model 171 and outputs information regarding the favorability of the package design.

Note that the process is not limited to the above. The favorability prediction model 171 may be trained using the product category and the score set for each product category. In this case, the server 1 inputs the package image to the network 16a. The network 16a extracts the feature amount of the package image and passes the extracted feature amount of the package image to the fully connected layer 16c. The server 1 inputs vector information obtained by combining the label indicating the product category and the score set for each category into one vector to the fully connected layer 16b. The label indicating the category may be, for example, 26 types of labels including beer, chuhai/cocktail, other alcohol, carbonated drinks, tea/black tea/coffee, instant drinks, nutritional drinks, other drinks, seasonings, cup noodles/bag noodles, soups, desserts, room temperature foods, frozen foods, refrigerated foods, chocolate/candy/gummy, ice cream, snacks, other sweets, cosmetics, shampoo, medicine, lip balm, sanitary products, pet food, and hot pot soup. Note that the 26 types of labels described above are only an example and are not limited to this.

The fully connected layer 16c predicts information about the favorability of the package design based on the features of the package image and the vector information input to the fully connected layer 16b. The prediction results are output to an output layer having neurons.

According to this variant example 1, it is possible to train the favorability prediction model 171 using scores set for each product category.

According to this variant example 1, by using the favorability prediction model 171 trained using the scores, it is possible to improve the prediction accuracy of information related to favorability.

(Embodiment 2)
The second embodiment relates to a form in which a package image including a product package design and an index indicating an area that has influenced information related to the favorability of the package design are output in association with each other. Note that a description of the contents that overlap with the first embodiment will be omitted. Note that in this embodiment, the index indicating an area that has influenced information related to the favorability is described in the form of a heat map, but this is not limiting and other data visualization techniques may be used.

Based on the multiple package images transmitted from the terminal 2, the server 1 generates a heat map corresponding to each package image. The server 1 transmits each generated heat map to the terminal 2 in association with the respective package image. The terminal 2 receives each package image and the heat map corresponding to each package image transmitted from the server 1. The terminal 2 displays each received package image and the heat map corresponding to each package image on the screen in association with each other.

Figure 12 is a flowchart showing the processing steps for outputting package images in association with a heat map. The control unit 21 of the terminal 2 acquires a plurality of package images using the storage unit 22 or the photographing unit 26 (step S221). The control unit 21 transmits the acquired plurality of package images to the server 1 via the communication unit 23 (step S222).

The control unit 11 of the server 1 receives a plurality of package images from the terminal 2 via the communication unit 13 (step S121). The control unit 11 inputs each of the received package images into the favorability prediction model 171 and outputs information related to the favorability of the package design contained in each package image (step S122). The control unit 11 extracts areas from each package image that influenced the information related to the favorability output by the favorability prediction model 171 (step S123). The control unit 11 generates a heat map based on indices indicating each of the extracted areas (step S124).

Specifically, the control unit 11 calculates the backpropagation of the fully connected layer of the favorability prediction model 171. The control unit 11 generates a heat map of the package image based on the package image whose features have been extracted by the intermediate layer of the favorability prediction model 171 obtained by backpropagation. The heat map can be generated by a known method such as CAM (Class Activation Mapping), Grad-CAM (Gradient-weighted Class Activation Mapping), or Grad-CAM++.

The control unit 11 transmits each of the generated heat maps to the terminal 2 via the communication unit 13 in association with each package image (step S125). The control unit 21 of the terminal 2 receives each package image and the heat maps corresponding to each package image transmitted from the server 1 via the communication unit 23 (step S223). The control unit 21 displays each of the received package images and the heat maps corresponding to each package image on the screen in association with each other via the display unit 25 (step S224), and ends the process.

Figure 13 is an explanatory diagram showing an example of a heat map display screen. The screen includes a package image display field 12a, a heat map display field 12b, and a file name display field 12c. The package image display field 12a is a display field that displays a package image. The heat map display field 12b is a display field that displays a heat map corresponding to the package image. The file name display field 12c is a display field that displays the file name of the package image.

The terminal 2 transmits a plurality of package images to the server 1. The server 1 generates a heat map corresponding to each of the package images based on the plurality of package images transmitted from the terminal 2. The server 1 transmits each of the generated heat maps to the terminal 2 in association with each of the package images. The terminal 2 receives each of the package images transmitted from the server 1 and the heat maps corresponding to each of the package images.

Terminal 2 displays on the screen each of the received package images in association with a heat map corresponding to the package image. Specifically, terminal 2 displays the received package image in package image display field 12a, displays a heat map corresponding to the package image in heat map display field 12b, and displays the file name of the package image in file name display field 12c. For example, based on the feature amount of the package image, the information regarding the favorability level may be expressed in the form of a heat map such that the area having the greatest influence (area that attracted attention) is red, the area having the second greatest influence (area that attracted some attention) is orange, and areas that have no influence (areas that are not attracted) are blue. Note that in FIG. 13, hatching is used instead of color for the example.

In addition, the terminal 2 may acquire information regarding the favorability of the package design contained in the package image, and display the acquired information regarding the favorability and a heat map corresponding to the package image on the screen at the same time.

According to this embodiment, it is possible to output a package image including the package design in association with an index showing the area that influenced the information regarding the favorability of the package design.

According to this embodiment, by displaying an indicator showing the areas that influenced the information related to favorability, it becomes possible to clearly see the important points in the package design.

(Embodiment 3)
The third embodiment relates to a form in which an appearance rate of image words in a package design of a product is output using artificial intelligence based on a package image including the package design of the product. Note that a description of the contents that overlap with the first and second embodiments will be omitted.

Fig. 14 is a block diagram showing an example of the configuration of the server 1 of the third embodiment. Note that a description of the contents overlapping with Fig. 2 will be omitted. The mass storage unit 17 stores an appearance ratio prediction model 173 (second learning model). The appearance ratio prediction model 173 is a predictor that predicts the appearance ratio of image words of the package design contained in the package image based on the package image, and is a learned model generated by machine learning.

Figure 15 is an explanatory diagram showing an example of a record layout of training data DB172 in embodiment 3. Note that the same reference numerals are used for the contents that overlap with those in Figure 3, and the description will be omitted. Training data DB172 includes an image word string. The image word string stores characters (text) that are used to express reasons for favorability when evaluating package designs.

Image words include, for example, "delicious" (no non-food related words), "cute", "simple", "good design elements", "nostalgic", "gentle", "sense of security and trust", "seasonal", "healthy", "effective", "luxurious", "good color", "new/unique", "sophisticated", "refreshing", "easy to understand", "noticeable", "memorable", "pretty" or "clean". Note that image words are not limited to those mentioned above, and any image words can be set according to actual needs.

Next, the image word specification process will be described.
16 is a flowchart showing a processing procedure for identifying image words. The control unit 11 of the server 1 acquires a set of evaluation texts that evaluate a package design (step S131). The set of evaluation texts may be, for example, a set of reviews or impressions written in any format about a package design, or may be evaluation information collected from responses to a questionnaire. When the set of evaluation texts is stored in the memory unit 12 or the mass storage unit 17, the control unit 11 acquires the set of evaluation texts from the memory unit 12 or the mass storage unit 17. The control unit 11 may acquire the set of evaluation texts from an external device via the communication unit 13.

The control unit 11 extracts a list of keywords from the acquired evaluation text set (step S132). Specifically, the control unit 11 extracts a list of keywords whose frequency of occurrence is equal to or greater than a predetermined frequency of occurrence (for example, 1000 times) from the acquired evaluation text set. The control unit 11 eliminates negative words or meaningless words from the extracted keyword list. Negative words include, for example, "sad," "painful," "unsatisfactory," "bad," "troubled," or "confused." For example, the control unit 11 performs a morphological analysis on the extracted keyword list, and further identifies negative words or meaningless words using an algorithm that scores each word that appears in the keyword list, such as tf-idf (Term Frequency-Inverse Document Frequency). The control unit 11 eliminates the identified negative words or meaningless words from the extracted keyword list.

The control unit 11 performs a vectorization process on the list of keywords from which negative words and meaningless words have been excluded (step S133). For example, the control unit 11 may vectorize each keyword included in the list of keywords using a known method for vectorizing words, such as Word2vec. Note that vectorization may be performed using Word2Vec, Doc2Vec (Paragraph2vec), LDA (Latent Dirichlet Allocation), NTSG (Neural Tensor Skip Gram), or the like.

The control unit 11 classifies (cluster) each keyword based on the characteristic information of each vectorized keyword (step S134). Clustering is a type of natural language processing in which vectorized keywords are divided into groups (clusters) of words that are close to the same concept based on their direction and size.

The control unit 11 identifies image words for the package design from each of the classified keywords (step S135), and ends the process. For example, the control unit 11 may accept input of image words for each cluster. Note that in the process of step S133, keywords that have not been vectorized (e.g., delicious) may be added to the image words by manual input. The identified image words are associated with package images and used when generating training data, which will be described later. Note that, although the process of identifying image words is executed on the server 1 side in this embodiment, it is not limited to this, and may be executed on the terminal 2 or other information processing terminal side.

Next, a process of outputting the appearance ratio of image words in a package design using the appearance ratio prediction model 173 will be described.
17 is an explanatory diagram for explaining the appearance ratio prediction model 173. The appearance ratio prediction model 173 is used as a program module that is a part of artificial intelligence software. The appearance ratio prediction model 173 is a predictor of the appearance ratio of image words in which a neural network has been constructed, in which a package image including a package design is input and the appearance ratio of image words in the package design is output.

The server 1 constructs the appearance ratio prediction model 173 by performing deep learning to learn the feature quantities of the parts that influenced the appearance ratio of the image words of the package design included in the package image. For example, the appearance ratio prediction model 173 is a CNN, and has an input layer that accepts the input of the package image, an output layer (softmax layer) that outputs the appearance ratio of the image words of the package design included in the package image using an activation function (softmax function in this embodiment), and an intermediate layer that has been trained by backpropagation.

In this embodiment, the occurrence rate prediction model 173 is described as being a CNN, but the occurrence rate prediction model 173 is not limited to a CNN and may be a trained model constructed with any learning algorithm, such as a neural network other than a CNN, an SVM, a Bayesian network, or a regression tree.

The server 1 acquires from the training data DB 172 a plurality of combinations of training data in which a package image including a package design is associated with an image word of the package design. The training data is data in which a package image is labeled with an image word of the package design included in the package image.

The server 1 performs learning using the acquired training data. Specifically, the server 1 inputs the package images, which are the training data, into the input layer, and acquires the occurrence rate of the image words of the package design contained in the package images from the output layer after arithmetic processing in the middle layer. The server 1 compares the prediction result output from the output layer (e.g., the image word with the highest occurrence rate) with the information (image word) labeled for the package image in the training data, i.e., the correct answer value, and optimizes the parameters used in the arithmetic processing in the middle layer so that the predicted value from the output layer approaches the correct answer value.

The server 1 performs the above process for each package image included in the training data to generate an appearance rate prediction model 173. As a result, for example, the server 1 can use the training data to train the appearance rate prediction model 173, thereby constructing a model that can output the appearance rate of image words in package designs.

When the server 1 acquires a package image, it inputs the acquired package image to the appearance rate prediction model 173. The server 1 performs a calculation process to extract features of each package image in the middle layer of the appearance rate prediction model 173. The server 1 inputs the extracted features to the output layer of the appearance rate prediction model 173, and acquires as an output the appearance rate of image words of the package design included in the package image.

As shown in the figure, the predicted results are output that show the respective occurrence rates of "cute," "simple," "seasonal," "...," and "healthy" for the package image as "0.82," "0.55," "0.35," "...," and "0.25."

In addition, an appearance rate prediction model 173 can be generated for each gender or age group. Specifically, the server 1 accepts a setting of the gender or age group, and acquires multiple pieces of training data to be used for generating the appearance rate prediction model 173 from the training data DB 172 according to the accepted gender or age group. The server 1 uses the acquired training data to generate an appearance rate prediction model 173 corresponding to the accepted gender or age group.

The server 1 stores the occurrence rate prediction model 173 generated for each gender or age group in the mass storage unit 17. The server 1 can output the occurrence rate of image words of the package design from the package image for each gender or age group using multiple occurrence rate prediction models 173 prepared for multiple genders or age groups.

Note that the process is not limited to the above. The occurrence rate prediction model 173 may be trained using a score set for each product category. In this case, the server 1 inputs a package image including the package design and a score set for each category to the occurrence rate prediction model 173, and outputs the occurrence rate of image words for the package design.

Figure 18 is an explanatory diagram showing an example of a display screen of the occurrence rate of image words. The screen includes a selection combo box 13a and a graph display field 13b. The selection combo box 13a is a combo box for receiving the selection of an image word. The graph display field 13b is a display field for displaying a graph (first graph) showing the relationship between the package image and the occurrence rate of the image word.

Terminal 2 acquires multiple package images to be evaluated. Terminal 2 accepts the selection of image words. For example, terminal 2 acquires 10 package images (1.jpg, 2.jpg, ..., 10.jpg) and accepts the selection of "delicious." Terminal 2 transmits the acquired multiple package images and the accepted image words to server 1. Server 1 receives the multiple package images and image words transmitted from terminal 2.

The server 1 inputs each received package image into the occurrence rate prediction model 173 and outputs the occurrence rates of the multiple image words that were set during training of the occurrence rate prediction model 173. The server 1 obtains, from the occurrence rates of the multiple image words that have been output, an occurrence rate that corresponds to a selected image word (e.g., looks delicious) in association with each package image.

Based on the package image and the occurrence rate of the selected image word, the server 1 generates a vertical bar graph showing the relationship between the package image and the occurrence rate of the image word. The server 1 transmits the generated vertical bar graph to the terminal 2. The terminal 2 receives the vertical bar graph transmitted from the server 1 and displays the received vertical bar graph in the graph display field 13b.

As shown, the vertical axis of the vertical bar graph indicates the occurrence rate of the selected image word. The horizontal axis indicates each package image and the file name of the package image sorted based on descending order of the occurrence rate of the image word. Specifically, based on descending order of the predicted occurrence rate of the image word, the server 1 displays the package image and file name in the lower part, displays vertical bars indicating the occurrence rate of the image word in the center, and displays the occurrence rate (value) of the image word and the order at the time of selection of the package image in the upper part. Note that while the occurrence rate of the image word is displayed in descending order, this is not limiting, and it may also be displayed in ascending order or without sorting.

Note that while Figure 18 shows a vertical bar graph, this is not limiting and other graph formats may also be used.

Figure 19 is a flowchart showing the processing steps when outputting the appearance rate of an image word. The control unit 21 of the terminal 2 acquires a plurality of package images using the memory unit 22 or the photographing unit 26 (step S241). The control unit 21 transmits the acquired plurality of package images to the server 1 via the communication unit 23 (step S242). The control unit 11 of the server 1 receives the plurality of package images transmitted from the terminal 2 via the communication unit 13 (step S141).

The control unit 11 inputs each received package image into the appearance rate prediction model 173 and outputs the appearance rates of multiple image words set during learning of the appearance rate prediction model 173 (step S142). The image words set during learning include, for example, "delicious" (none for non-food and drink products), "cute", "simple", "good design elements", "nostalgic", "gentle", "sense of security and trust", "seasonal", "healthy", "sense of effectiveness", "luxurious and high-quality", "good color", "new and unique", "sophisticated", "refreshing and cool", "easy to understand features", "noticeable and memorable", "beautiful and pretty" and "clean".

The control unit 21 of the terminal 2 receives a selection of an image word to be evaluated (e.g., delicious) through the input unit 24 (step S243). The control unit 21 transmits the received image word to be evaluated to the server 1 through the communication unit 23 (step S244). The control unit 11 of the server 1 receives the image word to be evaluated transmitted from the terminal 2 through the communication unit 13 (step S143). The control unit 11 obtains an appearance rate corresponding to the image word to be evaluated in association with each package image from the appearance rates of the multiple image words output by the appearance rate prediction model 173 (step S144).

The control unit 11 generates a graph showing the relationship between the package image and the occurrence rate of the image word to be evaluated, based on the package image and the occurrence rate of the image word to be evaluated (step S145). The control unit 11 transmits the generated graph to the terminal 2 via the communication unit 13 (step S146). The control unit 21 of the terminal 2 receives the graph transmitted from the server 1 via the communication unit 23 (step S245), displays the received graph on the display unit 25 (step S246), and ends the process.

According to this embodiment, it is possible to output the appearance rate of image words of a package design using the appearance rate prediction model 173 based on a package image including the package design.

According to this embodiment, by outputting the occurrence rate of image words for the package design, it becomes possible to clarify the reasons for the favorability of the package design, and to plan an appropriate package design.

(Embodiment 4)
The fourth embodiment relates to a form in which, based on a package image including the package design of a product, the standard deviation of the favorability of the package design is output using artificial intelligence. Note that a description of the contents that overlap with the first to third embodiments will be omitted.

There are cases where the same package design is rated with different degrees of favorability depending on gender or age. The difference (variation) in liking and disliking a package design can be interpreted using the standard deviation of favorability. The higher the standard deviation of favorability, the greater the difference in likes and dislikes; conversely, the lower the standard deviation of favorability, the smaller the difference in likes and dislikes.

Fig. 20 is a block diagram showing an example of the configuration of the server 1 of the fourth embodiment. Note that a description of the contents overlapping with Fig. 14 will be omitted. The mass storage unit 17 stores a standard deviation prediction model 174 (third learning model). The standard deviation prediction model 174 is a predictor that predicts the standard deviation of the favorability of the package design contained in the package image based on the package image, and is a learned model generated by machine learning.

Figure 21 is an explanatory diagram showing an example of a record layout of the training data DB 172 of embodiment 4. Note that the same reference numerals are used for the contents that overlap with those in Figure 3, and the description will be omitted. The training data DB 172 includes a standard deviation column. The standard deviation column stores the standard deviation of the favorability of the package design.

Figure 22 is an explanatory diagram explaining the standard deviation prediction model 174. The standard deviation prediction model 174 is used as a program module that is part of artificial intelligence software. The standard deviation prediction model 174 is a predictor of the standard deviation of favorability that has already constructed a neural network that takes a package image including a package design as input and outputs the standard deviation of the favorability of the package design.

The server 1 constructs a standard deviation prediction model 174 by performing deep learning to learn the feature quantities of the parts of the package image that influence the favorability of the package design. For example, the standard deviation prediction model 174 is a CNN, and has an input layer that accepts input of a package image that includes a package design, an output layer that outputs the standard deviation of the favorability of the package design, and an intermediate layer that has been trained by backpropagation.

In this embodiment, the standard deviation prediction model 174 is described as a CNN, but the standard deviation prediction model 174 is not limited to a CNN and may be a trained model constructed with any learning algorithm, such as a neural network other than a CNN, an SVM, a Bayesian network, or a regression tree.

The server 1 acquires from the training data DB 172 a number of combinations of training data in which package images including a package design are associated with standard deviations calculated based on information regarding the degree of favorability of the package design among a number of people. The training data is data in which package images including a package design are labeled with the standard deviation of the degree of favorability of the package design.

To create training data, for example, the server 1 obtains the degree of favorability of responses from 1,000 people for one package image on a five-point scale ranging from a maximum of 5 (like) to a minimum of 1 (dislike). The server 1 calculates the average value of the degrees of favorability of the responses from the 1,000 people. The server 1 calculates the standard deviation of the degrees of favorability of the responses from the 1,000 people based on the calculated average degree of favorability. The server 1 can create training data by labeling the package image with the standard deviation of the degrees of favorability of the package design from the 1,000 people.

The server 1 performs learning using the acquired training data. Specifically, the server 1 inputs the package images, which are the training data, into the input layer, and obtains the standard deviation of the favorability of the package design contained in the package images from the output layer after arithmetic processing in the intermediate layer. The server 1 compares the prediction result output from the output layer with the information labeled for the package image in the training data, i.e., the correct answer value, and optimizes the parameters used in the arithmetic processing in the intermediate layer so that the predicted value from the output layer approaches the correct answer value.

The server 1 performs the above process for each package image included in the training data to generate a standard deviation prediction model 174. As a result, for example, the server 1 can use the training data to train the standard deviation prediction model 174, thereby constructing a model that can output the standard deviation of the favorability of a package design.

When the server 1 acquires a package image including a package design, it inputs the acquired package image to the standard deviation prediction model 174. The server 1 performs a calculation process in the middle layer of the standard deviation prediction model 174 to extract features of each package image. The server 1 inputs the extracted features to the output layer of the standard deviation prediction model 174, and acquires the standard deviation of the favorability of the package design as an output. As shown in the figure, the standard deviation of the favorability, which is "1.010", is output from the standard deviation prediction model 174.

Note that the process is not limited to the above. The standard deviation prediction model 174 may be trained using scores set for each product category. In this case, the server 1 inputs a package image including the package design and the score set for each category into the standard deviation prediction model 174, and outputs the standard deviation of the favorability of the package design.

In addition, a standard deviation prediction model 174 can be generated for each gender or age group. Specifically, the server 1 accepts a setting of the gender or age group, and acquires multiple pieces of training data to be used for generating the standard deviation prediction model 174 from the training data DB 172 according to the accepted gender or age group. The server 1 uses the training data to generate the standard deviation prediction model 174 corresponding to the accepted gender or age group.

The server 1 stores the standard deviation prediction model 174 generated for each gender or age group in the mass storage unit 17. The server 1 can output the standard deviation of the favorability of the package design from the package image for each gender or age group using multiple standard deviation prediction models 174 prepared for multiple genders or age groups.

Figure 23 is an explanatory diagram showing an example of a display screen for the standard deviation of favorability. The screen includes a graph display field 14a and a favorability display field 14b. The graph display field 14a is a display field that displays a graph showing the relationship between the package image and the standard deviation of favorability. The favorability display field 14b is a display field that displays information related to favorability.

The terminal 2 acquires multiple package images to be evaluated and transmits the acquired multiple package images to the server 1. The server 1 receives the multiple package images transmitted from the terminal 2. The server 1 inputs each received package image into a favorability prediction model 171 and outputs information related to the favorability of the package design contained in each package image. The information related to the favorability is, for example, a weighted average value of the favorability. The server 1 inputs each received package image into a standard deviation prediction model 174 and outputs the standard deviation of the favorability of the package design contained in each package image.

Based on the package image and the standard deviation of the favorability, the server 1 generates a vertical bar graph showing the relationship between the package image and the standard deviation of the favorability. The server 1 transmits the generated vertical bar graph and the weighted average value of the favorability of each package design to the terminal 2. The terminal 2 receives the vertical bar graph and the weighted average value of the favorability of each package design transmitted from the server 1. The terminal 2 displays the received vertical bar graph in the graph display field 14a, and displays the received weighted average value of the favorability of each package design in the favorability display field 14b in association with the corresponding package image.

As shown, the vertical axis of the vertical bar graph indicates the standard deviation of the favorability of the package design. The horizontal axis indicates each package image sorted based on descending order of the standard deviation of the favorability. Specifically, based on the descending order of the predicted standard deviation of the favorability, the server 1 displays the package images and the file names of the package images in the lower part, displays a vertical bar indicating the standard deviation of the favorability in the middle part, and displays the standard deviation (value) of the favorability and the order at the time of selection of the package images in the upper part. The weighted average value of the favorability of the package design is displayed below the file name of the corresponding package image. Note that while the standard deviation of the favorability is displayed in descending order, this is not limiting, and it may also be displayed in ascending order or without sorting.

Note that while Figure 23 shows a vertical bar graph, this is not limited to this format and other graph formats may also be used.

Figure 24 is a flowchart showing the processing steps when outputting the standard deviation of the favorability. The control unit 21 of the terminal 2 acquires a plurality of package images using the storage unit 22 or the photographing unit 26 (step S251). The control unit 21 transmits the acquired plurality of package images to the server 1 via the communication unit 23 (step S252). The control unit 11 of the server 1 receives a plurality of package images from the terminal 2 via the communication unit 13 (step S151). The control unit 11 inputs each received package image into the standard deviation prediction model 174, and outputs the standard deviation of the favorability of the package design included in the package image (step S152).

The control unit 11 generates a graph showing the relationship between the package image and the standard deviation of the favorability based on the standard deviation of the favorability of the package image and the package design (step S153). The control unit 11 transmits the generated graph to the terminal 2 via the communication unit 13 (step S154). The control unit 21 of the terminal 2 receives the graph transmitted from the server 1 via the communication unit 23 (step S253), displays the received graph on the display unit 25 (step S254), and ends the process.

According to this embodiment, it is possible to output the standard deviation of the favorability of a package design using the standard deviation prediction model 174 based on a package image including the package design.

According to this embodiment, by outputting the standard deviation of the favorability level, it is possible to grasp the likes and dislikes of package designs.

(Embodiment 5)
The fifth embodiment relates to a form in which various prediction results are output based on a package image. Note that a description of the contents overlapping with the first to fourth embodiments will be omitted. The prediction results include information on the favorability output from the favorability prediction model 171, the appearance ratio of image words output from the appearance ratio prediction model 173, and the standard deviation of the favorability output from the standard deviation prediction model 174.

In this embodiment, the server 1 causes the terminal 2 to selectively display a first object for displaying information related to the degree of favorability, a second object for displaying the occurrence rate of image words, and a third object for displaying the standard deviation of the degree of favorability. The terminal 2 accepts the selection of the first object, the second object, or the third object, and displays on the screen the corresponding prediction result according to the accepted object.

Figure 25 is an explanatory diagram showing an example of a display screen for various prediction results. The screen includes a favorability tab 15a, a heat map tab 15b, an image word tab 15c, a standard deviation (variation) tab 15d, and a prediction result display section 15e.

The favorability tab 15a is a tab for displaying information related to favorability. The heat map tab 15b is a tab for displaying a heat map. The image word tab 15c is a tab for displaying the occurrence rate of image words. The standard deviation tab 15d is a tab for displaying the standard deviation of favorability. The prediction result display field 15e is a display field for displaying information related to favorability, a heat map, the occurrence rate of image words, or the standard deviation of favorability.

When the terminal 2 receives a touch operation on the favorability tab 15a, it displays information related to the favorability in the prediction result display field 15e. When the terminal 2 receives a touch operation on the heat map tab 15b, it displays a package image and a heat map corresponding to the package image in association with each other in the prediction result display field 15e. When the terminal 2 receives a touch operation on the image word tab 15c, it displays the occurrence rate of the image word in the prediction result display field 15e. When the terminal 2 receives a touch operation on the standard deviation tab 15d, it displays the standard deviation of the favorability in the prediction result display field 15e.

The display process for the information on favorability, the heat map, the occurrence rate of image words, and the standard deviation of favorability is the same as the display process in Figures 8, 13, 18, and 23, so a description thereof will be omitted.

According to this embodiment, it is possible to output information regarding the level of favorability, the occurrence rate of image words, or the standard deviation of the level of favorability, depending on the user's selection.

Note that the generation (construction), learning, and prediction processing of the favorability prediction model 171, the occurrence rate prediction model 173, and the standard deviation prediction model 174 described above was performed on the server 1 side, but this is not limiting and may also be performed on the terminal 2 side.

The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is indicated by the claims, not by the meaning described above, and is intended to include all modifications within the scope and meaning equivalent to the claims.

1. Information processing device (server)
REFERENCE SIGNS LIST 11 Control unit 12 Storage unit 13 Communication unit 14 Input unit 15 Display unit 16 Reading unit 17 Large-capacity storage unit 171 Likeability prediction model 172 Training data DB
173 occurrence rate prediction model 174 standard deviation prediction model 1a portable storage medium 1b semiconductor memory 1P control program 2 information processing terminal (terminal)
21 Control unit 22 Memory unit 23 Communication unit 24 Input unit 25 Display unit 26 Photography unit 2P Control program

Claims (18)

Obtain package images including the product's package design,
Accepts gender or age selection,
A program that causes a computer to execute a process of inputting the acquired package image into a first learning model that has been trained to output information regarding the degree of favorability of the package design corresponding to the received gender or age group when the package image is input, and outputting information regarding the degree of favorability of the package design corresponding to the received gender or age group. The program described in claim 1 executes a process of outputting information regarding the favorability of package designs for all genders and all age groups based on information regarding the favorability of package designs output from multiple first learning models prepared corresponding to each gender and age group. Obtain the score set for each product category,
The program according to claim 1 or 2, which executes a process of inputting the acquired score and the package image into the first learning model, and outputting information relating to the favorability of the package design. The program according to any one of claims 1 to 3, which causes a second learning model that has been trained to output the occurrence rate of image words in the package design when the package image is input to execute a process of inputting the package image and outputting the occurrence rate of the image words. The program according to claim 4, which executes a process of outputting the appearance rate of the image word from the package image for each gender or age group using a plurality of the second learning models prepared corresponding to a plurality of genders or age groups. acquiring a plurality of said package images;
Accepting a selection of image words for the package design;
inputting each of the package images into the second learning model and outputting the occurrence ratio of each of the received image words;
The program according to claim 4 or 5, which causes the program to execute a process of outputting a first graph showing a relationship between each of the package images and an appearance rate of each of the image words output by the second learning model. The program according to any one of claims 4 to 6, which causes a third learning model that has been trained to output the standard deviation of the favorability of the package design when the package image is input to execute a process of inputting the package image and outputting the standard deviation of the favorability. The program according to claim 7, which executes a process of outputting the standard deviation of the degree of favorability from the package image for each gender or age group using a plurality of the third learning models prepared corresponding to a plurality of genders or age groups. acquiring a plurality of said package images;
inputting each of the package images into the third learning model and outputting the standard deviation of the favorability level;
9. The program according to claim 7, further comprising: outputting a second graph showing a relationship between each of the package images and a standard deviation of each of the degrees of favorability output by the third learning model. displaying, in a selectable manner, a first object for outputting information related to the degree of favorability, a second object for outputting an appearance ratio of the image word, and a third object for outputting a standard deviation of the degree of favorability;
The program according to claim 7 , further comprising: receiving a selection of the first object, the second object, or the third object. acquiring a plurality of said package images;
inputting each of the package images into the first learning model and outputting information relating to the favorability of each of the package designs;
Extracting from each of the package images an area that influenced the information related to the favorability output by the first learning model;
The program according to claim 1 , further comprising: outputting each of the package images in association with an index indicating each of the extracted areas. Obtaining a plurality of combinations of training data in which a package image including a package design of a product is associated with a favorability level of the package design for each gender or age group;
A method for generating a learning model that causes a computer to execute a process of generating a learning model that outputs information regarding the favorability of the package design when the package image is input, based on multiple combinations of the acquired training data for each gender or age group. Obtaining a plurality of combinations of training data in which a score set for each product category, a package image including a package design of the product, and a favorability level of the package design are associated with each other;
A method for generating a learning model that causes a computer to execute a process of generating a learning model that outputs information regarding the favorability of the package design when the score and the package image are input, based on multiple combinations of the acquired training data. For each product category, input of brand name and company name is accepted to search for products.
obtaining the number of search hits received by a search engine for the brand name and company name;
The method for generating a learning model according to claim 13, further comprising the step of: calculating an average score of a plurality of brands in a product category based on the number of search hits obtained. Obtaining a plurality of combinations of training data in which package images including a package design of a product are associated with image words of the package design;
A method for generating a learning model, the method comprising: causing a computer to execute a process of generating a learning model that outputs the appearance rate of the image word when the package image is input, based on a plurality of combinations of the acquired training data. Obtaining a set of evaluation texts that evaluate the package design;
Extracting a list of keywords having a frequency of occurrence equal to or greater than a predetermined frequency from the acquired set of evaluation texts;
classifying each keyword included in the extracted keyword list based on feature information of each keyword included in the keyword list;
The method for generating a learning model according to claim 15, further comprising the step of: causing a computer to execute a process of identifying image words for the package design from the list of the classified keywords. Obtaining a plurality of combinations of training data in which a package image including a product package design is associated with a standard deviation of the degree of favorability of the package design by a plurality of people;
A method for generating a learning model, the method comprising: causing a computer to execute a process for generating a learning model that outputs the standard deviation of the favorability level when the package image is input, based on multiple combinations of the acquired training data. Obtain package images including the product's package design,
Accepts gender or age selection,
An information processing method comprising: inputting the acquired package image into a first learning model that has been trained to output information regarding the degree of favorability of the package design corresponding to the received gender or age group when the package image is input; and outputting information regarding the degree of favorability of the package design corresponding to the received gender or age group.

Images