JP7573239B2 - PROGRAM, INFORMATION PROCESSING METHOD, AND INFORMATION PROCESSING APPARATUS - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act Published on May 22, 2020 on the website of Confit's 2020 National Conference of the Japanese Society for Artificial Intelligence (34th) (https://confit.atlas.jp/guide/event/jsai2020/subject/1M3-GS-13-02/advanced) Published on June 9, 2020 at the 2020 National Conference of the Japanese Society for Artificial Intelligence (34th) (https://www.ai-gakkai.or.jp/jsai2020/) Published on June 20, 2020 on the PLUG AI website (https://hp.package-ai.jp/)

The present invention relates to a program, an information processing method, and an information processing device.

Conventionally, when evaluating designs, an evaluation device has been disclosed that extracts a first characteristic amount of a first design to be evaluated, calculates the similarity between the first design and each of a plurality of existing second designs based on the extracted first characteristic amount, and predicts customer impressions of the first design based on the calculated similarity amount and previously acquired information indicating customer impressions of each of the second designs (see, for example, Patent Document 1).

JP 2018-195078 A

However, the invention described in Patent Document 1 may not be able to appropriately suggest package images.

One aspect is to provide a program etc. that can appropriately suggest package images.

A program relating to one aspect causes a computer to execute a process of accepting the number of design elements and the number of designs contained in candidate package images, outputting a design element selection field for selecting a design element for the accepted number of design elements, outputting a design selection field for selecting a design for the accepted number of designs in correspondence with each design element selection field, generating a plurality of the candidate package images, inputting the generated package images into a first learning model that has been trained to output information regarding the favorability of the package image when a package image is input, outputting information regarding the favorability of each package image, and outputting a package image to be suggested from the plurality of package images based on the output information regarding the favorability.

On one hand, it makes it possible to suggest appropriate package images.

FIG. 1 is an explanatory diagram showing an overview of a package image suggestion system. FIG. 2 is a block diagram showing an example of the configuration of a server. FIG. 2 is a block diagram showing a configuration example of a terminal. FIG. 11 is an explanatory diagram for explaining an operation of outputting a package image to be proposed. FIG. 13 is an explanatory diagram for explaining a favorability prediction model. FIG. 13 is an explanatory diagram showing an example of a reception screen for the number of designs and the number of design elements. FIG. 13 is an explanatory diagram showing an example of a registration screen for a design element. FIG. 13 is an explanatory diagram showing an example of a display screen of a package image to be proposed. 10 is a flowchart showing a processing procedure for outputting a package image to be proposed. 10 is a flowchart showing a processing procedure for outputting a package image to be proposed. FIG. 11 is an explanatory diagram illustrating a favorability prediction model that is trained using scores. FIG. 11 is a block diagram showing an example of the configuration of a server according to a second embodiment. FIG. 1 is a diagram conceptually illustrating the configuration of StyleGAN. FIG. 13 is an explanatory diagram showing an example of a reception screen for a basic package image. FIG. 11 is an explanatory diagram showing an example of a style image reception screen. FIG. 13 is an explanatory diagram showing an example of a confirmation screen for generating a package image. FIG. 11 is an explanatory diagram showing an example of a display screen of a package image. 13 is a flowchart showing a processing procedure for generating a package image using a style image. FIG. 13 is a block diagram showing an example of the configuration of a server according to a third embodiment. FIG. 13 is an explanatory diagram for explaining an appearance ratio prediction model. FIG. 13 is a block diagram showing an example of the configuration of a server according to a fourth embodiment. FIG. 13 is an explanatory diagram illustrating a standard deviation prediction model.

The present invention will be described in detail below with reference to drawings showing an embodiment of the invention.

(Embodiment 1)
The first embodiment relates to a form in which a plurality of candidate package images are generated, and a package image to be proposed is output from the plurality of generated package images based on information regarding the favorability of the package images. Fig. 1 is an explanatory diagram showing an overview of a package image proposal system. The system of this embodiment includes an information processing device 1 and an information processing terminal 2, and each device transmits and receives information via a network N such as the Internet.

The information processing device 1 is an information processing device that processes, stores, and transmits/receives various types of information. The information processing device 1 is, for example, a server device, a personal computer, or a general-purpose tablet PC (personal computer). In this embodiment, the information processing device 1 is a server device, and for the sake of simplicity, will be referred to as server 1 below.

The information processing terminal 2 is a terminal device that receives and displays package images to be proposed, and stores (saves) the package images. The information processing terminal 2 is, for example, an information processing device such as a smartphone, a mobile phone, a tablet, or a personal computer terminal. For the sake of simplicity, the information processing terminal 2 will be read as terminal 2 below.

The server 1 according to this embodiment generates multiple candidate package images by rearranging the designs for multiple design elements. The server 1 inputs the multiple generated package images into a favorability prediction model and outputs information regarding the favorability of each package image. The server 1 extracts a package image to be proposed from the multiple candidate package images based on the output information regarding the favorability of each package image. The server 1 transmits the extracted package image to be proposed to the terminal 2.

Figure 2 is a block diagram showing an example of the configuration of the server 1. The server 1 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, a display unit 15, a reading unit 16, and a large-capacity storage unit 17. Each component is connected by a bus B.

The control unit 11 includes a processor such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), and GPU (Graphics Processing Unit), and performs various information processing, control processing, and the like related to the server 1 by reading and executing the control program 1P stored in the storage unit 12. The control program 1P can be deployed so that it is executed on a single computer or at one site, or distributed across multiple sites and on multiple computers interconnected by a communications network. Although the control unit 11 is described in FIG. 2 as being a single processor, it may also be a multi-processor.

The storage unit 12 includes memory elements such as RAM (Random Access Memory) and ROM (Read Only Memory), and stores the control program 1P or data required for the control unit 11 to execute processing. The storage unit 12 also temporarily stores data required for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing processing related to communication, and transmits and receives information to and from the terminal 2 via the network N.

The input unit 14 is an input device such as a mouse, keyboard, touch panel, or button, and outputs received operation information to the control unit 11. The display unit 15 is a liquid crystal display or an organic EL (electroluminescence) display, etc., and displays various information according to instructions from the control unit 11.

The reading unit 16 reads the portable storage medium 1a, which may be a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 16 and store it in the mass storage unit 17. The control unit 11 may also download the control program 1P from another computer via a network N or the like and store it in the mass storage unit 17. The control unit 11 may also read the control program 1P from the semiconductor memory 1b.

The large-capacity storage unit 17 includes a recording medium such as a hard disk drive (HDD) or a solid state drive (SSD). The large-capacity storage unit 17 includes a favorability prediction model 171. The favorability prediction model 171 is a predictor that predicts information related to the favorability of a package image based on the package image, and is a trained model generated by machine learning.

In this embodiment, the memory unit 12 and the large-capacity memory unit 17 may be configured as an integrated memory device. The large-capacity memory unit 17 may also be configured from multiple memory devices. Furthermore, the large-capacity memory unit 17 may be an external memory device connected to the server 1.

Server 1 may be executed on a single computer, or may be executed in a distributed manner on multiple computers, or may be executed in a distributed manner on virtual machines.

Figure 3 is a block diagram showing an example of the configuration of the terminal 2. The terminal 2 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, a display unit 25, and an image capture unit 26. Each component is connected by a bus B.

The control unit 21 includes an arithmetic processing device such as a CPU or MPU, and performs various information processing, control processing, etc. related to the terminal 2 by reading and executing the control program 2P stored in the memory unit 22. Note that while the control unit 21 is described as a single processor in FIG. 3, it may be a multiprocessor. The memory unit 22 includes memory elements such as RAM and ROM, and stores the control program 2P or data, etc. required for the control unit 21 to execute processing. The memory unit 22 also temporarily stores data, etc. required for the control unit 21 to execute arithmetic processing.

The communication unit 23 is a communication module for performing communication-related processing, and transmits and receives information to and from the server 1 via the network N. The input unit 24 may be a keyboard, a mouse, or a touch panel integrated with the display unit 25. The display unit 25 is a liquid crystal display or an organic EL display, etc., and displays various information according to instructions from the control unit 21.

The photographing unit 26 is, for example, a photographing device such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera. The photographing unit 26 may be composed of multiple photographing devices. The photographing unit 26 may not be built into the terminal 2, but may be externally connected directly to the terminal 2 and configured to be capable of photographing.

In addition, terminal 2 may also have the functions of server 1.

Figure 4 is an explanatory diagram explaining the operation of outputting a package image to be proposed. First, the process of generating candidate package images will be explained. In this embodiment, the server 1 uses a genetic algorithm to perform crossover and mutation processes on multiple design elements included in the package image to generate multiple candidate package images. A genetic algorithm is an evolutionary computing method that manipulates data based on evolutionary thinking and performs optimal solution search, learning, or inference.

The server 1 acquires from the terminal 2 the element names and element images of multiple design elements for each package design. Package designs are the designs of packaging and containers for foods, beverages, daily necessities, and other products. Design elements are the various design elements that make up a package design. For example, in the case of a beverage container, design elements include the background, cap, font or logo for indicating volume, etc.

The server 1 concatenates the multiple design elements it has acquired and expresses them in binary. The concatenated set is then treated as one chromosome, and this chromosome represents one individual. A genetic algorithm is applied to the individual thus represented. For the crossover process, the server 1 randomly crosses two design elements together multiple times for the multiple design elements it has received, thereby generating multiple candidate design elements after crossover.

For the mutation process, a single or multiple design elements are selected, for example, randomly from the multiple candidate design elements generated in the crossover process described above along with the multiple design elements received. For example, server 1 randomly selects three design elements and changes one point in the gene sequence of each of the three selected design elements to generate three mutated candidate design elements.

In this way, the server 1 performs crossover and mutation processes on multiple design elements to generate multiple candidate package images. Note that, although an example using a genetic algorithm is described in this embodiment, this is not limiting. For example, other methods such as randomly changing design elements may also be used.

Next, the server 1 evaluates each package image (individual) using information related to the favorability of each candidate package image that has been generated. The information related to the favorability includes a score that classifies the favorability from a maximum of 5 (like) to a minimum of 1 (dislike), or a weighted average of the favorability score from a maximum of 5 to a minimum of 1. Specifically, the server 1 inputs each of the generated package images into the trained favorability prediction model 171, and outputs information related to the favorability of each package image. The output process of the information related to the favorability using the favorability prediction model 171 will be described later.

The server 1 evaluates each package image (individual) based on the output information on the favorability. For example, the top N package images (e.g., 20) may be evaluated in descending order of the weighted average favorability as the most favorable individuals, or the package image whose weighted average favorability is equal to or greater than a predetermined threshold may be evaluated as the most favorable individual. The server 1 discards package images that are determined to have a low favorability. The server 1 extracts package images that are determined to have a high favorability (surviving individuals) as package images to be proposed.

The server 1 transmits the extracted package image to be proposed to the terminal 2 as a first generation package image. The terminal 2 receives and displays the first generation package image transmitted from the server 1. When the terminal 2 receives a request to generate a second generation package image based on the first generation package image, it transmits the request to generate the second generation package image to the server 1. In response to the request to generate a second generation package image transmitted from the terminal 2, the server 1 again performs crossover and mutation processes on multiple design elements contained in each package image based on the package image of the first generation, to generate candidate package images.

Then, the server 1 re-evaluates each of the generated candidate package images based on the favorability of the package image. The server 1 extracts the candidate package image determined to have a high favorability as a package image to be proposed. The server 1 transmits the extracted package image to be proposed to the terminal 2 as a second-generation package image.

In this way, the server 1 repeats the crossover and mutation processes based on the genetic algorithm until a package image of the final generation (e.g., the fifth generation) that satisfies the user is determined. By performing package image processing in this way, it is possible to generate a package image whose display format is closer to the user's preferences, i.e., a package image that is highly favorable to the user.

Next, the output process of information relating to favorability using the favorability prediction model 171 will be described. FIG. 5 is an explanatory diagram of the favorability prediction model 171. The favorability prediction model 171 is used as a program module that is part of artificial intelligence software. The favorability prediction model 171 is a predictor of information relating to favorability that has constructed a neural network that takes a package image as input and outputs information relating to the favorability of the package image.

The server 1 constructs (generates) the favorability prediction model 171 by performing deep learning to learn the features of the parts of the package image that affect the favorability of the package image. For example, the favorability prediction model 171 is a CNN (Convolution Neural Network) that has an input layer that accepts input of the package image, an output layer that outputs information related to the favorability of the package image, and an intermediate layer that has been trained by backpropagation.

The input layer has a number of neurons that accept input of pixel values of each pixel included in the package image, and passes the input pixel values to the intermediate layer. The intermediate layer has a number of neurons that extract features of the package image, and passes the extracted image features to the output layer. Specifically, the intermediate layer is configured by alternating convolution layers that convolve the pixel values of each pixel input from the input layer with pooling layers that map the pixel values convolved in the convolution layer, compressing the pixel information of the package image while ultimately extracting the features of the package image. The intermediate layer then predicts information related to the favorability of the package image using a fully connected layer whose parameters have been learned by backpropagation. The prediction results are output to the output layer, which has neurons.

In addition, the package image may be input to the input layer after passing through an alternating convolution layer and pooling layer to extract features.

In this embodiment, the network structure of ResNet50 (Residual Network50) may be applied as the favorability prediction model 171. ResNet50 is a CNN model with 50 layers. Note that, instead of the above-mentioned ResNet50, for example, VGG16, InceptionV3, DenseNet, InceptionResNetV2, etc. may be used.

In this embodiment, the favorability prediction model 171 is described as a CNN, but the favorability prediction model 171 is not limited to a CNN and may be a trained model constructed with any learning algorithm, such as a neural network other than a CNN, an SVM (Support Vector Machine), a Bayesian network, or a regression tree.

The server 1 acquires multiple combinations of training data in which package images are associated with the favorability levels of the package images. The training data is data in which package images are labeled with the weighted average of the favorability levels of the package images. Note that, in this embodiment, an example has been described in which the favorability levels are classified into five levels, from a maximum of 5 (like) to a minimum of 1 (dislike), but this is not limiting. The favorability levels may be set according to actual needs.

In the present embodiment, an example has been described in which pre-stored training data is acquired, but the present invention is not limited to this. For example, the server 1 may generate training data for learning favorability by labeling a weighted average of the favorability of each of a plurality of package images. Specifically, the server 1 acquires the favorability of responses from a plurality of people for one package image on a five-point scale ranging from a maximum of 5 to a minimum of 1. The server 1 calculates the weighted average of the favorability of the acquired responses from a plurality of people. The server 1 creates training data by labeling a package image with the weighted average of the favorability of the package image from a plurality of people.

For example, if 10 people provide responses regarding the level of favorability for package image A, 3 people provide a level of favorability of 5, 5 people provide a level of favorability of 4, and 2 people provide a level of favorability of 3. Server 1 calculates a weighted average level of favorability based on the responses regarding the level of favorability for package image A. The calculated weighted average level of favorability is "4.1" ((5 x 3 + 4 x 5 + 3 x 2 + 2 x 0 + 1 x 0)/10). Server 1 labels package image A with the weighted average level of favorability of "4.1".

The server 1 performs learning using the acquired training data. Specifically, the server 1 inputs a package image, which is training data, into the input layer, and acquires information about the favorability of the package image from the output layer after arithmetic processing in the intermediate layer. The information about the favorability includes a probability (likelihood) value indicating the likelihood of the favorability, or a weighted average value of the favorability. Figure 5 explains an example of the weighted average value of the favorability. As shown in the figure, the prediction result output from the output layer is a weighted average value of the favorability (from favorability level 5 to favorability level 1).

The server 1 compares the prediction results output from the output layer with the information labeled for the package images in the training data, i.e., the correct answer value, and optimizes the parameters used in the calculation process in the intermediate layer so that the predicted value from the output layer approaches the correct answer value. The parameters in question are, for example, the weights (coupling coefficients) between neurons and the coefficients of the activation functions used in each neuron. There are no particular limitations on the method of optimizing the parameters, but for example, the server 1 optimizes various parameters using the backpropagation method.

The server 1 performs the above process for each package image included in the training data to generate a favorability prediction model 171. As a result, for example, the server 1 can use the training data to learn the favorability prediction model 171, thereby constructing a model that can output information related to the favorability of a package image.

When the server 1 acquires a candidate package image generated based on a genetic algorithm, it inputs the acquired package image to the favorability prediction model 171. The server 1 performs a calculation process to extract features of the package image in the middle layer of the favorability prediction model 171. The server 1 inputs the extracted features to the output layer of the favorability prediction model 171 and acquires information related to the favorability of the package image as an output. As shown in the figure, the weighted average value of the favorability is output from the output layer. The weighted average value of the favorability is "3.84". The calculated weighted average value of the favorability may be corrected by rounding up, rounding down, or the like.

In the above example, the weighted average value of the favorability is output as the prediction result, but this is not limited to this. For example, the probability value itself indicating each favorability may be output as the prediction result. For example, the output layer outputs prediction results in which the probability values of "favorability 5", "favorability 4", "favorability 3", "favorability 2", and "favorability 1" for the package image are "0.05", "0.85", "0.03", "0.03", and "0.04", respectively. In addition, the prediction result may be output using a predetermined threshold. For example, when the server 1 determines that the probability value of "favorability 4" (0.85) is equal to or greater than a predetermined threshold (e.g., 0.80), it outputs a prediction result in which the favorability of the package image is "favorability 4". In addition, the above-mentioned threshold may not be used, and the favorability corresponding to the highest probability value from among the probability values of each favorability predicted by the favorability prediction model 171 may be output as the prediction result.

Note that the prediction process for information related to favorability is not limited to the prediction process using machine learning described above. For example, information related to favorability may be predicted using local feature extraction methods such as A-KAZE (Accelerated KAZE), SIFT (Scale Invariant Feature Transform), SURF (Speed-Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), and HOG (Histograms of Oriented Gradients).

Favourability prediction models 171 can also be generated for each gender or age group. Specifically, server 1 accepts a gender or age group setting, and acquires multiple pieces of training data to be used in generating favourability prediction models 171 according to the accepted gender or age group. Server 1 uses the acquired training data to generate a favourability prediction model 171 corresponding to the accepted gender or age group.

The server 1 stores the favorability prediction model 171 generated for each gender or age group in the mass storage unit 17. The server 1 can output information related to the favorability of the package image for each gender or age group using multiple favorability prediction models 171 prepared for multiple genders or age groups.

In addition, information regarding the overall (all genders and all ages) favorability can be output based on the favorability information output from multiple favorability prediction models 171 prepared for each gender or age. The overall includes, for example, men in their 20s, women in their 20s, men in their 30s, women in their 30s, men in their 40s, women in their 40s, men in their 50s, and women in their 50s.

Specifically, the server 1 acquires from the mass storage unit 17 each of a plurality of favorability prediction models 171 prepared corresponding to each gender and age group. The server 1 inputs a package image into each acquired favorability prediction model 171, and outputs a weighted average value of the favorability of the package image corresponding to each gender and age group. The server 1 calculates an average value of the weighted average values of the favorability based on the output weighted average values of the favorability of the package images corresponding to each gender and age group. The server 1 outputs the calculated average value of the weighted average values as the overall favorability.

Figure 6 is an explanatory diagram showing an example of a reception screen for the number of designs and the number of design elements. The screen includes a number of designs input field 11a, a number of design elements input field 11b, and a forward button 11c. The number of designs input field 11a is a text field for inputting the number of designs (design proposals). The number of design elements input field 11b is a text field for inputting the number of design elements. Note that, although an example is shown in which the number of designs and the number of design elements are each a maximum of 10, this is not limited to this. Note that the number of designs and the number of design elements are not limited to being input by the user, and may be determined in advance.

The forward button 11c is a button that transitions to a design element registration screen (Figure 7) described below. When the terminal 2 receives a touch (click) operation of the forward button 11c, it transmits to the server 1 the number of designs input in the design number input field 11a and the number of design elements input in the design element number input field 11b. The server 1 generates a design element registration screen according to the number of designs and the number of design elements transmitted from the terminal 2 and transmits it to the terminal 2. The terminal 2 receives the design element registration screen transmitted from the server 1 and transitions to the received design element registration screen.

Figure 7 is an explanatory diagram showing an example of a design element registration screen. The screen includes a design element selection field 12a, a design selection field 12b, and a registration button 12c. The server 1 generates a design element selection field 12a for selecting the number of design elements according to the number of design elements transmitted from the terminal 2. The server 1 generates a design selection field 12b for selecting the number of designs according to the number of designs transmitted from the terminal 2, in association with the generated design element selection field 12a.

The design element selection field 12a includes an element name input field 12a1 and an element type selection field 12a2. The element name input field 12a1 is a text field for inputting the name of the design element. Note that the element name input field 12a1 is not limited to a text field, but may be a selectable combo box set based on the names of pre-defined design elements. The element type selection field 12a2 is a combo box that accepts the selection of the type of design element. The types of design elements include text and non-text (not text).

The design selection field 12b includes a design name input field 12b1, an element image display field 12b2, an element image selection button 12b3, an element image deletion button 12b4, and an element image name input field 12b5. The design name input field 12b1 is a text field for inputting the name of the package design. The element image display field 12b2 is a display field for displaying the element images of the design elements. The element image selection button 12b3 is a button for selecting an element image. The element image deletion button 12b4 is a button for deleting an element image. The element image name input field 12b5 is a text field for inputting the name of the element image.

For example, if the number of design elements accepted in FIG. 6 is four, the server 1 generates four element name input fields 12a1 and four element type selection fields 12a2 included in the design element selection field 12a. If the number of designs accepted in FIG. 6 is five, the server 1 generates 20 (4×5) design name input fields 12b1, element image display fields 12b2, element image selection buttons 12b3, element image deletion buttons 12b4, and element image name input fields 12b5 included in the design selection field 12b, corresponding to the respective element name input fields 12a1 and element type selection fields 12a2.

The registration button 12c is a button that registers designs for multiple design elements set in the design element selection field 12a and the design selection field 12b on the server 1.

When terminal 2 receives an input operation in element name input field 12a1, it receives the input of the name of the design element. When terminal 2 receives a selection operation in element type selection field 12a2, it receives the selection of the type of design element that corresponds to the name of the design element input in element name input field 12a1.

In this embodiment, when multiple design elements are rearranged in the process of generating a package image using a genetic algorithm, a rule is established so that design elements of the type "text" do not overlap. For example, if the "500ml" design element is of the "text" type and both design A and design B contain "500ml," overlapping the "500ml" design element of design A and the "500ml" design element of design B does not constitute a valid design, and therefore the corresponding case (situation) is excluded.

When the terminal 2 receives an input operation in the design name input field 12b1, it receives the input of the name of the package design. When the terminal 2 receives a touch operation on the element image selection button 12b3, it receives the selection of an element image stored in the memory unit 22. The terminal 2 displays the received element image in the element image display field 12b2. The terminal 2 may also obtain an element image photographed via the photographing unit 26.

When the terminal 2 receives an input operation in the element image name input field 12b5, it receives the input of the name of the element image. When the terminal 2 receives a touch operation on the element image deletion button 12b4, it deletes the element image displayed in the element image display field 12b2 and deletes the name of the element image input in the element image name input field 12b5.

When the terminal 2 receives a touch operation of the registration button 12c, it transmits to the server 1 the name of the design element input in the element name input field 12a1, the type of design element selected in the element type selection field 12a2, the name of the package design input in the design name input field 12b1, the element image selected in the element image selection button 12b3, and the name of the element image input in the element image name input field 12b5.

Figure 8 is an explanatory diagram showing an example of a display screen for a package image to be proposed. Note that while Figure 8 shows an example of displaying a first generation package image generated by a genetic algorithm, this can be similarly applied to other package images. The screen includes an image ID display field 13a, an image display field 13b, a favorability display field 13c, an image save button 13d, a next generation image generation button 13e, and a switch button 13f.

The image ID display field 13a is a display field that displays the ID of the package image. The image display field 13b is a display field that displays the package image. The favorability display field 13c is a display field that displays information related to the favorability of the package image. That is, for each package image, the package image ID, the package image, and the favorability of the package image are simultaneously displayed.

The image save button 13d is a button for saving a package image. The next generation image generation button 13e is a button for generating a next generation package image. Note that in FIG. 8, a button for generating a second generation package image is shown as an example, but if a third generation package image is generated based on the second generation package image, for example, the next generation image generation button 13e becomes a button for generating the third generation package image. The switch button 13f is a button for switching between displaying package images of each generation generated in the past.

The server 1 performs crossover and mutation processing based on a genetic algorithm on the multiple design elements received via the terminal 2 in Fig. 7 to generate multiple candidate package images. For example, if the number of received design elements is 4 and the number of designs is 5, the server 1 generates 5 ^{(4 x 2)} package images based on a combination of the number of design elements and the number of designs. The server 1 randomly extracts a predetermined number (e.g., 50) of package images from the multiple package images generated. The server 1 outputs the extracted package images as candidate package images. The server 1 inputs the multiple candidate package images output to the favorability prediction model 171 and outputs information regarding the favorability of each package image.

The server 1 evaluates each package image based on the output information on the favorability. For example, a package image whose weighted average favorability is equal to or greater than a predetermined threshold may be evaluated as the most favorable individual. The server 1 extracts package images that are determined to have a high favorability as first-generation package images. The server 1 transmits the extracted package images, as well as information on the ID and favorability of each package image, to the terminal 2.

The terminal 2 receives the multiple package images, the ID of each package image, and information regarding the favorability level sent from the server 1, and displays them on the screen. Specifically, the terminal 2 displays the ID of each package image in the corresponding image ID display field 13a, displays each package image in the corresponding image display field 13b, and displays the favorability level of each package image in the corresponding favorability display field 13c.

When the terminal 2 receives a touch operation of the image save button 13d, the terminal 2 stores the multiple package images in the storage unit 22 or an external storage device in the specified image format. The image format may be any appropriate image format, such as bmp, tiff, jpg, jpeg, gif, or png format.

When terminal 2 receives a touch operation of next generation image generation button 13e, it sends to server 1 a request to generate a second generation package image based on the first generation package image. When terminal 2 receives a touch operation of switch button 13f, it switches between and displays the generated package images of each generation. For example, terminal 2 may switch between and display the package images of each generation in a sequential cyclical order (first generation package image, second generation package image, ..., Nth generation package image, first generation package image).

Figures 9 and 10 are flowcharts showing the processing steps when outputting package images to be proposed. The control unit 21 of the terminal 2 accepts input of the number of designs and the number of design elements by the user through the input unit 24 (step S201). The control unit 21 transmits the accepted number of designs and the number of design elements to the server 1 through the communication unit 23 (step S202).

The control unit 11 of the server 1 receives the number of designs and the number of design elements transmitted from the terminal 2 via the communication unit 13 (step S101). The control unit 11 generates design element selection fields for selecting design elements of the number of design elements and design selection fields for selecting designs of the number of designs in correspondence with each design element selection field according to the received number of designs and number of design elements (step S102). The control unit 11 transmits the generated design element selection fields and design selection fields to the terminal 2 via the communication unit 13 (step S103).

The control unit 21 of the terminal 2 receives the design element selection field and the design selection field sent from the server 1 via the communication unit 23 (step S203). The control unit 21 displays the received design element selection field and the design selection field on the display unit 25 (step S204). The control unit 21 accepts input of the number of design elements (element names and element types, etc.) and the number of designs (design names and element images, etc.) associated with each design element via the input unit 24 (step S205).

The control unit 21 accepts the setting of evaluation conditions for evaluating the package image via the input unit 24 (step S206). The evaluation conditions include evaluation by overall (all genders and all ages), evaluation by gender, evaluation by age, evaluation by gender and age (for example, women in their 20s), etc. Note that the evaluation conditions may be set to be one or a combination of two or more of overall, gender, age, or gender and age.

The control unit 21 transmits the design and evaluation conditions for the accepted design elements to the server 1 via the communication unit 23 (step S207). The control unit 11 of the server 1 receives the design and evaluation conditions for the design elements transmitted from the terminal 2 via the communication unit 13 (step S104). The control unit 11 performs crossover and mutation processes based on a genetic algorithm on the received multiple design elements to generate multiple package images that are candidates for the next generation (step S105).

The control unit 11 identifies a trained favorability prediction model 171 according to the received evaluation conditions (step S106). The control unit 11 inputs the generated package images into the identified trained favorability prediction model 171, and outputs (obtains) information regarding the favorability of each package image according to the set evaluation conditions (step S107).

The control unit 11 extracts a package image to be proposed from the multiple package images based on the obtained information on the favorability (step S108). Specifically, the control unit 11 evaluates each package image based on the obtained information on the favorability. For example, the top N package images (e.g., 20) may be evaluated in descending order of the weighted average favorability as the most favorable individuals. The control unit 11 extracts a package image that is determined to have a high favorability as a package image to be proposed. Note that the package image to be proposed that is initially extracted is considered to be a first-generation package image.

The control unit 11 transmits the extracted package image to be proposed, the package image's ID, and information regarding the favorability level to the terminal 2 via the communication unit 13 (step S109). The control unit 21 of the terminal 2 receives the package image to be proposed, the package image's ID, and information regarding the favorability level transmitted from the server 1 via the communication unit 23 (step S208). The control unit 21 displays the received package image to be proposed, the package image's ID, and information regarding the favorability level on the display unit 25 (step S209).

When the control unit 21 receives a request to generate a next-generation package image based on the package image of the currently generated generation through the input unit 24 (step S210), the control unit 21 extracts multiple design elements from the package image of the currently generated generation (step S211). Then, the control unit 21 returns to the process of step S207, and transmits the designs and evaluation conditions for the extracted multiple design elements to the server 1 through the communication unit 23.

According to this embodiment, by optimizing the combination of design elements using a genetic algorithm, it is possible to generate the final next-generation package image that satisfies users.

According to this embodiment, candidate package images generated using a genetic algorithm can be input to the favorability prediction model 171, and information regarding the favorability of the package image can be output.

According to this embodiment, by using both a genetic algorithm and a favorability prediction model 171 in combination, it is possible to extract a package image with a high favorability rating from multiple candidate package images generated based on the genetic algorithm and output it as the package image to be proposed.

<Modification 1>
We will explain the process of inputting multiple candidate package images generated based on a genetic algorithm into a favorability prediction model 171 that has been trained using scores set for each product category, and outputting information regarding the favorability of each package image.

Product categories include, for example, beer, chuhai/cocktails, carbonated drinks, tea/black tea/coffee, energy drinks, seasonings, instant noodles/bag noodles, soups, desserts, room temperature foods or frozen foods, etc. Other product categories may also include detergents, finishing agents, water bottles, cosmetics, etc.

First, the process of creating the score will be explained. The server 1 accepts input of the brand name and company name for searching for products for each product category. The brand name and company name are, for example, "Brand A AAA Co., Ltd." Note that the brand name and company name are not limited to "brand + company name" and may be, for example, "product name + company name" or "brand name + category name." The server 1 obtains the number of search hits for the accepted brand name and company name via a search engine. The search engine may be, for example, GOOGLE (registered trademark) or YAHOO (registered trademark), etc.

The server 1 calculates the logarithm of the number of search hits for the acquired brand name and company name, and outputs the calculated logarithm of the number of search hits as a score. The server 1 calculates the average score for multiple brands within the product category. The server 1 stores the calculated average score in the memory unit 12 or the mass memory unit 17 in association with the product category.

Next, a process of training the favorability prediction model 171 using the scores will be described.
Fig. 11 is an explanatory diagram for explaining a favorability prediction model 171 that is trained using scores. Note that explanations of contents that overlap with Fig. 5 will be omitted. The favorability prediction model 171 of Modification 1 applies the network structure of ResNet 50. The server 1 acquires a plurality of combinations of training data in which package images, scores corresponding to the package images, and favorability of the package images are associated with each other.

The server 1 performs learning using the acquired training data. Specifically, the server 1 inputs the package image, which is the training data, to the network 14a of the ResNet 50, excluding the fully connected layer (FC) 14c. The network 14a extracts the features of the package image, and passes the extracted features of the package image to the fully connected layer 14c, whose parameters have been learned by backpropagation. The server 1 inputs the score corresponding to the package image to the fully connected layer 14b. The fully connected layer 14c combines the features of the package image with the score input to the fully connected layer 14b, and predicts information related to the favorability of the package image based on the combined information. The prediction result is output to an output layer having neurons.

The server 1 compares the prediction results output from the output layer with the information labeled for the package image and score in the training data, i.e., the correct answer value, and optimizes the parameters used in the calculation process in the intermediate layer so that the predicted value from the output layer approaches the correct answer value. In this way, the server 1 uses the training data to train the favorability prediction model 171, and updates the trained favorability prediction model 171 in the large-capacity storage unit 17.

When the server 1 acquires a package image and a score corresponding to the package image, the server 1 inputs the acquired package image and score into the favorability prediction model 171 and outputs information regarding the favorability of the package image.

Note that the process is not limited to the above. The favorability prediction model 171 may be trained using the product category and the score set for each product category. In this case, the server 1 inputs the package image to the network 14a. The network 14a extracts the feature amount of the package image and passes the extracted feature amount of the package image to the fully connected layer 14c. The server 1 inputs vector information obtained by combining the label indicating the product category and the score set for each category into one vector to the fully connected layer 14b. The label indicating the category may be, for example, 26 types of labels including beer, chuhai/cocktail, other alcohol, carbonated drinks, tea/black tea/coffee, instant drinks, nutritional drinks, other drinks, seasonings, cup noodles/bag noodles, soups, desserts, room temperature foods, frozen foods, refrigerated foods, chocolate/candy/gummy, ice cream, snacks, other sweets, cosmetics, shampoo, medicine, lip balm, sanitary products, pet food, and hot pot soup. Note that the 26 types of labels described above are only an example and are not limited to this.

The fully connected layer 14c predicts information about the favorability of the package image based on the features of the package image and the vector information input to the fully connected layer 14b. The prediction result is output to an output layer having neurons.

Then, a process of outputting package images to be proposed using the favorability prediction model 171 trained using the scores will be described. The server 1 acquires the scores set for each product category and multiple candidate package images generated based on a genetic algorithm. The server 1 inputs the acquired scores and multiple package images into the favorability prediction model 171, and outputs information related to the favorability of each package image.

The server 1 extracts a package image to be proposed from a plurality of candidate package images based on the output information on the degree of favorability. The server 1 transmits the extracted package image to be proposed to the terminal 2. The terminal 2 receives and displays the package image to be proposed transmitted from the server 1.

According to this variant example 1, it is possible to output information regarding the favorability of candidate package images generated based on a genetic algorithm using a favorability prediction model 171 trained using scores set for each product category.

According to this variant example 1, by utilizing the favorability prediction model 171 trained using the scores, it is possible to improve the prediction accuracy of information related to favorability, and thus to suggest appropriate package images.

(Embodiment 2)
The second embodiment relates to a form in which a plurality of candidate package images are generated using a plurality of style images. Note that a description of the contents overlapping with the first embodiment will be omitted. A style image is an image used to realize a package design related to a specific style (subject or theme, etc.) in combination with a basic package image that is an original image. Styles include, for example, landscapes, countries, cultures, seasons, oceans, mountains, islands, towns, etc.

Fig. 12 is a block diagram showing an example of the configuration of the server 1 according to the second embodiment. Note that a description of the contents overlapping with Fig. 2 will be omitted. A generative model 172 is stored in the mass storage unit 17. The generative model 172 is a generator that generates multiple candidate package images by learning a basic package image and multiple style images using a StyleGAN (A Style-Based Generator Architecture for Generative Adversarial Networks) method, and is a trained model generated by machine learning.

Figure 13 is a diagram conceptually illustrating the structure of StyleGAN. StyleGAN is a type of generative adversarial network (GAN). The generator constructed by StyleGAN is composed of two networks: a mapping network f (hereinafter referred to as "network f") and a synthesis network g (hereinafter referred to as "network g"). Network f is composed of a neural network consisting of an 8-layer fully connected layer (FC) f1, and performs nonlinear conversion (mapping) of latent variables (z∈Z) to a latent space (w∈W).

Network g is a network that starts with a constant image of, for example, 4x4 size, and repeats the upsampling process to generate an image of, for example, 1024x1024 size as the final output. Network g consists of 18 layers, and the vector w∈W output from network f is referenced 18 times in network g.

Network g is a combination of multiple synthesis network parts g1. The first network part g1 includes a constant tensor layer 20a, two AdaIN layers 20b, and a convolution layer 20c. The second and subsequent network parts g1 include an upsampling layer 20d, two AdaIN layers 20b, and two convolution layers 20c. The constant tensor layer 20a is a layer that inputs a minimum resolution parameter (constant tensor) prepared in advance to the network g. The AdaIN layer 20b is a layer that applies the vector w∈W obtained from the nonlinear transformation in the network f to spatial data as information about the style. The convolution layer 20c is a convolution layer with a filter of size 3×3, for example. The upsampling layer 20d is a layer that performs upsampling processing of the image.

When the server 1 inputs a basic package image and multiple style images to the generative model 172, it converts the latent variables (z∈Z) used to generate the package image into a latent space (w∈W) through the network f. The server 1 inputs the converted latent space (w∈W) into the network g as information related to the style. The server 1 performs an affine transformation on the latent space (w∈W) through the AdaIN layer 20b to perform AdaIN processing.

Based on the style information applied to the latent space (w∈W) by the Adaln process, the server 1 extracts features of the style image by convolving the pixel values of each pixel through the convolution layer 20c. After that, the server 1 inputs noise into the latent space (w∈W) to perform probabilistic variation. Through the above-mentioned process, the style information of the style image can be applied to the basic package image.

The network g gradually increases the resolution of the generated image (package image) by performing an upsampling process through the upsampling layer 20d. As shown in the figure, the resolution is successively upsampled from, for example, 4x4, eventually becoming, for example, 1024x1024. Accordingly, by performing upsampling, information about different styles at each resolution can be applied to the latent space (w∈W), thereby controlling the style of the generated image (package image).

As a result, for example, when a basic package image and one style image are input to the generative model 172, multiple candidate package images can be output. Specifically, the server 1 inputs the basic package image and the style image to the generative model 172. The server 1 converts the latent variables (z∈Z) used to generate the package image into a latent space (w∈W) via the network f. The server 1 inputs the converted latent space (w∈W) to the network g as information related to the style.

The server 1 accepts a selection of multiple network parts g1 of the network g. Each network part g1 of the network g is classified based on the resolution, for example, into low resolution styles (Coarse styles) from 4x4 to 8x8, medium resolution styles (Middle styles) from 16x16 to 32x32, and high resolution styles (Fine styles) from 64x64 to 1024x1024. For example, the server 1 may accept a selection of any one of the low resolution styles, medium resolution styles, or high resolution styles. Alternatively, the server 1 may accept a selection of multiple network parts g1 and generate multiple different package images.

The server 1 inputs style-related information to each selected network unit g1. Each network unit g1 to which the style-related information has been input applies the style-related information obtained according to the corresponding resolution style to the basic package image.

Specifically, the network section g1 corresponding to the low resolution style generates a package image by applying style-related information obtained according to the low resolution style (4x4 to 8x8) to the basic package image. The network section g1 corresponding to the medium resolution style generates a package image by applying style-related information obtained according to the medium resolution style (16x16 to 32x32) to the basic package image. The network section g1 corresponding to the high resolution style generates a package image by applying style-related information obtained according to the high resolution style (64x64 to 1024x1024) to the basic package image.

The generative model 172 is not limited to being constructed using the StyleGan method, but may be constructed using methods such as PGGAN (Progressive Growing GAN), DCGAN (Deep Convolutional GAN), or CycleGAN. Other neural style transformations may also be used.

In this embodiment, GAN is used as the generation (learning) method for the generative model 172, but the generative model 172 is not limited to a trained model related to GAN, and may be a trained model using other deep learning methods such as VAE (Variational Auto Encoder) and U-NET (U-shaped neural network), decision trees, etc.

When the server 1 generates multiple candidate package images using the above-mentioned generative model 172, it extracts a package image to be proposed from the multiple package images based on the package image's favorability. The server 1 outputs the extracted package image to be proposed to the terminal 2. Note that the output process of the package image to be proposed is the same as in embodiment 1, and therefore will not be described here.

Figure 14 is an explanatory diagram showing an example of a basic package image reception screen. The screen includes an image display field 15a, an image selection button 15b, a delete button 15c, an image name input field 15d, and a proceed button 15e.

The image display field 15a is a display field that displays the basic package image that is to be the original image. The image selection button 15b is a button for selecting a basic package image. The delete button 15c is a button for deleting the selected basic package image. The image name input field 15d is a text field for inputting the name of the basic package image. The forward button 15e is a button for transitioning to a style image reception screen (Figure 15).

When the terminal 2 receives a touch operation of the image selection button 15b, it receives the selection of a basic package image stored in the memory unit 22. The terminal 2 displays the received basic package image in the image display field 15a. When the terminal 2 receives a touch operation of the delete button 15c, it deletes the basic package image displayed in the image display field 15a.

When the terminal 2 receives an input operation in the image name input field 15d, it receives the input of the name of the basic package image. When the terminal 2 receives a touch operation of the forward button 15e, it passes the basic package image selected by the image selection button 15b and the name of the basic package image input in the image name input field 15d to the style image reception screen (Figure 15), and transitions to the style image reception screen.

Figure 15 is an explanatory diagram showing an example of a style image reception screen. The screen includes a style selection combo box 16a, an image display field 16b, a forward button 16c, and a back button 16d.

The style selection combo box 16a is a combo box for selecting a style (landscape, country, culture, season, sea, mountain, island, city, etc.). The image display field 16b is a display field for displaying style images. Note that the number of selectable style images (e.g., five) may be determined in advance for each style. The forward button 16c is a button for transitioning to a confirmation screen for package image generation (Figure 16). The back button 16d is a button for transitioning to a basic package image reception screen (Figure 14).

When the terminal 2 receives a selection operation on the style selection combo box 16a, it acquires multiple style images corresponding to the selected style from the storage unit 22 or an external device. The terminal 2 displays each acquired style image in the corresponding image display field 16b. As shown in the figure, when the "Landscape" style is selected, the server 1 acquires five style images corresponding to the "Landscape" style and displays each acquired style image in the corresponding image display field 16b. When the terminal 2 receives a touch operation on the image display field 16b corresponding to the style image to be selected, it displays a frame indicating a selected state superimposed on the corresponding image display field 16b.

When terminal 2 receives a touch operation of forward button 16c, it transfers the basic package image and the name of the basic package image transferred from the basic package image reception screen (Fig. 14), as well as the name of the style selected in style selection combo box 16a, and the single or multiple style images selected in image display field 16b, to the package image generation confirmation screen (Fig. 16), and transitions to the package image generation confirmation screen. When terminal 2 receives a touch operation of back button 16d, it transitions to the basic package image reception screen (Fig. 14).

Figure 16 is an explanatory diagram showing an example of a confirmation screen for generating a package image. The screen includes a basic package image display field 17a, a style image display field 17b, a style display field 17c, a start button 17d, and a back button 17e.

The basic package image display field 17a is a display field that displays a basic package image. The style image display field 17b is a display field that displays a style image. The style display field 17c is a display field that displays the name of the style. The start button 17d is a button that generates a candidate package image. The back button 17e is a button that transitions to a style image reception screen (Figure 15).

The terminal 2 receives the basic package image, the style name, and the style image selected according to the style, which are passed from the style image reception screen (Figure 15). The terminal 2 displays the received basic package image in the basic package image display field 17a, displays the style name in the style display field 17c, and displays the selected style image in the style image display field 17b.

When the terminal 2 receives a touch operation of the start button 17d, it transmits the selected basic package image and style image to the server 1. The server 1 generates multiple candidate packages using the generation model 172 based on the basic package image and style image transmitted from the terminal 2. The server 1 inputs the multiple generated candidate package images to the favorability prediction model 171 and outputs information related to the favorability of each package image.

The server 1 transmits each of the candidate package images, the ID corresponding to each package image, and the favorability level to the terminal 2. The terminal 2 receives each of the package images, the ID corresponding to each package image, and the favorability level transmitted from the server 1, passes them to the package image display screen (FIG. 17), and transitions to the package image display screen.

When the device 2 receives a touch operation of the back button 17e, it transitions to a style image reception screen (Figure 15).

Figure 17 is an explanatory diagram showing an example of a display screen for a package image. The screen includes an image ID display field 18a, an image display field 18b, a favorability display field 18c, and an image save button 18d.

The image ID display field 18a is a display field that displays the ID of the package image. The image display field 18b is a display field that displays the package image. The favorability display field 18c is a display field that displays information related to the favorability of the package image. That is, for each package image, the package image ID, the package image, and the favorability of the package image are simultaneously displayed. The image save button 18d is a button that saves the package image.

The terminal 2 acquires each candidate package image, its ID, and its favorability from the server 1, and displays them on the screen. Specifically, the terminal 2 displays the ID of each package image in the corresponding image ID display field 18a, displays each package image in the corresponding image display field 18b, and displays the favorability of each package image in the corresponding favorability display field 18c.

When the terminal 2 receives a touch operation of the image save button 18d, it stores the multiple package images displayed in the image display field 18b in the memory unit 22 or an external storage device in the specified image format (e.g., bmp, tiff, jpg, jpeg, gif, or png).

Figure 18 is a flowchart showing the processing steps when generating a package image using a style image. The control unit 21 of the terminal 2 accepts the selection of a basic package image through the input unit 24 (step S221). The control unit 21 accepts the selection of a style through the input unit 24 (step S222). Depending on the accepted style, the control unit 21 acquires multiple style images corresponding to the style from the memory unit 22 (step S223). The control unit 21 displays the acquired multiple style images on the display unit 25 (step S224).

The control unit 21 receives a selection of a style image to be converted from among the multiple style images displayed through the input unit 24 (step S225). The control unit 21 transmits the received basic package image and the style image to be converted to the server 1 through the communication unit 23 (step S226). The control unit 11 of the server 1 receives the basic package image and the style image transmitted from the terminal 2 through the communication unit 13 (step S121).

Based on the received basic package image and style image, the control unit 11 generates multiple candidate package images using the generation model 172 (step S122). The control unit 11 inputs the multiple generated package images to the favorability prediction model 171 and outputs (obtains) information regarding the favorability of each package image (step S123).

The server 1 transmits each of the candidate package images, the ID corresponding to each package image, and the favorability level to the terminal 2 via the communication unit 13 (step S124). The control unit 21 of the terminal 2 receives each of the package images, the ID corresponding to each package image, and the favorability level transmitted from the server 1 via the communication unit 23 (step S227). The control unit 21 displays each of the received package images, the ID corresponding to each package image, and the favorability level on the display unit 25 (step S228), and ends the process.

According to this embodiment, it is possible to construct a generative model 172 using a style image.

In this embodiment, by using the generative model 172 to output candidate package images, it is possible to realize a variety of package designs based on style.

(Embodiment 3)
The third embodiment relates to a form in which the appearance rate of image words of a package image is output using artificial intelligence based on the package image of a candidate. Note that a description of the contents that overlap with the first and second embodiments will be omitted.

Fig. 19 is a block diagram showing an example of the configuration of the server 1 of the third embodiment. Note that a description of the contents overlapping with Fig. 12 will be omitted. The mass storage unit 17 stores an appearance rate prediction model 173 (second learning model). The appearance rate prediction model 173 is a predictor that predicts the appearance rate of image words of a package image based on the package image, and is a learned model generated by machine learning.

Image words are characters (text) used to express reasons for favorability when evaluating a package image. Image words include, for example, "delicious" (none for non-food and drink products), "cute", "simple", "good design elements", "nostalgic", "gentle", "sense of security and trust", "seasonal", "healthy", "feels effective", "luxurious and high-quality", "good color", "new and unique", "sophisticated", "refreshing and cool", "easy to understand characteristics", "noticeable and memorable", "pretty and pretty" or "clean". Note that image words are not limited to those mentioned above, and any image words can be set according to actual needs.

Next, the image word extraction process will be described. The server 1 acquires a set of evaluation texts that evaluate the package image. The set of evaluation texts may be, for example, a set of reviews or impressions written in any format about the package image, or it may be evaluation information collected from responses to a questionnaire. If the set of evaluation texts is stored in the memory unit 12 or the mass storage unit 17, the server 1 acquires the set of evaluation texts from the memory unit 12 or the mass storage unit 17. The server 1 may also acquire the set of evaluation texts from an external device.

The server 1 extracts a list of keywords from the acquired set of evaluation texts. For example, a list of keywords whose frequency of occurrence is equal to or greater than a predetermined frequency (e.g., 1000 times) may be extracted from the set of evaluation texts. The server 1 eliminates negative words or meaningless words from the extracted list of keywords. Negative words include, for example, "sad," "painful," "unsatisfactory," "tasteless," "troubled," or "confused." For example, the control unit 11 performs a morphological analysis on the extracted list of keywords, and further uses an algorithm for scoring each word that appears in the list of keywords, such as tf-idf (Term Frequency-Inverse Document Frequency), to extract negative words or meaningless words.

The server 1 removes the extracted negative words or meaningless words from the list of extracted keywords. The server 1 vectorizes each keyword included in the list of keywords from which the negative words and meaningless words have been removed, using a known method for vectorizing words, such as Word2vec, Doc2Vec (Paragraph2vec), LDA (Latent Dirichlet Allocation), or NTSG (Neural Tensor Skip Gram).

The server 1 classifies each keyword based on the feature information of each vectorized keyword (clustering). Clustering is a type of natural language processing in which vectorized keywords are divided into groups (clusters) of words that are close to the same concept based on their direction and size. The server 1 extracts image words for package images from each classified keyword. For example, manual input of image words for each cluster may be accepted. The extracted image words are associated with package images and used when generating training data, which will be described later.

Next, a process of outputting the appearance ratio of image words in package images using the appearance ratio prediction model 173 will be described.
20 is an explanatory diagram for explaining the appearance ratio prediction model 173. The appearance ratio prediction model 173 is used as a program module that is a part of artificial intelligence software. The appearance ratio prediction model 173 is a predictor of the appearance ratio of image words in which a neural network has been constructed, which receives a package image as input and outputs the appearance ratio of image words in the package image.

The server 1 constructs the appearance rate prediction model 173 by performing deep learning to learn the feature quantities of the parts that influenced the appearance rate of the image word of the package image. For example, the appearance rate prediction model 173 is a CNN, and has an input layer that accepts the input of the package image, an output layer (softmax layer) that outputs the appearance rate of the image word of the package image using an activation function (softmax function in this embodiment), and an intermediate layer that has been trained by backpropagation.

In this embodiment, the occurrence rate prediction model 173 is described as being a CNN, but the occurrence rate prediction model 173 is not limited to a CNN and may be a trained model constructed with any learning algorithm, such as a neural network other than a CNN, an SVM, a Bayesian network, or a regression tree.

The server 1 acquires a plurality of combinations of training data in which package images are associated with image words of the package images. The training data is data in which package images are labeled with image words of the package images.

The server 1 performs learning using the acquired training data. Specifically, the server 1 inputs the packaging images, which are the training data, into the input layer, and acquires the occurrence rate of the image word of the packaging image from the output layer after arithmetic processing in the intermediate layer. The server 1 compares the prediction result output from the output layer (e.g., the image word with the highest occurrence rate) with the information (image word) labeled with the packaging image in the training data, i.e., the correct answer value, and optimizes the parameters used in the arithmetic processing in the intermediate layer so that the predicted value from the output layer approaches the correct answer value.

The server 1 performs the above process for each package image included in the training data to generate an appearance rate prediction model 173. As a result, for example, the server 1 can use the training data to train the appearance rate prediction model 173, thereby constructing a model capable of outputting the appearance rate of image words for package images.

When the server 1 acquires a candidate package image, it inputs the acquired package image to the appearance rate prediction model 173. The server 1 performs a calculation process to extract features of the package image in the middle layer of the appearance rate prediction model 173. The server 1 inputs the extracted features to the output layer of the appearance rate prediction model 173, and acquires the appearance rate of the image word of the package image as an output.

As shown in the figure, the predicted results are output that show the respective occurrence rates of "cute," "simple," "seasonal," "...," and "healthy" for the package image as "0.82," "0.55," "0.35," "...," and "0.25."

In addition, an appearance rate prediction model 173 can be generated for each gender or age group. Specifically, the server 1 accepts a setting of the gender or age group, and acquires multiple pieces of training data to be used for generating the appearance rate prediction model 173 according to the accepted gender or age group. The server 1 uses the acquired training data to generate an appearance rate prediction model 173 corresponding to the accepted gender or age group.

The server 1 stores the occurrence rate prediction model 173 generated for each gender or age group in the mass storage unit 17. The server 1 can output the occurrence rate of image words of a package image from the package image for each gender or age group using multiple occurrence rate prediction models 173 prepared for multiple genders or age groups.

Note that the process is not limited to the above. The occurrence rate prediction model 173 may be trained using a score set for each product category. In this case, the server 1 inputs the package image and the score set for each category into the occurrence rate prediction model 173, and outputs the occurrence rate of the image word for the package image.

Furthermore, for the package images generated by the genetic algorithm in embodiment 1, it is possible to generate package images that are candidates for the next generation based on the appearance rate of a specific image word (e.g., delicious). Specifically, the server 1 acquires the identified image word. The server 1 inputs each package image generated by the genetic algorithm to the appearance rate prediction model 173, and outputs the appearance rate of the image word for each package image.

The server 1 acquires multiple package images with a high occurrence rate of the identified image word based on the occurrence rate of the image word output from the occurrence rate prediction model 173. The server 1 performs crossover and mutation processes based on multiple design elements contained in each acquired package image, and generates multiple package images that are candidates for the next generation.

Furthermore, for package images generated using the generation model 172 in embodiment 2, candidate package images can be output based on the appearance rate of a specific image word. Specifically, the server 1 acquires the identified image words. The server 1 inputs each package image generated using the generation model 172 to the appearance rate prediction model 173 and outputs the appearance rate of the image word for each package image. The server 1 outputs package images with a high appearance rate of the identified image word based on the appearance rate of the image word output from the appearance rate prediction model 173 as candidate package images.

According to this embodiment, it is possible to output the appearance rate of image words of candidate package images using the appearance rate prediction model 173.

According to this embodiment, by generating next-generation package images based on package images with a high occurrence rate of image words, it becomes possible to plan appropriate package designs based on clear reasons for favorability.

(Embodiment 4)
The fourth embodiment relates to a form in which, based on a candidate package image, artificial intelligence is used to output the standard deviation of the favorability of the package image. Note that a description of the contents that overlap with the first to third embodiments will be omitted.

Regarding likes and dislikes (variation) of package images, there are cases where the same package image is rated with different degrees of favorability depending on gender or age. Such differences in likes and dislikes of package images can be interpreted as the standard deviation of favorability. The higher the standard deviation of favorability, the greater the difference in likes and dislikes; conversely, the lower the standard deviation of favorability, the smaller the difference in likes and dislikes.

Fig. 21 is a block diagram showing an example of the configuration of the server 1 of the fourth embodiment. Note that a description of the contents overlapping with Fig. 19 will be omitted. The mass storage unit 17 stores a standard deviation prediction model 174 (third learning model). The standard deviation prediction model 174 is a predictor that predicts the standard deviation of the favorability of a package image based on the package image, and is a learned model generated by machine learning.

Figure 22 is an explanatory diagram explaining the standard deviation prediction model 174. The standard deviation prediction model 174 is used as a program module that is part of artificial intelligence software. The standard deviation prediction model 174 is a predictor of the standard deviation of favorability that has already constructed a neural network that takes a package image as input and outputs the standard deviation of the favorability of the package image.

The server 1 constructs a standard deviation prediction model 174 by performing deep learning to learn the feature quantities of the package image. For example, the standard deviation prediction model 174 is a CNN, and has an input layer that accepts input of a package image, an output layer that outputs the standard deviation of the favorability of the package image, and an intermediate layer that has been trained by backpropagation.

In this embodiment, the standard deviation prediction model 174 is described as a CNN, but the standard deviation prediction model 174 is not limited to a CNN and may be a trained model constructed with any learning algorithm, such as a neural network other than a CNN, an SVM, a Bayesian network, or a regression tree.

The server 1 acquires multiple combinations of training data in which package images are associated with standard deviations calculated based on information about the likeability of the package images by multiple people. The training data is data in which package images are labeled with the standard deviation of the likeability of the package images.

To create training data, for example, the server 1 obtains the favorability ratings of responses from 1,000 people for one package image on a five-point scale ranging from a maximum of 5 (like) to a minimum of 1 (dislike). The server 1 calculates the average value of the favorability ratings of the responses from the 1,000 people. The server 1 calculates the standard deviation of the favorability ratings of the responses from the 1,000 people based on the calculated average value of the favorability ratings. The server 1 can create training data by labeling a package image with the standard deviation of the favorability ratings of the package image from the 1,000 people.

The server 1 performs learning using the acquired training data. Specifically, the server 1 inputs the package images, which are the training data, into the input layer, and obtains the standard deviation of the favorability of the package images from the output layer after arithmetic processing in the intermediate layer. The server 1 compares the prediction result output from the output layer with the information labeled for the package images in the training data, i.e., the correct answer value, and optimizes the parameters used in the arithmetic processing in the intermediate layer so that the predicted value from the output layer approaches the correct answer value.

The server 1 performs the above process for each package image included in the training data to generate a standard deviation prediction model 174. As a result, for example, the server 1 can use the training data to train the standard deviation prediction model 174, thereby constructing a model that can output the standard deviation of the favorability of package images.

When the server 1 acquires a candidate package image, it inputs the acquired package image to the standard deviation prediction model 174. The server 1 performs a calculation process to extract features of the package image in the middle layer of the standard deviation prediction model 174. The server 1 inputs the extracted features to the output layer of the standard deviation prediction model 174, and acquires the standard deviation of the favorability of the package image as an output. As shown in the figure, the standard deviation of the favorability, which is "1.010", is output from the standard deviation prediction model 174.

In addition, a standard deviation prediction model 174 can be generated for each gender or age group. Specifically, the server 1 accepts a setting of gender or age group, and acquires multiple pieces of training data to be used for generating the standard deviation prediction model 174 according to the accepted gender or age group. The server 1 uses the training data to generate a standard deviation prediction model 174 corresponding to the accepted gender or age group.

The server 1 stores the standard deviation prediction model 174 generated for each gender or age group in the mass storage unit 17. The server 1 can output the standard deviation of the favorability of a package image from the package image for each gender or age group using multiple standard deviation prediction models 174 prepared for multiple genders or age groups.

Note that the process is not limited to the above. The standard deviation prediction model 174 may be trained using scores set for each product category. In this case, the server 1 inputs the package image and the score set for each category into the standard deviation prediction model 174, and outputs the standard deviation of the favorability of the package image.

Furthermore, for the package images generated by the genetic algorithm in embodiment 1, it is possible to generate package images that are candidates for the next generation based on the standard deviation of the favorability. Specifically, the server 1 acquires multiple package images with a small standard deviation of the favorability output from the standard deviation prediction model 174. The server 1 performs crossover processing and mutation processing based on multiple design elements contained in each acquired package image, and generates multiple package images that are candidates for the next generation.

Furthermore, for the package images generated by the genetic algorithm, it is possible to generate candidate package images for the next generation based on a combination of information about the likeability level, the frequency of occurrence of specific image words (e.g., delicious), and the standard deviation of the likeability level.

Specifically, first, the server 1 acquires multiple package images with high favorability output from the favorability prediction model 171. Next, the server 1 inputs the multiple acquired package images with high favorability into the appearance rate prediction model 173 and outputs the appearance rate of the image word. Based on the appearance rate of the image word output from the appearance rate prediction model 173, the server 1 acquires multiple package images with a high appearance rate of a specified image word (e.g., delicious).

Then, the server 1 inputs the multiple package images acquired into the standard deviation prediction model 174 and outputs the standard deviation of the favorability of each package image. The server 1 acquires multiple package images with a small standard deviation of the favorability based on the standard deviation of the favorability output from the standard deviation prediction model 174. The server 1 outputs the multiple acquired package images as package images that will be candidates for the next generation.

In the above example, the process of acquiring candidate package images was performed in the order of information on favorability, the occurrence rate of image words, and the standard deviation of favorability, but the order of the acquisition process is merely an example and may be performed in any order. The process of acquiring information on favorability, the occurrence rate of image words, and the standard deviation of favorability may be performed simultaneously. In this case, candidate package images are acquired based on the overall result of information on favorability, the occurrence rate of image words, and the standard deviation of favorability.

Furthermore, it is possible to generate package images that are candidates for the next generation based on a combination of any two of the information on favorability, the occurrence rate of image words, and the standard deviation of favorability. For example, when using a combination of information on favorability and the standard deviation of favorability, the server 1 first obtains multiple package images with high favorability output from the favorability prediction model 171.

Next, the server 1 inputs the multiple package images with high favorability obtained into the standard deviation prediction model 174 and outputs the standard deviation of the favorability of each package image. The server 1 obtains multiple package images with a small standard deviation of the favorability based on the standard deviation of the favorability output from the standard deviation prediction model 174. The server 1 outputs the multiple package images obtained as package images that will be candidates for the next generation.

Furthermore, for package images generated using the generation model 172 in embodiment 2, candidate package images can be output based on the standard deviation of the favorability. Specifically, the server 1 inputs each package image generated using the generation model 172 to the standard deviation prediction model 174, and outputs the standard deviation of the favorability of each package image. The server 1 outputs package images with a small standard deviation of the favorability output from the standard deviation prediction model 174 as candidate package images.

According to this embodiment, it is possible to output the standard deviation of the favorability of candidate package images using the standard deviation prediction model 174.

According to this embodiment, by generating next-generation package images based on package images with a small standard deviation of favorability, it becomes possible to plan appropriate package designs that take into account differences in likes and dislikes of package images.

The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is indicated by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

1. Information processing device (server)
REFERENCE SIGNS LIST 11 Control unit 12 Storage unit 13 Communication unit 14 Input unit 15 Display unit 16 Reading unit 17 Large-capacity storage unit 171 Favourability prediction model 172 Generation model 173 Appearance ratio prediction model 174 Standard deviation prediction model 1a Portable storage medium 1b Semiconductor memory 1P Control program 2 Information processing terminal (terminal)
21 Control unit 22 Memory unit 23 Communication unit 24 Input unit 25 Display unit 26 Photography unit 2P Control program

Claims (17)

Accept the number of design elements and the number of designs included in the candidate package images;
outputting a design element selection field for selecting a design element from the number of accepted design elements;
outputting a design selection field for selecting a design from the number of designs received, in correspondence with each design element selection field;
generating a plurality of candidate package images;
A first learning model that has been trained to output information regarding the favorability of a package image when the package image is input, inputting the generated package images into the first learning model and outputting information regarding the favorability of each package image;
A program for causing a computer to execute a process of outputting a package image to be proposed from the plurality of package images based on the output information on the degree of favorability. A plurality of designs are registered for a plurality of design elements included in the package image,
The program according to claim 1 , further comprising: generating a plurality of candidate package images by rearranging designs for each design element. Obtain multiple highly favorable package images,
3. The program according to claim 1 or 2, which executes a process of rearranging the designs of a plurality of design elements contained in the plurality of acquired package images based on a genetic algorithm to generate a plurality of package images that are candidates for the next generation. outputting information regarding each package image to be proposed, an ID of the package image, and a favorability level of the package image;
4. The program according to claim 1, further comprising a process for generating a plurality of package images as candidates for the next generation when a generation request is received for the output package image. A basic package image serving as an original image and a plurality of style images for performing style conversion are obtained;
5. The program according to claim 1, further comprising: inputting the acquired basic package image and each of the style images into a generative model that has been trained to generate a package image based on a basic package image and a style image, and executing a process of generating a plurality of candidate package images. Accepts style selection,
Output multiple style images corresponding to the accepted styles;
Accept the selection of multiple target style images from the multiple output style images,
The program according to claim 5 , which causes execution of a process of inputting the basic package image and a plurality of accepted style images to the generative model to generate a plurality of candidate package images. 7. The program according to claim 5 , further comprising: outputting information about each candidate package image, an ID of the package image, and a favorability level of the package image. The generative model has a synthesis network in which a plurality of synthesis network units each having a convolution layer are connected,
8. The program according to claim 5 , which causes a selected one of the synthesis network units to input information about the style image to generate a package image. 9. The program according to claim 1, further comprising a process for outputting a package image having a high weighted average favorability level based on a weighted average favorability level of each package image output by the first learning model. Obtain the score set for each product category,
inputting the obtained score and the plurality of candidate package images into the first learning model, and outputting information regarding the favorability of each package image;
10. The program according to claim 1, further comprising: outputting a package image to be proposed from the plurality of package image candidates based on the output information regarding the degree of favorability. The program according to any one of claims 1 to 10, which causes a second learning model that has been trained to output the occurrence rate of image words of the package image when the package image is input to execute a process of inputting the plurality of candidate package images and outputting the occurrence rate of image words of each package image. The program according to any one of claims 1 to 11, which causes a third learning model that has been trained to output the standard deviation of the favorability of the package image when the package image is input to execute a process of inputting the plurality of candidate package images and outputting the standard deviation of the favorability of each package image. Accept the number of design elements and the number of designs included in the candidate package images;
outputting a design element selection field for selecting a design element from the number of accepted design elements;
outputting a design selection field for selecting a design from the number of designs received, in correspondence with each design element selection field;
generating a plurality of candidate package images;
A first learning model that has been trained to output information regarding the favorability of a package image when the package image is input, inputting the generated package images into the first learning model and outputting information regarding the favorability of each package image;
An information processing method in which a computer executes a process of outputting a package image to be proposed from the plurality of package images based on the output information on the degree of favorability. An information processing device including a control unit,
The control unit is
Accept the number of design elements and the number of designs included in the candidate package images;
outputting a design element selection field for selecting a design element from the number of accepted design elements;
outputting a design selection field for selecting a design from the number of designs received, in correspondence with each design element selection field;
generating a plurality of candidate package images ;
A first learning model that has been trained to output information regarding the favorability of a package image when the package image is input, inputting the generated package images into the first learning model and outputting information regarding the favorability of each package image ;
A package image to be proposed is output from the plurality of package images based on the output information regarding the degree of favorability.
Information processing device. Accept the selection of a set number of design elements;
Accepting a selection of a set number of designs corresponding to each of the accepted design elements;
Submit your design for the accepted design elements;
obtaining a plurality of package images extracted based on information regarding the favorability of the package images from a plurality of candidate package images generated by rearranging the designs of the design elements;
Displaying each of the acquired candidate package images, an ID of the package image, and information regarding the favorability of the package image;
A program that causes a computer to execute a process of, when a request for generating a package image that is a candidate for the next generation of the displayed package image is received, transmitting the received generation request. Accept the selection of the basic package image that will be the original image,
Accepts style selection,
Obtain multiple style images corresponding to the accepted style;
Accepting selection of multiple target style images from the multiple acquired style images;
Send the accepted basic package image and multiple target style images,
obtaining a plurality of candidate package images generated based on the base package image and the target style image;
The program according to claim 15 , further comprising: displaying information about each of the acquired candidate package images, an ID of the package image, and a favorability level of the package image. Obtain the score set for each product category,
Generate multiple candidate package images,
inputting the acquired score and the generated multiple package images into a first learning model that has been trained to output information regarding the favorability of the package image when a score and a package image are input, and outputting information regarding the favorability of each package image;
A package image to be proposed is output from the plurality of package images based on the output information regarding the degree of favorability.
A program that causes a computer to carry out processing.

Images