JP7742785B2 - Information providing device, information providing method, and information providing program - Google Patents

Description

The present invention relates to an information provision device, an information provision method, and an information provision program.

Technologies that support outfit coordination are known in the past. For example, there is a technology that generates outfits based on rules regarding fashion combinations.

Japanese Patent Application Laid-Open No. 2013-235528

However, the above-mentioned conventional technology leaves room for improvement in terms of providing useful information to customers (referred to as "users"). For example, conventional technology makes it difficult to suggest makeup products to users that match their available hairstyles.

This application has been made in light of the above, and aims to provide an information provision device, an information provision method, and an information provision program that enable useful information to be provided to users.

In order to solve the above-mentioned problems and achieve the objectives, the information providing device of the present invention is characterized by comprising an acquisition unit that acquires hair style information related to hair styles that can be provided by a user, a calculation unit that calculates a compatibility level that indicates the degree of compatibility between the style and makeup based on the hair style information, and a generation unit that generates information to be provided to the user based on the compatibility level.

The information provision method of the present invention is executed by an information provision device and includes an acquisition step of acquiring hair style information related to hair styles that can be provided by a user, a calculation step of calculating a compatibility level indicating the degree of compatibility between the hairstyle and makeup based on the hair style information, and a generation step of generating information to be provided to the user based on the compatibility level.

The information provision program of the present invention is characterized in that it causes a computer to execute the following steps: an acquisition step for acquiring hair style information related to hair styles that can be provided by a user; a calculation step for calculating a compatibility level indicating the degree of compatibility between the hair style and makeup based on the hair style information; and a generation step for generating provision information to be provided to the user based on the compatibility level.

This invention can provide useful information to users.

FIG. 1 is a diagram illustrating an example of the configuration of an information providing system according to an embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of the information providing device according to the embodiment. FIG. 3 is a diagram illustrating an example of a mode information storage unit according to the embodiment. FIG. 4 is a diagram illustrating an example of a makeup information storage unit according to the embodiment. FIG. 5 is a diagram illustrating an example of a compatibility information storage unit according to the embodiment. FIG. 6 is a diagram illustrating an example of a provided information storage unit according to the embodiment. FIG. 7 is a diagram showing a specific example of the information providing process according to the embodiment. FIG. 8 is a flowchart illustrating an example of the flow of the information providing process according to the embodiment. FIG. 9 illustrates an example of a hardware configuration.

Below, detailed explanations will be given of the information provision device, information provision method, and information provision program (hereinafter, "embodiments") according to the present application, with reference to the drawings. Note that the information provision device, information provision method, and information provision program according to the present application are not limited to these embodiments. Furthermore, identical components in the following embodiments will be designated by the same reference numerals, and duplicate explanations will be omitted.

[Embodiment]
The configuration of the information providing system 100 according to the embodiment, the configuration of the information providing device 10, a specific example of the information providing process, and the flow of the information providing process will be described below, and finally the effects of this embodiment will be described.

1. Configuration of information provision system 100
The processing of an information provision system 100 according to this embodiment (referred to as "this system" as appropriate) will be described using Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the information provision system 100 according to this embodiment. Below, the example of the configuration of this system 100, the processing of this system 100, the set matching technology applied in this system 100, and the effects of this system 100 will be described in that order.

(1-1. Example of configuration of information providing system 100)
1 includes an information providing device 10, a user terminal 20, and a cosmetic information database 30. The information providing device 10, the user terminal 20, and the cosmetic information database 30 are communicably connected via a predetermined communication network (not shown) via wired or wireless communication. Note that the system 100 may include multiple information providing devices 10, multiple user terminals 20, or multiple cosmetic information databases 30.

(1-1-1. Information providing device 10)
The information providing device 10 is a device that transmits and receives data between the user terminal 20 and the cosmetic information database 30, and is realized, for example, by a server device or a cloud system. The example in Fig. 1 shows a case where the information providing device 10 is realized by a server device.

(1-1-2. User terminal 20)
The user terminal 20 is a device (computer) used by a user U to browse web pages related to makeup (appropriately referred to as "makeup") and to perform online shopping for cosmetics (appropriately referred to as "cosmetics") on the web. The user terminal 20 accepts operations by the user U. The user terminal 20 may be realized, for example, by a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like. The example in FIG. 1 shows a case where the user terminal 20 is realized by a smartphone.

In the following, user terminal 20 may be referred to as user U. In other words, user U may also be read as user terminal 20. User U may also be used to represent multiple customers.

(1-1-3. Cosmetic Information Database 30)
The makeup information database 30 is a storage device that stores makeup information related to makeup, which will be described later.

(1-2. Processing of Information Providing System 100)
The following describes steps S1 to S6 as the processing of the information providing system 100. Note that steps S1 to S6 below may be executed in a different order. Also, some of steps S1 to S6 below may be omitted.

(1-2-1. Processing of step S1)
In the present system 100, first, the information providing device 10 acquires, via the user terminal 20 of the user U, who may be a stylist at the beauty salon, hairstyle information related to hairstyles available to the user U at the beauty salon (step S1). Here, the hairstyle information refers to information related to the hairstyle in general, such as the hairstyle and hair color. The hairstyle information may also be information related to the hairstyle of a wig or toupee, and is not particularly limited. In the example of FIG. 1, the information providing device 10 acquires the hairstyle information from the user terminal 20, but it may also acquire the hairstyle information from a terminal device or database (not shown). That is, it may also acquire hairstyle information associated with the hair style included in an image selected by the user U from among images showing multiple hair styles displayed on the user terminal 20.

At this time, the information providing device 10 may acquire user information about the user U from the user terminal 20. Here, the user information refers to information including user attributes such as the user U's gender, age, generation, occupation, annual income, place of residence, marital status, whether or not they have children, videos of the user U, categories of interest to the user U, and behavioral history such as the user U's search history and browsing history on websites, purchase history in online shopping, and location information of the user U. Furthermore, the user information may also include information such as screen information of the user terminal 20 and biometric information of the user U, and is not particularly limited.

(1-2-2. Processing of step S2)
Second, in the present system 100, the information providing device 10 references the makeup information database 30 and acquires makeup information (step S2). Here, the makeup information refers to information related to makeup, such as cosmetics such as foundation, lipstick, and eyeshadow, tools and implements used in makeup application, and makeup application procedures, but is not limited thereto. In the example of FIG. 1, the information providing device 10 acquires the makeup information from the makeup information database 30, but it may also acquire the makeup information from a terminal device or database (not shown).

(1-2-3. Processing of step S3)
Third, in the present system 100, the information providing device 10 calculates a compatibility level indicating the degree of compatibility between the hair style and makeup that the user U can provide, based on the acquired hair style information and makeup information (step S3). For example, the information providing device 10 uses a machine learning model to calculate the compatibility level, which takes a value between 0 and 1 depending on the degree of compatibility between the combination of hair style and makeup. In this case, the information providing device 10 calculates the compatibility level using a machine learning model such as a deep neural network (DNN) that is trained to output the compatibility level for each combination of hair style and makeup when the information providing device 10 receives the hair style information and makeup information related to the hair style that the user U can provide. Alternatively, the information providing device 10 may calculate the compatibility level based on rules.

The above-mentioned step S3 will be explained using a specific example. For example, the information providing device 10 can calculate the compatibility level using a machine learning model trained for each attribute of the person to whom user U applies makeup, and a machine learning model trained for each situation of the person, as shown below.

The information providing device 10 can calculate compatibility using a machine learning model trained for each attribute, including the subject's hobbies and preferences. For example, the information providing device 10 generates multiple machine learning models for a subject who likes "XX (e.g., celebrity, era, type of drama (action, etc.))" and generates correct answer data for each machine learning model. Specifically, the machine learning model is trained using images that have been "liked" by a subject who likes "XX" or images that match "XX" as correct answer data. As a result, the information providing device 10 can select a machine learning model that matches the attributes of the subject specified by the user U and suggest makeup that is deemed to have a high compatibility rate by the selected machine learning model.

The information providing device 10 can also calculate compatibility using a machine learning model trained for each situation of the subject. For example, for each corporate recruitment situation, such as new graduate, job change, or mid-career recruitment, the information providing device 10 generates a machine learning model for each of new graduate, job change, and mid-career recruitment, using images of subjects who passed document screening during their job search as correct answer data. Therefore, for example, if user U selects "job change," the information providing device 10 can suggest makeup that is appropriate for the document screening at the time of job change.

(1-2-4. Processing of step S4)
Fourth, in the present system 100, information to be provided to the user U is generated based on the calculated compatibility, the acquired appearance information, and the user information of the user U (step S4). Here, the information to be provided is information about makeup identified from the makeup indicated by the makeup information, such as information about makeup that matches the hairstyle that the user U can wear, but is not limited to such information. At this time, the information providing device 10 identifies makeup indicated by the makeup information that has a compatibility level equal to or greater than a predetermined threshold, and generates the information to be provided. For example, the information providing device 10 identifies an image of makeup that has a compatibility level of "0.7" or greater with the hairstyle that the user U can wear, and generates the information to be provided about the image and the cosmetics required for the makeup indicated in the image.

The above-mentioned step S4 will now be explained using a specific example. For example, the information providing device 10 can generate information to be provided based on the user information of user U as shown below.

The information providing device 10 can generate information to be provided based on the user attributes indicated in the user information. For example, the information providing device 10 can prioritize suggesting makeup candidates that are similar to the makeup preferences registered in advance by the user U. Furthermore, the information providing device 10 can also prioritize suggesting makeup images that are relatively similar to the makeup characteristics of a sample image provided by the user U to a hair and makeup booking site, etc. Therefore, the information providing device 10 can suggest makeup that is highly similar from among the candidates identified from the cosmetic information based on the user attributes, etc.

The information providing device 10 can also generate information to be provided based on the behavioral history indicated by the user information. For example, if the information providing device 10 can estimate the cosmetics that user U has purchased based on the behavioral history, such as user U's purchase history, it will suggest makeup based on those cosmetics.

(1-2-5. Processing of step S5)
Fifth, in the present system 100, the information providing device 10 transmits the generated provided information to the user terminal 20 of the user U (step S5). For example, the information providing device 10 transmits to the user terminal 20 of the user U provided information including an image of makeup that has a compatibility rating of 0.7 or higher with the hair and makeup that the user U can provide, and information on the cosmetics and makeup tools required for the makeup.

(1-2-6. Processing of step S6)
Sixth, in the present system 100, the information providing device 10 learns the calculated compatibility degree (step S6). For example, the information providing device 10 uses a machine learning model to learn using backpropagation or the like so that, when information indicating a good compatibility between a combination of a hair style that the user U can provide and makeup is input, the information providing device 10 outputs the compatibility degree of makeup for the hair style as a numerical value approaching "1." Furthermore, the information providing device 10 uses a machine learning model to learn using backpropagation or the like so that, when information indicating a bad compatibility between a combination of a hair style that the user U can provide and makeup is input, the information providing device 10 outputs the compatibility degree of makeup for the hair style as a numerical value approaching "0."

(1-3. Set matching technology applied in the information provision system 100)
Here, the set matching technology applied in the information provision system 100 described above will be described in detail. A technology called set matching is known as an example of a technology for calculating a score indicating the degree of harmony between sets (hereinafter referred to as a "matching score" as appropriate) based on a set of information indicating feature quantities such as images. Set matching is a technology that uses, for example, deep learning, and estimates that the higher the matching score, the higher the compatibility between the sets. Therefore, set matching can quantitatively evaluate the compatibility between the sets. Calculation of a matching score using set matching is disclosed, for example, in "Exchangeable Deep Neural Networks for Set-to-Set Matching and Learning" by Y. Saito, T. Nakamura, H. Hachiya, and K. Fukumizu. Note that set matching is just an example, and embodiments are not limited to those using this technology.

The information providing device 10 applies a technique for calculating matching scores between sets, such as set matching, to calculate a matching score (compatibility) for each combination of hair style and makeup. Specifically, the information providing device 10 calculates the compatibility with makeup for each hair style that the user U can provide. At this time, the information providing device 10 can estimate that the higher the compatibility, the better the compatibility between the hair style that the user U can provide and the makeup. Therefore, the information providing device 10 identifies the makeup that is calculated to have the highest compatibility for one hair style. Furthermore, the information providing device 10 estimates that the identified makeup for one hair style is the makeup that is most compatible among the makeups included in the makeup information. This allows the information providing device 10 to quantitatively determine the compatibility between hair styles and makeup, rather than relying on the user U's subjective judgment of appearance, etc.

The information providing device 10 also trains the machine learning model as follows. For example, the information providing device 10 identifies images posted to a social networking site (SNS) that have been favorably rated by viewers of the SNS. Next, the information providing device 10 identifies the hair style (hairstyle, hair color) and makeup in the identified images. The information providing device 10 may use various identification models or obtain identification results through crowdsourcing, etc. The information providing device 10 uses the above identification results as correct answer data. The information providing device 10 may also use the results of identification from images posted by user U as correct answer data. The information providing device 10 then trains the machine learning model so that it outputs a high score when a hair style and makeup that are correct answer data are input, and outputs a low score when a hair style and makeup that are not included in the correct answer data are input. The information providing device 10 may generate such a machine learning model for each attribute of the subject to whom user U applies makeup, or for each situation of the subject. Furthermore, the information providing device 10 may reflect the degree of compatibility taking into account multiple perspectives by manipulating the output values of multiple models as the output of the machine learning model.

(1-4. Effects of the information provision system 100)
As described above, in the information provision system 100, the information provision device 10 acquires hairstyle information regarding the hairstyles that the user U can provide, calculates a compatibility score indicating the degree of compatibility between the hairstyles that the user U can provide and makeup based on the hairstyle information, and generates information to be provided to the user U based on the compatibility score. The information provision device 10 calculates the compatibility score using a machine learning model that has learned the degree of compatibility between the combination of the hair style and the makeup applied by a subject with that hair style. Therefore, the system 100 can estimate makeup that matches the hair and makeup that the user U can provide or desires to provide, and suggest cosmetics to the user U. In other words, the system 100 can provide useful information to the user U, such as a hairdresser who makes hair and makeup suggestions on a hair and makeup booking site or the like.

In addition, in this system 100, the information providing device 10 further acquires user information about the user U who provides the hair style, identifies makeup based on the acquired user information, and generates the information to be provided. As a result, this system 100 can suggest to the user U hairstyles and hair and makeup that the user U can provide, and that have a high affinity with hairstyles that are highly rated on social media, or hair and makeup that is favored by celebrities and other famous people. In other words, this system 100 can provide more specific and useful information to the user U for whom hair and makeup suggestions are made.

Furthermore, in the present system 100, the information providing device 10 calculates the compatibility level using a machine learning model that has learned the compatibility level between a hair style and makeup applied by a subject with that hair style. The information providing device 10 calculates the compatibility level by using a machine learning model that has learned the compatibility level from video images including hair styles and makeup, a machine learning model that has learned the compatibility level for each attribute of the subject receiving the makeup, or a machine learning model that has learned the compatibility level for each situation of the subject. Therefore, the present system 100 generates various machine learning models trained using corrective data collected from social media, television, movies, etc., and can suggest more accurate makeup to the user U who will be applying the makeup. In other words, the present system 100 enables the user U to receive hair and makeup suggestions to be provided with more specific and useful information with a high degree of accuracy.

2. Configuration of information providing device 10
The configuration of the information providing device 10 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example configuration of the information providing device 10 according to the embodiment. As shown in Fig. 2, the information providing device 10 includes a communication unit 11, a storage unit 12, and a control unit 13. The information providing device 10 may also include an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator of the information providing device 10, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

(2-1. Communication unit 11)
The communication unit 11 is realized by, for example, a network interface card (NIC), etc. The communication unit 11 is connected to a predetermined communication network via a wired or wireless connection, and transmits and receives information to and from various devices.

(2-2. Storage unit 12)
The storage unit 12 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 2 , the storage unit 12 according to the embodiment includes a behavior information storage unit 12a, a cosmetic information storage unit 12b, a compatibility information storage unit 12c, and a provided information storage unit 12d. The storage unit 12 stores various types of information referenced by the control unit 13 when it operates, and various types of information acquired when the control unit 13 operates.

(2-2-1. Behavior information storage unit 12a)
The aspect information storage unit 12a stores various information (aspect information) related to the aspect of hair that can be provided by the user U. An example of the information stored in the aspect information storage unit 12a will be described below with reference to FIG. 3. FIG. 3 is a diagram showing an example of the aspect information storage unit 12a according to the embodiment. In the example of FIG. 3, the aspect information storage unit 12a has items such as "aspect ID" and "aspect information."

"Appearance ID" indicates identification information for identifying the hair appearance. "Appearance information" is information about the hair appearance, indicating the general hairstyle, such as hair style and hair color, or information about the hair appearance when using a wig or toupee, etc.

In other words, Figure 3 shows an example in which the aspect information for the hair aspect identified by the aspect ID "HID#1" is "Aspect Information #1," and the aspect information for the hair aspect identified by the aspect ID "HID#2" is "Aspect Information #2."

(2-2-2. Cosmetic information storage unit 12b)
The makeup information storage unit 12b stores various types of makeup-related information (makeup information). An example of the information stored in the makeup information storage unit 12b will be described below with reference to FIG. 4. FIG. 4 is a diagram illustrating an example of the makeup information storage unit 12b according to the embodiment. In the example of FIG. 4, the makeup information storage unit 12b has fields such as "makeup ID" and "makeup information."

"Makeup ID" refers to identification information used to identify makeup. "Makeup information" refers to information related to makeup, such as cosmetics such as foundation, lipstick, and eye shadow, as well as the tools and implements used for makeup application, and makeup procedures.

That is, Figure 4 shows an example in which the makeup information for the makeup identified by the makeup ID "DID#1" is "makeup information #1," and the makeup information for the makeup identified by the makeup ID "DID#2" is "makeup information #2."

(2-2-3. Compatibility Information Storage Unit 12c)
The compatibility information storage unit 12c stores the compatibility (compatibility information) of each makeup application with respect to the hair style calculated by the calculation unit 13b of the control unit 13. An example of information stored in the compatibility information storage unit 12c will be described below with reference to FIG. 5. FIG. 5 is a diagram showing an example of the compatibility information storage unit 12c according to the embodiment. In the example of FIG. 5, the compatibility information storage unit 12c has items such as "style ID,""makeupID," and "compatibility."

"Appearance ID" indicates identification information for identifying the hair appearance. "Makeup ID" indicates identification information for identifying the makeup. "Compatibility" is a numerical value indicating the degree of compatibility of each makeup with the hair appearance.

In other words, Figure 5 shows an example in which the hair style identified by style ID "HID#1" has makeup information "makeup information #1" and a compatibility level "compatibility level #1," while the hair style identified by style ID "HID#2" has makeup information "makeup information #2" and a compatibility level "compatibility level #2."

(2-2-4. Provided information storage unit 12d)
The provided information storage unit 12d stores the provided information generated by the generation unit 13c of the control unit 13. An example of information stored in the provided information storage unit 12d will now be described with reference to FIG. 6. FIG. 6 is a diagram illustrating an example of the provided information storage unit 12d according to the embodiment. In the example of FIG. 6, the provided information storage unit 12d has items such as "mode ID" and "provided information."

"Style ID" indicates identification information for identifying the style of hair. "Provided information" is information about makeup that is compatible with the hair style that user U can provide, such as cosmetics, makeup tools, and makeup procedures to be provided to user U.

In other words, Figure 6 shows an example in which the provided information about the hair style identified by the style ID "HID#1" is "Provided Information #1," and the provided information about the hair style identified by the style ID "HID#2" is "Provided Information #2."

(2-3. Control unit 13)
The control unit 13 is realized by, for example, a central processing unit (CPU) or a micro processing unit (MPU) executing various programs (corresponding to an example of an information processing program) stored in a storage device inside the information providing device 10 using RAM as a work area. The control unit 13 is also realized by, for example, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

As shown in FIG. 2, the control unit 13 has an acquisition unit 13a, a calculation unit 13b, a generation unit 13c, a transmission unit 13d, and a learning unit 13e, and realizes or executes the information processing functions and actions described below. Note that the internal configuration of the control unit 13 is not limited to the configuration shown in FIG. 2, and may be other configurations as long as they perform the information processing described below. Furthermore, the connection relationships between the processing units in the control unit 13 are not limited to the connection relationships shown in FIG. 2, and may be other connection relationships.

(2-3-1. Acquisition unit 13a)
The acquisition unit 13a acquires, from the user terminal 20, information on the hair style, hair color, and other hair characteristics associated with the image selected by the user U as the aspect information.

The acquisition unit 13a also acquires makeup-related information. For example, the acquisition unit 13a acquires information related to makeup, such as cosmetics, makeup tools, and makeup application procedures, from the makeup information database 30. At this time, the acquisition unit 13a may acquire, from the makeup information database 30, makeup information associated with the service provider of the hair and makeup booking site. The acquisition unit 13a may also acquire makeup information including makeup that has been registered in advance as being highly compatible with each hair style.

The acquisition unit 13a also acquires user information about the user U who provides the hair style.
For example, the acquisition unit 13a acquires user information from the user terminal 20 of user U, such as user attributes including categories of interest to user U, and behavioral history of user U, such as search history, browsing history, purchase history, and location information.

The acquisition unit 13a stores the acquired behavior information in the behavior information storage unit 12a. The acquisition unit 13a also stores the acquired makeup information in the makeup information storage unit 12b. Furthermore, the acquisition unit 13a can also store the acquired user information in the storage unit 12.

(2-3-2. Calculation unit 13b)
The calculation unit 13b calculates a compatibility degree indicating the degree of compatibility between a combination of hair style and makeup based on the appearance information. For example, to explain a calculation method, the calculation unit 13b calculates the compatibility degree using a machine learning model that has learned the degree of compatibility between a combination of a hair style and makeup applied by a subject having hair with that style. As a specific example, the calculation unit 13b calculates the compatibility degree using a machine learning model such as a DNN that has been trained to output a compatibility degree for each makeup item with respect to the hair style when the appearance information and makeup information selected by the user U are input. Furthermore, the calculation unit 13b may calculate the compatibility degree for each makeup item with respect to the hair style selected by the user U on a rule-based basis.

Furthermore, regarding the calculated numerical values, the calculation unit 13b calculates the compatibility degree so that it takes a numerical value between 0 and 1 depending on the degree of compatibility for each makeup. The calculation unit 13b may also calculate the compatibility degree so that it takes a numerical value between 0 and 100% depending on the degree of compatibility for each makeup, and the range and unit of the calculated numerical value are not particularly limited.

To explain this using a specific example, if the compatibility between hairstyle H and cosmetic M1 is "0.9," the compatibility between hairstyle H and cosmetic M2 is "0.8," and the compatibility between hairstyle H and cosmetic M3 is "0.7," the calculation unit 13b will estimate that cosmetic M1, which has the highest compatibility, is highly compatible with hairstyle H. This allows the calculation unit 13b to appropriately optimize the combination of clothing items and hair styles.

Furthermore, regarding the learning data of the machine learning model used by calculation unit 13b, calculation unit 13b calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility from video images including hair styles and makeup. As a specific example, calculation unit 13b calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility from video images, still images, etc. that simultaneously show hair styles and applied makeup, collected from websites, SNS, television, movies, etc.

Furthermore, the calculation unit 13b calculates the compatibility level using a machine learning model that has learned the degree of compatibility for each attribute of the subject to be made up. As a specific example, the calculation unit 13b calculates the compatibility level using multiple machine learning models generated for each user attribute, including the subject's age, favorite celebrities, types of dramas, and other hobbies and interests.

The calculation unit 13b also calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility for each situation of the subject to whom makeup is applied. As a specific example, the information providing device 10 calculates the degree of compatibility for each situation, such as new graduate, job change, mid-career recruitment, etc., using a machine learning model that has learned images of subjects who passed document screening during their job hunting activities.

The calculation unit 13b acquires hairstyle information related to the hairstyle selected by the user U from the hairstyle information storage unit 12a. The calculation unit 13b also acquires makeup information indicating makeup from the makeup information storage unit 12b. Meanwhile, the calculation unit 13b stores the calculated compatibility in the compatibility information storage unit 12c.

(2-3-3. Generation unit 13c)
Based on the calculated compatibility, the generation unit 13c generates information to be provided to the user U. For example, the generation unit 13c identifies cosmetics that have a compatibility level equal to or higher than a predetermined threshold among the cosmetics indicated in the acquired cosmetic information, and generates information to be provided. Explaining this using a specific example, the generation unit 13c identifies cosmetics M1, M2, M3, ..., which are cosmetics necessary for makeup for hairstyle H, which is a hair style that the user U can provide, and identifies cosmetics M1, M2, and M3 that have a compatibility level with hairstyle H of "0.7" or higher, and generates information to be provided that includes images of cosmetics M1, M2, and M3.

The generation unit 13c also identifies makeup based on the acquired user information of the user U and generates information to be provided. For example, the generation unit 13c identifies makeup that has been registered as a preference of the user U from among makeup with a compatibility level equal to or higher than a predetermined threshold, and generates information to be provided that includes the makeup.

The generation unit 13c acquires the compatibility information from the compatibility information storage unit 12c. The generation unit 13c also stores the generated provided information in the provided information storage unit 12d.

(2-3-4. Transmitting unit 13d)
The transmitting unit 13d transmits the information to be provided generated by the generating unit 13c to the user U. For example, the transmitting unit 13d transmits to the user terminal 20 of the user U an image including makeup whose compatibility with a hair style that the user U can provide is equal to or exceeds a predetermined threshold. At this time, the transmitting unit 13d may transmit information about cosmetics required for the makeup and makeup steps along with the image including the makeup.

The transmission unit 13d acquires the provided information from the provided information storage unit 12d. The transmission unit 13d may also transmit the provided information to a server, operator terminal, or database (not shown).

(2-3-5. Learning unit 13e)
The learning unit 13e learns the machine learning model so as to output the compatibility of each makeup with the hair style when the appearance information and makeup selected by the user U are input. At this time, the learning unit 13e may learn the machine learning model by backpropagation or the like.

For example, when information indicating that a combination of makeup and a hair style that user U can provide is compatible is input to the machine learning model for each user U, the attributes of user U, and the situation of user U, the learning unit 13e trains the machine learning model so that the compatibility degree of makeup for that hair style is output as a number approaching "1."

On the other hand, the learning unit 13e trains the machine learning model so that when information indicating that a combination of makeup and a hair style that user U can provide is incompatible is input to the machine learning model for each user U, user U's attributes, and user U's situation, the compatibility of the makeup with that hair style is output as a numerical value approaching "0."

[3. Specific examples of information provision processing]
Next, a specific example of the information provision process according to the embodiment will be described. Hereinafter, the learning process of the machine learning model and the process of generating the information to be provided will be described, and then a specific example of the information to be provided that is transmitted to the user terminal 20 of the user U will be described.

(3-1. Learning process of machine learning model)
The learning process of the machine learning model according to the embodiment will be described. By executing this process, the information providing device 10 can generate a trained machine learning model to be used in the process of generating information to be provided, which will be described later. Below, the supervised data collection process, image data extraction process, and image data learning process will be described in this order.

(3-1-1. Correct data collection process)
First, the information providing device 10 collects correct answer data from SNS. Here, the collected correct answer data is, for example, an image in which a hairstyle and makeup are photographed at the same time, and the evaluation satisfies a predetermined condition. More specifically, the collected correct answer data is an image posted by a predetermined person, an image with more than a predetermined number of "likes," an image with more than a predetermined number of comments or favorable content, etc. Similarly, the information providing device 10 can also collect correct answer data from moving images such as television commercials and movies.

(3-1-2. Image data extraction process)
Next, the information providing device 10 extracts hair and makeup parts (e.g., face, hands, and feet) from the collected correct answer data image. At this time, the information providing device 10 may extract makeup parts such as eyebrows, eyes, nose, mouth, and nails.

(3-1-3. Image data learning process)
The information providing device 10 then trains the machine learning model so that when a combination of an image of a hair portion and an image of each makeup portion extracted from the same image (i.e., a combination that is likely to be evaluated well) is input, the machine learning model outputs information indicating a high compatibility (e.g., a score of 1). The information providing device 10 also trains the machine learning model so that when a combination of an image of a hair portion and an image of a makeup portion extracted from a different image (i.e., a combination that is not known to be evaluated well), the machine learning model outputs information indicating a low compatibility (e.g., a score of 0 or a random number equal to or less than a predetermined value (0.7)). Through the above-described processing, the information providing device 10 generates a machine learning model that has learned the above-described set matching.

(3-2. Generation process of provided information)
The process of generating information to be provided according to the embodiment will be described. By executing this process, the information providing device 10 can, for example, provide makeup looks posted by each stylist, and when a user U selects a makeup look, provide a reservation service for the makeup salon to which the selected stylist belongs (a stylist reservation service). The following describes the process of obtaining appearance information, the process of generating candidate information, the process of identifying appearance, and the process of transmitting information to be provided, in that order.

(3-2-1. Behavior Information Acquisition Process)
First, the information providing device 10 is a server that provides information on, for example, makeup. At this time, the information providing device 10 acquires images of hair and makeup registered or selected by the user U. That is, through the above-described process, the information providing device 10 acquires images (hair style information) of hair styles that the user U can provide.

(3-2-2. Candidate Information Generation Process)
Next, the information providing device 10 generates an image by cutting out the hair portion from the acquired image, and generates candidate information by combining each makeup portion with the image cut out from the hair portion. At this time, the information providing device 10 may accept the makeup image (makeup information) from each user U, or may obtain it from the makeup information database 30 that collects images of makeup portions.

(3-2-3. Mode Identification Processing)
Next, the information providing device 10 inputs each of the generated candidate information into a machine learning model and identifies makeup, etc. included in the candidate information whose output score exceeds a predetermined threshold. In other words, through the above-described process, the information providing device 10 identifies makeup that is highly compatible with the hair style that the user U can wear.

(3-2-4. Provision information transmission process)
The information providing device 10 then generates provided information by adding a link to a cosmetics purchasing site or the like to the image of the identified makeup, etc., and transmits one or more pieces of provided information to the user terminal 20. As a result, the information providing device 10 can provide the user U with information such as, "Here is the makeup that goes well with this hairstyle. Would you like to purchase the cosmetics?" The information providing device 10 can also provide the above information to the user U via a server of a hair and makeup reservation site where the user U provides information on beauty salons, etc. In this case, the information providing device 10 can also transmit the provided information directly to the user terminal 20 of the user U.

(3-3. Specific examples of provided information)
A specific example of the information to be provided according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a diagram showing a specific example of the information providing process according to the embodiment. Below, a specific example of the information to be provided that is transmitted to and displayed on the user terminal 20 of the user U will be described.

(3-3-1. Display screen W1)
First, we will explain the display screen W1, which shows the hair styles, hair colors, and other hair features that the user U can provide, as the provided information displayed on the user terminal 20 of the user U. The display screen W1 displays an image H1 of the hairstyle H, which is the hair style that the user U uploaded to the hair and makeup booking site, and a screen H2 that shows the treatment fee, treatment details, etc. for that hair style.

(3-3-2. Display screen W2)
Next, a display screen W2 will be described, which contains information about makeup and other items that can be provided by the user U for hair styles such as hairstyles and hair colors, as provided information displayed on the user terminal 20 of the user U. The display screen W2 displays screens M1, M2, and M3 containing information about cosmetics X, Y, and Z under the heading "cosmetics that go well with this hair." At this time, the user U can view details about each cosmetic and images of makeup using the cosmetics by clicking on the screens M1, M2, and M3 containing information about cosmetics X, Y, and Z. The user U can also transition to a purchasing site for each cosmetic by clicking on the screens M1, M2, and M3 containing information about cosmetics X, Y, and Z.

[4. Information provision process flow]
The procedure of information processing of the information providing device 10 according to the embodiment will be described with reference to FIG. 8. FIG. 8 is a flowchart showing an example of the flow of the information providing process according to the embodiment. Note that the following steps S101 to S106 may be executed in a different order. Also, some of the following steps S101 to S106 may be omitted.

(4-1. Behavior information collection process)
First, the acquisition unit 13a of the information providing device 10 executes a behavior information acquisition process (step S101). For example, the acquisition unit 13a acquires behavior information on the hair behavior that the user U can provide from the user terminal 20.

(4-2. Cosmetic information collection processing)
Second, the acquisition unit 13a of the information providing device 10 executes a makeup information acquisition process (step S102). For example, the acquisition unit 13a refers to the makeup information database 30 and acquires makeup information related to makeup.

(4-3. Compatibility Calculation Process)
Third, the calculation unit 13b of the information providing device 10 executes a compatibility calculation process (step S103). For example, the calculation unit 13b calculates the compatibility of each makeup application with the hair style that the user U can provide, based on the acquired hairstyle information and makeup information.

(4-4. Provision information generation process)
Fourth, the generation unit 13c of the information providing device 10 executes a provision information generation process (step S104). For example, the generation unit 13c generates provision information to be provided to the user U based on the calculated compatibility degree and user information.

(4-5. Provision information transmission process)
Fifth, the transmitting unit 13d of the information providing device 10 executes a process of transmitting information to be provided (step S105). For example, the transmitting unit 13d transmits the generated information to be provided to the user terminal 20 of the user U.

(4-6. Compatibility learning process)
Sixth, the learning unit 13e of the information providing device 10 executes a compatibility learning process (step S106). For example, the learning unit 13e learns the calculated compatibility of each makeup with the hair style that the user U can provide.

5. Effects of the embodiment
Finally, the effects of the embodiment will be described below: Effects 1 to 7 corresponding to the processing according to the embodiment will be described below.

(5-1. Effect 1)
In the process according to the embodiment described above, aspect information regarding the hair style that the user U can provide is acquired, and a compatibility level indicating the degree of compatibility between the hair style and the makeup is calculated based on the acquired product information. Therefore, this process can provide useful information to the user U.

(5-2. Effect 2)
In the process according to the embodiment described above, the compatibility degree is calculated using a machine learning model that has learned the compatibility degree between a combination of a hair style and makeup applied by a subject having that hair style. Therefore, this process can more effectively provide useful information to the user U.

(5-3. Effect 3)
In the process according to the embodiment described above, the compatibility degree is calculated using a machine learning model that has learned the compatibility degree from video images including hair styles and makeup. Therefore, this process can provide useful information to the user U more effectively and with high accuracy.

(5-4. Effect 4)
In the process according to the embodiment described above, the compatibility degree is calculated using a machine learning model that has learned the compatibility degree for each attribute of the target person to be made up. Therefore, in this process, useful information can be provided to the user U with high accuracy for each attribute of the target person.

(5-5. Effect 5)
In the process according to the embodiment described above, the compatibility degree is calculated using a machine learning model that has learned the compatibility degree for each situation of the subject to be made up. Therefore, in this process, useful information can be provided to the user U with high accuracy for each situation of the subject.

(5-6. Effect 6)
In the process according to the present embodiment described above, the cosmetic information related to the cosmetic is further acquired, and the cosmetic indicated by the cosmetic information is identified as having a compatibility level equal to or greater than a predetermined threshold, and information to be provided is generated. Therefore, by using the cosmetic information, the process can more effectively provide useful information to the user U.

(5-7. Effect 7)
In the process according to the present embodiment described above, user information about the user U who provides the hair style is further acquired, makeup is identified based on the user information, and information to be provided is generated. Therefore, by using the user information, the process can more effectively provide useful information to the user U.

[Hardware configuration]
The information providing device 10 according to the above-described embodiment is realized by a computer 1000 having a configuration as shown in Fig. 9, for example. The information providing device 10 will be described below as an example. Fig. 9 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output I/F (Interface) 1060, an input I/F 1070, and a network I/F 1080 are connected via a bus 1090.

The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and secondary storage device 1050, programs read from the input device 1020, and the like, and executes various processes. The arithmetic unit 1030 is realized, for example, by a CPU (Central Processing Unit), MPU (Micro Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), etc.

The primary storage device 1040 is a memory device, such as RAM (Random Access Memory), that primarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, etc. The secondary storage device 1050 may be internal storage or external storage. The secondary storage device 1050 may also be a removable storage medium such as a USB memory or an SD (Secure Digital) memory card. The secondary storage device 1050 may also be cloud storage (online storage), NAS (Network Attached Storage), a file server, etc.

The output I/F 1060 is an interface for transmitting information to be output to an output device 1010, such as a display, projector, printer, or other device that outputs various types of information, and is implemented, for example, by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (High Definition Multimedia Interface). The input I/F 1070 is an interface for receiving information from various input devices 1020, such as a mouse, keyboard, keypad, buttons, scanner, or other device, and is implemented, for example, by USB.

Furthermore, the output I/F 1060 and the input I/F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. In other words, the output device 1010 and the input device 1020 may be wireless devices.

Furthermore, the output device 1010 and the input device 1020 may be integrated, such as a touch panel. In this case, the output I/F 1060 and the input I/F 1070 may also be integrated as an input/output I/F.

The input device 1020 may be a device that reads information from, for example, optical recording media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memories.

The network I/F 1080 receives data from other devices via the network N and sends it to the computing device 1030, and also transmits data generated by the computing device 1030 to other devices via the network N.

The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output I/F 1060 and the input I/F 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

For example, when the computer 1000 functions as the information providing device 10, the arithmetic unit 1030 of the computer 1000 realizes the functions of the control unit 130 by executing a program loaded onto the primary storage device 1040. The arithmetic unit 1030 of the computer 1000 may also load a program acquired from another device via the network I/F 1080 onto the primary storage device 1040 and execute the loaded program. The arithmetic unit 1030 of the computer 1000 may also link with other devices via the network I/F 1080 and call and use program functions, data, etc. from other programs on the other devices.

〔others〕
Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments.

Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown.

Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

For example, the information providing device 10 described above may be implemented using multiple server computers, and depending on the function, the configuration can be flexibly changed, such as by calling an external platform using an API (Application Programming Interface) or network computing.

Furthermore, the above-described embodiments and variations can be combined as appropriate to the extent that the processing content is not contradictory.

In addition, the above-mentioned "section, module, unit" can be interpreted as "means" or "circuit." For example, a control unit can be interpreted as a control means or a control circuit.

REFERENCE SIGNS LIST 10 Information providing device 11 Communication unit 12 Storage unit 12a Appearance information storage unit 12b Cosmetic information storage unit 12c Compatibility information storage unit 12d Provided information storage unit 13 Control unit 13a Acquisition unit 13b Calculation unit 13c Generation unit 13d Transmission unit 13e Learning unit 20 User terminal 30 Cosmetic information database 100 Information providing system

Claims (8)

an acquisition unit that acquires hair style information relating to hair styles that can be provided by a user;
a calculation unit that calculates a compatibility level indicating a degree of compatibility between the appearance and makeup based on the appearance information;
a generation unit that generates information to be provided to the user based on the compatibility degree;
Equipped with
The acquisition unit further acquires preferences of the user as user information related to the user who provides the aspect;
the generation unit identifies the makeup similar to the preference of the user and generates the provided information.
An information providing device characterized by: the calculation unit calculates the compatibility degree using a machine learning model that has learned the compatibility degree between a combination of a hair style and makeup applied by a subject having the hair style;
2. The information providing device according to claim 1. the calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility from video images including hair styles and makeup.
3. The information providing device according to claim 2. the calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility for each attribute of the subject to be made up.
4. The information providing device according to claim 2 or 3. the calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility for each situation of the subject to be made up.
5. The information providing device according to claim 2, wherein the information providing device is a device for providing information to a user. The acquisition unit further acquires makeup information related to makeup applied to the subject,
the generation unit identifies makeup indicated by the makeup information that has a compatibility level equal to or higher than a predetermined threshold, and generates the information to be provided.
6. The information providing device according to claim 1, wherein the information providing device is a device for providing information to a user. An information providing method executed by an information providing device, comprising:
an acquisition step of acquiring hair style information relating to hair styles that can be provided by a user;
a calculation step of calculating a compatibility degree indicating a degree of compatibility between the appearance and makeup based on the appearance information;
a generating step of generating information to be provided to the user based on the compatibility degree;
Including,
The acquiring step further acquires preferences of the user as user information related to the user who provides the aspect;
the generating step identifies the makeup similar to the preference of the user and generates the provided information.
1. An information providing method comprising: an acquisition step of acquiring aspect information relating to hair aspects that can be provided by a user;
a calculation step of calculating a compatibility degree indicating a degree of compatibility between the appearance and makeup based on the appearance information;
a generation step of generating information to be provided to the user based on the compatibility degree;
on the computer ,
The obtaining step further obtains a preference of the user as user information related to the user who provides the aspect;
the generation step includes identifying the makeup similar to the user's preference and generating the provided information.
An information providing program characterized by: