JP7779760B2 - Information providing device, information providing method, and information providing program - Google Patents

Description

The present invention relates to an information provision device, an information provision method, and an information provision program.

Technologies that support outfit coordination are known in the past. For example, there is a technology that generates outfits based on rules regarding fashion combinations.

Japanese Patent Application Laid-Open No. 2013-235528

However, the above-mentioned conventional technology leaves room for improvement in terms of providing useful information to customers (referred to as "users"). For example, conventional technology makes it difficult to suggest makeup to users that matches their hairstyle.

This application has been made in light of the above, and aims to provide an information provision device, an information provision method, and an information provision program that enable useful information to be provided to users.

In order to solve the above-mentioned problems and achieve the objectives, the information providing device of the present invention is characterized by comprising an acquisition unit that acquires aspect information related to a hair aspect selected by a user, a calculation unit that calculates a compatibility level indicating the degree of compatibility between the aspect and a body ornament based on the aspect information, and a generation unit that generates information to be provided to the user based on the compatibility level.

Furthermore, the information provision method according to the present invention is an information provision method executed by an information provision device, and is characterized by including an acquisition step of acquiring aspect information related to a hair aspect selected by a user, a calculation step of calculating a compatibility level indicating the degree of compatibility between the aspect and a body ornament based on the aspect information, and a generation step of generating information to be provided to the user based on the compatibility level.

The information provision program of the present invention is characterized in that it causes a computer to execute the following steps: an acquisition step for acquiring hairstyle information related to a hairstyle selected by a user; a calculation step for calculating a compatibility level indicating the degree of compatibility between the hairstyle and a body ornament based on the hairstyle information; and a generation step for generating provision information to be provided to the user based on the compatibility level.

This invention can provide useful information to users.

FIG. 1 is a diagram illustrating an example of the configuration of an information providing system according to an embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of the information providing device according to the embodiment. FIG. 3 is a diagram illustrating an example of a mode information storage unit according to the embodiment. FIG. 4 is a diagram illustrating an example of a decoration information storage unit according to the embodiment. FIG. 5 is a diagram illustrating an example of a compatibility information storage unit according to the embodiment. FIG. 6 is a diagram illustrating an example of a provided information storage unit according to the embodiment. FIG. 7 is a diagram showing a specific example of the information providing process according to the embodiment. FIG. 8 is a flowchart illustrating an example of the flow of the information providing process according to the embodiment. FIG. 9 illustrates an example of a hardware configuration.

Below, detailed explanations will be given of the information provision device, information provision method, and information provision program (hereinafter, "embodiments") according to the present application, with reference to the drawings. Note that the information provision device, information provision method, and information provision program according to the present application are not limited to these embodiments. Furthermore, identical components in the following embodiments will be designated by the same reference numerals, and duplicate explanations will be omitted.

[Embodiment]
The configuration of the information providing system 100 according to the embodiment, the configuration of the information providing device 10, a specific example of the information providing process, and the flow of the information providing process will be described below, and finally the effects of this embodiment will be described.

1. Configuration of information provision system 100
The processing of an information provision system 100 according to this embodiment (referred to as "this system" as appropriate) will be described using Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the information provision system 100 according to this embodiment. Below, the example of the configuration of this system 100, the processing of this system 100, the set matching technology applied in this system 100, and the effects of this system 100 will be described in that order.

(1-1. Example of configuration of information providing system 100)
1 includes an information providing device 10, a user terminal 20, a website server 30, and a decoration information database 40. The information providing device 10, the user terminal 20, the website server 30, and the decoration information database 40 are communicatively connected via a predetermined communication network (not shown) via wired or wireless communication. Note that the system 100 may include multiple information providing devices 10, multiple user terminals 20, multiple website servers 30, or multiple decoration information databases 40.

(1-1-1. Information providing device 10)
The information providing device 10 is a device that transmits and receives data between the user terminal 20, the website server 30, and the decoration information database 40, and is realized, for example, by a server device or a cloud system. The example in Fig. 1 shows a case where the information providing device 10 is realized by a server device.

(1-1-2. User terminal 20)
The user terminal 20 is a device (computer) used by a user U to browse web pages, do online shopping, make reservations at beauty salons or stylists, etc. The user terminal 20 accepts operations by the user U. The user terminal 20 may be realized, for example, by a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), etc. The example in FIG. 1 shows a case where the user terminal 20 is realized by a smartphone.

In the following, user terminal 20 may be referred to as user U. In other words, user U may also be read as user terminal 20. User U may also be used to represent multiple customers.

(1-1-3. Website Server 30)
The website server 30 is a device that transmits and receives data to and from the information providing device 10 and the user terminal 20, and is realized, for example, by a server device or a cloud system. The example in Fig. 1 shows a case where the information providing device 10 is realized by a server device. The website server 30 is a device used by a service provider that provides the user U with services such as internet shopping on the web and a search service for beauty salons and stylists.

(1-1-4. Decoration Information Database 40)
The ornament information database 40 is a storage device that stores ornament information related to body ornaments, which will be described later. The ornament information database 40 may store ornament information as part of the website server 30.

(1-2. Processing of Information Providing System 100)
The following describes steps S1 to S7 as the processing of the information providing system 100. Note that steps S1 to S7 below may be executed in a different order. Also, some of the processing of steps S1 to S7 below may be omitted.

(1-2-1. Processing of step S1)
In this system 100, first, the user U selects the hair style that he or she desires for a haircut or hair styling (or "hair and makeup" as appropriate) at a beauty salon or the like via the user terminal 20, and transmits the selection of the hair style to the website server 30 (step S1).

(1-2-2. Processing of step S2)
Secondly, in the present system 100, the information providing device 10 acquires from the website server 30, aspect information relating to the hair aspect selected by the user U (step S2). Here, aspect information refers to information relating to the hair aspect indicating the overall hairstyle, such as the hairstyle and hair color. The aspect information may also be information relating to the hair aspect using a wig or toupee, and is not particularly limited. In the example of FIG. 1, the information providing device 10 acquires the aspect information from the website server 30, but it may also acquire the aspect information from the user terminal 20 or a terminal device or database (not shown).

At this time, the information providing device 10 may acquire user information about the user U from the user terminal 20. Here, the user information refers to information including user attributes such as the user U's gender, age, generation, occupation, annual income, place of residence, marital status, whether or not they have children, videos of the user U, and categories of interest to the user U, as well as the user U's behavioral history such as their website search history and browsing history, their online shopping purchase history, their reservation history on reservation sites, and their location information. Furthermore, the user information may also include screen information of the user terminal 20, biometric information of the user U, and the like, and is not particularly limited.

(1-2-3. Processing of step S3)
Third, in the present system 100, the information providing device 10 refers to the decoration information database 40 and acquires decoration information (step S3). Here, the decoration information refers to information about body decorations applied through decorative practices such as makeup and nail art. The decoration information may also be information about the body after aesthetic treatments, cosmetic medicine, etc., and is not particularly limited. In the example of FIG. 1, the information providing device 10 acquires decoration information from the decoration information database 40, but it may also acquire aspect information from the website server 30 or a terminal device or database (not shown).

(1-2-4. Processing of step S4)
Fourth, in the present system 100, the information providing device 10 calculates a compatibility score indicating the degree of compatibility between the hair style selected by the user U and the body decoration based on the acquired style information and decoration information (step S4). For example, the information providing device 10 uses a machine learning model to calculate the compatibility score, which takes a value between 0 and 1 depending on the degree of compatibility between the combination of the hair style and the body decoration. In this case, the information providing device 10 calculates the compatibility score using a machine learning model such as a deep neural network (DNN) that is trained to output a compatibility score for each combination of the hair style and the body decoration when the style information and decoration information related to the hair style selected by the user U are input. Alternatively, the information providing device 10 may calculate the compatibility score based on rules.

The above-mentioned step S4 will be explained using a specific example. For example, the information providing device 10 can calculate the compatibility level using a machine learning model trained for each user U, a machine learning model trained for each attribute of user U, and a machine learning model trained for each situation of user U, as shown below.

The information providing device 10 can calculate compatibility using a machine learning model trained for each user U. For example, the information providing device 10 allows the machine learning model to learn how to match hairstyles and makeup for images that the user U has viewed on a social networking service (SNS) and given high ratings, such as "likes." As a result, the information providing device 10 can suggest makeup salons such as beauty salons or nail salons when the user U selects a hairstyle.

In addition, the information providing device 10 can calculate compatibility using a machine learning model trained for each attribute, including user U's hobbies and preferences. For example, the information providing device 10 generates multiple machine learning models for a user who likes "XX (e.g., celebrity, era, type of drama (action, etc.))" and generates correct answer data for each machine learning model. Specifically, the machine learning model is trained using images that have been "liked" by a subject who likes "XX" or images that match "XX" as correct answer data. As a result, when user U accesses a website, the information providing device 10 can select a machine learning model that matches user U's user attributes and suggest makeup and nail art that are deemed to have a high degree of compatibility by the selected machine learning model.

In addition, the information providing device 10 can calculate compatibility using a machine learning model trained for each situation of user U. For example, for each corporate recruitment situation, such as new graduates, job changes, and mid-career recruitment, the information providing device 10 generates a machine learning model for each of new graduates, job changes, and mid-career recruitment, using images of candidates who passed the document screening process as correct answer data. Therefore, for example, if user U selects "job changes," the information providing device 10 can suggest makeup and stylists that are appropriate for the document screening process at the time of job changes.

(1-2-5. Processing of step S5)
Fifth, in the present system 100, information to be provided to the user U is generated based on the calculated compatibility, the acquired appearance information, and the user information of the user U (step S5). Here, the information to be provided refers to information about the body decoration identified from the body decoration indicated by the decoration information, such as body decorations such as makeup and nail art that match the hairstyle selected by the user U, and information about makeup salons, nail salons, etc. that can provide the body decorations, but is not limited to such information. At this time, the information providing device 10 identifies body decorations indicated by the decoration information that have a compatibility level equal to or greater than a predetermined threshold, and generates the information to be provided. For example, the information providing device 10 identifies an image of makeup that has a compatibility level of "0.7" or higher with the hairstyle selected by the user U, and generates the information to be provided about the image and makeup salons that can provide the makeup shown in the image.

The above-mentioned step S5 will be explained using a specific example. For example, the information providing device 10 can generate information to be provided based on user information about user U, as shown below, or based on business information about a makeup salon, nail salon, or the like.

The information providing device 10 can generate information to be provided based on the user attributes indicated in the user information. For example, the information providing device 10 can identify the makeup characteristics of user U (e.g., makeup color, makeup areas, makeup products) based on an image of user U, and suggest a group of makeup looks with relatively similar makeup characteristics. The information providing device 10 may also prioritize suggesting makeup looks that are similar to the makeup preferences registered in advance by user U. Furthermore, even if the makeup characteristics or preferences of user U are unknown, the information providing device 10 can prioritize suggesting makeup images that are relatively similar to the makeup characteristics of a sample image selected by user U on a hair and makeup booking site, etc. Therefore, the information providing device 10 can suggest makeup looks that are highly similar from among the candidates identified from the decoration information based on user attributes, etc.

The information providing device 10 can also generate information to be provided based on the behavioral history indicated by the user information. For example, if the information providing device 10 can estimate the hair and makeup that user U has previously applied based on the user U's behavioral history, such as their reservation history, it will suggest makeup and nail art based on that hair and makeup. Specifically, if user U has a history of booking hair coloring, the information providing device 10 obtains from the website server 30 images of the hair and makeup selected by user U and images of coloring that has previously been applied, and identifies suggested body decorations by combining the hair and makeup selected by user U and the previously applied hair and makeup with various makeup and nail art.

Furthermore, the information providing device 10 can generate provided information based on business information. For example, the information providing device 10 obtains business information such as information on makeup salons and nail salons, as well as information on reservation sites for such salons, from the website server 30, and generates suggested information including this information. As a result, the information providing device 10 can not only suggest makeup that matches the hair and makeup applied by the user U, but also provide information on makeup salons and nail salons, as well as stylists, makeup artists, and manicurists who can achieve this makeup.

(1-2-6. Processing of step S6)
Sixth, in the present system 100, the information providing device 10 transmits the generated information to the user terminal 20 of the user U (step S6). For example, the information providing device 10 transmits to the user terminal 20 of the user U information to be provided, including an image of makeup that has a compatibility rating of 0.7 or higher with the hair and makeup selected by the user U and information on makeup salons that can provide the makeup shown in the image. At this time, the information providing device 10 can also provide the generated information to the user U by transmitting the information to the website server 30.

(1-2-7. Processing of step S7)
Seventh, in the present system 100, the information providing device 10 learns the calculated compatibility degree (step S7). For example, the information providing device 10 uses a machine learning model to learn using backpropagation or the like so that, when information indicating a good compatibility between the hair style selected by the user U and the body ornament is input, the information providing device 10 outputs the compatibility degree of the body ornament for the hair style as a numerical value approaching "1." Furthermore, when information indicating a bad compatibility between the hair style selected by the user U and the body ornament is input, the information providing device 10 uses a machine learning model to learn using backpropagation or the like so that the information providing device 10 outputs the compatibility degree of the body ornament for the hair style as a numerical value approaching "0."

(1-3. Set matching technology applied in the information provision system 100)
Here, the set matching technology applied in the information provision system 100 described above will be described in detail. A technology called set matching is known as an example of a technology for calculating a score indicating the degree of harmony between sets (hereinafter referred to as a "matching score" as appropriate) based on a set of information indicating feature quantities such as images. Set matching is a technology that uses, for example, deep learning, and estimates that the higher the matching score, the higher the compatibility between the sets. Therefore, set matching can quantitatively evaluate the compatibility between the sets. Calculation of a matching score using set matching is disclosed, for example, in "Exchangeable Deep Neural Networks for Set-to-Set Matching and Learning" by Y. Saito, T. Nakamura, H. Hachiya, and K. Fukumizu. Note that set matching is just an example, and embodiments are not limited to those using this technology.

The information providing device 10 applies a technique for calculating matching scores between sets, such as set matching, to calculate a matching score (compatibility) for each combination of hair style and body ornament. Specifically, the information providing device 10 calculates the compatibility between the body ornament and each hair style selected by the user U. At this time, the information providing device 10 can estimate that the higher the compatibility, the higher the compatibility between the hair style selected by the user U and the body ornament. Therefore, the information providing device 10 identifies the body ornament that has been calculated to have the highest compatibility for the particular hair style. Furthermore, the information providing device 10 estimates that the identified body ornament for the particular hair style is the body ornament that has the highest compatibility among the body ornaments included in the ornamentation information. This allows the information providing device 10 to quantitatively determine the compatibility between the hair style and the body ornament, rather than relying on the user U's sensory judgment of appearance, etc.

The information providing device 10 also trains the machine learning model as follows. For example, the information providing device 10 identifies images posted on SNS that user U has given favorable reviews. Next, the information providing device 10 identifies the hair style (hairstyle, hair color) and body accessories (makeup, nail art) in the identified images. At this time, the information providing device 10 may use various identification models or obtain identification results through crowdsourcing, etc. The information providing device 10 uses the above identification results as correct answer data. The information providing device 10 may also use the results of identification from images posted by user U as correct answer data. The information providing device 10 then trains the machine learning model so that it outputs a high score when hair style and body accessories that are correct answer data are input, and outputs a low score when hair style and body accessories that are not included in the correct answer data are input. At this time, the information providing device 10 may generate such a machine learning model for each attribute of user U or for each user situation. Furthermore, the information providing device 10 may reflect the degree of compatibility taking into account multiple perspectives by manipulating the output values of multiple models as the output of the machine learning model.

(1-4. Effects of the information provision system 100)
As described above, in the information providing system 100, the information providing device 10 acquires hairstyle information related to a hair style selected by the user U, calculates a compatibility score indicating the degree of compatibility between the hair style selected by the user U and a body accessory based on the hairstyle information, and generates information to be provided to the user U based on the compatibility score. The information providing device 10 calculates the compatibility score using a machine learning model that has learned the degree of compatibility between the hair style and the body accessory applied to a subject with that hair style. Therefore, the system 100 can estimate makeup and nail art that matches the hair and makeup desired by the user U and suggest stylists to the user U. In other words, the system 100 can provide useful information to the user U on e-commerce (EC) sites, hair and makeup reservation sites, and the like.

In addition, in the present system 100, the information providing device 10 further acquires user information about the user U who selected a hair style, identifies body ornamentation based on the acquired user information, and generates information to be provided. The information providing device 10 also generates information to be provided that includes business information about businesses that can provide body ornamentation. As a result, the present system 100 can suggest to the user U makeup that is highly compatible with hairstyles that are popular on social media, etc., or hair and makeup favored by celebrities and other famous people, as well as make more specific suggestions, such as suggestions about stylists who can achieve the makeup. In other words, the present system 100 makes it possible to provide more specific and useful information to users U of e-commerce sites, etc.

Furthermore, in the present system 100, the information providing device 10 calculates the compatibility level using a machine learning model that has learned the compatibility level between a combination of hair style and body ornaments worn by a subject with that hair style. The information providing device 10 calculates the compatibility level using a machine learning model that has learned the compatibility level from video images including the hair style and body ornament, a machine learning model that has learned the compatibility level for each user U, a machine learning model that has learned the compatibility level for each attribute of user U, or a machine learning model that has learned the compatibility level for each situation of user U. Therefore, the present system 100 generates various machine learning models trained using corrective data collected from social media, television, movies, etc., and can suggest more accurate makeup looks to the user U. In other words, the present system 100 enables the provision of more specific and useful information with high accuracy to users U on e-commerce sites, etc.

2. Configuration of information providing device 10
The configuration of the information providing device 10 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example configuration of the information providing device 10 according to the embodiment. As shown in Fig. 2, the information providing device 10 includes a communication unit 11, a storage unit 12, and a control unit 13. The information providing device 10 may also include an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator of the information providing device 10, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

(2-1. Communication unit 11)
The communication unit 11 is realized by, for example, a network interface card (NIC), etc. The communication unit 11 is connected to a predetermined communication network via a wired or wireless connection, and transmits and receives information to and from various devices.

(2-2. Storage unit 12)
The storage unit 12 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 2 , the storage unit 12 according to the embodiment includes a mode information storage unit 12a, a decoration information storage unit 12b, a compatibility information storage unit 12c, and a provided information storage unit 12d. The storage unit 12 stores various information referenced by the control unit 13 when it operates and various information acquired when the control unit 13 operates.

(2-2-1. Behavior information storage unit 12a)
The aspect information storage unit 12a stores various information (aspect information) related to the hair aspect selected by the user U. An example of the information stored in the aspect information storage unit 12a will be described below with reference to FIG. 3. FIG. 3 is a diagram showing an example of the aspect information storage unit 12a according to the embodiment. In the example of FIG. 3, the aspect information storage unit 12a has items such as "aspect ID" and "aspect information."

"Appearance ID" indicates identification information for identifying the hair appearance. "Appearance information" is information about the hair appearance, indicating the general hairstyle, such as hair style and hair color, or information about the hair appearance when using a wig or toupee, etc.

In other words, Figure 3 shows an example in which the aspect information for the hair aspect identified by the aspect ID "HID#1" is "Aspect Information #1," and the aspect information for the hair aspect identified by the aspect ID "HID#2" is "Aspect Information #2."

(2-2-2. Decoration information storage unit 12b)
The decoration information storage unit 12b stores various information related to body decorations (decoration information). An example of the information stored in the decoration information storage unit 12b will be described with reference to FIG. 4. FIG. 4 is a diagram illustrating an example of the decoration information storage unit 12b according to the embodiment. In the example of FIG. 4, the decoration information storage unit 12b has fields such as "decoration ID,""decorationinformation," and "business information."

"Accessory ID" refers to identification information for identifying body accessories. "Accessory information" refers to information related to body accessories based on treatments such as makeup, nail art, and aesthetics. "Business information" refers to information about the attributes of makeup salons, nail salons, and aesthetic salons that can provide body accessories, as well as information about the stylists, makeup artists, manicurists, and aestheticians employed at those makeup salons, etc.

In other words, Figure 4 shows an example in which the body decoration identified by decoration ID "DID#1" has decoration information "Decoration Information #1" and business information "Business Information #1," while the body decoration identified by decoration ID "DID#2" has decoration information "Decoration Information #2" and business information "Business Information #2."

(2-2-3. Compatibility Information Storage Unit 12c)
The compatibility information storage unit 12c stores the compatibility (compatibility information) of each body ornament with respect to the hair style calculated by the calculation unit 13b of the control unit 13. An example of information stored in the compatibility information storage unit 12c will be described below with reference to FIG. 5. FIG. 5 is a diagram illustrating an example of the compatibility information storage unit 12c according to an embodiment. In the example of FIG. 5, the compatibility information storage unit 12c has fields such as "Style ID,""OrnamentID," and "Compatibility."

"Style ID" indicates identification information for identifying the style of hair. "Accessory ID" indicates identification information for identifying the body accessory. "Compatibility" is a numerical value indicating the degree of compatibility of each body accessory with the hair style.

In other words, Figure 5 shows an example in which the hair style identified by style ID "HID#1" has decoration information "Decoration Information #1" and a compatibility level "Compatibility Level #1," while the hair style identified by style ID "HID#2" has decoration information "Decoration Information #2" and a compatibility level "Compatibility Level #2."

(2-2-4. Provided information storage unit 12d)
The provided information storage unit 12d stores the provided information generated by the generation unit 13c of the control unit 13. An example of information stored in the provided information storage unit 12d will now be described with reference to FIG. 6. FIG. 6 is a diagram illustrating an example of the provided information storage unit 12d according to the embodiment. In the example of FIG. 6, the provided information storage unit 12d has items such as "mode ID" and "provided information."

"Style ID" indicates identification information for identifying the hair style. "Provided information" is information about makeup, nail art, and other body decorations that are compatible with the hair style selected by user U, as well as information about makeup salons, nail salons, etc. that can provide such body decorations.

In other words, Figure 6 shows an example in which the provided information about the hair style identified by the style ID "HID#1" is "Provided Information #1," and the provided information about the hair style identified by the style ID "HID#2" is "Provided Information #2."

(2-3. Control unit 13)
The control unit 13 is realized by, for example, a central processing unit (CPU) or a micro processing unit (MPU) executing various programs (corresponding to an example of an information processing program) stored in a storage device inside the information providing device 10 using RAM as a work area. The control unit 13 is also realized by, for example, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

As shown in FIG. 2, the control unit 13 has an acquisition unit 13a, a calculation unit 13b, a generation unit 13c, a transmission unit 13d, and a learning unit 13e, and realizes or executes the information processing functions and actions described below. Note that the internal configuration of the control unit 13 is not limited to the configuration shown in FIG. 2, and may be other configurations as long as they perform the information processing described below. Furthermore, the connection relationships between the processing units in the control unit 13 are not limited to the connection relationships shown in FIG. 2, and may be other connection relationships.

(2-3-1. Acquisition unit 13a)
The acquisition unit 13a acquires, as the appearance information, information on the hair style selected by the user U, such as the hairstyle and hair color selected by the user U, from the website server 30.

The acquisition unit 13a also acquires decoration information related to body decoration. For example, the acquisition unit 13a acquires information related to body decoration such as makeup, nail art, and beauty treatments from the decoration information database 40. At this time, the acquisition unit 13a may acquire decoration information associated by a service provider on the website server 30 from the decoration information database 40. The acquisition unit 13a may also acquire decoration information from the website server 30 that includes body decorations that have been registered in advance as body decorations that are highly compatible with each hair style.

The acquisition unit 13a also acquires user information about the user U who selected the hair style.
For example, the acquisition unit 13a acquires user information from the user terminal 20 of user U, such as user attributes including user U's video images, categories of interest, etc., and user U's behavioral history such as search history, browsing history, purchase history, reservation history, and location information.

Furthermore, the acquisition unit 13a may acquire business information including information on registered makeup salons such as beauty salons and stylists such as hairdressers from the website server 30. The acquisition unit 13a can also acquire business information including information on makeup salons and stylists associated with body decorations from the decoration information database 40.

The acquisition unit 13a stores the acquired mode information in the mode information storage unit 12a. The acquisition unit 13a also stores the acquired decoration information and business information in the decoration information storage unit 12b. Furthermore, the acquisition unit 13a can also store the acquired user information in the storage unit 12.

(2-3-2. Calculation unit 13b)
The calculation unit 13b calculates a compatibility degree indicating the degree of compatibility between a combination of a hair style and a body ornament based on the style information. For example, the calculation method will be described. The calculation unit 13b calculates the compatibility degree using a machine learning model that has learned the degree of compatibility between a combination of a hair style and a body ornament worn by a subject having hair of that style. As a specific example, the calculation unit 13b calculates the compatibility degree using a machine learning model such as a DNN that has been trained to output a compatibility degree for each body ornament for each hair style when the style information and ornament information selected by the user U are input. Alternatively, the calculation unit 13b may calculate the compatibility degree for each body ornament for the hair style selected by the user U on a rule-based basis.

Furthermore, with regard to the calculated numerical values, the calculation unit 13b calculates the compatibility level so that it takes a numerical value between 0 and 1 depending on the degree of compatibility for each body ornament. Furthermore, the calculation unit 13b may calculate the compatibility level so that it takes a numerical value between 0 and 100% depending on the degree of compatibility for each body ornament, and the range and unit of the calculated numerical value are not particularly limited.

To explain this using a specific example, if the compatibility between hairstyle H and makeup D1 is "0.9," the compatibility between hairstyle H and makeup D2 is "0.8," and the compatibility between hairstyle H and makeup D3 is "0.7," the calculation unit 13b will estimate that makeup D1, which has the highest compatibility, is highly compatible with hairstyle H. This allows the calculation unit 13b to appropriately optimize the combination of hair style and body ornamentation.

Furthermore, regarding the learning data of the machine learning model used by calculation unit 13b, calculation unit 13b calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility from video images including hair styles and body ornaments. As a specific example, calculation unit 13b calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility from video images, still images, etc. that simultaneously show hair styles and applied body ornaments, collected from websites, SNS, television, movies, etc.

Furthermore, regarding the type of machine learning model used by the calculation unit 13b, the calculation unit 13b calculates the compatibility level using a machine learning model that has learned the degree of compatibility for each user U. As a specific example, the calculation unit 13b calculates the compatibility level using a machine learning model that has learned to match hairstyles and makeup in images that the user U has viewed on SNS and given high ratings such as "likes."

Furthermore, the calculation unit 13b calculates the compatibility level using a machine learning model that has learned the degree of compatibility for each attribute of the user U. As a specific example, the calculation unit 13b calculates the compatibility level using multiple machine learning models generated for each user attribute, including the user U's age, favorite celebrities, types of dramas, and other hobbies and interests.

The calculation unit 13b also calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility for each situation of the user U. As a specific example, the information providing device 10 calculates the degree of compatibility for each situation, such as new graduate, job change, mid-career recruitment, etc., using a machine learning model that has learned images of candidates who passed document screening during their job hunting activities.

The calculation unit 13b acquires hairstyle information related to the hairstyle selected by the user U from the hairstyle information storage unit 12a. The calculation unit 13b also acquires ornament information indicating body ornaments from the ornament information storage unit 12b. Meanwhile, the calculation unit 13b stores the calculated compatibility degree in the compatibility degree information storage unit 12c.

(2-3-3. Generation unit 13c)
Based on the calculated compatibility, the generation unit 13c generates information to be provided to the user U. For example, the generation unit 13c identifies body decorations indicated by the acquired decoration information that have a compatibility level equal to or higher than a predetermined threshold, and generates information to be provided. Explaining this using a specific example, the generation unit 13c identifies makeups D1, D2, D3, ..., which are body decorations for hairstyle H, which is a hair style selected by the user U, that have a compatibility level with hairstyle H of "0.7" or higher, and generates information to be provided that includes images of makeups D1, D2, D3, etc.

The generation unit 13c also identifies body ornaments based on the acquired user information of the user U and generates information to be provided. For example, the generation unit 13c identifies body ornaments registered as preferences of the user U from among body ornaments with a compatibility level equal to or greater than a predetermined threshold, and generates information to be provided that includes the body ornaments.

Furthermore, the generation unit 13c generates information to be provided that includes business information related to businesses that can provide body decorations. For example, the generation unit 13c generates information to be provided that includes information about the makeup salon or stylist that provided the image of the body decoration, which is a body decoration with a compatibility level equal to or higher than a predetermined threshold. In this case, the generation unit 13c may generate information that includes a link to the reservation page of the makeup salon, etc., as information about the makeup salon or stylist.

The generation unit 13c acquires the compatibility information from the compatibility information storage unit 12c. The generation unit 13c also stores the generated provided information in the provided information storage unit 12d.

(2-3-4. Transmitting unit 13d)
The transmitting unit 13d transmits the information to be provided generated by the generating unit 13c to the user U. For example, the transmitting unit 13d transmits to the user terminal 20 of the user U an image including a body decoration whose compatibility with the hair style selected by the user U is equal to or greater than a predetermined threshold. At this time, the transmitting unit 13d may transmit, together with the image including the body decoration, images of makeup salons, stylists, etc. that can provide the body decoration, as well as business information such as available appointment dates.

The transmission unit 13d acquires the provided information from the provided information storage unit 12d. The transmission unit 13d may also transmit the provided information to the website server 30 or a business operator terminal/database (not shown).

(2-3-5. Learning unit 13e)
The learning unit 13e learns the machine learning model so as to output the compatibility of each body ornament with the hair style when the style information and ornament information selected by the user U are input. At this time, the learning unit 13e may learn the machine learning model by backpropagation or the like.

For example, when information indicating that a combination of a hair style selected by user U and a body ornament is compatible with each other is input to the machine learning model for each user U, the attributes of user U, and the situation of user U, the learning unit 13e trains the machine learning model so that the compatibility degree of the body ornament with the hair style is output as a numerical value approaching "1."

On the other hand, the learning unit 13e trains the machine learning model so that when information indicating a poor compatibility between the hair style selected by user U and the body ornament is input to the machine learning model for each user U, user U's attributes, and user U's situation, the compatibility degree of the body ornament for that hair style is output as a numerical value approaching "0."

[3. Specific examples of information provision processing]
Next, a specific example of the information provision process according to the embodiment will be described. Hereinafter, the learning process of the machine learning model and the process of generating the information to be provided will be described, and then a specific example of the information to be provided that is transmitted to the user terminal 20 of the user U will be described.

(3-1. Learning process of machine learning model)
The learning process of the machine learning model according to the embodiment will be described. By executing this process, the information providing device 10 can generate a trained machine learning model to be used in the process of generating information to be provided, which will be described later. Below, the supervised data collection process, image data extraction process, and image data learning process will be described in this order.

(3-1-1. Correct data collection process)
First, the information providing device 10 collects correct answer data from SNS. Here, the collected correct answer data is, for example, an image in which a hairstyle and makeup are photographed at the same time, and the evaluation satisfies a predetermined condition. More specifically, the collected correct answer data is an image posted by a predetermined person, an image with more than a predetermined number of "likes," an image with more than a predetermined number of comments or favorable content, etc. Similarly, the information providing device 10 can also collect correct answer data from moving images such as television commercials and movies.

(3-1-2. Image data extraction process)
Next, the information providing device 10 extracts hair and makeup parts (e.g., face, hands, and feet) from the collected correct answer data image. At this time, the information providing device 10 may extract makeup parts such as eyebrows, eyes, nose, mouth, and nails.

(3-1-3. Image data learning process)
The information providing device 10 then trains the machine learning model so that when a combination of an image of a hair portion and an image of each makeup portion extracted from the same image (i.e., a combination that is likely to be evaluated well) is input, the machine learning model outputs information indicating a high compatibility (e.g., a score of 1). The information providing device 10 also trains the machine learning model so that when a combination of an image of a hair portion and an image of a makeup portion extracted from a different image (i.e., a combination that is not known to be evaluated well), the machine learning model outputs information indicating a low compatibility (e.g., a score of 0 or a random number equal to or less than a predetermined value (0.7)). Through the above-described processing, the information providing device 10 generates a machine learning model that has learned the above-described set matching.

(3-2. Generation process of provided information)
The process of generating information to be provided according to the embodiment will be described. By executing this process, the information providing device 10 can, for example, provide makeup looks posted by each stylist, and when a user U selects a makeup look, provide a reservation service for the makeup salon to which the selected stylist belongs (a stylist reservation service). The following describes the process of obtaining appearance information, the process of generating candidate information, the process of identifying appearance, and the process of transmitting information to be provided, in that order.

(3-2-1. Behavior Information Acquisition Process)
First, the information providing device 10 is a server that provides information about, for example, a makeup salon. On the other hand, the website server 30 is a server for a website that is, for example, a hair and makeup reservation site for a beauty salon or the like. Here, a user U references the website that is the hair and makeup reservation site. At this time, the website server 30 transmits an image of the hair and makeup that the user U has searched for and selected to the information providing device 10. In other words, through the above-described process, the information providing device 10 obtains an image (style information) of the hair style selected by the user U.

(3-2-2. Candidate Information Generation Process)
Next, the information providing device 10 generates an image by cutting out the hair portion from the acquired image, and generates candidate information by combining each makeup portion with the image cut out from the hair portion. At this time, the information providing device 10 may accept the makeup image (decoration information) from each stylist, or may obtain it from the decoration information database 40 that collects images of makeup portions.

(3-2-3. Mode Identification Processing)
Next, the information providing device 10 inputs each of the generated candidate information into a machine learning model, and identifies makeup, etc. included in the candidate information whose output score exceeds a predetermined threshold. In other words, through the above-described processing, the information providing device 10 identifies body ornaments that are highly compatible with the hair style selected by the user U.

(3-2-4. Provision information transmission process)
The information providing device 10 then generates information to be provided by adding links to reservation pages and related products to the identified images of makeup, etc., and transmits one or more pieces of generated information to the website server 30. As a result, the website server 30 can provide the user U with information such as, "Here is the makeup that goes well with this hairstyle. Would you like to make a reservation?" The information providing device 10 can also provide the user U with information associating an image of the makeup application with a web page for purchasing the makeup products, or information associating an image of the aesthetic treatment with a reservation page for the aesthetic treatment. In this case, the information providing device 10 can also transmit the information to be provided directly to the user terminal 20 of the user U.

(3-3. Specific examples of provided information)
A specific example of the information to be provided according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a diagram showing a specific example of the information providing process according to the embodiment. Below, a specific example of the information to be provided that is transmitted to and displayed on the user terminal 20 of the user U will be described.

(3-3-1. Display screen W1)
First, we will explain the display screen W1, which shows the hair style, hair color, and other hair features selected by the user U, as information displayed on the user terminal 20 of the user U. The display screen W1 displays an image H1 of hairstyle H, which is the hair style selected by the user U on the hair and makeup booking site, and a screen H2 showing the treatment fee, treatment details, etc. for that hair style. At this time, the user U can view the makeup image and information about the beauty salon (makeup salon) on the display screen W2 (described later) and then decide whether or not to make a reservation for hair and makeup for the above hairstyle H.

(3-3-2. Display screen W2)
Next, we will explain the display screen W2, which contains information displayed on the user terminal 20 of the user U, including body accessories such as makeup for the hairstyle, hair color, and other hair features selected by the user U, as well as business information such as beauty salons. The display screen W2 displays screens D1, D2, and D3 for information on makeups X, Y, and Z, as well as screens B1, B2, and B3 for information on beauty salons X, Y, and Z that can provide each makeup, under the heading "makeup that matches this hairstyle." At this time, the user U can view images and details of each makeup by clicking on screens D1, D2, and D3, which contain information on makeups X, Y, and Z. Furthermore, the user U can view information on each beauty salon and its associated hairdresser (stylist) or transition to a beauty salon reservation screen by clicking on screens B1, B2, and B3, which contain information on beauty salons X, Y, and Z.

[4. Information provision process flow]
The procedure of information processing of the information providing device 10 according to the embodiment will be described with reference to FIG. 8. FIG. 8 is a flowchart showing an example of the flow of the information providing process according to the embodiment. Note that the following steps S101 to S106 may be executed in a different order. Also, some of the following steps S101 to S106 may be omitted.

(4-1. Behavior information collection process)
First, the acquisition unit 13a of the information providing device 10 executes a behavior information acquisition process (step S101). For example, the acquisition unit 13a acquires behavior information related to the hair behavior selected by the user U from the website server 30.

(4-2. Decoration information collection process)
Second, the acquisition unit 13a of the information providing device 10 executes an ornament information acquisition process (step S102). For example, the acquisition unit 13a refers to the ornament information database 40 and acquires ornament information related to body ornaments.

(4-3. Compatibility Calculation Process)
Third, the calculation unit 13b of the information providing device 10 executes a compatibility calculation process (step S103). For example, the calculation unit 13b calculates the compatibility of each body ornament with the hair style selected by the user U based on the acquired style information and ornament information.

(4-4. Provision information generation process)
Fourth, the generation unit 13c of the information providing device 10 executes a provision information generation process (step S104). For example, the generation unit 13c generates provision information to be provided to the user U based on the calculated compatibility degree and user information.

(4-5. Provision information transmission process)
Fifth, the transmitting unit 13d of the information providing device 10 executes a process of transmitting information to be provided (step S105). For example, the transmitting unit 13d transmits the generated information to be provided to the user terminal 20 of the user U.

(4-6. Compatibility learning process)
Sixth, the learning unit 13e of the information providing device 10 executes a compatibility learning process (step S106). For example, the learning unit 13e learns the calculated compatibility of each body ornament with the hair style selected by the user U.

5. Effects of the embodiment
Finally, the effects of the embodiment will be described below: Effects 1 to 9 corresponding to the processing according to the embodiment will be described below.

(5-1. Effect 1)
In the process according to the embodiment described above, aspect information regarding the hair style selected by the user U is acquired, and a compatibility level indicating the degree of compatibility between the hair style and the body ornament is calculated based on the acquired aspect information. Therefore, this process can provide useful information to the user U.

(5-2. Effect 2)
In the process according to the above-described embodiment, the compatibility degree is calculated using a machine learning model that has learned the degree of compatibility between a combination of a hair style and a body ornament worn by a subject having that hair style. Therefore, this process can more effectively provide useful information to the user U.

(5-3. Effect 3)
In the process according to the embodiment described above, the compatibility degree is calculated using a machine learning model that has learned the compatibility degree from video images including hair styles and body ornaments. Therefore, this process can provide useful information to the user U more effectively and with higher accuracy.

(5-4. Effect 4)
In the process according to the embodiment described above, the compatibility level is calculated using a machine learning model that has learned the compatibility level for each user U. Therefore, in this process, useful information can be provided to each user U with high accuracy.

(5-5. Effect 5)
In the process according to the embodiment described above, the compatibility degree is calculated using a machine learning model that has learned the compatibility degree for each attribute of the user U. Therefore, in this process, useful information can be provided to the user U with high accuracy for each attribute of the user U.

(5-6. Effect 6)
In the process according to the embodiment described above, the compatibility level is calculated using a machine learning model that has learned the compatibility level for each situation of the user U. Therefore, in this process, useful information can be provided to the user U with high accuracy for each situation of the user U.

(5-7. Effect 7)
In the process according to the present embodiment described above, ornament information related to the body ornament is further acquired, and body ornaments indicated by the ornament information that have a compatibility level equal to or greater than a predetermined threshold are identified, and information to be provided is generated. Therefore, in this process, by using the ornament information, useful information can be provided to the user U more effectively.

(5-8. Effect 8)
In the process according to the present embodiment described above, user information about the user U who selected the hair style is further acquired, and the body ornament is identified based on the user information, and information to be provided is generated. Therefore, by using the user information, the process can more effectively provide useful information to the user U.

(5-9. Effect 9)
In the process according to the present embodiment described above, provision information including business information about businesses that can provide body decorations is generated, so that useful information including business information can be provided to the user U more effectively.

[Hardware configuration]
The information providing device 10 according to the above-described embodiment is realized by a computer 1000 having a configuration as shown in Fig. 9, for example. The information providing device 10 will be described below as an example. Fig. 9 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output I/F (Interface) 1060, an input I/F 1070, and a network I/F 1080 are connected via a bus 1090.

The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and secondary storage device 1050, programs read from the input device 1020, and the like, and executes various processes. The arithmetic unit 1030 is realized, for example, by a CPU (Central Processing Unit), MPU (Micro Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), etc.

The primary storage device 1040 is a memory device, such as RAM (Random Access Memory), that primarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, etc. The secondary storage device 1050 may be internal storage or external storage. The secondary storage device 1050 may also be a removable storage medium such as a USB memory or an SD (Secure Digital) memory card. The secondary storage device 1050 may also be cloud storage (online storage), NAS (Network Attached Storage), a file server, etc.

The output I/F 1060 is an interface for transmitting information to be output to an output device 1010, such as a display, projector, printer, or other device that outputs various types of information, and is implemented, for example, by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (High Definition Multimedia Interface). The input I/F 1070 is an interface for receiving information from various input devices 1020, such as a mouse, keyboard, keypad, buttons, scanner, or other device, and is implemented, for example, by USB.

Furthermore, the output I/F 1060 and the input I/F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. In other words, the output device 1010 and the input device 1020 may be wireless devices.

Furthermore, the output device 1010 and the input device 1020 may be integrated, such as a touch panel. In this case, the output I/F 1060 and the input I/F 1070 may also be integrated as an input/output I/F.

The input device 1020 may be a device that reads information from, for example, optical recording media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memories.

The network I/F 1080 receives data from other devices via the network N and sends it to the computing device 1030, and also transmits data generated by the computing device 1030 to other devices via the network N.

The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output I/F 1060 and the input I/F 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

For example, when the computer 1000 functions as the information providing device 10, the arithmetic unit 1030 of the computer 1000 realizes the functions of the control unit 130 by executing a program loaded onto the primary storage device 1040. The arithmetic unit 1030 of the computer 1000 may also load a program acquired from another device via the network I/F 1080 onto the primary storage device 1040 and execute the loaded program. The arithmetic unit 1030 of the computer 1000 may also link with other devices via the network I/F 1080 and call and use program functions, data, etc. from other programs on the other devices.

〔others〕
Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments.

Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown.

Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

For example, the information providing device 10 described above may be implemented using multiple server computers, and depending on the function, the configuration can be flexibly changed, such as by calling an external platform using an API (Application Programming Interface) or network computing.

Furthermore, the above-described embodiments and variations can be combined as appropriate to the extent that the processing content is not contradictory.

In addition, the above-mentioned "section, module, unit" can be interpreted as "means" or "circuit." For example, a control unit can be interpreted as a control means or a control circuit.

REFERENCE SIGNS LIST 10 Information providing device 11 Communication unit 12 Storage unit 12a Behavior information storage unit 12b Decoration information storage unit 12c Compatibility information storage unit 12d Provided information storage unit 13 Control unit 13a Acquisition unit 13b Calculation unit 13c Generation unit 13d Transmission unit 13e Learning unit 20 User terminal 30 Website server 40 Decoration information database 100 Information providing system

Claims (10)

an acquisition unit that acquires hair style information related to a hair style selected by a user;
a calculation unit that calculates a compatibility degree indicating a degree of compatibility between the aspect and the body ornament based on the aspect information;
a generation unit that generates information to be provided to the user based on the compatibility degree;
Equipped with
the acquisition unit further acquires a preference of the user as user information related to the user who selected the mode;
the generation unit identifies the body ornament registered as a preference of the user, and generates the provided information including the identified body ornament .
An information providing device characterized by: the calculation unit calculates the degree of compatibility using a machine learning model that has learned the degree of compatibility between a combination of a hair style and a body ornament worn by a subject having hair of that style.
2. The information providing device according to claim 1. the calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility from video images including hair styles and body ornaments;
3. The information providing device according to claim 2. the calculation unit calculates the degree of compatibility using the machine learning model that has learned the degree of compatibility for each user;
4. The information providing device according to claim 2 or 3. the calculation unit calculates the compatibility degree using the machine learning model that has learned the compatibility degree for each attribute of the user;
5. The information providing device according to claim 2, wherein the information providing device is a device for providing information to a user. the calculation unit calculates the compatibility degree using the machine learning model that has learned the compatibility degree for each user situation;
6. The information providing device according to claim 2, wherein the information providing device is a device for providing information to a user. the acquisition unit further acquires ornament information related to body ornaments;
the generation unit identifies the body decorations indicated by the decoration information that have a compatibility level equal to or higher than a predetermined threshold, and generates the information to be provided.
7. The information providing device according to claim 1, wherein the information providing device is a device for providing information to a user. the generation unit generates the provision information including business information related to a business that can provide the body decoration.
8. The information providing device according to claim 1, wherein the information providing device is a device for providing information to a user. An information providing method executed by an information providing device, comprising:
an acquisition step of acquiring hair style information related to a hair style selected by a user;
a calculation step of calculating a compatibility degree indicating a degree of compatibility between the aspect and the body ornament based on the aspect information;
a generating step of generating information to be provided to the user based on the compatibility degree;
Including,
The acquiring step further acquires a preference of the user as user information related to the user who selected the mode,
the generating step identifies the body ornament registered as the user's preference, and generates the provided information including the identified body ornament .
1. An information providing method comprising: an acquisition step of acquiring hair style information related to a hair style selected by a user;
a calculation step of calculating a compatibility degree indicating a degree of compatibility between the aspect and the body ornament based on the aspect information;
a generation step of generating information to be provided to the user based on the compatibility degree;
on the computer,
The obtaining step further obtains a preference of the user as user information related to the user who selected the mode;
the generating step includes identifying the body ornament registered as the user's preference, and generating the provided information including the identified body ornament .
An information providing program characterized by: