JP7570104B2 - Makeup support device and program - Google Patents

Description

The present invention relates to a makeup support device, a custom-made cosmetics system, and a program.

Conventionally, stamps (cosmetic stamps) used to apply makeup to facial features (eyebrows, mouth, cheeks, etc.) have been known (see Patent Document 1). This type of cosmetic stamp is ideal for people who want to apply makeup easily and in a time-saving manner.

JP 2017-213339 A

However, for those who have never used cosmetic stamps or who have little experience using them, it is difficult to imagine how the makeup will look when applied with cosmetic stamps (the final image). In addition, because facial features, bone structure, and other facial characteristics vary widely, there is also the problem that commercially available (ready-made) cosmetic stamps may not necessarily suit oneself.

The present invention was made in consideration of the above circumstances, and its purpose is to provide a makeup support device, a cosmetics customization system, and a program that allow a user to check the finish of their makeup without actually using the cosmetics, and to order cosmetics that suit them based on the results of that check.

To solve the above problems, the present invention adopts the following configuration.

The makeup support device according to the first aspect of the present invention comprises:
A face image acquisition means for acquiring an image including a face;
A makeup pattern acquisition means for acquiring an eyebrow makeup pattern from a plurality of eyebrow makeup patterns prepared in advance;
a simulation image generating means for generating a simulation image in which eyebrow makeup based on the eyebrow makeup pattern acquired by the makeup pattern acquiring means is applied to an eyebrow portion of a face included in the image acquired by the face image acquiring means;
a simulation image display means for displaying the simulation image generated by the simulation image generating means;
an order information generating means for generating order information for an eyebrow makeup stamp that can be pressed against an eyebrow part of a face to apply eyebrow makeup, based on an eyebrow makeup pattern included in the simulation image displayed by the simulation image display means;
and a transmitting means for transmitting the order information generated by the order information generating means to a destination of an order for the eyebrow makeup stamp .

According to this, a simulation image is generated and displayed in which eyebrow makeup is applied to the eyebrow part of the face included in the image acquired by the face image acquisition means (also called "face image") based on the eyebrow makeup pattern acquired by the makeup pattern acquisition means, so that the user can check the finish of the eyebrow makeup applied to his/her face through this simulation image. Here, the eyebrow makeup pattern applied to the eyebrow part of the face image (face in the image) is acquired from a plurality of eyebrow makeup patterns prepared in advance, so that the user can check various eyebrow makeup patterns. This makes it possible to check the finish of the eyebrow makeup by various eyebrow makeup patterns through the simulation image. Also, order information for the eyebrow makeup stamp is generated based on the eyebrow makeup pattern included in the displayed simulation image, i.e., the eyebrow makeup pattern applied to the face (face image) in the displayed simulation image (the eyebrow makeup pattern that was the basis for generating the simulation image), and the order information is transmitted to the ordering destination of the eyebrow makeup stamp. This makes it possible for a user who has performed a simulation of eyebrow makeup to order an eyebrow makeup stamp that suits him/her based on the results of the simulation (custom-made).

The makeup support device of the present invention is the makeup support device of the present invention,
a makeup state recognition means for recognizing a state of eyebrow makeup applied to a face included in an image acquired by the face image acquisition means when the face includes actual eyebrow makeup;
a second order information generating means for generating order information for the eyebrow makeup stamp having specifications corresponding to the actual eyebrow makeup based on the state of the eyebrow makeup recognized by the makeup state recognizing means ;
and a second transmission means for transmitting the order information generated by the second order information generation means to an order destination of the eyebrow makeup stamp.
The gist of the present invention is as follows.

According to this, when a face with actual eyebrow makeup is included in a face image, the state of the applied eyebrow makeup is recognized, and based on the recognized state of eyebrow makeup , order information for an eyebrow makeup stamp with specifications corresponding to the actual eyebrow makeup is generated , and the order information is sent to an ordering destination of the eyebrow makeup stamp . This allows a user to order (custom-made) an eyebrow makeup stamp of their choice based on an image of a face with actual eyebrow makeup (also called a "make-up face image").

The program of means 3 is a program for causing a computer to function as each of the means of the makeup support device of means 1 or 2 .

According to the present invention, it is possible to provide a makeup support device, a cosmetics customization system, and a program that allow a user to check the finish of their makeup without actually using the cosmetics, and to order cosmetics that suit them based on the results of that check.

FIG. 1 is a configuration diagram of a system according to an embodiment. FIG. 2 is a configuration diagram of a terminal device. FIG. 2 is a configuration diagram of a sales company device. FIG. 2 is a configuration diagram of a manufacturer device. FIG. 4 is a functional block diagram of a processing unit of the terminal device. 13 is a flowchart of an eyebrow makeup application execution process. 13 is a flowchart of an eyebrow makeup application execution process. 13 is a flowchart of an order management process. 1A is an explanatory diagram of a first process display, and FIG. 1B is an explanatory diagram of a second process display. 1C is an explanatory diagram of the third step display, and FIG. 1D is an explanatory diagram of the fourth step display. 1 is a table showing eyebrow shape patterns. FIG. 13 is a diagram showing an example of the configuration of an eyebrow stamp. 13 is a flowchart of an eyebrow makeup application execution process 2 according to the second modified example. 13 is a flowchart of an eyebrow makeup application execution process 2 according to the second modified example.

Next, one embodiment of the present invention will be described using an example.

This embodiment shows a device that enables simulation of eyebrow makeup (eyebrow make-up) for applying makeup to the eyebrow area of the face, and a system related to ordering eyebrow makeup stamps using the device.

[Overall system configuration]
1, the system according to the present embodiment includes a terminal device 1 used by a user, a sales company device 2 provided at a sales company that sells eyebrow cosmetic stamps, a manufacturer device 3 provided at a manufacturer that manufactures eyebrow cosmetic stamps, and a network such as the Internet 4. Note that the sales company and the manufacturer may be the same or different parties.

The terminal device 1 is a portable terminal device such as a smartphone or tablet that is used by a user to simulate eyebrow makeup or order (purchase) eyebrow makeup stamps. The terminal device 1 can also be configured as a notebook or desktop personal computer (PC). The terminal device 1 is also referred to as a "user device."

The sales business device 2 is a device such as a computer managed by a business that sells products over the Internet (online shopping) and provides various associated services, and has server functions for operating websites and providing various services. The products referred to here include eyebrow cosmetic stamps. The sales business device 2 is also referred to as a "service providing server."

The manufacturer device 3 is a device such as a computer managed by a business that manufactures products (eyebrow cosmetic stamps in this embodiment) sold by the sales business, and has a server function that manages manufacturing requests (orders) from the sales business (sales business device 2) and information about manufactured products. The manufacturer device 3 is also called a "manufacturing management server." The manufacturer device 3 can include a manufacturing device for the product.

The seller device 2 and the manufacturer device 3 can each be configured with one or more computers or other devices. Furthermore, if the seller and manufacturer are the same business, the seller device 2 and the manufacturer device 3 can be installed under the management of the same business, or the seller device 2 and the manufacturer device 3 can be configured with a single device.

[Terminal Device Configuration]
Next, the configuration of the terminal device 1 will be described with reference to Fig. 2. As shown in the figure, the terminal device 1 includes a processing unit 10, a storage unit 11, a display unit 12, an operation input unit 13, a communication unit 14, a camera (imaging unit) 15, a battery (power supply unit) 16, etc.

The processing unit 10 executes various processes for controlling the operation of the terminal device 1, and is composed of a CPU (Central Processing Unit) and the like. The processing unit 10 is also referred to as the "control unit" (the same applies to the processing unit 20 and processing unit 30 described below).

The storage unit 11 stores the OS (Operating System) that operates the terminal device 1, various application programs that can be executed by the terminal device 1, and various other data necessary for the operation of the terminal device 1, and is composed of a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory).

The application programs (application software) stored in the memory unit 11 include programs such as an eyebrow makeup simulation application (also called an "eyebrow makeup application") provided by the sales business device 2, and a web application (also called a "web application") that connects to a web server and enables browsing, operation, etc. of web sites (web pages).

The memory unit 11 also stores data related to the eyebrow makeup app, such as multiple types of eyebrow makeup patterns (multiple makeup patterns) provided by the sales company device 2. The eyebrow makeup patterns include eyebrow shape patterns that represent the shape of the eyebrows (also called "eyebrow shape") and eyebrow color patterns that represent the color of the eyebrows (also called "eyebrow color").

The display unit 12 displays various images, screens, characters, etc., and is composed of, for example, a liquid crystal display. This display unit 12 has a touch panel function. The display unit 12 is also called a "display panel."

The operation input unit 13 is composed of buttons, switches, etc. that are operated by the user of the terminal device 1 to input various instructions. Instructions input by the operation input unit 13 include input by buttons, icons, keyboards, input areas, etc. displayed on the display unit 12 equipped with a touch panel function. A signal (operation signal, instruction signal) based on the instruction input by the operation input unit 13 is input to the processing unit 10.

The communication unit 14 performs data communication with other devices via the network 4, for example, performing wireless communication conforming to the IEEE 802.11 standard. The other devices include the sales company device 2.

The camera 15 becomes available when the camera function is activated under the control of the processing unit 10, and captures images of subjects such as people and scenery based on the operation (instruction) of the user using the terminal device 1. The lenses 15a (see FIG. 9(A)) of the camera 15 are provided on the front and back of the terminal device 1.

The battery 16 is composed of a rechargeable battery or the like, and supplies power to each of the components of the mobile terminal 1, such as the processing unit 10, memory unit 11, display unit 12, operation input unit 13, communication unit 14, and camera 15.

[Configuration of the sales company device]
Next, the configuration of the sales company device 2 will be described with reference to Fig. 3. As shown in Fig. 3, the sales company device 2 includes a processing unit 20, a storage unit 21, a display unit 22, an operation unit 23, a communication unit 24, and the like.

The processing unit 20 is configured with a CPU and the like, similar to the processing unit 10 of the terminal device 1, and executes various processes for controlling the operation of the sales company device 2 (service provision server).

The memory unit 21 stores the OS (Operating System) that runs the sales business device 2, various application programs, and various other data necessary for the operation of the sales business device 2, and is composed of a ROM (Read Only Memory), which is a non-volatile memory, a RAM (Random Access Memory), which is a volatile memory, a hard disk drive (HDD), which is an auxiliary storage device, etc.

The data stored in the memory unit 21 includes data necessary for operating websites and providing various services. Specifically, the data includes programs for operating websites such as a website providing an eyebrow makeup app and a website selling eyebrow makeup stamps, a program for the eyebrow makeup app that is installed and executed on the terminal device 1, and multiple types of eyebrow makeup patterns that can be used with the eyebrow makeup app.

In this embodiment, an eyebrow makeup pattern database DB1 that manages (stores) multiple types of eyebrow makeup patterns is provided, and the database DB1 is stored in the storage unit 21. The eyebrow makeup pattern database DB1 stores multiple eyebrow shape patterns (six types, patterns P1 to P6 in this embodiment) with different eyebrow shapes, for example, as shown in FIG. 11. It also stores data (eyebrow color patterns) on multiple eyebrow colors (three colors, gray, natural brown, and dark brown, in this embodiment) that can be set (used) for each eyebrow shape.

The types and number of eyebrow makeup patterns (eyebrow shape patterns, eyebrow color patterns) managed in the eyebrow makeup pattern database DB1 can be changed (updated) or increased or decreased by the seller operating the seller device 2. This allows, for example, a trendy eyebrow makeup pattern to be added after the fact, or an eyebrow makeup pattern that is losing popularity to be deleted after the fact.

The types and number of eyebrow makeup patterns are not limited to those shown in this embodiment, and more patterns may be provided.

The memory unit 21 also stores a customer information database DB2 and an order information database DB3.

The customer information database DB2 is a database that manages (stores) information on users (customers) who order (purchase) eyebrow makeup stamps using the eyebrow makeup app.

The order information database DB3 is a database that manages (stores) information about the order contents (also called "order information") when ordering eyebrow makeup stamps using the eyebrow makeup app.

A user of the terminal device 1 can operate the terminal device 1 to connect to the sales provider device 2 (service provision server), download data required for the operation of the eyebrow makeup app, such as the eyebrow makeup app program and eyebrow makeup patterns (also referred to as "eyebrow makeup app related data"), and install the data in the terminal device 1, thereby executing (using) the eyebrow makeup app on the terminal device 1. In addition, update programs for the eyebrow makeup app and update data for eyebrow makeup patterns can also be connected to the sales provider device 2, downloaded, and installed (updated) in the terminal device 1.

In addition, if the sales business device 2 is configured to include two or more (multiple) computers (servers), the various data exemplified as being stored in the memory unit 21 may be distributed and managed (stored) in multiple computers.

The display unit 22 displays various images, screens, characters, etc., and is composed of a liquid crystal display or the like.

The operation unit 23 is operated to input various data and instructions, and is composed of a keyboard, mouse, etc.

The communication unit 24, like the communication unit 14 of the terminal device 1, communicates data with other devices via the network 4. The other devices include the terminal device 1 and the manufacturer device 3.

[Manufacturer Device Configuration]
Next, the configuration of the manufacturer device 3 will be described with reference to Fig. 4. As shown in Fig. 4, the manufacturer device 3 includes a processing unit 30, a storage unit 31, a display unit 32, an operation unit 33, a communication unit 34, and the like.

The processing unit 30, memory unit 31, display unit 32, operation unit 33, and communication unit 34 are basically configured in the same manner as the processing unit 20, memory unit 21, display unit 22, operation unit 23, and communication unit 24 of the sales business device 2. Therefore, a detailed description of the configuration that is the same as that of the sales business device 2 will be omitted.

The memory unit 31 stores various data necessary for the operation of the manufacturer device 3, as well as an order management database DB4 and a product management database DB5.

The order management database DB4 is a database that manages information (also called "order information") based on a manufacturing request (order information) when order information sent from the terminal device 1 to the sales business device 2 is sent to the manufacturing business device 3, and a manufacturing request for a product (in this embodiment, an eyebrow cosmetic stamp) is accepted upon receipt of the order information by the manufacturing business device 3.

The product management database DB5 is a database that manages information (also called "product information") about manufacturing requests (order information) from the sales business device 2, i.e., about goods (products) manufactured based on the order information managed in the order management database DB4.

[Eyebrow makeup app execution process]
Next, the eyebrow makeup application execution process (S100) executed by the terminal device 1 in which the eyebrow makeup application related data is installed will be described with reference to the flowcharts shown in Fig. 6 and Fig. 7. In this specification, "S" in S100 and the like ("S" in the flowchart) means a step.

The eyebrow makeup app execution process (S100) is executed (started) by the processing unit 10 when the user operates the terminal device 1 to launch the eyebrow makeup app. The eyebrow makeup app can be launched, for example, by tapping an eyebrow makeup app icon (not shown) displayed on the display unit 12 of the terminal device 1 in which the eyebrow makeup app related data has been installed.

The processing unit 10 of the terminal device 1 in which the eyebrow makeup app related data is installed functions as each of the means 101 to 112 shown in FIG. 5 by starting (executing) the eyebrow makeup app. This terminal device 1 corresponds to an example of the "makeup support device" of the present invention. Note that instruction input by the user (instruction input by the operation input unit 13) while executing (using) the eyebrow makeup app is detected by the instruction input detection means 101.

As shown in Figure 6, in the eyebrow makeup application execution process (S100), first, the application screen is displayed on the display unit 12 of the terminal device 1 (S101), and the camera 15 is started (S103). The display content (also called the "first step display") on the display unit 12 (application screen) at this time is shown in Figure 9 (A).

As shown in the figure, the display unit 12 on which the app screen is displayed is divided into a main display area 12a and a sub-display area 12b, which has a smaller display range (area size) than the main display area 12a. In other words, the app screen is broadly divided into the main display area 12a and the sub-display area 12b. In this embodiment, the screen configuration is such that the sub-display area 12b is located below the main display area 12a. Note that the configuration of the app screen, including the display contents described below, is not limited to this embodiment, and can be set appropriately depending on the size (screen size) and shape of the display unit (display screen) of the terminal device (user device).

In the first step display, an image captured by the camera 15 through the lens 15a (here, an image including the user's face) is displayed in the main display area 12a. In addition, an OK button BT1 for inputting an image capture command and a selection button BT2 for inputting an command to select data to be used in the eyebrow makeup simulation from data such as photographs stored in the memory unit 11 (also called a "selection command") are displayed in the secondary display area 12b.

In the first step display, the user can tap the OK button BT1 or the selection button BT2 to select whether to perform an eyebrow makeup simulation based on the face imaged at that time or based on a face imaged in the past.

Returning to FIG. 6, next in S105, it is determined whether or not an image capture instruction has been input by tapping the OK button BT1 in the first step display (S105). As a result, if it is determined that an image capture instruction has been input (YES in S105), an image of the image capture target (here, the user's face) is captured by the camera 15 (S107). The processing of S107 is performed by the processing unit 10 functioning as the image capture means 102.

On the other hand, if it is determined that an image capture instruction has not been input (NO in S105), it is determined whether or not a selection instruction has been input by tapping the selection button BT2 (S109). As a result, if it is determined that a selection instruction has not been input (NO in S109), the process returns to S105, and if it is determined that a selection instruction has been input (YES in S109), data selection processing is performed (S111).

The data selection process (S111) is a process in which a list of data such as photographs stored in the memory unit 11 is displayed on the display unit 12 (not shown), and one piece of data is selected from the list based on a selection operation (selection instruction input) by the user.

Following the processing of S107 or S111, in S113, an image including a face (facial image) is acquired from the captured data (captured data) or selected data (selected data) (S113), and the acquired facial image is displayed in the main display area 12a (S115), while an eyebrow shape sample icon SA1 and an eyebrow color sample icon SA2 based on the eyebrow makeup pattern stored in the storage unit 11 are displayed in the sub-display area 12b (S117). The eyebrow shape sample icon SA1 and the eyebrow color sample icon SA2 are collectively referred to as "eyebrow makeup samples".

The processes of S113, S115, and S117 are performed by the processing unit 10 functioning as the face image acquisition means 103, the face image display means 104, and the makeup pattern display means 105, respectively. The eyebrow makeup sample displayed in the sub-display area 12b indicates the eyebrow makeup pattern stored in the memory unit 11.

Figure 9 (B) shows the display content (also called the "second step display") on the display unit 12 (application screen) when the face image and eyebrow makeup sample are displayed.

As shown in the figure, the face image FA displayed in the main display area 12a is displayed as a still image showing the user's front face. Also, the eyebrow shape sample icon SA1 displayed in the sub-display area 12b indicates an eyebrow shape based on an eyebrow makeup pattern (eyebrow shape pattern) stored in the memory unit 11.

In this embodiment, three eyebrow shape sample icons SA1 corresponding to three of the six patterns P1 to P6 shown in FIG. 11 are displayed in the secondary display area 12b, and the user can select a preferred eyebrow shape by tapping one of the eyebrow shape sample icons SA1.

In addition, by swiping left or right on the display portion of the display unit 12 (display panel) where the eyebrow shape sample icon SA1 is displayed, the eyebrow shape sample icon SA1 can be scrolled left or right within the secondary display area 12b, thereby changing the type of eyebrow shape sample icon SA1 displayed in the secondary display area 12b, i.e., the type of eyebrow makeup pattern (eyebrow shape).

Note that Figure 9 (B) shows the state in which eyebrow shape sample icons SA1 corresponding to P4 to P6 out of the six patterns P1 to P6 shown in Figure 11 are displayed.

In addition, in the second step display, the eyebrow color sample icon SA2 displayed in the secondary display area 12b indicates the eyebrow color based on the eyebrow makeup pattern (eyebrow color pattern) stored in the memory unit 11. In this embodiment, three eyebrow color sample icons SA2 corresponding to three types of eyebrow colors (gray, natural brown, dark brown) are displayed in the secondary display area 12b, and the user can select the preferred eyebrow color by tapping one of the eyebrow color sample icons SA2.

In this embodiment, the eyebrow color sample icon SA2 is not active (enabled) until an eyebrow shape is selected by tapping the eyebrow shape sample icon SA1, and once an eyebrow shape is selected, the eyebrow color sample icon SA2 is configured to be active (enabled). This is to give priority to the selection of eyebrow shape over the selection of eyebrow color.

Returning to FIG. 6, next in S119, it is determined whether or not an instruction to select an eyebrow makeup sample has been input in the second step display (S119). Here, it is determined whether or not both an instruction to select an eyebrow shape by tapping the eyebrow shape sample icon SA1 and an instruction to select an eyebrow color by tapping the eyebrow color sample icon SA2 have been input (whether or not the selection of eyebrow shape and eyebrow color has been completed).

If it is determined in S119 that an instruction to select an eyebrow makeup sample has not been input (NO in S119), the process waits until an instruction to select an eyebrow makeup sample is input. On the other hand, if it is determined that an instruction to select an eyebrow makeup sample has been input (YES in S119), the process reads and acquires an eyebrow makeup pattern corresponding to the selected eyebrow makeup sample from the storage unit 11 (S121). The process of S121 is performed by the processing unit 10 functioning as the makeup pattern acquisition means 106.

Next, in S123, a simulation image is generated in which makeup (virtual makeup) based on the eyebrow makeup pattern acquired in S121 is applied to the eyebrow portion of the face image FA being displayed in the main display area 12a, i.e., the face image acquired in S113 (S123). The eyebrow makeup simulation image can be generated, for example, by combining the face image FA being displayed in the display area 12a (the face image acquired in S113) with an eyebrow image based on the eyebrow makeup pattern (eyebrow shape pattern, eyebrow color pattern) acquired in S121. The processing of S123 is performed by the processing unit 10 functioning as the simulation image generating means 107.

Next, in S125, the simulation image generated in S123 is displayed in the main display area 12a (S125). The display content (also called the "third step display") on the display unit 12 (application screen) at this time is shown in FIG. 10 (C).

As shown in the figure, the simulation image SM is displayed in the main display area 12a, and the eyebrow shape sample icon SA1 and eyebrow color sample icon SA2 are displayed in the sub-display area 12b, similar to the second step display (see FIG. 9(B)).

The simulation image SM shown in FIG. 10(C) shows a case where eyebrow makeup based on pattern P5 has been applied to the eyebrow area of the face image FA shown in FIG. 9(B). The processing of S125 is performed by the processing unit 10 functioning as the simulation image display means 108.

In addition, in the third step display, in addition to the simulation image SM, the main display area 12a displays guide lines GU, which serve as a guide for adjusting the position and balance of the eyebrows after makeup (makeup eyebrows), a guide setting button BT3 that allows the guide lines GU to be switched between hidden (OFF) and displayed (ON) in both directions, and a decision button BT4 for inputting a decision instruction to finalize the results of the eyebrow makeup simulation. By default, the guide lines GU are displayed, and by tapping the guide setting button BT3, the user can select whether to hide (OFF) or display (ON) the guide lines GU.

The guide line GU consists of one vertical guide line GU1 that extends vertically (in the middle) between the left and right eyebrows, and two horizontal guide lines GU2, GU3 that extend horizontally (horizontally) above and below the eyebrows.

The user can move the eyebrow image included in the simulation image SM by touching with a finger a portion of the simulation image SM corresponding to the eyebrow portion (eyebrow image) on the display unit 12 (display panel) on which the simulation image SM is displayed and dragging and dropping it. In addition, the size (dimension) of the eyebrow image included in the simulation image SM can be enlarged or reduced by touching with a finger a portion of the simulation image SM corresponding to the eyebrow portion (eyebrow image) and pinching out or in. In other words, an instruction to adjust the eyebrow image is input by touch operations such as dragging and dropping, pinching out, and pinching in. At this time, if the guide lines GU are displayed, the position, balance, etc. of the eyebrow image can be adjusted using the guide lines GU as a reference (guideline). Such display (display control) of the guide lines GU is achieved by the processing unit 10 functioning as the guide display means 109.

When adjusting the eyebrow image, the adjustable range of the eyebrow image may be limited (restricted) within the range of the guide line GU. For example, in FIG. 10(C), the eyebrow image R corresponding to the right eyebrow may be movable within the range of the two horizontal guide lines GU2 and GU3 on the left side of the vertical guide line GU1, and the eyebrow image L corresponding to the left eyebrow may be movable within the range of the two horizontal guide lines GU2 and GU3 on the right side of the vertical guide line GU1. This prevents the balance of the appearance of the eyebrow position, etc. from becoming unnatural, making it possible to simulate eyebrow makeup that looks good. In addition, with regard to the adjustment of the eyebrow image, in addition to adjusting (changing) the position and size of the eyebrow image, adjustments related to the shape of the eyebrow, such as the thickness and length, may also be made by touch operation. This makes it possible to improve the quality and accuracy of the eyebrow makeup simulation.

In addition, in the third step display, an instruction to select an eyebrow makeup sample can be input by tapping the eyebrow shape sample icon SA1 or eyebrow color sample icon SA2 displayed in the secondary display area 12b. In other words, the eyebrow shape and eyebrow color can be changed even after the simulation image SM is displayed. Note that after the simulation image SM is displayed, both sample icons SA1 and SA2 are active (enabled), and neither is given priority.

Next, in S127 shown in FIG. 7, in the third display step (simulation image display screen), it is determined whether an instruction to select an eyebrow makeup sample has been input (S127). Here, it is determined whether an instruction to select an eyebrow shape by tapping the eyebrow shape sample icon SA1 or an instruction to select an eyebrow color by tapping the eyebrow color sample icon SA2 has been input. As a result, if it is determined that an instruction to select an eyebrow makeup sample has been input (YES in S127), the process returns to S121 (see FIG. 6) and the process from S121 is performed again.

On the other hand, if it is determined that an instruction to select an eyebrow makeup sample has not been input (NO in S127), it is determined whether an instruction to adjust the eyebrow image has been input (S129). As described above, an instruction to adjust the eyebrow image is input by performing an operation such as drag and drop or pinch out or pinch in on the part corresponding to the eyebrow image.

If it is determined in S129 that an instruction to adjust the eyebrow image has been input (YES in S129), a process is performed in which the eyebrow image included in the simulation image SM is moved, enlarged, or reduced in accordance with the input (eyebrow image adjustment process) (S131). The process of S131 is performed by the processing unit 10 functioning as the makeup pattern adjustment means 110.

If it is determined in S129 that an instruction to adjust the eyebrow image has not been input (NO in S129), it is determined whether a confirmation instruction has been input by tapping the decision button BT4 (S133). As a result, if it is determined that a confirmation instruction has not been input (NO in S133), the process returns to S127.

On the other hand, if it is determined that a confirmation instruction has been input (YES in S133), a confirmation screen is displayed on the display unit 12 to allow the user to confirm the result of the eyebrow makeup simulation that has been decided (confirmed) (S135). The display content of the display unit 12 at this time (also referred to as the "fourth step display") is shown in Figure 10 (D).

As shown in the figure, in the fourth step display (confirmation screen), a simulation image SM is displayed in the main display area 12a as the result of the eyebrow makeup simulation. At this time, the display contents of the main display area 12a are such that the guide lines GU, guide setting button BT3, and decision button BT4 displayed in the main display area 12a in the third step display (see FIG. 10(C)) are hidden, and the simulation image SM displayed in the third step display immediately before moving to the fourth step display is displayed as the result of the eyebrow makeup simulation. The processing of S135 related to the display of the simulation image SM in the fourth step display (confirmation screen) is also performed by the processing unit 10 functioning as the simulation image display means 108.

In addition, in the fourth step display, the secondary display area 12b displays a message such as "Are you sure?", a back button BT5 for inputting an instruction to return to the previous step display (third step display) (also called a "return instruction"), and an order button BT6 for inputting an instruction to order an eyebrow makeup stamp that can be used to apply the eyebrow makeup shown in the simulation image SM (eyebrow makeup simulation result) being displayed in the primary display area 12a (also called an "order instruction").

Returning to FIG. 7, next in S137, it is determined whether an order instruction has been input by tapping the order button BT6 (S137). As a result, if it is determined that an order instruction has not been input (NO in S137), it is determined whether a return instruction has been input by tapping the back button BT5 (S139), and if it is determined that a return instruction has been input (YES in S139), the process returns to S125 (see FIG. 6) and the process from S125 is performed again. At this time, the display content on the display unit 12 of the terminal device 1 becomes that shown in FIG. 10 (C) (third step display).

On the other hand, if it is determined in S139 that a return command has not been input (NO in S139), it is determined whether or not an instruction to end the eyebrow makeup app has been input (S141). The instruction to end the eyebrow makeup app can be given, for example, by swiping the app screen of the eyebrow makeup app being displayed on the display unit 12 of the terminal device 1 in a predetermined direction (e.g., upward) or by displaying a home screen (not shown) on the display unit 12.

If it is determined in S141 that an end instruction has not been input (NO in S141), the process returns to S137, and if it is determined that an end instruction has been input (YES in S141), an application end process is performed to end the eyebrow makeup application (S155). The eyebrow makeup application is ended in this manner without issuing an order instruction (S137) when the user only performs an eyebrow makeup simulation.

In contrast, if it is determined in the above-mentioned S137 that an order instruction has been input (YES in S137), order information relating to an order for an eyebrow makeup stamp based on the results of this eyebrow makeup simulation is generated (S143), and an order screen (not shown) is displayed on the display unit 12 (S145). The processing of S143 is performed by the processing unit 10 functioning as the order information generating means 111.

The order screen (not shown) displays, for example, the order information generated in S143, input fields for inputting information about the user who is the orderer (purchaser) (also referred to as "user information"), and a send button for inputting an instruction to send the user information and order information (also referred to as a "send instruction"). Once the required items have been entered into the input fields, the send button becomes active (enabled), and the send instruction can be entered by tapping the send button. By making the send button active (enabled) upon completion of input into the input fields, it is possible to prevent erroneous sending of information.

The order information is generated based on the eyebrow makeup pattern (eyebrow makeup pattern) included in the simulation image SM (eyebrow makeup simulation result) displayed in the main display area 12a in the fourth step display (confirmation screen). Here, if the size, shape, etc. of the eyebrow image is adjusted (changed) by the eyebrow image adjustment process (S131) described above, the adjusted (changed) eyebrow makeup pattern is reflected in the generation of the order information.

The order information includes information indicating, for example, the order date, product name, product specifications (eyebrow shape, eyebrow color, size, etc.), product unit price, quantity, product cost, etc. The "product specifications" in the order information specifies the eyebrow makeup pattern (eyebrow makeup pattern) included in the simulation image SM (eyebrow makeup simulation result). The user information includes information indicating the name, address (delivery destination of the product), contact information (telephone number, email address, etc.), payment method, etc. of the user who ordered (purchased) the product (here, eyebrow makeup stamp). Note that the order information and user information may be collectively referred to as "order information." Hereinafter, the order information and user information are also collectively referred to as "order information."

Next, in S147, it is determined whether a send instruction has been input by tapping a send button (not shown) (S147). As a result, if it is determined that a send instruction has not been input (NO in S147), it is determined whether an instruction to end the eyebrow makeup app has been input (S151). If it is determined that an instruction has not been input (NO in S151), the process returns to S147, and if it is determined that an instruction has been input (YES in S151), the app termination process is performed (S155). The eyebrow makeup app is terminated without transmitting order information when the user only performs an eyebrow makeup simulation.

On the other hand, if it is determined in S147 that a transmission instruction has been input (YES in S147), the order information is transmitted to the order destination (the sales company device 2 in this embodiment) (S149). The processing in S149 is performed by the processing unit 10 functioning as the transmission means 112.

Next, in S153, it is determined whether an instruction to end the eyebrow makeup app has been input (S153), and if it is determined that an instruction to end the app has not been input (NO in S153), the process waits until an instruction to end the app is input, and if it is determined that an instruction to end the app has been input, the process to end the app is performed (S155).

[Order Management Processing]
Next, the order management process (S200) executed by the sales company device 2 (service providing server) will be described with reference to the flowchart shown in Fig. 8. The order management process (S200) is a process executed by the processing unit 20.

As shown in FIG. 8, in the order management process (S200), first, it is determined whether or not the order information from the terminal device 1, i.e., the order information sent by the process of S149 described above, has been received (S201). As a result, if it is determined that the order information has not been received (NO in S201), this process ends without performing the processes from S203 onwards.

On the other hand, if it is determined that the order information has been received (YES in S201), an order reception number is issued (generated) and the order reception number is returned to the terminal device 1 that sent the order information (S203). This allows the user to confirm that the current order has been properly received. The order reception number can be issued (generated) automatically by the program. Note that the processing of S201 is performed by the processing unit 20 functioning as a receiving means.

Next, in S205, the user information from the order information received in S201 is registered (stored) as customer information in the customer information database DB2 (S205). This allows the sales company to manage information (age, order history, etc.) about users (customers) who have ordered (purchased) eyebrow makeup stamps using the eyebrow makeup app.

Note that the registration of information in the customer information database DB2 in S205 may be performed either by a program or by a sales company's person in charge (operator) manually entering the information.

Next, in S207, the order information received in S201 is registered (stored) in order information database DB3 (S207). Here, the order information is linked (associated) with the order reception number issued (generated) in S203 and the customer information (user information) registered in customer information database DB2 in S205, and registered (stored) in order information database DB3. This allows the sales company to manage information related to eyebrow makeup stamps ordered using the eyebrow makeup app (order date, eyebrow makeup stamp specifications, order history, etc.).

Note that the registration of information in the order information database DB3 in step S207 may be performed by a program or by a sales company's person in charge (operator) manually entering the information.

Next, in S209, a process is executed to request the manufacturer (manufacturer device 3) to manufacture the eyebrow cosmetic stamp based on the order information received in S201 (S209). This manufacturing request can be made in any manner, such as by transferring the order information received in S201 to the manufacturer device 3, converting the order information received in S201 into a format that can be received (accepted) by the manufacturer device 3 and sending it to the manufacturer device 3, or inputting information about the product to be requested to be manufactured (information about the eyebrow cosmetic stamp based on the order information received in S201) into a dedicated input form (manufacturing request form) that can be used between the sales company and the manufacturer.

The above is the specific processing content of the order management processing (S200) by the seller device 2 (processing unit 20), but when the manufacturing request is sent to the manufacturer device 3 by the processing of S209 and the manufacturing request is accepted by the manufacturer device 3 (manufacturer), information based on the manufacturing request is managed as order information in the order management database DB4. The order information includes information indicating, for example, the order date, the order origin (seller), the delivery destination (address, name, etc. of the user who is the purchaser), the product name, the specifications of the product (eyebrow cosmetic stamp) (eyebrow shape, eyebrow color, size, etc.), the unit price of the product, the quantity, the product price, etc. The order information also includes information that overlaps with the order information.

The manufacturer manufactures the eyebrow cosmetic stamp based on the order information managed in the order management database DB4. Specifically, the manufacturer manufactures the eyebrow cosmetic stamp based on the eyebrow shape specified based on the order information managed in the order management database DB4, i.e., the order information (manufacturing request) from the sales company device 2. The manufacturer also mixes cosmetic powder based on the eyebrow color specified based on the order information (manufacturing request) from the sales company device 2, and manufactures eyebrow cosmetic stamp powder (not shown) by soaking a sponge or the like in the mixed powder. The eyebrow cosmetic stamp and eyebrow cosmetic stamp powder manufactured in this manner are stored in a dedicated storage case (not shown) and delivered to the user. In this way, in this embodiment, the eyebrow cosmetic stamp ordered using the eyebrow cosmetic app is delivered as a set with the eyebrow cosmetic stamp powder.

In the product management database DB5, information about the products (eyebrow stamps, powder for eyebrow stamps) manufactured by the manufacturers is managed as product information. The product information includes, for example, information indicating the specifications (eyebrow shape, eyebrow color, size, etc.) of the products (eyebrow stamps, powder for eyebrow stamps), the delivery date, the ordering party (sales company), the delivery destination, etc.

If a manufacturer receives a request (additional order) from a distributor or user later to manufacture the same product (eyebrow stamp, powder for eyebrow stamp) as a previously manufactured product, the manufacturer can manufacture the same product (eyebrow stamp, powder for eyebrow stamp, or a set of these) based on the product information managed in product management database DB5. Also, if a distributor or user later orders replacement parts (consumables) for a previously manufactured product (eyebrow stamp, powder for eyebrow stamp), the manufacturer can manufacture the corresponding replacement parts (consumables) based on the product information managed in product management database DB5.

By connecting the sales business device 2 that executes the above-mentioned order management process (S200) and the terminal device 1 that executes the above-mentioned eyebrow makeup application execution process (S100) via the network 4, a custom-made eyebrow makeup stamp system (corresponding to an example of the "custom-made cosmetics system" of the present invention) is constructed. Note that the sales business device 2 in this case corresponds to an example of the "order receiving device" of the present invention.

Here, an example of the configuration of an eyebrow cosmetic stamp will be described with reference to FIG. 12. As shown in the figure, the eyebrow cosmetic stamp 500 is composed of a main body 501 that is horizontally long (rectangular) in a plan view, and a stamp member 502 that is affixed to the bottom surface of the main body 501. Note that while FIG. 12 shows the eyebrow cosmetic stamp 500 for the left eyebrow, it is actually sold as a set for the left and right eyebrows, and the eyebrow cosmetic stamp for the right eyebrow appears symmetrically to the eyebrow cosmetic stamp 500 for the left eyebrow shown in FIG. 12.

The main body 501 is made of transparent plastic or the like. At both the left and right ends of the main body 501, there are gripping portions 501a for gripping with fingers when using the eyebrow stamp 500.

The stamp member 502 is made of porous rubber, sponge, or the like, and is molded into an eyebrow shape according to the eyebrow shape pattern. One side of the stamp member 502 is the stamp surface (stamp portion) 502a that is pressed against the eyebrow area of the face.

When using the eyebrow stamp 500, pinch the tabs 501a on both the left and right ends with your fingers and press (stamp) the stamp surface 502a, which has eyebrow stamp powder on it, onto the eyebrows of your face reflected in a mirror. This applies the eyebrow powder on the stamp surface 502a to the eyebrows, applying eyebrow makeup. The main body 501 of the eyebrow stamp 500 shown in this configuration example is transparent, making it easy for the user to see where to stamp.

[Effects of the embodiment]
In the terminal device 1 (makeup support device) in which the eyebrow makeup application related data of this embodiment described above is installed, a simulation image SM is generated in which eyebrow makeup is applied to the eyebrow portion of the face included in the face image acquired in the process of S113 (face image acquisition means 103) based on the eyebrow makeup pattern acquired in the process of S121 (makeup pattern acquisition means 106), and is displayed on the display unit 12 (main display area 12a). Therefore, the user can easily simulate eyebrow makeup by operating the terminal device 1, and can check the finish of the eyebrow makeup applied to his or her face through the simulation image SM.

Here, eyebrow makeup patterns (eyebrow makeup patterns) to be applied to the eyebrow portion of the facial image (face in the image) are provided by the sales business device 2 (service provision server) and stored (prepared in advance) in the memory unit 11 of the terminal device 1, and the user can select a desired pattern from the multiple eyebrow makeup patterns stored in the memory unit 11. The eyebrow makeup pattern selected by the user is then acquired, and a simulation image SM with eyebrow makeup based on that eyebrow makeup pattern is displayed, allowing the user to try out and check various makeup patterns. This makes it possible to check the finish of the makeup with various makeup patterns through the simulation image SM.

In this embodiment, order information for an eyebrow makeup stamp is generated based on the eyebrow makeup pattern included in the simulation image SM displayed on the display unit 12 (main display area 12a) of the terminal device 1, i.e., the eyebrow makeup pattern applied to the face (facial image) in the simulation image SM (the eyebrow makeup pattern that was the basis for generating the simulation image SM), and the order information is sent to the sales company device 2 (the order destination for the eyebrow makeup stamp). This allows a user who has performed a simulation of eyebrow makeup using the eyebrow makeup app (terminal device 1) to order an eyebrow makeup stamp that suits them based on the results of the simulation, i.e., to have an eyebrow makeup stamp custom-made.

In this embodiment, a terminal device 1 used by a user and a seller device 2 (order receiving device) that accepts orders for eyebrow cosmetic stamps are connected via a network 4, and an order (order information) for an eyebrow cosmetic stamp from a user using the terminal device 1 is accepted by the seller device 2, and a system is constructed in which a manufacturer manufactures the user's eyebrow cosmetic stamp based on the order (order information). This makes it possible to realize a custom-made eyebrow cosmetic stamp system that allows users to easily simulate eyebrow makeup (check the finished look) and order eyebrow cosmetic stamps that suit them.

There are several possible variations of the embodiment described above. The variations are described below. Note that the following explanation focuses on the differences from the embodiment described above, and does not include the commonalities between the embodiment and the embodiment.

[Modification 1]
Variation 1 is obtained by adding to the above-mentioned embodiment a configuration (so-called "recommendation function") for extracting facial feature points from the face image acquired by the above-mentioned process of S113 (face image acquisition means) and presenting eyebrow makeup patterns (eyebrow makeup samples) suited to the user's face based on the extracted feature points.

The eyebrow makeup app program (eyebrow makeup app related data) according to the first variant includes an algorithm for recognizing and extracting facial feature points, such as the overall face shape included in a face image, and the positions and shapes of the elements (features) that make up the face, such as the eyes, nose, and mouth. Publicly known facial image feature point extraction techniques can be used as techniques for extracting facial feature points.

In addition, the eyebrow makeup patterns (eyebrow makeup patterns managed in the eyebrow makeup pattern database DB1) stored in the storage unit 11 as data related to the eyebrow makeup app are created according to various facial features (particularly the overall face shape, eye shape, skin color, etc.) and are stored after being classified (grouped) according to those features. In this case, features (feature points) can include, for example, faces such as round, inverted triangular, long, square, oval, square, etc.; eyes such as round, almond, droopy, upturned, single, double, deep-set, etc.; and skin colors such as fair skin, dark skin, etc.

When the processing unit 10 of the terminal device 1 in which the eyebrow makeup app related data according to the first modification is installed acquires a face image by the process of S113, the processing unit 10 then extracts facial feature points contained in the face image acquired in S113. This feature point extraction process is performed by the processing unit 10 functioning as a feature point extraction means.

Next, the processing unit 10 acquires one or more eyebrow makeup patterns corresponding to the extracted facial feature points, i.e., eyebrow makeup patterns suitable for the face from which the feature points were extracted, from among the eyebrow makeup patterns stored in the memory unit 11, based on the extracted facial feature points, and displays an eyebrow makeup sample based on the acquired eyebrow makeup pattern in the secondary display area 12b. This process of displaying the eyebrow makeup sample is performed by the processing unit 10 functioning as a makeup pattern presentation means, and it can be said that the process of S117 described above (makeup pattern display means 105) has a function of presenting eyebrow makeup patterns (recommendation function).

Next, when one of the eyebrow makeup samples (eyebrow makeup patterns) displayed (presented) in the secondary display area 12b is selected according to the facial feature points (S119), the processing unit 10 reads and acquires the eyebrow makeup pattern corresponding to the selected eyebrow makeup sample from the memory unit 11, generates a simulation image with eyebrow makeup applied based on the acquired eyebrow makeup pattern, and displays it in the primary display area 12a (S121 to S125).

According to the above-described first modification, eyebrow makeup patterns (eyebrow makeup samples) that match the facial features of the user are presented, and a simulation image can be generated and displayed using an eyebrow makeup pattern selected from the list, or an eyebrow makeup stamp can be ordered. This makes it possible to improve the functionality of the eyebrow makeup app by adding a function (recommendation function) that recommends eyebrow makeup patterns (eyebrow makeup samples) that are suitable for the user.

The presentation of eyebrow makeup patterns according to facial features (recommendation function) may be done as follows.

For example, information on currently popular eyebrow makeup and/or information on users who have used an eyebrow makeup app to perform eyebrow makeup simulations or order eyebrow makeup stamps (age, order history, facial features, etc.) may be accumulated and trained on an AI (artificial intelligence), various eyebrow makeup patterns may be created based on the learning results, and an eyebrow makeup pattern that corresponds to the facial image acquired in the process of S113 may be acquired and presented from the created eyebrow makeup patterns. In this case, the learning by the AI (artificial intelligence) may be performed, for example, on the sales company device 2 (service providing server) side, and the user may operate the terminal device 1 to access the sales company device 2 (service providing server) and periodically download the latest eyebrow makeup patterns (eyebrow makeup app related data) and install them in the terminal device 1. This makes it possible to present eyebrow makeup patterns that are more suitable for the user.

[Modification 2]
In the second modification, when eyebrow makeup is actually applied to the face in the face image acquired by the process of S113 (face image acquisition means) described above, a configuration is added to the above embodiment to recognize the state of the applied eyebrow makeup and generate order information for the eyebrow makeup stamp based on the recognition result. Note that the second modification can also be applied to the first modification.

The program of the eyebrow makeup app (eyebrow makeup app related data) according to the second modification includes an algorithm for recognizing the state of eyebrow makeup when makeup (eyebrow makeup) is applied to the eyebrow portion of the face in the face image. The state of eyebrow makeup includes at least the shape (contour) and color of the eyebrows (makeup eyebrows) that constitute the eyebrow makeup. As a technology for recognizing the state of eyebrow makeup, a publicly known face image recognition technology can be used. For example, the state of eyebrow makeup can be recognized by identifying the part where makeup is applied (e.g., the eyebrow portion) based on the difference (difference) between the bare skin color and the makeup color in the face image and extracting the shape, color, etc. of the identified part.

When the eyebrow makeup app is started, the processing unit 10 of the terminal device 1 having the eyebrow makeup app related data according to the second modification installed displays a menu screen (not shown) on the display unit 12 of the terminal device 1 to allow the user to select whether to perform eyebrow makeup simulation or eyebrow makeup state recognition. If eyebrow makeup simulation is selected, the processing unit 10 executes the above-mentioned eyebrow makeup app execution process (S100). On the other hand, if eyebrow makeup state recognition is selected, the processing unit 10 executes the eyebrow makeup app execution process 2 (S1000) shown in Figs. 13 and 14. The eyebrow makeup app execution process 2 (S1000) will be described below.

[Eyebrow makeup application execution process 2]
In the eyebrow makeup application execution process 2 (S1000), the processes of S1001 to S1013 shown in Fig. 13 are the same as the processes of S101 to S113 in the above-mentioned embodiment. Note that the face image acquired in S1013 is a face image (a made-up face image) in which actual makeup (at least eyebrow makeup in this case) is applied to the user's face.

In S1015 following S1013, the state of eyebrow makeup applied to the face in the face image acquired in S1013 is recognized (S1015). At this time, the shape and color of the eyebrows (makeup eyebrows) that constitute the eyebrow makeup are recognized. The processing of S1015 is performed by the processing unit 10 functioning as makeup state recognition means.

Next, in S1017, based on the state of eyebrow makeup recognized in S1015, eyebrow makeup stamp information is displayed on the display unit 12 (main display area 12a) of the terminal device 1 (S1017). The processing in S1017 is performed by the processing unit 10 functioning as an eyebrow makeup stamp information display means.

The eyebrow stamp information includes at least information indicating the shape of the eyebrows (eyebrow shape pattern) identified based on the state of eyebrow makeup recognized in S1015, and information indicating the color of the eyebrows (eyebrow color) identified based on the state of eyebrow makeup recognized in S1015. In addition, the eyebrow stamp information may be displayed on the display unit 12 with information (image data) indicating an overall image of an eyebrow stamp having an eyebrow-shaped stamp member corresponding to the shape of the eyebrows identified based on the state of eyebrow makeup recognized in S1015, and information (image data) indicating powder for the eyebrow stamp of the eyebrow color identified based on the state of eyebrow makeup recognized in S1015.

After processing S1017, the processing unit 10 performs processing from S1019 to S1037 shown in FIG. 14. The processing from S1019 to S1037 is basically the same as the processing from S137 to S155 in the embodiment described above (see FIG. 7). However, if it is determined in S1021 of this modified example 2 that a return instruction has been input (YES in S1021), the processing returns to S1001 shown in FIG. 13. In other words, the acquisition of a face image, recognition of the state of eyebrow makeup, etc. are redone.

In addition, in S1025 of this second modified example, order information for the eyebrow makeup stamp is generated based on the eyebrow makeup stamp information displayed in the process of S1017, i.e., the state of eyebrow makeup recognized in the process of S1015. This makes it possible to order an eyebrow makeup stamp based on eyebrow makeup actually applied to the user's face (eyebrow area). In this case, the user can actually apply eyebrow makeup using, for example, an eyebrow pencil or mascara, and then order and obtain an eyebrow makeup stamp that allows the user to apply the eyebrow makeup as a stamp. The process of S1025 is performed by the processing unit 10 functioning as a second order information generating means.

According to the above-described second modification, if the face in the face image acquired by the process of S1013 has actual eyebrow makeup applied, the state of the applied eyebrow makeup is recognized, and order information for the eyebrow makeup stamp is generated based on the recognized state of the eyebrow makeup, and the order information is sent to the sales business device 2 (the order destination for the eyebrow makeup stamp). This allows the user to order (custom-made) an eyebrow makeup stamp of their choice based on the image of the face with actual eyebrow makeup applied (made-up face image).

In the second modification, the eyebrow makeup stamp information is displayed based on the state of eyebrow makeup recognized by the process of S1015 (makeup state recognition means), and the order information for the eyebrow makeup stamp is generated (S1017, S1025). However, based on the state of eyebrow makeup recognized by the process of S1015 (makeup state recognition means), an eyebrow makeup pattern corresponding to the state of eyebrow makeup may be obtained from the storage unit 11 (see S121). In this case, a simulation image based on the obtained eyebrow makeup pattern is generated and displayed, making it possible to perform a simulation based on eyebrow makeup actually applied to the user's face (eyebrow area) (see S123, S125), and it becomes possible to order an eyebrow makeup stamp based on the results of the simulation (see S137, S143, S147, S149). In addition, it is also possible to adjust the eyebrow image as in the previous embodiment (see S129, S131).

The configuration according to this modified example 2, which is added to the above-mentioned embodiment, can also be implemented independently of the embodiment. In other words, it is possible to have an eyebrow makeup app that does not have an eyebrow makeup app execution process (S100) but has an eyebrow makeup app execution process 2 (S1000) (another example of modified example 2). The inventions related to such another example of modified example 2 (related inventions) are shown below.

(Related invention 1)
The system includes a user device (e.g., a terminal device 1) used by a user, and an order receiving device (e.g., a sales company device 2) that receives an order for a cosmetic product (e.g., an eyebrow cosmetic stamp) from the user,
A cosmetics custom-made system configured so that the user device and the order receiving device can be connected via a network,
The user device comprises:
A facial image acquisition means for acquiring an image including a face of a user with makeup (e.g., eyebrow makeup);
a makeup condition recognition means for recognizing a state of makeup applied to a face included in an image acquired by the face image acquisition means;
an order information generating means for generating order information for cosmetics based on the state of makeup recognized by the makeup state recognizing means;
an order information sending means for sending the order information generated by the order information generating means to a destination of an order for cosmetics,
the order receiving device includes an order information receiving means for receiving the order information transmitted by the order information transmitting means,
A custom-made cosmetics system, characterized in that cosmetics for the user are manufactured based on the order information received by the order information receiving means.

(Related invention 2)
In the cosmetics custom-made system of the above-mentioned related invention 1,
The cosmetic custom-made system is characterized in that the cosmetic is an eyebrow makeup stamp that can apply makeup to the eyebrows by pressing it against the eyebrows of the face.

[Modification 3]
The third modification adds to the above-mentioned embodiment a configuration for calculating the actual size of the face included in the face image acquired by the process of S113 (face image acquisition means) and a configuration for including information on the calculated face size in the order information for the eyebrow cosmetic stamp. Note that the third modification can be applied to both the first and second modifications.

The program of the eyebrow makeup app (eyebrow makeup app related data) according to the third variation includes an algorithm for calculating the actual size of the face in the face image acquired in the process of S113. The size of the face can be calculated, for example, by dividing it into the vertical size and the horizontal size. Specifically, the distance of a straight line connecting the top of the head and the tip of the chin can be calculated as the vertical size of the face (vertical size), and the distance of a straight line connecting the left and right temples can be calculated as the horizontal size of the face (horizontal size).

When the processing unit 10 of the terminal device 1 in which the eyebrow makeup app related data according to the third modification example is installed acquires a face image by the process of S113, the processing unit 10 then calculates the actual size of the face included in the face image acquired in S113. This calculation process is performed by the processing unit 10 functioning as a face size calculation means.

The processing unit 10 then retains the calculated information regarding the face size (also referred to as "face size information") until an order instruction is input by the user, and when an order instruction is input (YES in S137), the processing unit 10 includes the face size information in the order information generated based on the input of the order instruction (S143). After this, when a transmission instruction is input by the user (YES in S147), the processing unit 10 transmits the order information including the face size information to the ordering destination of the eyebrow cosmetic stamp (sales business device 2) (S149).

According to the above-described third modification, the actual size of the face included in the face image acquired by the process of S113 (face image acquisition means) is calculated, and information regarding the calculated size is included in the order information for the eyebrow cosmetic stamp. This makes it possible to order (custom-made) an eyebrow cosmetic stamp suited to the size of the user's face.

Although the above describes examples and variations of the present invention, the present invention is not limited to these.

For example, the simulation image SM displayed on the display unit 12 (main display area 12a) of the terminal device 1 may be displayed as an image (3D image) that can be rotated in three dimensions based on a touch operation on the display panel. In this case, the shape of the eyebrows can be adjusted to match the contours of the face contained in the simulation image SM. In addition, by rotating the simulation image SM within the display unit 12 to change the direction of the face, the shape of the eyebrows can be changed accordingly. In this way, a 3D simulation of eyebrow makeup is possible, making it possible to provide a highly realistic eyebrow makeup simulation.

Furthermore, the simulation images generated during the execution of the eyebrow makeup simulation by the eyebrow makeup app, and data such as the eyebrow makeup patterns (eyebrow shape patterns, eyebrow color patterns) on which the simulation images were generated may be stored as created information in the storage unit 11 of the terminal device 1 (created information storage means). In this way, it becomes possible to re-generate simulation images generated (created) in the past based on the created information, and it becomes possible to check the results of the eyebrow makeup simulation again or order eyebrow makeup stamps based on the re-generated simulation images. This makes it possible to improve the convenience of the eyebrow makeup app.

In addition, the system (e.g., a service providing server) may collect statistics on eyebrow makeup patterns selected by users based on the order history of eyebrow makeup stamps using the eyebrow makeup app (makeup pattern statistical means), and based on the statistical results, information on the selection tendency of eyebrow makeup patterns (also referred to as "selection tendency information") may be displayed on the display unit 12 of the terminal device 1. The selection tendency information may be, for example, information indicating eyebrow makeup patterns with relatively high selection rates, information indicating the relationship between the selected eyebrow makeup pattern and the user demographic (age, occupation, etc.), information indicating the selection rate (sales ranking) of eyebrow makeup patterns in a ranking format, etc. In this way, it is possible to inform users of the selection tendency of eyebrow makeup patterns (popular eyebrow makeup patterns, etc.). This makes it possible to enhance the functions of the eyebrow makeup app.

In addition, in the above-mentioned embodiment and modified example, eyebrow makeup app related data is installed in the terminal device 1 used by the user, and the processing unit 10 of the terminal device 1 performs processes related to the eyebrow makeup app, such as the eyebrow makeup app execution process (S100) and the eyebrow makeup app execution process 2 (S1000), but the processes may be performed on the sales company device 2 (service providing server) side.

Specifically, for example, when a user uses the terminal device 1 to perform an eyebrow makeup simulation or to order eyebrow makeup stamps, the user connects to the sales company device 2 via the network 4 and accesses a website (also referred to as an "eyebrow makeup related site") that provides services related to eyebrow makeup (eyebrow makeup simulation, sales of eyebrow makeup stamps, etc.). Then, based on an instruction from the terminal device 1 that has accessed the eyebrow makeup related site, the processing unit 20 of the sales company device 2 performs the above-mentioned eyebrow makeup application execution process (S100) and eyebrow makeup application execution process 2 (S1000), etc., and returns the execution results of each process to the terminal device 1, so that each process display, etc. (screens shown in Figures 9 and 10, etc.) described in the above embodiment are displayed on the display unit 12 of the terminal device 1.

With this configuration, as long as the communication environment (network environment) is in place, it is possible to perform eyebrow makeup simulations and order eyebrow makeup stamps, as in the above-mentioned embodiments, without having to install eyebrow makeup app-related data on the terminal device 1.

In addition, in the above-mentioned embodiment and modified example, an example was described in which the present invention was applied to a device and system targeted at eyebrow makeup, but the present invention can also be applied to devices and systems targeted at makeup applied to facial parts other than eyebrows, such as the mouth and cheeks. Examples of makeup patterns other than eyebrows include patterns based on lipstick makeup (lip makeup patterns) and patterns based on cheek makeup (cheek makeup patterns). Also, examples of cosmetics other than eyebrow makeup stamps include lipstick stamps and cheek stamps. The present invention can also be applied to cosmetics other than stamp-type cosmetics.

1 Terminal device, 2 Seller device, 3 Manufacturer device, 4 Network, 10 Processing unit, 11 Memory unit, 12 Display unit, 13 Operation input unit, 14 Communication unit, 15 Camera (imaging means), 16 Battery, 20 Processing unit, 21 Memory unit, 22 Display unit, 23 Operation unit, 24 Communication unit, 30 Processing unit, 31 Memory unit, 32 Display unit, 33 Operation unit, 34 Communication unit, 101 Instruction input detection means, 102 Imaging means, 103 Face image acquisition means, 104 Face image display means, 105 Makeup pattern display means, 106 Makeup pattern acquisition means, 107 Simulation image generation means, 108 Simulation image display means, 109 Guide display means, 110 Makeup pattern adjustment means, 111 Order information generation means, 112 Transmission means, 500 Eyebrow makeup stamp, 501 Main body, 501a Knob, 502 stamp member, 502a stamp surface.

Claims (3)

A face image acquisition means for acquiring an image including a face;
A makeup pattern acquisition means for acquiring an eyebrow makeup pattern from a plurality of eyebrow makeup patterns prepared in advance;
a simulation image generating means for generating a simulation image in which eyebrow makeup based on the eyebrow makeup pattern acquired by the makeup pattern acquiring means is applied to an eyebrow portion of a face included in the image acquired by the face image acquiring means;
a simulation image display means for displaying the simulation image generated by the simulation image generating means;
an order information generating means for generating order information for an eyebrow makeup stamp that can be pressed against an eyebrow part of a face to apply eyebrow makeup, based on an eyebrow makeup pattern included in the simulation image displayed by the simulation image display means;
a transmission means for transmitting the order information generated by the order information generating means to a destination of an order for the eyebrow makeup stamp . a makeup state recognition means for recognizing a state of eyebrow makeup applied to a face included in an image acquired by the face image acquisition means when the face includes actual eyebrow makeup;
a second order information generating means for generating order information for the eyebrow makeup stamp having specifications corresponding to the actual eyebrow makeup based on the state of the eyebrow makeup recognized by the makeup state recognizing means;
and a second transmission means for transmitting the order information generated by the second order information generation means to an order destination of the eyebrow makeup stamp.
The makeup support device according to claim 1 . A program for causing a computer to function as each of the means of the makeup support device according to claim 1 or 2.

Images