JP7536348B2 - Massage evaluation method, massage evaluation device, and massage evaluation program - Google Patents

Description

The present invention relates to a massage evaluation method, a massage evaluation device, and a massage evaluation program.

2. Description of the Related Art Conventionally, a massage evaluation method is known in which a massage effect is confirmed after the massage is given to a user.
For example, Patent Document 1 discloses a method for determining the strain distribution of facial skin using images taken before and after a cosmetic treatment on the face, such as a facial massage, and for quantitatively analyzing the effects of the cosmetic treatment using this strain distribution as an evaluation index.

JP 2017-9598 A

However, many conventional massages are difficult to achieve effective results. Furthermore, conventional massage evaluation methods evaluate changes in the shape of the entire face, making it difficult to confirm partial changes.

The present invention aims to provide a method for evaluating a massage that can check partial changes in the face after an effective massage has been administered.

In order to solve the above problems, the massage evaluation method according to the present invention includes a functional agent application step of applying a functional agent having at least one of the functions of promoting blood circulation, promoting fat melting, or promoting fascial relaxation to the surface of the user's face, a lymph node stimulation step of performing a lymph node massage that physically stimulates the part of the user's face where the lymph nodes are located after the functional agent application step, a fascial stimulation step of performing a fascial massage that physically stimulates the part of the user's face where the fascia is located after the functional agent application step, and an image processing step of confirming the effects of the lymph node massage and fascial massage using an image processing system that includes a face shape evaluation unit that evaluates changes in the proportions of the user's face based on the positions of the user's skeleton, muscles, and fat from image data of the user's face, and a display processing unit that outputs information indicating the changes in the proportions of the user's face evaluated by the face shape evaluation unit.

In addition, in the lymph node stimulation step, a beauty device having a pin-shaped pressing part biased by a biasing member may be used to press the part of the user's face where the lymph nodes are located with the pressing part.

In addition, in the fascia stimulation step, a beauty device having a pin-shaped pressing part biased by a biasing member may be used to press the part of the user's face where the fascia is located with the pressing part.

The functional agent may also have the ingredients glaucine, okra seed extract, and niacinamide.

The functional agent may further contain a component of glecon hesperidin.

The massage evaluation device of the present invention also includes a functional agent application step of applying a functional agent having at least one of the functions of promoting blood circulation, promoting fat melting, or promoting fascial relaxation to the surface of the user's face, a lymph node stimulation step of physically stimulating the part of the user's face where the lymph nodes are located after the functional agent application step, and a fascia stimulation step of physically stimulating the part of the user's face where the fascia is located after the functional agent application step, a facial shape evaluation unit that evaluates changes in the proportions of the user's face based on the positions of the user's skeleton, muscles, and fat from image data of the user's face after the functional agent application step, and a display processing unit that outputs information indicating the changes in the proportions of the user's face evaluated by the facial shape evaluation unit.

The massage evaluation program of the present invention causes a computer to realize a functional agent application step of applying a functional agent having at least one of the functions of promoting blood circulation, promoting fat melting, or promoting fascial relaxation to the surface of the user's face, a lymph node stimulation step of physically stimulating the part of the user's face where the lymph nodes are located after the functional agent application step, and a fascia stimulation step of physically stimulating the part of the user's face where the fascia is located after the functional agent application step, a facial shape evaluation function of evaluating changes in the proportions of the user's face based on the positions of the user's skeleton, muscles, and fat from image data of the user's face after the massage, and a display processing function of outputting information showing the changes in the proportions of the user's face evaluated by the facial shape evaluation unit.

In the massage evaluation method of the present invention, by carrying out the functional agent application step, the components contained in the functional agent are applied to the user's face, which can provide at least one of the following effects: blood circulation promotion effect, fat melting effect, and fascia relaxation effect, thereby enhancing the effect of the massage.
In addition, in the lymph node stimulation step and the fascia stimulation step, the lymph nodes and fascia are stimulated, thereby stimulating the lymph nodes, promoting the flow of lymphatic fluid, and promoting the metabolism of waste products interposed between the muscle and fascia, thereby effectively eliminating the attachment between the fascia and the muscle.

After performing the massage in this manner, an image processing step is performed using an image processing system to evaluate changes in the proportions of the user's face from image data captured of the user's face. This makes it possible to check partial changes in the face after an effective massage has been performed.

1 is a schematic diagram showing an example of the configuration of an image processing system according to an embodiment of the present invention; FIG. 2 is a diagram showing a state in which a user is using the mirror device shown in FIG. 1 . FIG. 2 is a block diagram showing an example of the configuration of the image processing system shown in FIG. 1 . FIG. 2 is a diagram illustrating a processing flow of the entire image processing system. FIG. 4 is a diagram illustrating evaluation items that can be evaluated by the skin condition evaluation unit. 4 is a block diagram showing an example of the configuration of a skin condition evaluation unit. FIG. 13 is a diagram showing an example of an evaluation table that serves as an evaluation criterion when a skin condition evaluation unit detects blemishes. FIG. 8A is a diagram showing an example of a processing result in the image processing system, and FIG. 8B is a partially enlarged view of FIG. 8A. 2 is a block diagram showing a configuration example of the mobile terminal shown in FIG. 1 . FIG. 13 is a diagram showing a processing flow in a skin condition evaluation unit. 11A and 11B are diagrams illustrating another example of a processing result in the image processing system. 13 is a diagram showing an example of an evaluation table that serves as an evaluation criterion when a skin condition evaluation unit detects pores. FIG. 13 is a diagram showing an example of an evaluation table that serves as an evaluation criterion when the skin condition evaluation unit detects dark circles, redness, and blemishes. FIG. 13 is a diagram showing an example of an evaluation table that serves as an evaluation criterion when a skin condition evaluation unit detects texture, fine lines, pores, and wrinkles. FIG. 2 is a block diagram showing an example of the configuration of a face shape evaluation unit shown in FIG. 1 . 14A and 14B are diagrams showing an example of each vertex recognized by the vertex recognition unit shown in FIG. 13, in which (a) is a front view and (b) is a side view of the imaging data. FIG. 11 is a diagram showing a processing flow in a face shape evaluation unit. 13 is a schematic diagram showing a process in which the vertex recognition unit recognizes vertices on a cheek. FIG. FIG. 11 is a diagram showing an example of the contents displayed by a display processing unit. 13A and 13B are diagrams illustrating other examples of the display contents by the display processing unit. FIG. 2A is an external view of the beauty device used in the massage step, and FIG. FIG. 2 is a diagram showing steps (a) to (d) of the first step in a first example of a massage method. A figure showing each step (a) to (d) of the second step in the first example of the massage method. A diagram showing each step (a) to (c) of the third step in the first example of the massage method. A figure showing each step (a) to (d) of the fourth step in the first example of the massage method. A figure showing each step (a) to (d) of the fifth step in the first example of the massage method. A figure showing each step (a) to (c) of the sixth step in the first example of the massage method. A diagram showing steps (a) to (d) of the first step in a second example of a massage method. A diagram showing each step (a) to (d) of the second step in the second example of the massage method. A diagram showing each step (a) to (d) of the third step in the second example of the massage method. A figure showing each step (a) to (b) of the first step in a third example of a massage method. A figure showing each step (a) to (b) of the second step in the third example of the massage method. A figure showing each step (a) to (b) of the third step in the third example of the massage method. A figure showing each step (a) to (b) of the fourth step in the third example of the massage method. A figure showing each step (a) to (b) of the first step in a fourth example of a massage method.

(Overall composition)
An image processing system 100 according to an embodiment of the present invention will be described with reference to the drawings.
Fig. 1 is a schematic diagram showing an example of the configuration of an image processing system 100 according to an embodiment of the present invention. Fig. 2 is a diagram showing a state in which a user 5 is using the mirror device 2 shown in Fig. 1.

As shown in FIG. 1, the image processing system 100 is a system that performs image processing on image data capturing an image of the face of a user 5 in order to assist the user 5 in promoting beauty, and provides advice regarding beauty promotion by displaying the results to the user 5.
When actually used by a user 5, the user 5 sits in front of the mirror device 2 as shown in FIG. 2, and various analyses described below are performed using image data of the face of the user 5 captured by the imaging unit 21.

As shown in FIG. 1, the image processing system 100 includes a mirror device 2 and an image analysis device 1 that are connected to each other via a network 3. In the illustrated example, multiple mirror devices 2 are provided. The mirror devices 2 include a store terminal 2A and a personal terminal 2B.

The store terminal 2A is a terminal used in a store that provides beauty promotion guidance to the user 5, and can be used by the user 5 when he or she visits the store.
The personal terminal 2B is a terminal that is assumed to be used mainly by the user 5 at home, and can be used by the user in his/her daily life, for example, when grooming himself/herself or before going to bed.
In the illustrated example, a mobile terminal 4 of a user 5 is connected to the network 3. The mobile terminal 4 is connected to the network 3 via wireless communication.

For example, the image processing system 100 of the present invention uses a shop terminal 2A installed in a shop that provides beauty-related services, such as an esthetic salon, to capture an image of the face of the user 5 and display the evaluation results.
The image processing system 100 may be used, for example, to suggest measures that the user 5 should take in the future to improve the condition of his or her facial skin.

Furthermore, the image processing system 100 can display the results of daily imaging and evaluation of the face of the user 5 using a personal terminal 2B provided at the user's home.
That is, the image processing system 100 may be used under the control of a store operator 6 or may be used by the user 5 himself/herself.

FIG. 3 is a block diagram showing an example of the configuration of the image processing system 100. As shown in FIG.
3, the mirror device 2 includes a display unit 20, an imaging unit 21, and a communication unit 22. The display unit 20 is provided on the surface of the mirror device 2, and serves as a mirror surface and is a display capable of displaying data. That is, the display unit 20 displays various analysis results by the image analyzing device 1, which will be described later.

The area in which data is displayed and the area which becomes a mirror surface can be adjusted in the display unit 20. That is, for example, the entire surface of the display unit 20 may be a mirror surface, or data may be displayed on the entire surface of the display unit 20.
Moreover, data can be displayed on the mirror surface of the display unit 20, or half of the area of the display unit 20 may be made a mirror surface and data may be displayed on the remaining half.

The imaging unit 21 captures an image in front of the display unit 20. The imaging unit 21 is not particularly limited as long as it is a device that can capture an image of the face of the user 5 and acquire image data when the user 5 is located in front of the display unit 20. The imaging unit 21 has an imaging element such as a CMOS or a CCD.
The communication unit 22 transmits the image data captured by the imaging unit 21 to the communication unit 23 of the image analysis device 1 .

When the imaging unit 21 captures an image of the face of the user 5, the display unit 20 displays the outline of image data of the face of the user 5 captured in the past.
That is, when capturing images habitually, it is preferable that the position of the face of the user 5 does not change significantly with respect to the imaging unit 21. For this reason, the display unit 20 displays the outline of image data of the face of the user 5 captured in the past so that the user 5 positioned in front of the imaging unit 21 can align the position of his or her face with the imaging unit 21. At this time, an image of the face of the user 5 captured by the imaging unit 21 is displayed on the display screen, and the imaging unit 21 acquires the image data after the user 5 aligns the position of his or her face.

The image data acquired by the mirror device 2 may be 2D data or 3D data. In this embodiment, a configuration in which the imaging unit 21 acquires image data as 2D data will be described.
The mirror device 2 may be, for example, a 3D camera in which a plurality of image capturing units 21 are arranged at intervals, or may be configured to include one image capturing unit 21 and a distance sensor.

The communication unit 22 of the mirror device 2 communicates with the communication unit 23 of the image analyzing device 1 via the network 3 shown in FIG.
The network 3 is a network for connecting the mirror device 2, the image analyzing device 1, and the mobile terminal 4 to one another, and is, for example, a wireless network or a wired network.

Specifically, network 3 may be a wireless LAN (WLAN), a wide area network (WAN), ISDNs (integrated service digital networks), wireless LANs, LTE (long term evolution), LTE-Advanced, fourth generation (4G), fifth generation (5G), CDMA (code division multiple access), WCDMA (registered trademark), Ethernet (registered trademark), etc.

In addition, network 3 is not limited to these examples, and may be, for example, a Public Switched Telephone Network (PSTN), Bluetooth (registered trademark), Bluetooth Low Energy, an optical fiber line, an Asymmetric Digital Subscriber Line (ADSL) line, a satellite communication network, or any other network.

The network 3 may be, for example, NB-IoT (Narrow Band IoT) or eMTC (enhanced Machine Type Communication). Note that NB-IoT and eMTC are wireless communication methods for IoT, and are networks that enable long-distance communication at low cost and with low power consumption.

Network 3 may also be a combination of these examples. Network 3 may also include a plurality of different networks that combine these examples.
For example, the network 3 may include a wireless network based on LTE and a wired network such as an intranet, which is a closed network.

(Image analysis device)
As shown in Figure 3, the image analysis device 1 includes a skin condition evaluation unit 30, a facial shape evaluation unit 50, a communication unit 23, a future prediction unit 60, an evaluation result providing unit 61, a confirmation content reporting unit 62, a memory unit 63, and a user identification unit 64.
The image analysis device 1 analyzes image data of the face of the user 5 captured by the imaging unit 21 .

The skin condition evaluation unit 30 is a functional unit that evaluates the skin condition of the user 5. The configuration of the skin condition evaluation unit 30 will be described in detail later with reference to FIG.
The facial shape evaluation unit 50 is a functional unit that evaluates changes in facial promotion of the user 5. The configuration of the facial shape evaluation unit 50 will be described in detail later with reference to FIG.

The future prediction unit 60 uses at least one of the skin health condition of the user 5 stored in the skin condition evaluation unit 30 and the facial shape evaluation of the user 5 stored in the facial shape evaluation unit 50 to make a future prediction of the face of the user 5. The future prediction unit 60 generates composite data of the face of the user 5, referring to the past history, to show what effects can be expected if the proposed measures are continuously implemented in the future, and displays the composite data on the display unit 20 of the mirror device 2.

The evaluation result providing unit 61 analyzes the level of beauty promotion of the user 5 from the image data and provides the analysis result to the personal terminal 2B. The analysis result here refers to each analysis result performed by the skin condition evaluation unit 30 and the facial shape evaluation unit 50 described later.
Here, the degree of beauty promotion refers to the degree of progress in the process of aiming for improvement in beauty items such as skin condition and facial promotion from the current state to the future.

The confirmation content reporting unit 62 reports to the store terminal 2A confirmation history information regarding the confirmation history confirmed by the user 5 using the personal terminal 2B, among the analysis results provided to the user 5. The confirmation history information can be confirmed by obtaining a data log, which is a history of the usage status of the personal terminal 2B.

The confirmation content reporting unit 62 counts the number of times the user 5 has confirmed each analysis result performed by the skin condition evaluation unit 30 and the facial shape evaluation unit 50 within a predetermined period, and reports the count to the store terminal 2A. The same content may also be reported to the personal terminal 2B.
Furthermore, the confirmation content reporting section 62 reports to the store terminal 2A at least one of the maintenance method and the maintenance product displayed by the display section 20. The maintenance method and the maintenance product will be described in detail later.

The storage unit 63 stores the contents confirmed by the user 5 using the store terminal 2A when visiting a store together with the confirmation history information. That is, the contents confirmed by the user 5 are stored in the storage unit 63 by tallying up the contents confirmed by the user 5 using the personal terminal 2B at home and the contents confirmed by the user 5 using the store terminal 2A at the store.
The storage unit 63 also stores face data of each user 5 and the ID of the user 5 .

The user identification unit 64 identifies the user 5 who is using the device. The user identification unit 64 may accept input of the ID of the user 5 from, for example, a keyboard on a touch panel displayed on the display unit 20, or may identify the user 5 by, for example, referring to the captured image data of the face of the user 5 in the storage unit 63.

The overall processing flow of the image processing system 100 will now be described with reference to Fig. 4. Fig. 4 is a diagram showing the overall processing flow of the image processing system 100.
4, the user first performs user authentication (S10: user authentication step). The user 5 may input a user ID using a touch panel keyboard displayed on the display unit 20, or the user identification unit 64 may refer to the information of the user 5 stored in the storage unit 63 based on the image data of the face captured by the imaging unit 21 to perform the user authentication.

Next, the skin condition evaluation unit 30 evaluates the skin condition of the user 5 (S11: skin condition evaluation step). This will be described later.
Next, the facial shape evaluation unit 50 evaluates the change in the facial promotion of the user 5 (S12: facial shape evaluation step). The details of this will be described later.

Next, the evaluation result providing unit 61 transmits each analysis result to the personal terminal 2B, thereby providing the evaluation result to the user 5 (S13: evaluation result providing step). The contents of each analysis result will be described later.
Next, the confirmation content reporting unit 62 reports the contents confirmed by the user 5 to the store terminal 2A by transmitting the contents confirmed by the user 5 to the store terminal 2A (S14: confirmation content reporting step). This allows the store staff to confirm what beauty-related items the user 5 is interested in.

(Skin Condition Evaluation Department)
Next, a detailed description will be given of the configuration of the skin condition assessment unit 30. The skin condition assessment unit 30 assesses the skin health condition of the user 5 based on the skin color of the user 5 from the image data.
The skin condition evaluation unit 30 detects abnormal parts on the skin of the user 5 based on the skin color of the user 5, and displays the number of abnormal parts together with the past history. The number of abnormal parts can be grasped by counting the number of parts that exceed a preset threshold value for the hue of the skin color, for example.

Fig. 5 is a diagram illustrating evaluation items that can be evaluated by the skin condition evaluation unit 30. Fig. 5 shows face regions that are detected by the skin condition evaluation unit 30 for each abnormality item within the image captured by the imaging unit 21.
As shown in Fig. 5, the skin condition evaluation unit 30 has a function of detecting abnormalities in the skin condition. The skin condition evaluation unit 30 can detect abnormalities in the skin condition such as fine lines, wrinkles, age spots, enlarged pores, rough skin (texture), redness, and dark circles under the skin. The skin condition evaluation unit 30 sets each area shown in Fig. 5, and performs detection processing on each of these areas as detection areas corresponding to each abnormality item to be detected.

Next, the configuration of the skin condition evaluation unit 30 will be described in detail with reference to Fig. 6. Fig. 6 is a block diagram of the skin condition evaluation unit 30. In this description, of the multiple evaluation functions possessed by the skin condition evaluation unit 30, evaluation of blemishes will be described as an example.
6, the skin condition evaluation unit 30 includes a device-side communication unit 31, a data storage unit 32, a data processing unit 33, and a device-side display unit 34. The skin condition evaluation unit 30 is an information processing device that analyzes the facial skin condition of the user 5 from image data obtained by capturing an image of the face of the user 5.

The device-side communication unit 31 is a communication interface that transmits and receives various types of data via the network 3. The various types of data include image data, processing data, and improvement data. In other words, the device-side communication unit 31 functions as a receiving unit that receives image data transmitted from the communication unit 22 of the mirror device 2.

Here, the image data refers to data obtained by capturing an image of the face of the user 5 using the mirror device 2. The processed data refers to data obtained by identifying and marking the positions of blemishes in the image data using an evaluation result display unit 33C, which will be described later.
The improvement data refers to data that is displayed to the user 5 by the improvement data generating unit 33F (described later) by changing the hue of the pigmentation abnormality area based on an assumption that the pigmentation in the pigmentation abnormality area has been improved in the image data.

The data storage unit 32 has a function of storing various control programs required for the operation of the data processing unit 33 and various data received from the outside by the device side communication unit 31. The data storage unit 32 also has an evaluation table that serves as a reference when a skin color evaluation unit 33A, which will be described later, evaluates the skin color of the user 5.
The data storage unit 32 is realized by various storage media such as a HDD, an SSD, or a flash memory.

By executing the control program stored in the data storage unit 32, the data processing unit 33 realizes each function to be realized by the image processing system 100. The functions mentioned here include a skin color evaluation function, a pigmentation identification function, an evaluation result display function, a depth estimation function, a measure proposal function, and an improvement data generation function.
The device-side display unit 34 is a monitor device that displays the details of the operation of the image analyzing device 1 and the results of processing.

The data processing unit 33 is a computer that controls each part of the image analyzing device 1, and may be, for example, a central processing unit (CPU), a microprocessor, an ASIC, an FPGA, or the like.
It should be noted that the data processing unit 33 is not limited to these examples, and may be any computer that controls each part of the image analyzing device 1.

The data processing unit 33 includes a skin color evaluation unit 33A, an abnormal skin identification unit 33B, an evaluation result display unit 33C, a depth estimation unit 33D, a measure proposal unit 33E, and an improvement data generation unit 33F.
When detecting blemishes, the skin color evaluation unit 33A uses image data obtained by capturing an image of the skin of the user 5 to divide an arbitrary location of the skin of the user 5 into a plurality of stages.

The multiple stages are preset categories for classifying the skin condition of the user 5, and are expressed, for example, as stage 1 to stage 4, with the higher stage number indicating that the pigmentation abnormality is occurring deeper in the skin and the severity of the pigmentation abnormality. Note that separate groups of stages may be prepared for races with different skin colors.

The skin color evaluation unit 33A classifies the skin color of the user 5 into a plurality of stages according to the skin hue value (for example, RGB value). This will be described in detail with reference to FIG.
7 is an example of an evaluation table that serves as an evaluation criterion for the skin color evaluation unit 33A when detecting blemishes. Note that the hue value is not limited to the RGB value, but may be a CMYK value or other index value.

In the example of the evaluation table shown in FIG. 7, the classification of the hue and stage for the type of pigment spot among the pigmentation abnormalities is described. The hue can be expressed by RGB values, for example. That is, in this figure, the hue is expressed by color, but the corresponding RGB value data may also be provided.

For example, as shown in Figure 7, in the case of senile lentigo, the pigmentation abnormality tends to occur on the cheekbones, and when it is dark brown to brown, it is judged to be at stage 1. When the pigmentation abnormality is light dark brown to light brown, it is judged to be at stage 2. When the pigmentation abnormality is light dark brown to light brown, it is judged to be at stage 3.

Next, in the case of post-inflammatory hyperpigmentation, which tends to occur all over the face, if the pigmentation abnormality is brown to brown, it is judged to be in stage 1. If the pigmentation abnormality is light brown to light brown, it is judged to be in stage 2.
Furthermore, if the color is light brown to light brown, it is judged to be at stage 3. If the color is blue to gray, it is judged to be at stage 4. Here, the type of pigment spot is judged by the operator 6 based on the position and appearance of the pigmentation abnormality.

It should be noted that this evaluation table is merely an example, and a different evaluation table can be used as the evaluation table for evaluating blemishes.
Furthermore, the skin color evaluation unit 33A changes the evaluation table to be referred to depending on the type of skin abnormality (fine lines, wrinkles, age spots, enlarged pores, rough skin, redness, or dark circles) (see FIGS. 13 and 14). This will be described later.

The skin abnormality identification unit 33B identifies an area on the skin of the user 5 where a pigmentation abnormality, including spots caused by pigmentation, has occurred based on the stages classified by the skin color evaluation unit 33A. Here, pigmentation refers to a pigmentation abnormality in the skin caused by the accumulation of black melanin in the epidermis.

Here, we will explain the mechanism by which pigmentation abnormalities such as age spots (pigment spots) occur on the skin. When the skin is exposed to stimuli such as ultraviolet rays, melanocytes (melanin-forming cells), an internal tissue of the skin, produce black melanin. This black melanin has the role of protecting the skin, but in healthy skin, it is excreted from the body over time.

On the other hand, when the skin's metabolic cycle is disrupted or when a large amount of melanin is produced, some of the melanin is not excreted from the body and remains and accumulates in the epidermis. Among these pigment abnormalities, those caused by inflammation or ultraviolet rays are called pigmentation or pigment spots (spots).

The evaluation result display unit 33C displays the position of the image data for each stage classified by the skin color evaluation unit 33A. The evaluation result display unit 33C also marks the pigmentation abnormality identified by the skin abnormality identification unit 33B. This generates processed data in which the image data is displayed and marked to indicate the blemish position. The marking is performed by applying a mark of a color set for each stage to a portion whose hue corresponds to any of stages 1 to 4.
Furthermore, the evaluation result display unit 33C can display and mark the positions of multiple image data captured at different times for the same user 5 for each stage, and display them side by side.

An example of the processed data will now be described with reference to Fig. 8. Fig. 8(a) is a diagram showing an example of the processing result in the image processing system 100, and Fig. 8(b) is a partially enlarged diagram of Fig. 8(a). As shown in Fig. 8(a) and Fig. 8(b), the positions of a plurality of blemishes are specified for any position on the skin of the face of the user 5. Then, at each position, a pigmentation abnormality is represented by marking. Some of these markings cannot be visually identified.
8(a) and 8(b), the lightest mark M1 indicates stage 1, the darkest mark M3 indicates stage 3, and the medium-shape mark M2 indicates stage 2. No stage 4 marks are visible in this figure.

The depth estimation unit 33D estimates the depth from the skin surface of the pigmentation abnormality in the skin of the user 5 based on the hue value of the pigmentation abnormality identified by the skin abnormality identification unit 33B. Here, it is generally known that the depth from the skin surface of the location where the pigmentation abnormality occurs varies depending on the hue of the blemish or pigmented spot.

For example, in the evaluation table for age spots (pigment spots) shown in Fig. 7, pigmentation abnormalities corresponding to stage 1 are determined to have occurred in the upper layer of the epidermis, and pigmentation abnormalities corresponding to stage 2 are determined to have occurred in the middle layer of the epidermis, while pigmentation abnormalities corresponding to stage 3 are determined to have occurred in the lower layer of the epidermis, and pigmentation abnormalities corresponding to stage 4 are determined to have occurred from the lower layer of the epidermis to the dermis layer.
Here, the upper side refers to the side of the inside of the skin that faces the surface, and the lower side refers to the side of the inside of the skin that faces the inside of the body. The criteria for determining such color and depth can be set arbitrarily.

The measure suggestion unit 33E proposes measures to promote the improvement of pigmentation based on the depth of the pigmentation abnormality estimated by the depth estimation unit 33D. The measures include the use of the beauty device 70, the use of an introduction beauty serum, the use of a carbonic acid pack, UV care, etc. The appropriate measure is determined based on the depth of the pigmentation abnormality. In addition, for pigmentation abnormalities formed deep inside the skin, it may be suggested that the patient be examined by a medical institution.

The improvement data generating unit 33F assumes a state in which pigmentation in the pigmentation abnormality area has been improved for the image data, changes the hue of the pigmentation abnormality area, and displays this to the user 5. In other words, it has a function of visually expressing what effect will be obtained when the proposed measures are taken for a certain period of time.

The improvement data generating unit 33F uses, for example, past history data to estimate how the pigmentation abnormality will change if the measure is implemented for a certain period of time from a similar state. Based on this estimation, the improvement data generating unit 33F generates improvement data from the image data.

Next, the configuration of the portable terminal 4 will be described with reference to Fig. 9. Fig. 9 is a block diagram showing an example of the configuration of the portable terminal 4.
The mobile terminal 4 includes a terminal communication unit 41 , a terminal memory unit 42 , a terminal processing unit 43 , a camera 44 , and a terminal display unit 45 .

The terminal-side communication unit 41 is a communication interface that transmits and receives various types of data via the network 3. The various types of data include image data and data indicating the comparison results. In other words, the terminal-side communication unit 41 receives various types of information from the image analysis device 1.

The device storage unit 42 has a function of storing various control programs and various data required for the operation of the device processing unit 43. The device storage unit 42 is realized by various storage media such as an HDD, an SSD, or a flash memory.
By executing a control program stored in the device storage unit 42 , the device processing unit 43 may realize at least a part of the functions to be realized by the image processing system 100 .

The device processing unit 43 is a computer that controls each part of the mobile device 4, and may be, for example, a central processing unit (CPU), a microprocessor, an ASIC, an FPGA, etc. Note that the device processing unit 43 is not limited to these examples, and may be any computer that controls each part of the mobile device 4.

The terminal processing unit 43 includes a reception unit 43 A. The reception unit 43 A receives the image data and the comparison results transmitted from the image analyzing device 1 and displays them on the terminal side display unit 45 .
The camera 44 can capture an image by operation of the user 5. Instead of the mirror device 2 according to this embodiment, image data may be acquired by the camera 44 of the mobile terminal 4 and transmitted to the image analysis device 1.
The terminal display unit 45 is a monitor device that displays information indicating the comparison results processed by the image analyzing device 1. The terminal display unit 45 can display image data together with the comparison results.

Next, the contents of the process performed by the skin condition evaluation unit 30 will be described with reference to Fig. 10. Fig. 10 is a diagram showing a process flow in the image analysis device 1.
As shown in FIG. 10, first, image data of the face of the user 5 captured by the mirror device 2 is acquired (image acquisition step: S501).
Next, the skin color evaluation unit 33A classifies the skin of the user 5 into a plurality of stages with reference to the evaluation table (skin color evaluation step: S502).

Next, the skin abnormality identification unit 33B identifies areas of the skin of the user 5 where pigmentation abnormalities occur (pigmentation identification step: S503). Then, the evaluation result display unit 33C identifies blemishes or pigmented spots by marking the pigmentation abnormalities (evaluation result display step: S504).

Next, the depth estimation unit 33D estimates the depth of the pigmentation abnormality from the skin surface (depth estimation step: S505). At this time, the depth estimation unit 33D refers to pre-stored correspondence data between the color of the pigmentation abnormality and the depth from the skin surface.

Next, the measure suggesting unit 33E proposes a measure to improve pigmentation to the user 5 (measure suggesting step: S506).
Finally, the improvement data generating unit 33F generates and displays improvement data (improvement data generating step: S507). The improvement data generating unit 33F assumes a state in which pigmentation in the pigmentation abnormality area has been improved for the image data, changes the hue of the pigmentation abnormality area, and displays it to the user 5. This allows the user 5 to visually grasp what effect will be obtained when a measure is taken, and motivates him or her to continue taking the measure.

Next, another example of the processing result will be described with reference to FIG. 11. FIG. 11 is a diagram showing another example of the processing result in the image processing system 100. As shown in FIG. 11, even if there are more blemishes than in FIG. 8 described above, the pigmentation abnormality areas are marked with a classification into multiple stages. Then, for the multiple confirmed blemishes, the number of blemishes can be evaluated for each corresponding stage. In addition, the current state of the user 5 can be objectively displayed by tallying up the number of blemishes and comparing it with an ideal value or an average value.

Here, a method for detecting skin abnormalities other than blemishes will be described with reference to Fig. 12 to Fig. 14. Fig. 12 is a diagram showing an example of an evaluation table that serves as an evaluation criterion when evaluating pores by the skin condition evaluation unit 30. In this evaluation table, the occurrence mechanism, skin characteristics, skin quality (quality of skin around pores), palpation (feel when touched), prone area (areas where pores frequently appear), causes, and countermeasures are classified according to the shape of the pores.
When detecting pores, the abnormal skin identifying section 33B identifies the positions of the pores from the information on the skin color detected by the skin color evaluating section 33A.

At this time, information indicating the color range of the pores is stored in advance in the evaluation table, and the pores are detected by referring to this value. In addition, when detecting the pores, the skin abnormality specifying unit 33B evaluates the shape of the pores and classifies them into a plurality of types.
For example, as shown in Fig. 12, pores can be classified into four types: dry pores, sagging pores, clogged pores, and shape-memory pores. Maintenance information on what measures can be taken to make the pores classified in this way less noticeable is written in the countermeasures column located at the right end of Fig. 12. The countermeasures suggestion unit 33E presents the countermeasures together with the evaluation results, allowing the user 5 to take advantage of the countermeasures.

FIG. 13 is a diagram showing an example of an evaluation table that serves as an evaluation criterion when detecting dark circles, redness, and blemishes.
When detecting dark circles under the eyes, the abnormal skin specification unit 33B detects the position of the dark circles under the eyes from the information on the skin color detected by the skin color evaluation unit 33 A. At this time, information indicating the range of the color of the dark circles under the eyes is stored in advance in the evaluation table, and the dark circles under the eyes are detected by referring to this value.

When detecting redness, the abnormal skin identification unit 33B detects as redness those parts of the overall skin color detected by the skin color evaluation unit 33A where the red hue is prominent. The unit then checks the areas where there is redness, and if there is redness around the cheeks, it is determined to be capillary dilation, if there is redness over the entire face, it is determined to be due to dryness, and if there is strong localized redness, it is determined to be due to acne. Then, measures to be taken are prepared for each type of redness.

When detecting blemishes, as described above, the blemishes are classified into stages according to the color of the blemish detected by the skin color evaluation unit 33 A. It is known that the depth of the blemish from the skin surface to the location varies depending on the color of the blemish, so the measures to be taken for each blemish are different.
In the case of dark circles, redness, and blemishes, the measure suggesting unit 33E presents to the user 5 the maintenance information and measure information stored in the evaluation table as solutions, together with the evaluation results.

13 and 14, the evaluation table shows maintenance products recommended for maintenance together with the maintenance information. This information is stored in the data storage unit 32. The measure proposal unit 33E may present the maintenance products together with the maintenance information, or may present only the maintenance products.
That is, the display unit 20 of the mirror device 2 can display at least one of a maintenance method and a maintenance product together with the evaluation result.

FIG. 14 is a diagram showing an example of an evaluation table that serves as an evaluation standard when detecting texture, fine lines, pores, and wrinkles.
When evaluating the texture (rough skin), the skin color evaluation unit 33A divides the cheek skin into minute area elements based on differences in color, evaluates the density, and assigns a score. If the score is 50 points or more, it is determined to be hyperkeratosis, and if it is between 30 and 49 points, it is determined to be dry skin. If it is 29 points or less, it is determined to be normal.

When evaluating fine wrinkles, the skin color evaluation unit 33A detects lines formed around the eyes that have a different hue from the surroundings. That is, wrinkles formed around the eyes are referred to as wrinkles in this description. If there is one or more lines extending horizontally below the eye, the line is determined to be a sagging wrinkle. Also, if there are three or more lines on the left and right outer sides of the eye in the area below the corner of the eye, the line is determined to be a fine wrinkle.
The main cause of sagging wrinkles is thought to be muscle weakening, while the main cause of fine wrinkles is thought to be thinning of the epidermis and dermis. The evaluation of pores is as described above, so a detailed explanation will be omitted here.

When evaluating wrinkles, the skin color evaluation unit 33A detects lines that form around the forehead and cheeks and have a different hue from the surroundings. That is, in this explanation, wrinkles that form around the forehead and cheeks are called fine lines. Lines on the forehead are judged to be expression wrinkles, and lines on the cheeks are judged to be large wrinkles. Expression wrinkles are mainly caused by everyday facial expressions and weakened muscles in the scalp and around the eyes. Large wrinkles are mainly caused by weakened muscles in the cheeks and around the mouth.

In this way, the skin condition evaluation unit 30A can detect areas where skin abnormalities are occurring for each type of skin abnormality (fine lines, wrinkles, age spots, enlarged pores, rough skin, redness, or dark circles).

(Facial shape evaluation unit)
Next, the configuration of the face shape evaluation unit 50 shown in Fig. 3 will be described with reference to Fig. 15. Fig. 15 is a block diagram showing the configuration of the face shape evaluation unit 50.
The facial shape evaluation unit 50 evaluates changes in the facial proportions of the user 5 based on the image data and the positions of the bones, muscles, and fat of the user 5. The facial shape evaluation unit 50 calculates the area of a predetermined region defined on the face of the user 5 and displays the area of the predetermined region together with the past history.

The facial shape evaluation unit 50 includes a device-side communication unit 51, a data storage unit 52, a device-side processing unit 53, and a device-side display unit 54. The facial shape evaluation unit 50 is an information processing device that analyzes the facial condition of the user 5 from imaging data that captures the face of the user 5.

The device-side communication unit 51 is a communication interface that transmits and receives various types of data via the network 3. The various types of data include imaging data and data indicating the comparison results. In other words, the device-side communication unit 51 functions as a receiving unit that receives imaging data.

The data storage unit 52 has a function of storing various control programs required for the operation of the device processing unit 53 and various data received from the outside by the device side communication unit 51. The data storage unit 52 also stores at least one reference area data.
The data storage unit 52 is realized by various storage media such as a HDD, an SSD, or a flash memory.

By executing the control program stored in the data storage unit 52, the device processing unit 53 realizes each function to be realized by the image processing system 100. The functions here include a vertex recognition function, a region definition function, an area calculation function, an area comparison function, and a result display function.
The device-side display unit 54 is a monitor device that displays the contents of the operations of the face shape evaluation unit 50 and the results of the processing.

The device processing unit 53 is a computer that controls each unit of the face shape evaluation unit 50, and may be, for example, a central processing unit (CPU), a microprocessor, an ASIC, an FPGA, or the like.
It should be noted that the device processing unit 53 is not limited to these examples, and may be any computer that controls each unit of the face shape evaluation unit 50.

The device processing unit 53 includes a vertex recognition unit 53A, a region definition unit 53B, an area calculation unit 53C, an area comparison unit 53D, and a display processing unit 53E.
The vertex recognition unit 53A recognizes the positions of the two fixed points Pf and one movable point Pm from image data capturing an image of the face of the user 5.

Here, the fixed point Pf is a vertex that is determined depending on the facial bone structure. Since the fixed point Pf is determined depending on the facial bone structure, the position of the fixed point Pf changes only slightly over time.
Note that the term "fixed" here does not mean that the position does not change at all, but rather that the amount of change is extremely small compared to a movable point Pm, which will be described later.

On the other hand, the movable point Pm is a vertex that is determined depending on the facial muscles and fat, and its position changes downward, for example, as the facial muscles weaken with age and as fat accumulates on the face.
In addition, the position of the movable point Pm changes upward when the facial muscles are stimulated, making the facial muscles stronger or reducing the amount of facial fat. Such a change in the position of the movable point Pm changes the proportions of the face, which greatly affects the impression that the face gives to others.

Here, each vertex recognized by the vertex recognition unit 53A in this embodiment will be described with reference to FIG.
16 is a diagram showing each vertex recognized by the vertex recognition unit 53A, in which (a) is a front view and (b) is a side view of the imaging data. Note that this content is merely an example, and each vertex recognized by the vertex recognition unit 53A can be changed as desired. In other words, taking into consideration the skeletal structure and muscle structure of the user 5, the vertices of the face that are easy to recognize can be used for evaluation.

As shown in Fig. 16, the apex recognition unit 53A recognizes two fixed points Pf and one movable point Pm for one delimited area. The two fixed points Pf are the apexes identified by the deep nose point P1 and the apex P2 of the temple, and the one movable point Pm is the apex P3 on the cheek. The deep nose point P1 is shared by a pair of delimited areas on the left and right. A specific method for identifying each apex will be described later.
In this embodiment, the positions of the deep nose point P1 and the apex P2 of the temple are equal to each other in the vertical direction. The apex P3 on the cheek is located lower than the deep nose point P1 and the apex P2 of the temple.

The apex recognition unit 53A recognizes the apexes defined by the subnose point P4 and the subear point P5 as two fixed points Pf, and recognizes the apex P6 under the cheek as one movable point Pm. The subnose point P4 is shared by a pair of defined regions on the left and right. A specific method for identifying each apex will be described later.
In this embodiment, the subnasal point P4 and the subaural point P5 are positioned at the same vertical position. The subcheek apex P6 is located lower than the subnasal point P4 and the subaural point P5.

The means for recognizing each vertex in the vertex recognition unit 53A may be a method for identifying absolute coordinates relative to spatial coordinates set for the imaging data, or a method for identifying relative coordinates based on one of the three vertices that define the defined area.
In this embodiment, since the imaging data is 3D data, the coordinate values are also expressed three-dimensionally.

The area demarcation unit 53B defines a triangular demarcation area by lines connecting the vertices whose positions have been recognized by the vertex recognition unit 53A. The area demarcation unit 53B also defines a pair of left and right demarcation areas based on the midline O1 of the face.
The demarcated region defined by the region defining unit 53B may be a two-dimensional region or a three-dimensional region. In this embodiment, the demarcated region is a three-dimensional region.

In this embodiment, the region defining unit 53B defines two types of defined regions at an interval in the vertical direction of the face. Here, the defined region located on the upper side is defined as an upper defined region A1, and the defined region located on the lower side is defined as a lower defined region A2.
That is, the area defining section 53B defines a pair of upper and lower defining areas A1 and A2 on the left and right.

The reason why the upper defined area A1 and the lower defined area A2 are spaced apart in the vertical direction is to evaluate the entire face across the entire area in the vertical direction, so that the evaluation by the upper defined area A1 and the lower defined area A2 covers the entire face. Therefore, there is no problem if parts of the upper defined area A1 and the lower defined area A2 overlap each other.

The area calculation unit 53C calculates the area of the demarcated region by using the coordinate data of each vertex specified by the region definition unit 53B.
The area comparison unit 53D compares the area of the demarcated region calculated by the area calculation unit 53C with a reference area that is known as the area of a region corresponding to the demarcated region.

The area comparison section 53D can use, as the reference area, for example, the area of a defined region defined from image data of the user 5 captured a certain period of time before the image data was captured, that is, in the past.
Furthermore, the area comparison unit 53D can use, as the reference area, the area of the demarcated area in an ideal model of a face desired by the user 5. In this way, the reference area can be set arbitrarily as long as it can be compared with the area of the demarcated area at the current time.

Here, an example of a method for creating an ideal model as a face desired by the user 5 will be described.
The ideal model is created using past imaging data. Approximately 100 pieces of original data are prepared by visually specifying ideal demarcated regions for the past imaging data. The ideal model can be created by using this original data and performing deep learning processing.

Next, a guideline for comparing the areas will be described by taking the area measured previously as the reference area as an example. In the upper defined area A1 of this embodiment, the apex P3 on the cheek, which is the movable point Pm, is located below the deep nose point P1 and the apex P2 of the temple, which are fixed points Pf.
Also, in the lower defined area A2, the apex P6 under the cheek, which is the movable point Pm, is located lower than the subnasal point P4 and the subear point P5.

For this reason, when the apex P3 on the cheek, which is the movable point Pm, and the apex P6 below the cheek move downward, the areas of the upper defined area A1 and the lower defined area A2 each increase.
On the other hand, when the apex P3 on the cheek and the apex P6 below the cheek, which are the movable points Pm, move upward, the areas of the upper defined area A1 and the lower defined area A2 each become smaller.

In other words, in a configuration in which the position of the movable point Pm is located below the position of the fixed point Pf, as in this embodiment, if the area of the defined area is smaller than the reference area, which is the area measured last time, the movable point Pm will have moved upward.
This means that the facial proportions have improved, either because the facial muscles have become stronger or because the facial fat has decreased.

On the other hand, if the area of the defined region is larger than the reference area, which is the area measured previously, then the movable point Pm has moved downward.
This means that the facial proportions have worsened due to weaker facial muscles or excess facial fat.
In this way, by checking the amount of change in the demarcated region, user 5 can quantitatively grasp whether the proportions of his or her face are improving or worsening.

In this embodiment, the configuration has been described in which the position of the movable point Pm is located below the position of the fixed point Pf for each of the upper defined area A1 and the lower defined area A2, but this is not limited to this configuration. The position of the movable point Pm may be located above the position of the fixed point Pf.

In this case, the result of comparing the area of the defined area with the reference area will be the opposite of what was explained above. That is, if the area of the defined area is larger than the reference area, the facial proportions are improving, and if the area of the defined area is smaller than the reference area, the facial proportions are worsening.

In addition, if the area of the defined region in the ideal model of the face desired by user 5 is taken as the reference area, it is possible to determine whether the facial proportions have improved by checking how close the area has come to the reference area.

The display processing unit 53E displays the comparison result, obtained by the area comparison unit 53D comparing the area of the defined area with the reference area, on the device-side display unit 54 and on the terminal-side display unit 45 (described later) of the mobile terminal 4. Specific examples of the display contents displayed by the display processing unit 53E will be described later.

Next, the control flow of the image processing system 100 and the contents of the processing in the image processing system 100 will be described with reference to FIGS.
FIG. 17 is a diagram showing a processing flow in the image processing system 100, and FIG. 18 is a schematic diagram showing a process in which the apex recognition unit 53A recognizes the apex P3 on the cheek.

As shown in FIG. 17, in the beauty promotion method according to the present embodiment, first, image data of the face of the user 5 acquired by the imaging unit 21 of the mirror device 2 is received (S601: image receiving step).
In the image receiving step, in order to suppress changes due to facial expressions of the user 5, it is desirable to always maintain the same expression, for example by lightly biting the molars together.

Next, the vertex recognition unit 53A recognizes each vertex by using the imaging data transmitted from the imaging unit 21 (S602: vertex recognition step).
In the vertex recognition step, the positions of two fixed points Pf and one movable point Pm are recognized as three vertices that constitute one demarcated area. Here, one embodiment of a specific method for determining each vertex will be described. Note that this description is merely an example, and each vertex may be determined by other methods.

As shown in Fig. 18, the apex recognition unit 53A evaluates the image data three-dimensionally to recognize each apex. First, for the deep nose point P1 which is one of the fixed points Pf among the three apexes constituting the upper defined area A1, the deepest part of the nose root of the face is identified and recognized as the deep nose point P1.
Next, for the apex P2 of the temple that constitutes the other fixed point Pf, the most recessed part of the temple part of the face is recognized as the apex P2 of the temple. Note that the apex P2 of the temple may be the part of the outermost edge of the face in the left-right direction when viewed from the front, through which a straight line connecting the deep nose point P1 and the center of the pupil or the inner corner of the eye passes.

Furthermore, for the vertex P3 on the cheek that constitutes the movable point Pm, the part of the upper part of the cheek that is the most elevated part near the vertical line outside the pupil is recognized as the vertex P3 on the cheek. In this case, as shown in Fig. 18, the most elevated part may be recognized as the vertex P3 on the cheek by projecting contour lines onto the imaging data.
By performing this process on both the left and right sides, the vertices that make up the pair of left and right upper defined areas A1 are recognized.

Next, as shown in Figure 16, for the deep nose point P1 which is one of the fixed points Pf of the three vertices that make up the lower defined area A2, the most recessed part of the lower part of the nose on the face is recognized as the subnasal point P4.
Next, for the sub-ear point P5 which constitutes the other fixed point Pf, the most recessed part of the face located below the ear is recognized as the sub-ear point P5.

Furthermore, for the vertex P6 under the cheek that constitutes the movable point Pm among the three vertices, the most elevated part of the lower part of the cheek of the face, which is located near the vertical line outside the pupil and bulges to the sides of the corners of the mouth, is recognized as the vertex P6 under the cheek. When recognizing the vertex P6 under the cheek, there are also cases where the most elevated part is recognized as the vertex P6 under the cheek by projecting a contour line onto the imaging data.
By performing this process on both the left and right sides, the vertices that make up the pair of left and right lower defined areas A2 are recognized.

It is also possible to perform image processing to identify each vertex that constitutes each defined area, for example by comparing the position of each vertex in the facial data of multiple people that have been registered in advance with the captured image data, without adopting the method of recognizing each vertex described above.
The position of each vertex may be specified by superimposing the latest imaging data on the past imaging data. The position of each vertex may be specified by the operator 6 selecting an appropriate point on the imaging data as each vertex.

Next, the area definition unit 53B defines a defined area using the vertex data identified in the vertex recognition step (S603: area definition step).
In the area definition step, a triangular defined area is defined by lines connecting each vertex.

Next, the area calculation unit 53C calculates the area of the defined area defined in the area defining step (S604: area calculation step).
In the area calculation step, the area of the defined region is calculated using the coordinate data of each vertex.

Next, the area comparison unit 53D compares the area of the demarcated region calculated in the area calculation step with a reference area (S605: area comparison step).
In the area comparison step, the area of the defined area is compared with a reference area that is known as the area of the area corresponding to the defined area. In this description, the area of the defined area obtained from a previous measurement result is set as the reference area.

Finally, the display processing unit 53E outputs information indicating the comparison result (S606: display processing step).
In the display processing step, the comparison result between the area of the defined area compared by the area comparison unit 53D and the reference area is displayed on the device display unit 54 and the terminal display unit 45. The comparison result may also include information on the current result and information proposing measures (such as facial massage) that the user 5 should take in the future. The comparison result does not have to be displayed on the terminal display unit 45.
By making such a comparison, as in the first embodiment described above, it is possible to quantitatively evaluate changes in facial proportions due to aging and improvement measures, which can contribute to beauty promotion.

Next, the results of the evaluation by the face shape evaluation unit 50 and the effects thereof will be described with reference to FIGS.
Fig. 19 is a diagram showing an example of the display contents by the display processing unit 53E, (a) imaging data from two months ago, and (b) imaging data at the time of evaluation. Fig. 20 is a diagram showing another example of the display contents by the display processing unit 53E, (a) imaging data from two months ago, and (b) imaging data at the time of evaluation. In Fig. 19 and Fig. 20, the same image data is arranged above and below.

In one example of the comparison results shown in FIG. 19, the area of the upper defined area A1 has decreased by approximately 23%, and the area of the lower defined area A2 has decreased by approximately 53% compared to two months ago. This has resulted in a youthful, plump look, and an improved appearance.

In another example of the comparison results shown in FIG. 20, the area of the upper defined area A1 has decreased by approximately 21.5%, and the area of the lower defined area A2 has decreased by approximately 25% compared to two months ago. This gives a youthful, plump look and improves the appearance. This gives a well-balanced, graceful expression, and improves the appearance.

Next, a method for evaluating a massage performed using such an image processing system 100 will be described. In this method for evaluating a massage, a predetermined massage is given to the face of the user 5, and then the effect of the massage is evaluated using the image processing system 100. This allows the effectiveness of the effect that the massage has on the user 5 to be confirmed.

The massage evaluation method is carried out by executing a massage step and an image processing step.
The massage step is performed by executing a functional agent application step, a lymph node stimulation step, and a fascia stimulation step. In this explanation, the massage step is performed on the face, but the massage step may be performed on parts other than the face.

In the functional agent application step, a functional agent is applied to the surface of the face of the user 5. The functional agent is a drug that has at least one of the following functions: promoting blood circulation, promoting fat melting, or promoting fascial relaxation. Specifically, it is preferable that the functional agent contains one of the following components: glaucine (blood circulation promoter), okra seed extract (fat melting promoter, fascial relaxation promoter), niacinamide (blood circulation promoter), and grecon hesperidin (blood circulation promoter). Note that the components are not limited to these, and any component that has the above-mentioned effect can be used.

In the functional agent application step, the functional agent is applied to the face, particularly to areas that the user 5 wishes to tighten, such as the cheeks, around the eyes, etc. Then, the lymph node stimulation step is carried out.
In the lymph node stimulating step of performing a lymph node massage, after the functional agent applying step, the parts of the face of the user 5 where the lymph nodes are located are physically stimulated. The lymph nodes on the face are located under the ears.

In the lymph node stimulation step, the lymph nodes are stimulated using a cosmetic device 70 shown in Fig. 21. Fig. 21(a) is an external view of the cosmetic device 70 used in the massage step. Fig. 21(b) is a cross-sectional view of the cosmetic device 70.
21, the cosmetic device 70 includes pin-shaped pressing parts 72, 74 biased by biasing members 73, 75. The pressing parts 72, 74 are pressed against the skin of the face of the user 5 in the area where the lymph nodes are located, thereby stimulating the lymph nodes. Here, the structure of the cosmetic device 70 will be described in detail.

The cosmetic device 70 includes a cosmetic device main body 71 , a first pressing portion 72 , a first biasing member 73 , a second pressing portion 74 , and a second biasing member 75 .
The cosmetic device main body 71 has a cylindrical shape formed into a multi-stage cylindrical shape whose diameter increases stepwise from one side to the other side in the axial direction.

The cosmetic device main body 71 comprises a small diameter cylindrical portion 71A located on one side in the axial direction, an intermediate cylindrical portion 71B axially connected to the small diameter cylindrical portion 71A, and a large diameter cylindrical portion 71C located on the other side in the axial direction.
A protruding portion 71D that protrudes radially inward is formed on the inner circumferential surface of the large diameter cylindrical portion 71C at a portion that is axially connected to the intermediate cylindrical portion 71B.

The first pressing portion 72 is provided at one end of the cosmetic device body 71 and serves to press against the face or body of the user, and is composed of one first pin.
The first biasing member 73 is housed inside one end of the cosmetic device main body 71 and biases the first pressing portion 72 toward the outside of the cosmetic device main body 71. The first biasing member 73 is a coil spring.

The second pressing portion 74 is provided at the other end of the cosmetic device main body 71 and is used to press against the user's face or body, and is composed of three second pins and a holding portion that holds the three second pins together.
The second biasing member 75 is housed inside the other end of the cosmetic device body 71 and abuts against the end of the holding part, thereby biasing the second pressing part 74 toward the outside of the cosmetic device body 71. The second biasing member 75 is a coil spring.

The first biasing member 73 is disposed inside the intermediate cylindrical portion 71B and abuts against the first pin while abutting against the protruding portion 71D.
The second biasing member 75 is disposed inside the large diameter cylindrical portion 71C and abuts against the end of the holding portion while abutting against the protruding portion 71D.

When using the cosmetic device 70, one of the pressing parts 72, 74 is pressed against the face of the user 5 to stimulate the face of the user 5. The first and second pressing members 73, 75 have different pressing forces, and the user 5 can select which pressing part 72, 74 to use.

For example, when the first pressing portion 72 is pressed against the face of the user 5, the first pressing portion 72 is displaced against the biasing force from the first biasing member 73. At this time, the length of the first biasing member 73, which is a coil spring, becomes shorter and the amount of deformation increases, so that the biasing force that the first pressing portion 72 receives from the first biasing member 73 gradually increases.

As a result, the force with which the first pressing portion 72 presses against the face of the user 5 gradually increases. Then, the first pressing portion 72 may be pressed against the face until the limit of the displacement range of the first biasing member 73 is reached, or the first pressing portion 72 may stop being pressed against the face of the user 5 just before pain is felt. By repeating this operation multiple times, it is possible to provide a stimulus to the face of the user 5.
In the lymph node stimulation step, not only the lymph nodes in the face but also the lymph nodes near the collarbone are stimulated, which helps to eliminate lymphatic congestion in the lymph nodes.

Next, a fascia stimulation step is performed to perform a fascia massage. The fascia stimulation step is performed using the beauty device 70 used in the lymph node stimulation step. In the fascia stimulation step, the top of the head, cheeks, neck, etc. are repeatedly pressed by the pressing parts 72, 74 of the beauty device 70. This allows the fascia that has adhered to the muscle to be peeled off.
The lymph node stimulation step and the fascia stimulation step may be performed repeatedly, or the fascia stimulation step may be performed first and then the lymph node stimulation step.

Next, an image processing step is performed, in which the face shape evaluation step by the face shape evaluation unit 50 described above and a display processing step of outputting information showing changes in the facial proportions of the user 5 evaluated by the face shape evaluation unit 50 are performed.
In the display processing step, the display processing unit 53E displays the evaluation result on the display unit 20. In this way, the effects of the lymph node massage and fascia massage can be confirmed.

Here, the usefulness of stimulating the lymph nodes in the massage step will be explained.
Adhesion between muscle and fascia may occur due to waste products between them. To remove this waste, it is effective to improve the flow of lymph fluid between the muscle and fascia. By stimulating the lymph nodes that store lymph fluid, the flow of lymph fluid can be improved.

By improving lymphatic flow in this way while stimulating the fascia, waste products trapped between the muscle and fascia can be effectively removed by the flow of lymphatic fluid. This effectively removes adhesions of fascia to muscle.

Next, the usefulness of stimulating lymph nodes and fascia using a beauty device 70 equipped with pressure sections 72, 74 in the massage step will be described. For example, in conventional massages, stimulation is given by rolling a roller member over the skin surface in the area where the fascia is located.

However, when stimulating the surface of the skin with a roller member, the pressure tends to disperse as the roller rotates, making it difficult for the stimulation to reach the deeper parts of the body where the target fascia and lymph nodes are located. This creates an issue in that waste products that are between the fascia and the muscles cannot be effectively removed.

On the other hand, when the pin-shaped pressure parts 72, 74 are used to stimulate the outside of the area where the fascia is located, the pressure from the tips of the pin-shaped pressure parts 72, 74 is concentrated on the area that comes into contact with the tips of the pressure parts 72, 74. This makes it easier for the pressure to reach deep inside the body where the fascia and lymph nodes are located.

In addition, the pressing force of the biasing members 73, 75 increases with the displacement of the pressing portions 72, 74, so that the pressing force can be adjusted by adjusting the degree of displacement of the pressing portions 72, 74. Furthermore, since the biasing members 73, 75 bias the pressing portions 72, 74 with a generally constant force, there is an advantage in that the pressing force of the pressing portions 72, 74 can be easily kept generally constant.

Next, a first example of a specific procedure for a massage method will be described with reference to Figs.
First, as shown in Fig. 22(a), a stimulation is applied to the venous angle near the clavicle using a cosmetic device 70. At this time, the pressing parts 72 and 74 of the cosmetic device 70 are pressed against the venous angle. This state is preferably maintained for about 5 seconds.
Next, as shown in Fig. 22(b), the pressing parts 72, 74 of the cosmetic device 70 are placed against the central part of the clavicle where the subclavian lymph nodes are located, and the pressing parts 72, 74 are repeatedly pressed for about 20 seconds. (Lymph node stimulation step) The pressing parts 72, 74 are preferably pressed at a speed of about 5 times per second.

22(c), the pressing parts 72, 74 of the cosmetic device 70 are placed under the mastoid process, and the pressing parts 72, 74 are repeatedly pressed for about 10 seconds. It is preferable to press the pressing parts 72, 74 at a speed of about 3 to 5 times per second.
Next, as shown in Fig. 22(d), the pressing parts 72, 74 of the cosmetic device 70 are placed against the back of the head, and the pressing parts 72, 74 are repeatedly pressed for about 10 seconds. It is preferable to press the pressing parts 72, 74 at a speed of about 3 to 5 times per second. It is preferable to press the part of the back of the head that is located above the nape at the same height as the center of the ear.

Next, as shown in Fig. 23(a), the functional agent is applied to the entire nape of the neck on one side. A few cc of the functional agent is applied while spreading it out. This makes it possible to eliminate unevenness in the application of the functional agent.
Next, as shown in Figure 23(b), stretch the parts of your body in front of your ears up and down. Repeat this movement about five times.

Next, as shown in Fig. 23(c), the functional agent is applied around one temple. A few cc of the functional agent is applied while spreading it out.
Next, as shown in Fig. 23(d), place four fingers on the temples and pull the fingers diagonally upward (towards the temporal region) to stretch the scalp. At this time, it is preferable to keep the fingers pulled up for about five seconds.

Next, the temporal region is stimulated. As shown in Fig. 24(a), a portion of the temporal region that is at the same height as the temple and above the ear is identified. Then, as shown in Fig. 24(b), the pressing portions 72, 74 of the cosmetic device 70 are placed against the temple, and the pressing portions 72, 74 are repeatedly pressed for about 10 seconds. It is preferable to press the pressing portions 72, 74 at a speed of about 3 to 5 times per second.
24C, the pressing parts 72, 74 are pressed repeatedly around the temples in the same manner, preferably at a speed of about 3 to 5 times per second for about 10 seconds.

Next, as shown in Fig. 25(a), stimulation is applied to six points on the temporal region. This action is repeated about 30 times for each point.
Next, as shown in Fig. 25(b), place four fingers on the side of the head and massage in an arc. This movement is repeated about five times.

Next, as shown in Fig. 25(c), stimulation is applied to five points on the forehead from the center of the hairline to the temples. This action is preferably performed about 30 times for each point.
Next, as shown in Fig. 25(d), place four fingers on the forehead and massage in an arcing motion. This movement is repeated about five times.

Next, as shown in Fig. 26(a), while applying pressure with the thumb from the top of the head, slide it down to behind the ear. This movement is performed three times along each of the three lines.
Next, as shown in Fig. 26(b), the back of the ear is pulled in the direction of the arrow. This movement is performed for 5 seconds in each direction.

Next, as shown in Fig. 26(c), stimulation is applied to eight locations on the A and B parts using the beauty device 70. This operation is performed 30 times for each location.
Next, as shown in FIG. 26(d), place four fingers on the hairline and comb the hair toward the back of the head three times.

Next, as a finishing touch, first place your wrist under your ear as shown in Figure 27(a). Next, move your wrist to the side of the head above your ear as shown in Figure 27(b), and place the other hand along the head. Finally, use both hands to pull the head up as shown in Figure 27(c). This completes the entire massage sequence.

Next, a second example of the specific procedure of the massage method will be described. This massage is mainly for the fascia of the head. It has been confirmed that releasing adhesions in the fascia of the head affects the silhouette of the face.
First, as shown in Figure 28(a), grasp the scalp above the ear with the four fingers of both hands. Next, draw circles with the four fingers of both hands to move the temporal muscle above the ear. In this case, draw three circles three times each.

Next, as shown in FIG. 28(b), the head is stimulated by the pressure sections 72, 74 of the beauty device 70 toward the Baihui acupoint located on the top of the head.
Next, as shown in Figure 28(c), grasp the scalp at the hairline with the four fingers of both hands. Next, draw circles with the four fingers of both hands as if moving the frontalis muscle. In this case, draw three circles three times each.

28(d), the hairline is stimulated from the center to the temples with the pressure sections 72, 74 of the cosmetic device 70. Next, the thumb is applied with pressure from the center of the hairline to the temples and is alternately moved to lift the hairline.
Next, as shown in Fig. 29(a), apply pressure with your thumb from the hairline to the top of the head and slide it down to behind your ear, drawing each of the three lines three times.

Next, as shown in FIG. 29(b), place four fingers at the hairline and comb the hair toward the back of the head three times.
Next, as shown in FIG. 29(c), the back of the ear is pulled five times in each direction of the arrows.

Next, as shown in FIG. 29(d), the lower mastoid process is stimulated ten times on each side using the pressure sections 72 and 74 of the cosmetic device 70.

Next, as shown in FIG. 30(a), the ear is pinched between the index finger and middle finger, and rubbed strongly five times while applying pressure from above and below.
Next, as shown in Fig. 30(b), the entire back of the head is stimulated by the pressure sections 72, 74 of the cosmetic device 70. At this time, stimulation is given along three lines five times each.

Next, as shown in FIG. 30(c), pinch the sternocleidomastoid muscle from the base of the neck with the thumb and index finger on one side, slide it down to the depression at the back of the head, and apply light pressure for about five seconds.
Next, as shown in FIG. 30(d), to stretch the neck, place one wrist on top of the ear and the other wrist around the acupressure point called Fengchi on the back of the head.
Then, while applying pressure to the inside, pull it up for about 5 seconds. This completes the series of massage steps according to the second example.

Next, a third example of the specific procedure of the massage method will be described with reference to Figs. 31 to 34. This massage is mainly for the abdominal area of the body. The abdominal area massage is intended to improve poor posture caused by hunched back and forward shoulders.
This massage can remove the stiffness and adhesions in the muscles and fascia around the abdomen, and is expected to have the effect of relieving stiff shoulders, correcting hunched posture, and tightening the stomach.

First, as shown in Fig. 31(a), three points under the clavicle from the center of the clavicle to the coracoid process are pressed using the first pressing part 72 of the cosmetic device 70. Then, a functional agent is applied, and the second pressing part 74 is pressed about five times from above.
Next, as shown in FIG. 31(b), the point shown in the figure, called the coracoid process, is pressed with the first pressing part 72 of the cosmetic device 70 for about 30 seconds.

Next, as shown in FIG. 32(a), the intercostal cartilage located below the sternoclavicular joint is pressed up to the upper chest area using the second pressing part 74 about four or five times.
32(b), the indurated part of the upper arm is pressed with the first pressing part 72 of the cosmetic device 70. Then, hands are placed on both shoulders and pressure is applied to open the shoulders.

Next, as shown in FIG. 33( a ), with the arm raised, the inside of the armpit is pressed with the first pressing portion 72 of the cosmetic device 70 for about one minute.
Next, as shown in FIG. 33(b), the area located three finger widths outward from the solar plexus is pressed with the first pressing part 72 of the cosmetic device 70 for about one minute per area.

Next, as shown in FIG. 34(a), the flank area from the side of the solar plexus to the side of the navel is pressed with the first pressing part 72 of the cosmetic device 70 for about one minute at each location.
34(b), the first pressing part 72 of the cosmetic device 70 is pressed from the solar plexus along the ribs at five points on each side for about one minute. Then, a functional agent is applied and the second pressing part 74 is pressed.

Next, a fourth example of a specific procedure for the massage method will be described with reference to Fig. 35. This massage is mainly performed on the chest area of the body.
Chest massage aims to improve poor posture caused by hunched back and forward shoulders. This massage improves blood flow and lymph flow around the collarbone, making it easier for nutrients to reach the bust.

First, the central area under the collarbone is pressed for about one minute with the first pressing part 72 of the cosmetic device 70. Then, as shown in Fig. 35(a), the depression behind the earlobe is pressed for about one minute with the first pressing part 72 of the cosmetic device 70.
35(b), the lower part of the mastoid process is pressed for about one minute with the first pressing part 72 of the cosmetic device 70. After that, a functional agent is applied to the area where the sternocleidomastoid muscle is located.
Then, after this, the massage of Figs. 30 to 32 is performed as described above.

As described above, according to the image processing system 100 of this embodiment, the skin condition evaluation unit 30 evaluates the skin health condition of the user 5 based on the skin color of the user 5 using image data captured by the imaging unit 21 of the mirror device 2 of the face of the user 5. This makes it possible to quantitatively evaluate skin abnormalities.

Then, the facial shape evaluation unit 50 evaluates the change in the facial proportions of the user 5 based on the image data and the positions of the skeleton, muscles, and fat of the user 5. Here, since the change in the facial proportions is evaluated based on the relative positions of the skeleton, muscles, and fat of the user 5, it is possible to easily judge whether the change is good or bad before and after the change in comparison with a mode in which the change in the entire face is evaluated. This makes it possible to uniquely judge the change in the shape of the face.

In addition, the display unit 20 of the mirror device 2 displays the evaluation results of the skin condition evaluation unit 30 and the facial shape evaluation unit 50, so that the user 5 can check the condition of the facial skin and changes in proportions when using the mirror device 2 to groom himself/herself, ensuring convenience for the user 5.

The future prediction unit 60 also predicts the future of the face of the user 5 using at least one of the skin health condition of the user 5 stored in the skin condition evaluation unit 30 and the facial shape evaluation of the user 5 stored in the facial shape evaluation unit 50. This makes it possible to motivate the user 5 to promote beauty care.

The skin color evaluation unit 33A in the skin condition evaluation unit 30 uses image data of the skin of the user 5 to classify any part of the skin of the user 5 into a number of pre-set stages according to its hue value. The skin abnormality identification unit 33B then identifies the position of the pigmentation abnormality on the skin of the user 5 based on the stage classified by the skin color evaluation unit 33A. This allows the degree of pigmentation such as blemishes and pigmented spots to be quantitatively evaluated for beauty purposes.

The evaluation result display unit 33C also displays the position of each part of the image data for each stage classified by the skin color evaluation unit 33A, and marks the pigmentation abnormality parts identified by the skin abnormality identification unit 33B. This allows the user 5 to visually grasp the evaluation results, ensuring convenience for the user 5.

The evaluation result display unit 33C also displays and marks the positions of the different stages for multiple image data taken at different times for the same user 5, and displays them side by side. This allows the user 5 to visually grasp the changes over time in their skin condition.

In addition, the depth estimation unit 33D estimates the depth of the pigmentation abnormality from the skin surface of the skin of the user 5 based on the hue value of the pigmentation abnormality, which can provide information for consideration of measures to improve the pigmentation abnormality.

In addition, the measure suggestion unit 33E suggests measures to promote improvement of pigmentation based on the depth of the pigmentation abnormality, so that appropriate measures can be suggested according to the condition of the pigmentation abnormality.

In addition, the improvement data generating unit 33F assumes a state in which pigmentation in the pigmentation abnormality area has been improved for the image data, changes the hue of the pigmentation abnormality area, and displays this to the user 5. This makes it possible to visually convey to the user 5 the effect that can be expected from the proposed measures, thereby motivating the user 5 to continue to implement the measures.

Furthermore, according to the image processing system 100 of this embodiment, the vertex recognition unit 53A recognizes the positions of two fixed points Pf that are determined depending on the facial bone structure, and one movable point Pm that is determined depending on the facial muscles and fat, from the image data capturing an image of the face of the user 5.

Next, the area definition unit 53B defines a triangular defined area by drawing straight lines connecting the identified vertices, and the area calculation unit 53C calculates the area of the defined area. The area comparison unit 53D then compares the area of the defined area with a reference area. This allows for a quantitative evaluation of changes in facial proportions due to aging and improvement measures, which can contribute to promoting beauty.

In addition, in this case, the defined area is defined by two fixed points Pf and one movable point Pm, so compared to a configuration in which the defined area is defined by, for example, two or three movable points Pm, it is possible to suppress variation in identifying the position of the movable point Pm, which is difficult to recognize, and perform accurate evaluation.

By evaluating the area of the demarcated region, it is possible to handle larger numerical values and increase the amount of change, compared to, for example, a case in which the position of the movable point Pm is evaluated based on the distance from the fixed point Pf.
This makes it easier for user 5 to recognize the degree of change in the face, even if the change in facial proportions before and after a certain period of time is difficult to discern when checking the image data, and motivates the user 5 to promote beauty treatments.

Moreover, most existing facial proportion analyses rely on bone structure (such as the so-called golden ratio of the face), which is a congenital factor that cannot be changed, making it difficult to increase motivation for beauty treatments other than plastic surgery. On the other hand, image processing system 100 does not rely only on bone structure, but evaluates changes in muscles and fat caused by aging that can be improved through self-care, so it can increase user 5's motivation for beauty treatments.

The image processing system 100 also includes a display processing unit 53E that outputs information indicating the results of the comparison between the area of the defined area and the reference area, so that the quantitative evaluation results can be easily confirmed by displaying them, for example, on the mobile terminal 4 of the user 5.

In addition, the area definition unit 53B defines a pair of left and right defined areas based on the facial midline O1, so beauty promotion can be carried out to achieve well-balanced facial proportions.

Further, the area defining unit 53B defines a defined area by two fixed points Pf defined by the deep nose point P1 and the apex P2 of the temple, and one movable point Pm defined by the apex P3 on the cheek.
This makes it possible to quantitatively evaluate the proportions around the upper cheeks of the face and check for changes in, for example, sagging of the upper cheeks (such as the nasolabial folds that form between the nasolabial folds and cheekbones), which can become a concern with age.

The apex recognition unit 53A also evaluates the imaging data three-dimensionally to recognize the deep nose point P1, the apex P2 of the temple, and the apex P3 on the cheek. This makes it easy to recognize each apex, regardless of the shape of the user's 5 face.

In addition, the area definition unit 53B defines two types of defined areas spaced apart in the upper and lower directions of the face, so that by evaluating the upper and lower sides of the face separately, the proportions of the entire face can be quantitatively evaluated, making it possible to promote beauty even more effectively.

In addition, the area definition unit 53B defines the defined area using two fixed points Pf, identified by the subnasal point P4 and the subauricular point P5, and one movable point Pm, identified by the apex P6 under the cheek, so that the proportions around the lower cheeks of the face can be quantitatively evaluated, and changes in sagging of the lower cheeks, which can become a concern with age, can be confirmed, for example.

In addition, the vertex recognition unit 53A evaluates the imaging data three-dimensionally to recognize the subnose point P4, the subear point P5, and the vertex P6 under the cheek, so that each vertex can be easily recognized regardless of the facial shape of the user 5.

In addition, when the area comparison unit 53D uses the area of the defined region of the user 5 at a certain time before the imaging data was captured as the reference area, it is possible to quantitatively evaluate how the facial proportions change over time. This allows the cosmetic effect to be accurately understood.

In addition, when the area comparison unit 53D uses the area of a defined region in an ideal model representing the face desired by the user 5 as the reference area, it is possible to quantitatively confirm how close the user 5 has come to the goal. This maintains the motivation of the user 5 for beauty treatments and allows for effective beauty treatment promotion.

In addition, since the image processing system 100 is equipped with a facial shape evaluation unit 50, it is possible to easily obtain image data of the face of the user 5 and evaluate the image data using the facial shape evaluation unit 50.

The device-side display unit 54 has a function of being able to display on the display surface the imaging data of the face of the user 5 facing the display surface. This allows the user 5 to check, as if looking in a mirror, the imaging data of the face of the user 5 captured by the imaging unit 21 of the mirror device 2, and the evaluation results of the change in proportions performed by the device processing unit 53 using the imaging data.

In addition, in the massage evaluation method according to the present invention, by executing the functional agent application step, the components contained in the functional agent are applied to the face of the user 5, which can provide at least one of the following effects: blood circulation promotion effect, fat melting effect, and fascia relaxation effect. This can enhance the effect of the massage.

In addition, in the lymph node stimulation step and fascia stimulation step, the lymph nodes and fascia are stimulated, stimulating the lymph nodes to promote lymphatic flow, promote the metabolism of waste products interposed between the muscle and fascia, and effectively eliminate the attachment between the fascia and the muscle.

After performing the massage in this manner, an image processing step is performed using the image processing system 100, and the effect of the massage can be reliably evaluated by evaluating the change in the proportions of the face of the user 5 from the image data of the face of the user 5.

Furthermore, in the lymph node stimulation step of the massage evaluation method, the lymph nodes are stimulated using a cosmetic device 70 having pin-shaped pressing portions 72, 74 biased by biasing members 73, 75, so that even if the lymph nodes are located deep in the body, the lymph nodes can be reliably stimulated to promote the flow of lymphatic fluid.
This promotes the metabolism of waste products between the muscle and fascia, and effectively removes the attachment between the fascia and the muscle.

In addition, if the functional agent contains a glaucine component, it can have a blood circulation promoting effect. If the functional agent contains an okra seed extract component, it can have a blood circulation promoting effect.

Furthermore, when the functional agent contains a niacinamide component, it can exert an effect of promoting fat melting.
Furthermore, when the functional agent contains the component glecon hesperidin, it can exert a fascial relaxation effect.

In addition, in the fascia stimulation step of the massage evaluation method, a beauty device 70 having pin-shaped pressing parts 72, 74 biased by biasing members 73, 75 is used to press the pressing parts 72, 74 against the part of the face of the user 5 where the fascia is located. Therefore, compared to a configuration in which a roller member is rolled to stimulate the surface of the skin, for example, it is possible to reliably stimulate the fascia located deep within the skin.

In addition, when the pin-shaped pressing parts 72, 74 are biased by a coil spring as a biasing member, the force with which the pressing parts 72, 74 are displaced can be kept constant, and the pressing force of the pressing parts 72, 74 can be kept constant.

Moreover, the evaluation result providing unit 61 provides the analysis result to the personal terminal 2B, and the confirmation content reporting unit 62 reports to the store terminal 2A confirmation history information of the analysis result provided to the user 5. The storage unit 63 also stores the contents confirmed by the user 5 using the store terminal 2A when he or she visited the store together with the confirmation history information.
This allows the store staff to check what content of the analysis performed on the user's image the user 5 is interested in, and by giving advice on the content that the user 5 is interested in, beauty promotion can be carried out efficiently.

In addition, since the image analysis device 1 is equipped with a skin condition evaluation unit 30, a facial shape evaluation unit 50, and a user identification unit 64, it is possible to evaluate changes in skin condition and facial promotion for each user 5 using the device, and multiple users 5 can use one mirror device 2.

In addition, the memory unit 63 stores the facial data of each user 5 and the ID of the user 5, and the user identification unit 64 identifies the user 5 by referring to the memory unit 63 for the captured image data of the user's face.
This enables user authentication to be performed simply by the user 5 having his/her face photographed, and, for example, enables the user 5 to omit the operation of inputting his/her own ID, thereby ensuring convenience for the user 5.

In addition, the confirmation content reporting section 62 tally up the number of times the user 5 confirms each analysis result performed by the skin condition evaluation section 30 and the facial shape evaluation section 50 within a specified period of time, and reports this to the shop terminal 2A.
Therefore, it is possible to statistically grasp what beauty items the users 5 as a whole are interested in based on the confirmation history information of the multiple users 5. This can be useful for the store to provide guidance regarding beauty promotion.

Furthermore, the confirmation content reporting section 62 reports at least one of the maintenance method and the maintenance product displayed on the display section 20 to the store terminal 2A.
This allows store staff to know the maintenance methods and maintenance products that the user 5 is interested in, which contributes to the sales activities carried out by the store.

It goes without saying that the image processing system 100 is not limited to the above embodiment, and may be realized by other methods. Various modified examples will be described below.
For example, the control program of the above embodiment may be provided in a state stored in a computer-readable storage medium. The storage medium is capable of storing the control program in a "non-transient tangible medium". The storage medium may include any suitable storage medium such as a HDD or an SDD, or a suitable combination of two or more of these. The storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile. Note that the storage medium is not limited to these examples, and may be any device or medium capable of storing the control program.

Furthermore, the image processing system 100 can implement each of the functions described in the embodiments by, for example, reading out a control program stored in a storage medium and executing the read out control program.
The control program may be provided to the image processing system 100 via any transmission medium (such as a communication network or broadcast waves). The image processing system 100 executes a control program downloaded via the Internet or the like to realize the functions of the multiple functional units shown in each embodiment.

The control program may also be implemented using, for example, a scripting language such as ActionScript or JavaScript (registered trademark), an object-oriented programming language such as Objective-C or Java (registered trademark), or a markup language such as HTML5.

At least a portion of the processing in the image processing system 100 may be realized by cloud computing consisting of one or more computers. In addition, each functional unit of the image processing system 100 may be realized by one or more circuits that realize the functions shown in the above embodiments, and the functions of multiple functional units may be realized by one circuit.

In addition, in the above embodiment, an example was described in which the image analysis device 1 and the mobile terminal 4 were separate devices, but this is not limited to this aspect. The mobile terminal 4 may have some or all of the functions of the image analysis device 1.

In addition, although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that a person skilled in the art would easily be able to make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure. For example, the functions included in each means, step, etc. can be rearranged so as not to cause logical inconsistencies, and multiple means, steps, etc. can be combined into one or divided. In addition, the configurations shown in each embodiment may be combined as appropriate.

1 Image analysis device 2 Mirror device 30 Skin condition evaluation unit 50 Facial shape evaluation unit 70 Cosmetic device 100 Image processing system

Claims (5)

A display unit that can display data and has a mirror surface;
An imaging unit that captures an image in front of the display unit;
a face shape evaluation unit for evaluating changes in the proportions of the user's face;
a display processing unit that outputs information indicating a change in the proportions of the user's face evaluated by the face shape evaluation unit to the display unit,
the display unit displays an outline of image data of a face of the user captured in the past so that a user positioned in front of the imaging unit can align the position of the face when the imaging unit captures an image of the face of the user;
When evaluating a change in the proportions of the user's face based on the positions of the user's bone structure, muscles, and fat from image data of the user's face captured by the imaging unit, the facial shape evaluation unit
A functional agent application step of applying a functional agent having at least one of the functions of promoting blood circulation, promoting fat melting, and promoting fascia relaxation to a surface of the user's face;
a lymph node stimulating step of performing a lymph node massage to physically stimulate a part of the user's face where the lymph nodes are located after the functional agent applying step;
A fascial stimulation step of performing a fascial massage to physically stimulate a portion of the user's face where the fascia is located after the functional agent application step;
This massage evaluation device evaluates the face of a user after treatment in which the functional material application step, the lymph node stimulation step, and the fascia stimulation step are performed, and for each of the image data after the treatment and the image data of the user's face captured in the past, two fixed points whose positions change little over time and a movable point are set, and the areas of the regions obtained by connecting the two fixed points and the movable point are compared to evaluate the user's face. The massage evaluation device according to claim 1, characterized in that in the lymph node stimulation step, a beauty device having a pin-shaped pressing part biased by a biasing member is used, and the pressing part presses the part of the user's face where the lymph nodes are located. The massage evaluation device according to claim 1 or 2, characterized in that in the fascia stimulation step, a beauty device having a pin-shaped pressing part biased by a biasing member is used, and the pressing part presses the part of the user's face where the fascia is located. The massage evaluation device according to any one of claims 1 to 3, characterized in that the functional agent contains the ingredients glaucine, okra seed extract, and niacinamide. The massage evaluation device according to claim 4, characterized in that the functional agent further contains a component of grecon hesperidin.

Images