JP7626353B2 - Skin condition estimation device - Google Patents

Description

The present invention relates to a skin condition estimation device.

Methods for objectively evaluating skin conditions have been developed. These methods are used in cosmetic sales, dermatology, and beauty salon interviews, etc.

Patent Document 1 discloses a method for evaluating skin condition by measuring components in a keratin sample collected from a subject. The results of the skin condition evaluation can be used, for example, by a beauty professional to provide appropriate skin care advice to the subject.

Japanese Patent Application Publication No. 10-216106

In the method disclosed in Patent Document 1, components contained in the collected keratin are directly measured. This requires extracting and isolating the components contained in the keratin, which is very time-consuming, so there is still a need for a method and device that can more easily evaluate the condition of the skin. The present invention has been made to solve this problem, and its purpose is to provide a device, method, program, etc. that can estimate the condition of the skin with a simpler procedure.

The skin condition estimation device according to the present invention includes an image acquisition unit and an estimation unit. The image acquisition unit acquires an image of a stratum corneum sample taken from the skin surface using a microscope. The estimation unit inputs the acquired image into a trained machine learning model, deriving an output from the machine learning model, and estimates at least one of an index representing the skin condition and advice regarding skin care habits suitable for the skin based on the output.

The output derived from the machine learning model may include at least one of data representing an index indicating the condition of the skin and data representing advice regarding skin care habits suitable for the skin. The data may be numerical data, text data, image data, etc.

In the estimation device, the skin condition is estimated based on an image of a stratum corneum sample taken from the surface of the skin, captured by a microscope. Therefore, there is no need to directly measure specific components contained in the collected stratum corneum sample. Therefore, at least one of an index representing the skin condition of the person who provided the stratum corneum sample and advice on skin care habits suitable for the skin is estimated without going through a time-consuming process.

In the estimation device, the image may include an image captured by a polarizing microscope.

In the estimation device, the index representing the skin condition may be a unit amount of ceramide representing the amount of ceramide per keratinocyte contained in the stratum corneum sample.

In the estimation device, the estimation unit may estimate at least one of an index representing the skin condition and advice on skin care habits suitable for the skin, using a machine learning model trained using learning data, which is a data set of an image of a stratum corneum sample taken from the surface of the skin captured by a microscope and ground truth data, and the ground truth data may be at least one of an index representing the skin condition based on a value measured from the stratum corneum sample, a data set of the amount of ceramide and the amount of protein measured from the stratum corneum sample, and advice on skin care habits suitable for the skin.

The index representing the skin condition based on the value measured from the stratum corneum sample included in the correct answer data may be data representing the index, and the advice regarding skin care habits suitable for the skin may be data representing the advice. The data may be numerical data, text data, image data, etc.

The estimation device may further include a result output unit that outputs the result estimated by the estimation unit.

The method for estimating a skin condition according to the present invention includes the following (1) and (2).
(1) Obtaining an image of a stratum corneum sample taken from the surface of the skin using a microscope.
(2) Inputting the acquired image into a trained machine learning model to derive an output from the machine learning model, and estimating at least one of an indicator representing the skin condition and advice regarding skin care habits suitable for the skin based on the output.

In the estimation method, the image may include an image captured by a polarizing microscope.

In the estimation method, the index representing the skin condition may be a unit amount of ceramide representing the amount of ceramide per keratinocyte contained in the stratum corneum sample.

The skin condition estimating program according to the present invention causes a computer to execute the following (1) and (2).
(1) Obtaining an image of a stratum corneum sample taken from the surface of the skin using a microscope.
(2) Inputting the acquired image into a trained machine learning model to derive an output from the machine learning model, and estimating at least one of an indicator representing the skin condition and advice regarding skin care habits suitable for the skin based on the output.

The output derived from the machine learning model may include at least one of data representing an index indicating the condition of the skin and data representing advice regarding skin care habits suitable for the skin. The data may be numerical data, text data, image data, etc.

In the estimation program, the image may include an image captured by a polarizing microscope.

In the estimation program, the index representing the skin condition may be a unit amount of ceramide representing the amount of ceramide per keratinocyte contained in the stratum corneum sample.

The learning device according to the present invention includes a memory unit and a learning unit. The memory unit stores learning data, which is a data set of an image of a stratum corneum sample taken from the surface of the skin captured by a microscope and correct answer data. The learning unit uses the learning data to train a machine learning model. The correct answer data is at least one of an index representing the skin condition based on values measured from the stratum corneum sample, a data set of the amount of ceramide and the amount of protein measured from the stratum corneum sample, and advice on skin care habits suitable for the skin.

In the learning device, the images may include images captured by a polarizing microscope.

In the learning device, the index representing the skin condition may be a unit amount of ceramide representing the amount of ceramide per keratinocyte contained in the stratum corneum sample.

The method for generating a trained model according to the present invention includes the following steps (1) and (2).
(1) A step of preparing learning data, which is a data set of images taken by a microscope of a stratum corneum sample taken from the surface of the skin and ground truth data.
(2) A step of performing machine learning using the learning data so that when an image of a stratum corneum sample taken from the surface of the skin taken with a microscope is input, data corresponding to the ground truth data is output.
In addition, the correct answer data may be at least one of an index representing the condition of the skin based on values measured from the stratum corneum sample, a dataset of the amounts of ceramide and protein measured from the stratum corneum sample, and advice regarding skin care habits suitable for the skin.

The index representing the skin condition based on the value measured from the stratum corneum sample included in the correct answer data may be data representing the index, and the advice regarding skin care habits suitable for the skin may be data representing the advice. The data may be numerical data, text data, image data, etc.

In the method for generating a trained model, the image may include an image captured by a polarizing microscope.

In the method for generating a trained model, the index representing the skin condition may be a unit amount of ceramide representing the amount of ceramide per keratinocyte contained in a stratum corneum sample.

According to the present invention, the skin condition of a subject is estimated based on an image of a stratum corneum sample taken from the subject, captured by a microscope. This makes it possible to easily estimate the skin condition and provide advice on skin care habits suitable for the subject.

FIG. 2 is a block diagram showing the electrical configuration of the estimation device according to the embodiment of the present invention. FIG. 1 is a diagram for explaining the principle of the present invention. FIG. 1 is a diagram for explaining the principle of the present invention. FIG. 1 is a diagram for explaining the principle of the present invention. FIG. 2 is a block diagram showing the electrical configuration of a learning device according to an embodiment of the present invention. 13 is a flowchart showing the procedure of an estimation process. 3A to 3C are diagrams illustrating the operation of an estimation device according to an embodiment of the present invention.

1. Overview of the estimation device
Hereinafter, a skin condition estimation device, an estimation method, an estimation program, and a method for generating a machine-learned model according to one embodiment of the present invention will be described with reference to the drawings.

The estimation device 1 shown in FIG. 1 is one embodiment of the estimation device according to the present invention. In terms of hardware, the estimation device 1 is a general-purpose computer. An estimation program 20 provided from a computer-readable recording medium such as a CD-ROM or USB memory or via a network or the like is installed in the estimation device 1. The estimation program 20 according to this embodiment is application software that estimates an index representing the skin condition and advice regarding skin care habits suitable for that skin from an image of a stratum corneum sample. The estimation program 20 causes the estimation device 1 to execute steps included in the operation described below.

The estimation device 1 includes a memory unit 2, a control unit 3, a display unit 6, an input unit 4, and a communication unit 5. These units 2 to 5 are connected to each other via a communication line 8 such as a bus line or a cable, and can communicate with each other as needed. The communication unit 5 communicates with an external device such as a microscope. The display unit 6 is composed of a liquid crystal display or the like, and displays information to the user, which will be described later. The input unit 4 is composed of a mouse, keyboard, and touch panel, and accepts input from the user to the estimation device 1. The memory unit 2 is a non-volatile storage area composed of a hard disk, flash memory, etc. The control unit 3 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and may include a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), and other AI-dedicated chips. The user of the estimation device 1 may be a person who accepts consultations regarding the skin condition of a subject who provides a stratum corneum sample from the skin surface, and recommends products and treatments suitable for the subject.

The storage unit 2 stores a skin condition estimation program 20. The estimation program 20 of this embodiment derives an index representing the skin condition from the estimation model 22 by inputting an image of a stratum corneum sample into a trained machine learning model (estimation model 22) trained in the learning device 100 described below. The storage unit 2 also stores a database 21 that stores advice on skin care habits in association with an index representing the estimated skin condition. The estimation program 20 of this embodiment reads out advice on appropriate skin care habits from the database 21 based on the derived index representing the skin condition, and displays it on the display unit 6.

The control unit 3 reads and executes the estimation program 20 stored in the memory unit 2, and thereby virtually operates as an image acquisition unit 30, an estimation unit 31, and a result output unit 32. The image acquisition unit 30 reads an image of a stratum corneum sample acquired using an external microscope. The estimation unit 31 estimates the subject's skin condition using the estimation model 22. The result output unit 32 causes the display unit 6 to display an index representing the skin condition, and in addition to or instead of this, advice regarding skin care habits. The detailed operation of each unit will be described later.

2. Principles of skin condition estimation
The principle of skin condition estimation according to an embodiment of the present invention will be described below. The estimation device 1 estimates the skin condition based on this principle.

In the stratum corneum of the skin, keratinocytes overlap to form multiple layers. The space between each keratinocyte is filled with intercellular lipids. One of these intercellular lipids is ceramide, which retains water to keep the skin moist and also acts as a barrier against external stimuli. In general, the more ceramide present in the stratum corneum, the higher the skin's moisturizing power and barrier function.

The stratum corneum can be easily collected by, for example, attaching a stratum corneum sampling tool having an adhesive layer to the skin surface and peeling it off, that is, by tape stripping. The inventor observed a stratum corneum sample attached to a stratum corneum sampling tool under a microscope and found that information about the state of the skin appears as an image feature in the image taken by the microscope. Figure 2 is a diagram explaining this. The inventor discovered that the CIA value, which is the sum of the brightness values of an image (polarized microscope image) of a stratum corneum sample taken by a polarizing microscope for each pixel, correlates with the actual amount of ceramide contained in the stratum corneum sample. Based on the results of the experiments conducted by the inventor, a graph plotting the CIA value and the actual amount of ceramide is shown in Figure 2. The inventor also discovered that the DIA value, which is the sum of the brightness values of an image (bright field image) of a stratum corneum sample taken by a bright field microscope for each pixel, correlates with the actual amount of protein contained in the stratum corneum sample. Based on the results of the experiments conducted by the inventor, a graph plotting the DIA value and the actual amount of protein is shown in Figure 2. This protein is the main component that makes up keratinocytes. The actual amount of ceramide was measured by subjecting a sample solution prepared containing a stratum corneum specimen to a known HPLC (high performance liquid chromatography). The actual amount of protein was measured by subjecting a sample solution prepared containing a stratum corneum specimen to a known protein assay BCA kit.

Hereinafter, the amount of ceramide contained in the stratum corneum sample may be referred to as the absolute ceramide amount, and the amount of protein contained in the stratum corneum sample may be referred to as the absolute protein amount. The value obtained by dividing the absolute ceramide amount by the absolute protein amount is referred to as the unit ceramide amount. It can be said that the unit ceramide amount is an amount that indicates the amount of ceramide per keratinocyte. As shown in Figure 2, the value obtained by dividing the CIA value by the DIA value (CIA/DIA) correlates with the amount of ceramide obtained by dividing the actual amount of ceramide contained in the stratum corneum sample by the actual amount of protein (unit ceramide amount based on the actual amount).

As explained above, the condition of the skin can be evaluated by the amount of ceramide present in the stratum corneum. However, because the ease with which the stratum corneum can be peeled off by tape stripping varies from person to person, the absolute amount of ceramide may not necessarily adequately reflect the condition of the skin. Therefore, it is preferable to evaluate the condition of the skin using the unit ceramide amount. The higher the unit ceramide amount, the higher the skin's moisturizing ability and barrier function, and the lower the unit ceramide amount, the lower the skin's moisturizing ability and barrier function.

The inventor further applied the above findings and found that there is a relationship between the feature amount of the polarizing microscope image extracted by machine learning and the actual amount of ceramide and the unit ceramide amount. Figure 3A is a diagram explaining this. Figure 3A shows a graph in which the estimated ceramide amount estimated from the polarizing microscope image using the machine learning model constructed by the inventor is plotted against the actual amount of ceramide measured by the above method. As can be seen from this graph, the actual amount of ceramide and the estimated ceramide amount have a correlation coefficient of about 0.81. Also, Figure 3A shows a graph in which the estimated unit ceramide amount estimated from the polarizing microscope image using a similar machine learning model is plotted against the unit ceramide amount calculated based on the actual amount. As can be seen from this graph, the unit ceramide amount based on the actual amount and the estimated unit ceramide amount have a correlation coefficient of about 0.65. In other words, the unit ceramide amount can be estimated by the image processing method based on the polarizing microscope image and the bright field image, but when machine learning is used, it can be estimated with good accuracy based only on the polarizing microscope image.

The inventor further found that there is a relationship between the feature amount of the bright-field image extracted by the same machine learning as above and the actual measured amount of protein. Figure 3B shows a graph in which the estimated protein amount estimated from the bright-field image using the same machine learning model as in Figure 3A is plotted against the actual measured amount of protein measured by the above method. As can be seen from this graph, the actual measured amount of protein and the estimated amount of protein have a correlation coefficient of about 0.88. In this way, the protein amount can be estimated with good accuracy based on the bright-field image. Therefore, in the same machine learning model, it is also possible to output the absolute ceramide amount from the input of the polarizing microscope image and output the absolute protein amount from the input of the bright-field image, and estimate the unit ceramide amount based on these outputs.

Based on these findings, the inventors have constructed a machine learning model that, when a polarized microscope image is input, outputs values corresponding to an estimated absolute ceramide amount and an estimated unit ceramide amount. The estimation device 1 uses a trained machine learning model (estimation model 22) constructed based on the above findings to estimate an index representing a skin condition and advice regarding skin care habits based on a microscope image of a stratum corneum sample (see FIG. 6). The following describes the learning of the estimation model by the learning device 100 according to one embodiment of the present invention.
3. Configuration of the learning device

Figure 4 is a block diagram showing the electrical configuration of the learning device 100. The learning device 100 is a general-purpose computer in terms of hardware, and may be the same device as the estimation device 1. The learning device 100 includes a memory unit 200, a control unit 300, a display unit 600, an input unit 400, and a communication unit 500. These units 200 to 500 are connected to each other via a communication line 800 such as a bus line or cable, and can communicate as appropriate. The display unit 600 is composed of a liquid crystal display or the like. The input unit 400 is composed of a mouse, keyboard, and touch panel, and accepts input from a user to the learning device 100. The memory unit 200 is a non-volatile storage area composed of a hard disk, flash memory, etc. The control unit 300 is composed of a CPU, ROM, RAM, etc., and may include a GPU, FPGA, ASIC, and other AI-specific chips.

A learning program 210 provided from a computer-readable recording medium such as a CD-ROM or USB memory or via a network is installed in the storage unit 200. The learning program 210 has a function of constructing a machine learning model and training the constructed machine learning model.

The memory unit 200 stores learning data 220 prepared for learning. The learning data 220 is a large number of data sets of images of stratum corneum samples captured by a microscope and correct answer data. The learning data 220 of this embodiment is a large number of data sets in which data of polarized light microscope images of stratum corneum samples are associated with absolute ceramide amounts and unit ceramide amounts measured from the same stratum corneum samples as correct answer data. The unit ceramide amount data is calculated in advance based on the absolute ceramide amounts measured directly from the stratum corneum samples and the absolute protein amounts measured directly from the stratum corneum samples. All of the polarized light microscope image data included in the learning data 220 of this embodiment has the same number of pixels and is converted to 8-bit grayscale.

The method of collecting the stratum corneum sample to create the learning data 220 is not particularly limited, but in this embodiment, a stratum corneum sample collected by tape stripping from the skin surface of the subject's arm is used. The stratum corneum collection tool used for tape stripping is not particularly limited, but it is preferable to use one that has low birefringence and excellent translucency.

The polarizing microscope image is an image of the stratum corneum attached to the exfoliant using a polarizing microscope. The magnification of the polarizing microscope is 100x in this embodiment. However, the magnification is not limited to this and may be changed as appropriate as long as it is possible to extract features of the polarizing microscope image related to the skin condition.

The method for measuring the absolute ceramide amount and absolute protein amount is not particularly limited, and known methods can be used. In the measurement method according to this embodiment, a sample solution is first prepared from a sample containing the collected stratum corneum using hexane, and ceramide is extracted from the precipitate obtained by centrifuging the sample solution. The extracted ceramide is then enzymatically decomposed to obtain a sample solution in which components derived from ceramide are fluorescently modified. The obtained sample solution is subjected to HPLC, and the measured amount of ceramide is regarded as the absolute amount of ceramide. In addition, the sample solution after measuring the amount of ceramide is used to measure the amount of protein using a known protein assay BCA kit, and the absolute amount of protein is regarded as the absolute amount of protein.

When the control unit 300 detects that the user has performed a predetermined operation via the input unit 400, it starts the learning program 210. The learning unit 310 constructs a machine learning model through a predetermined operation by the user. The type of machine learning model is not particularly limited, but in this embodiment, it is a neural network as shown in FIG. 3. The number of intermediate layers of the neural network is not particularly limited and can be set by the user as appropriate. When the neural network is constructed, the learning unit 310 reads the learning data 220 through a predetermined operation by the user, and learns the neural network using the data of the polarized microscope image as input and the absolute ceramide amount and unit ceramide amount (i.e., data corresponding to the correct answer data) as output. The number of learning epochs is not particularly limited and can be set appropriately. In addition, learning may be repeated until the error due to verification meets a predetermined standard, rather than a predetermined number of times. The learning unit result output unit 320 displays the learning result for each time on the display unit 600, and stores the parameters at the end of learning in the learning result storage area 230 of the memory unit 200. The machine learning model to which the learned parameters are applied is stored in the estimation device 1 as the estimation model 22, and is applied to the processing of the estimation device 1. In other words, the estimation model 22 is an inference program incorporating the learned parameters, and causes a computer to output or estimate the unit ceramide amount and absolute ceramide amount by inputting a polarized microscope image. The estimation model 22 itself may be stored as an inference program in a non-volatile storage medium independent of the storage unit of the estimation device 1. Hereinafter, the estimation model 22 may be referred to as a neural network 22.

4. Operation of the Estimation Device
The operation of the estimation device 1 to which the neural network 22 trained by the learning device 100 is applied will be described below with reference to Figures 5 and 6. The estimation device 1 estimates a unit ceramide amount from the feature amount of an image of a stratum corneum sample, and outputs advice on skin care habits tailored to the subject based on the estimated unit ceramide amount. The estimation device 1 is connected to an external microscope (not shown) so as to be able to communicate with it via wired or wireless communication. When the estimation program 20 of the estimation device 1 is started and preparations are made to read an image captured by the microscope, the process reaches the start state shown in the flowchart of Figure 5.

First, an image is prepared for reading by the estimation device 1. Specifically, the user collects a stratum corneum sample from the skin surface of the subject's arm by tape stripping. The stratum corneum sampler used for collection is preferably the same as that used when creating the learning data 220. Next, the user uses a microscope to capture an image of the stratum corneum sample attached to the stratum corneum sampler. The magnification of the microscope can be, for example, 100 times. In this embodiment, the microscope used at this time is a polarizing microscope.

In step S1, the control unit 3 receives a command from the user and causes the image acquisition unit 30 to read an image of the stratum corneum sample. The image acquisition unit 30 reads the image and temporarily stores the image data in a RAM or the like.

In step S2, the image acquisition unit 30 converts the temporarily stored image into an image of the same format and size as the image data contained in the learning data 220. In this embodiment, the image acquisition unit 30 converts the temporarily stored image into an 8-bit grayscale image.

When the image conversion is completed, the process continues to step S3. In step S3, the estimation unit 31 uses the neural network 22 trained by the learning device 100 to estimate an index representing the skin condition from the index indicating the characteristics of the loaded image. In this embodiment, the index representing the skin condition is the unit ceramide amount. More specifically, when the estimation unit 31 inputs the loaded image to the neural network 22, values corresponding to the estimated ceramide amount and the unit ceramide amount are output from the neural network 22. Hereinafter, the unit ceramide amount at this time may be referred to as the estimated unit ceramide amount. Herein, the unit ceramide amount may be the value of the unit ceramide amount itself, or may represent a predetermined category classified according to the value of the unit ceramide amount. In this embodiment, the output value of the neural network 22 represents the value of the ceramide amount and the unit ceramide amount.

In step S4 of this embodiment, the estimation unit 31 classifies the estimated unit ceramide amount into a category according to its value based on the estimated unit ceramide amount. The categories are, for example, ceramide levels classified in stages from the largest unit ceramide amount. More specifically, they may be ceramide levels 1 to 5, etc., classified in five stages from the largest unit ceramide amount. The categories may also represent moisture-related states, such as "moisturized," "normal," and "dry," classified in stages from the largest unit ceramide amount. The categories may also represent skin barrier function states, such as "high," "normal," and "low," classified in stages from the largest unit ceramide amount. When learning the estimation model, learning may be performed by including data representing these categories in the correct answer data, and data representing the category into which the subject's skin condition is classified may be included in the output derived from the neural network 22.

In the next step S5, the result output unit 32 reads out from the database 21 advice on skin care habits associated with the index representing the skin condition, i.e., the unit ceramide amount, and displays it on the display unit 6 together with the estimated unit ceramide amount. The display mode of the index representing the skin condition is not particularly limited. For example, it can be displayed in the form of a numerical value representing the estimated unit ceramide amount, a character string representing a category into which the value of the estimated unit ceramide amount is classified, an icon representing a category into which the value of the estimated unit ceramide amount is classified, and a graphic indicating the location of the category to which the value of the estimated unit ceramide amount corresponds on a scale representing the entire category. In the example shown in FIG. 6, a character string such as "ceramide level: 4" is displayed as a character string representing the category into which the value of the estimated unit ceramide amount is classified. In addition, the advice on skin care habits can include information specifying a product for further improving or maintaining the skin condition of the subject according to the skin condition. The product includes cosmetics, quasi-drugs, dietary supplements, foods, medicines, etc. The advice on skin care habits may also include information on lifestyle habits that may affect the condition of the skin, such as face washing methods, diet, and sleep time, as well as an introduction to products to improve these lifestyle habits. In the example shown in FIG. 6, an external view of the product is displayed along with a character string that identifies the product, such as "The recommended product is XX." When learning the estimation model, learning may be performed by including data representing advice on skin care habits in the correct answer data, and the data representing advice on skin care habits that is estimated to be suitable for the subject's skin may be included in the output derived from the neural network 22.

The user can present the subject with the indicators representing the skin condition displayed by the estimation device 1 and advice regarding skin care habits. The subject can recognize his/her own skin condition and obtain advice regarding skin care habits suitable for his/her skin condition without being bound by his/her own or the user's subjective opinion. The user can also provide further explanations regarding the subject's skin condition, recommended products, and skin care habits, and answer questions from the subject.

<5. Application form>
The estimation device 1 according to an embodiment of the present invention can be used by a beauty professional engaged in beauty services to sell skin care products to a customer in, for example, an aesthetic salon, a cosmetic store, a clinic, etc. In other words, the user of the embodiment can be a beauty professional, and the subject can be a customer who receives beauty services.

6. Modifications
Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the present invention. The following modified examples can be combined as appropriate.

(1) The output of the estimation model 22 of the estimation device 1, which uses a polarized microscope image as input, may be only the estimated unit ceramide amount, or may be only advice regarding skin care habits. Also, the output of the estimation model 22 may be the estimated unit ceramide amount and advice regarding skin care habits, or may be the estimated unit ceramide amount, the estimated ceramide amount, and advice regarding skin care habits. The correct answer data of the learning data 220 may be the same type of data as the output of the estimation model 22. For example, when the output of the estimation model 22 is a unit ceramide amount, the correct answer data may be the unit ceramide amount. Also, when the output of the estimation model 22 is advice regarding skin care habits, the correct answer data may be advice regarding skin care habits. Similarly, when the estimation model 22 has multiple types of outputs, the correct answer data may be the multiple types of data.

(2) The result output unit 32 may display only one of the indicators representing the skin condition or advice regarding skin care habits on the display unit 6. In cases such as when only the indicators representing the skin condition are displayed on the display unit 6, the configuration of the database 21 may be omitted in the estimation device 1.

(3) The subject who collects the stratum corneum sample, the subject who images the stratum corneum sample under a microscope, and the subject who operates the estimation device 1 may each be different subjects. Furthermore, the subject who collects the stratum corneum sample may not be the user of the estimation device 1, but the subject himself. In this case, a remote subject can collect his or her own stratum corneum sample and send the collected stratum corneum sample to the user who uses the estimation device 1. The user can have the estimation device 1 estimate the subject's skin condition from the sent stratum corneum sample. The user can provide at least one of the indicators representing the skin condition output by the estimation device 1 and advice on skin care habits to the subject in the form of a document, electronic data, audio data, etc.

(4) The result output unit 32 may output at least one of the indicators representing the skin condition and the advice regarding skin care habits onto a medium such as paper using a printer. In this case, the display unit 6 may be omitted from the estimation device 1.

(5) In the above embodiment, an example was shown in which polarizing microscope image data was used as the image data input to the neural network 22 and the image data included in the learning data 220, but the microscope image data is not limited to this. As long as the image data has features related to at least one of the absolute protein amount and absolute ceramide amount contained in the stratum corneum sample, image data from various types of microscopes can be used. For example, as described above, the feature amount of the bright field image is related to the absolute protein amount. Therefore, the estimation model 22 may be configured to receive bright field image data and polarizing microscope image data as input, and to output the absolute protein amount and absolute ceramide amount. In this case, the output may further include at least one of the estimated unit ceramide amount and advice on skin care habits. Alternatively, the estimation unit 31 may estimate at least one of an index representing the skin condition and advice on skin care habits based on the absolute protein amount and absolute ceramide amount output from the estimation model 22. The estimation model 22 may also be configured to receive bright field image data and polarizing microscope image data as input, and to output the index representing the skin condition and advice on skin care habits.

(6) Furthermore, the data set constituting the training data 220 may be appropriately changed according to the input and output of the estimation model 22. For example, the training data 220 may be a data set in which bright field image data and polarized microscope image data are combined with absolute protein amount and absolute ceramide amount data as correct answer data. Furthermore, the correct answer data may include at least one of an estimated unit ceramide amount and advice on skin care habits. Furthermore, the training data 220 may be a data set in which bright field image data and polarized microscope image data are combined with at least one of an index representing a skin condition and advice on skin care habits as correct answer data.

(7) The estimation device 1 may store the image of the stratum corneum sample acquired by the image acquisition unit 30 in the memory unit 2. At this time, data identifying the subject, data on the date the image was captured, the estimated skin condition, etc. may be stored in association with the image. In such a configuration, for example, a user can compare the skin condition of the same subject over time, and verify the effectiveness of advice regarding skin care habits practiced by the subject.

(8) The learning device 100 and the estimation device 1 use a neural network as a machine learning model to estimate the skin condition, but the machine learning model is not limited to this, and any other known model such as a support vector machine, a regression model, or a decision tree model may be used.

(9) The estimation device 1 may not include the estimation unit 31, and the function of the estimation unit 31 may be provided in another piece of hardware. For example, the estimation model 22 stored in a server that provides a cloud service may estimate the unit amount of ceramide.

(10) The learning device 100 may not include the learning unit 310, and the functions of the learning unit 310 may be provided in other hardware. For example, a program stored in a server that provides a cloud service may construct and learn a machine learning model.

<7. Features>
(1) According to the embodiment, the estimation device 1 estimates the skin condition of a subject from an image of a stratum corneum sample collected from the subject. In this manner, the skin condition can be estimated without directly analyzing the components of the stratum corneum sample, which is convenient and easy to use for both the user and the subject, and significantly reduces the time burden.

(2) According to the estimation device 1 of the above embodiment, the skin condition is estimated based on a large amount of data and is not dependent on the subjective opinion of the user or the subject. This reduces the effort required of the user to evaluate the skin condition through a medical interview or component analysis of the stratum corneum. This allows the user to focus more on suggesting skin care habits, including product introductions, in order to improve or maintain the subject's skin condition.

Reference Signs List 1 Estimation device 2 Storage unit 3 Control unit 4 Input unit 5 Communication unit 100 Learning device

Claims (12)

an image acquisition unit that acquires an image of a stratum corneum sample taken from the surface of the skin using a microscope;
an estimation unit that inputs the acquired image into a trained machine learning model, derives an output from the machine learning model, the output being a unit ceramide amount representing the amount of ceramide per keratinocyte contained in the stratum corneum sample , and estimates advice on skin care habits suitable for the skin based on the output .
A skin condition estimation device. The image includes an image captured by a polarizing microscope.
The estimation device according to claim 1 . the estimation unit estimates advice regarding skin care habits suitable for the skin using a machine learning model trained using learning data, which is a data set of an image of a stratum corneum sample taken from the surface of the skin captured by a microscope and ground truth data ;
The correct data is the unit ceramide amount based on a value measured from a stratum corneum sample.
The estimation device according to claim 1 or 2. Further comprising a result output unit that outputs the result estimated by the estimation unit.
The estimation device according to any one of claims 1 to 3. Obtaining an image of a stratum corneum sample taken from the surface of the skin using a microscope;
inputting the acquired image into a trained machine learning model to derive an output from the machine learning model, the output being a unit ceramide amount representing an amount of ceramide per keratinocyte contained in the stratum corneum sample ; and estimating advice regarding a skin care habit suitable for the skin based on the output .
A method for estimating skin condition. The image includes an image captured by a polarizing microscope.
The estimation method according to claim 5. Obtaining an image of a stratum corneum sample taken from the surface of the skin using a microscope;
inputting the acquired image into a trained machine learning model, deriving an output from the machine learning model which is a unit amount of ceramide representing an amount of ceramide per keratinocyte contained in the stratum corneum sample , and estimating advice regarding a skin care habit suitable for the skin based on the output ;
A program to estimate skin condition. The image includes an image captured by a polarizing microscope.
The estimation program according to claim 7. A memory unit that stores learning data, which is a data set of an image of a stratum corneum sample taken from the surface of the skin captured by a microscope and ground truth data;
a learning unit that uses the learning data to learn a machine learning model so that when the image is input to the machine learning model, data corresponding to the correct answer data is output;
The correct data is a unit amount of ceramide representing the amount of ceramide per keratinocyte contained in the stratum corneum sample.
Learning device. The image includes an image captured by a polarizing microscope.
The learning device according to claim 9. A step of preparing learning data which is a data set of an image of a stratum corneum sample taken from the surface of the skin captured by a microscope and ground truth data;
and performing machine learning using the learning data so that, when an image of a stratum corneum sample taken from the surface of the skin captured by a microscope is input, data corresponding to the ground truth data is output;
The correct data is a unit amount of ceramide representing the amount of ceramide per keratinocyte contained in the stratum corneum sample.
How to generate a trained model. The image includes an image captured by a polarizing microscope.
A method for generating a trained model according to claim 11.