JP7679929B2 - Method for generating data for human skin tactile sensation, device for generating data for human skin tactile sensation, method for evaluating human skin tactile sensation, device for evaluating human skin tactile sensation, device for presenting human skin tactile sensation, and method for presenting human skin tactile sensation - Google Patents

Description

The present invention relates to a method for generating data for human skin tactile sensation, a device for generating data for human skin tactile sensation, a method for evaluating human skin tactile sensation, a device for evaluating human skin tactile sensation, and a method for presenting human skin tactile sensation.

A conventional method for evaluating human skin is to photograph the skin and perform image analysis of the skin surface pattern to evaluate the apparent skin condition.

In recent years, in the cosmetics field, there has been a demand for tactile devices that allow multiple people to share their skin condition. However, conventional methods that simply analyze images of two-dimensional skin patterns do not anticipate converting the images into acceleration information (vibrations), making it difficult to reproduce a skin feel that can be shared by multiple people.

On the other hand, in order to present different textures even with the same material, a tactile sensation reproduction device has been proposed that can change the tactile sensation when touching a material with a fingertip by applying depth-wise vibrations to the contact material (for example, non-patent document 1).

However, while non-patent document 1 considers a hard contact material whose surface does not change, in the case of skin, the material itself has softness and viscoelasticity that causes it to deform, so if the reproduction of the tactile sensation caused by vibration for a contact material that does not change is applied to a skin model, errors will occur in the reproduced tactile sensation.

In addition, non-patent document 2 considers the friction between a finger and an artificial skin model in order to analyze the sensation of soft skin felt by a finger, and shows that the tendency for the contact area and normal force to change differs depending on the surface hardness of the finger and the artificial skin model that is the subject of contact.

Furthermore, Patent Document 1 proposes that in order to evaluate skin to which cosmetics have been applied, a vibration sensor be used to measure the vibrations that occur when the skin is touched with a finger as an evaluation criterion, and a relationship be established in advance by matching the sensory evaluation of the feel of the applied cosmetics with the change in frequency of the measured vibrations.

WO2019/030803 publication

Shuhei Asano, Shogo Okamoto, and Yoji Yamada. Increase and decrease of texture roughness by vibrotactile stimulation. IEEE Trans. Human-Mach. Syst. 45, 393-398, 2015. Kouki Inoue, Shogo Okamoto, Yasuhiro Akiyama, Yoji Yamada, Effect of material hardness on friction between fingertips and dry and lubricated joint skin, IEEE Transactions on Haptics, vol.13, no.1, pp.123-129, 2020

Here, if one were to apply the friction technology in Non-Patent Document 2 to reproducing the feel of skin, it would be necessary to distinguish between cases based on the surface hardness of the finger and the object in contact, and because the surface hardness of each person's fingertips differs, it would be difficult to share various skin feel sensations among multiple people.

In addition, in the method of Patent Document 1, a vibration sensor was used to collect vibrations when actually touching the skin with a finger as a sample for evaluation criteria, but there was a risk that the collected vibrations would be influenced by the characteristics of the person touching (such as the hardness of the finger and the way of touching).

In view of the above circumstances, the present invention aims to provide a method for generating human skin tactile data that collects information on human skin without touching the skin or using a special tactile sensor, minimizes errors due to the characteristics of the person touching the skin, and generates human skin tactile data that can be shared by multiple people.

In order to solve the above problems, in one aspect of the present invention,
acquiring an image of skin, the image being human skin or a skin model;
and generating human skin tactile data estimated as vibrations generated when touching the human skin or a skin model, using the acquired skin image.

According to one aspect, in a method for generating human skin tactile data that generates human skin tactile data that can be shared by multiple people, human skin information can be collected without touching the skin or using a special tactile sensor, minimizing errors due to the characteristics of the person touching it.

1 is a schematic flow diagram showing a method for generating human skin tactile data according to the present invention. FIG. 1 is a functional block diagram of a human skin tactile data generating device according to a first embodiment of the present invention. 4 is a detailed flow of a data generating process according to the first embodiment. 11 shows an example of a range of photographing a human skin gel, a three-dimensional image of the human skin gel simulating a person in their twenties, and a displacement curve calculated in S21 and S22 of the first embodiment. 13 shows a three-dimensional image of human skin gel simulating a person in his 40s, and an example of a displacement curve calculated in S21 and S22 of the first embodiment. 13 shows a three-dimensional image of human skin gel simulating a person in his 70s, and an example of a displacement curve calculated in S21 and S22 of the first embodiment. 13 is a graph showing an example of the frequency spectrum of a displacement curve obtained by converting the horizontal axis of the displacement curve from a distance axis to a time axis in S23 in the first embodiment. 10 is a graph showing an example of an acceleration spectrum obtained by converting the spectrum of the displacement curve into acceleration in S24 in the first embodiment. 11 is a graph showing an example of corrected acceleration information obtained by converting the acceleration spectrum by applying a low-pass filter in S25 in the first embodiment. 1 is a schematic flow diagram showing a method for presenting a human skin tactile sensation using human skin tactile sensation data generated by the human skin tactile sensation data generating method of the present invention. FIG. 1 is an external view showing an example of a human skin tactile sensation providing device for realizing the tactile sensation providing process of the present invention. FIG. 12 is a hardware block diagram of the human skin tactile sensation presentation device of FIG. 11 . A diagram showing different types of human skin gels being directly touched with a finger to distinguish them. 13 is a diagram showing the skin type discrimination results when the human skin gel is directly touched with a finger. FIG. 13 is a diagram showing a state in which vibration data is created by the human skin tactile sensation data generation method including image acquisition of the present invention, and the vibration data is presented as a tactile sensation by a human skin tactile sensation presentation device. FIG. 13 is a diagram showing the results of skin type discrimination when a tactile sensation is presented by a human skin tactile sensation presentation device using vibration data generated by the method of the present invention. As a comparative example, this figure shows a state in which vibration data is created by touching an artificial finger AF equipped with an accelerometer α to a skin sample in a human skin tactile sensation presentation device, and the vibration data is presented as a tactile sensation by the human skin tactile sensation presentation device. FIG. 13 is a diagram showing the results of skin type discrimination when a tactile sensation is reproduced by a human skin tactile sensation presentation device using vibration data generated by a method of a comparative example. FIG. 11 is a functional block diagram of a human skin tactile data generating device according to a second embodiment of the present invention. 13 is a detailed flow of a data generating process according to the second embodiment. 13 shows an example of a luminance image of human skin and an example of a displacement curve using the luminance calculated in S201, S202, and S203 of the second embodiment. 1 is a schematic flowchart showing a human skin tactile sensation evaluation method including a human skin tactile sensation data generation method of the present invention. FIG. 1 is a block diagram of a human skin texture evaluation device for implementing the human skin texture evaluation method of the present invention. FIG. 1 is a schematic diagram showing a beauty advisor presenting skin quality to a customer during counseling using the human skin texture evaluation device of the present invention. 1 is a schematic diagram of a human skin texture evaluation and presentation system using the human skin texture evaluation method of the present invention.

Hereinafter, a description will be given of an embodiment of the present invention with reference to the drawings. In the following drawings, the same components are designated by the same reference numerals, and duplicate explanations may be omitted.

The present invention relates to a method for generating data for human skin tactile sensation, i.e., a method for generating data for rendering (representing, reproducing, regenerating, presenting) the tactile sensation of human skin. The present invention also relates to a device for generating data for human skin tactile sensation, a method for evaluating the tactile sensation of human skin, a device for presenting the tactile sensation of human skin, and a method for presenting the tactile sensation of human skin, which use the method for generating data for human skin tactile sensation.

In this specification, "human skin feel" refers to the sensation felt by the fingers or palm when touching human skin by stroking or rubbing it with the fingers or palm.

<Method of generating data for human skin texture>
FIG. 1 is a schematic flow diagram showing a method for generating human skin tactile data according to the present invention.

As shown in Figure 1, in the human skin tactile data generation method of the present invention, first, in step S1, an image of skin is acquired (image acquisition process). The "skin" from which an image is acquired includes not only actual human skin, but also skin models such as skin replicas (hard artificial skin that reproduces the unevenness of the surface of human skin) and human skin gel (soft artificial skin that expresses the surface and softness of human skin). In addition, the actual human skin may be bare skin, or it may be skin after a specific cosmetic product has been applied.

The image acquisition means is a three-dimensional imaging means capable of acquiring a three-dimensional image in the first embodiment described below, and is a two-dimensional imaging means capable of acquiring a two-dimensional image in the second embodiment described below. As a modified example, it may be a two-dimensional imaging means capable of acquiring a two-dimensional UV image, a two-dimensional imaging means capable of acquiring a two-dimensional optical interference image, or a two-dimensional imaging means with an adjusted focus.

Then, in step S2, the acquired skin image is used to generate data for human skin feel (data generation process).

Here, human skin tactile data refers to tactile data felt on the fingers or palm, estimated as the vibrations that occur when touching skin (human skin or a skin model). Human skin tactile data is vibration data (presentation data (also called reproduction data, expression data, or rendered data)) that serves as the source data for the virtual texture when presenting the tactile sensation, and is also comparison data used for comparison with evaluation criteria, etc., during evaluation.

In the present invention, the details of the data generation process of step S2 will be described in the first embodiment, the second embodiment, and a modified example. Below, an example of an apparatus capable of realizing the human skin tactile data generation method of FIG. 1 will be described.

<First embodiment of the human skin tactile data generating device>
FIG. 2 is a functional block diagram of a human skin tactile data generating device according to a first embodiment of the present invention.

As shown in FIG. 2, the human skin tactile data generating device 100 of the first embodiment comprises a three-dimensional imaging means 1 which is an image acquisition unit, and an information processing device 2 which is a data generating unit.

The three-dimensional imaging means 1 acquires an image of human skin or skin that is a skin model. In this embodiment, the three-dimensional imaging means 1 is a means capable of capturing a three-dimensional image of a living body's skin or skin that is an artificial skin model, such as a three-dimensional microscope (e.g., a laser biomicroscope), an ultrasonic cross-sectional imaging device, or a 3D camera.

The information processing device 2 uses the acquired skin image to generate human skin tactile data estimated as vibrations that occur when touching human skin or a skin model. The information processing device 2 is, for example, a PC (personal computer) with specific software installed or a dedicated three-dimensional skin image analysis device.

The information processing device 2 in this embodiment is executable to have a surface roughness information acquisition unit 21, an acceleration information conversion unit 22, and a human skin tactile data output unit 23 (hereinafter simply referred to as the output unit).

The surface roughness information acquisition unit 21 acquires skin surface roughness information from the skin image. In detail, the surface roughness information acquisition unit 21 has a surface roughness extraction unit 211 that extracts the skin surface roughness (height on the skin cross section) at each position from the three-dimensional skin image, and a displacement curve acquisition unit 212 that acquires a displacement curve that connects information on the surface roughness (height on the skin cross section) at each position on a specified straight line.

The acceleration information conversion unit 22 converts the skin surface roughness information acquired by the surface roughness information acquisition unit 21 into acceleration information. The acceleration information conversion unit 22 has a displacement spectrum conversion unit 221, an acceleration spectrum conversion unit 222, and an acceleration information correction unit 223. Details of these functions will be described later with reference to FIG. 3.

The output unit 23 sets the acceleration information calculated by the acceleration information conversion unit 22 as human skin tactile data (human skin expression data) and outputs it.

Note that Figure 2 shows a configuration in which the functions of the surface roughness information acquisition unit 21, acceleration information conversion unit 22, and output unit 23 are realized by a single information processing device 2, but these calculation functions may be executed using two or more information processing devices.

(Data Generation Process in the First Example)
Details of the data generation process executable by the information processing device 2 shown in Fig. 2 will be described with reference to Fig. 3 to Fig. 9. Fig. 3 is a detailed flow of the data generation process according to the first embodiment.

The information processing device 2 of Figure 1 is capable of executing the following data generation processes: a process of acquiring skin surface roughness information from a skin image (S21, S22), a process of converting the skin surface roughness information into acceleration information (S23, S24, S25), and a process of setting the acceleration information to human skin tactile data (S26).

Here, the inventors of the present application generated data for human skin feel using sample A of human skin gel simulating a person in their 20s, sample B of human skin gel simulating a person in their 40s, and sample C of human skin gel simulating a person in their 70s. Figures 4 to 9 show the data generated at each stage of the process.

The human skin gels used in samples A, B, and C are an example of a skin model (artificial skin) that mimics a specific skin condition, and are made of ultra-soft urethane resin that is as soft as human skin. The human skin gel used in the following calculations was Bioskin (registered trademark) (manufactured by Bealux).

In more detail, in the process of acquiring the skin image shown in Figure 1 in this embodiment, a three-dimensional image of the surface of human skin or a skin model is acquired as the skin image using a laser microscope, which is an example of a three-dimensional imaging means 1 shown in Figure 2.

Below, as an example of a laser microscope for acquiring images, a 3D measuring microscope (VR-3100, Keyence) that acquires 3D images of specific areas of a sample surface by laser scanning was used to acquire 3D images of an area of 24 mm wide and 18 mm long on the human skin gel. Figure 4(a) shows the photographed area and horizontal centerline on human skin gel sample A. The horizontal centerline is an example of a straight line at a specified position that can be extracted from the cross-sectional height information contained in the 3D image.

In the three-dimensional image acquired here, height information for each position on the skin cross section can be seen depending on the color. For example, Fig. 4(b) is a three-dimensional image of human skin gel A simulating a person in their 20s, Fig. 5(a) is a three-dimensional image of human skin gel B simulating a person in their 40s, and Fig. 6(a) is a three-dimensional image of human skin gel C simulating a person in their 70s. Here, in the figures, white parts indicate a high skin surface position (height on the skin cross section), gray parts indicate a medium skin surface position, and black parts indicate a low skin surface position, i.e., concave.

By extracting the color of the three-dimensional image for each position in this manner, in step S21, the height on the skin cross section at each position is extracted from the three-dimensional image of the skin, i.e., the surface roughness at each position of the skin.

Referring to the scale bars on the left of Figures 4(b), 5(a), and 6(a), in human skin gel A simulating a person in their 20s shown in Figure 4(b), the skin surface position is in the range of -0.037 mm to +0.045 mm, in human skin gel B simulating a person in their 40s shown in Figure 5(a), the skin surface position is in the range of -0.048 mm to +0.049 mm, and in human skin gel C simulating a person in their 70s shown in Figure 6(a), the skin surface position is in the range of -0.124 mm to +0.11 mm. This shows that the unevenness of the skin surface increases in the following order: human skin gel C (70s) > human skin gel B (40s) > human skin gel A (20s).

The displacement curve is obtained by connecting horizontally the skin surface positions (heights on the skin cross section) on a straight line at a specific position (for example, the horizontal center line indicated by the arrow in Figure 4(a)) that can be extracted from the cross-sectional height information contained in such a three-dimensional image.

That is, in step S22, based on the surface roughness at each position on the skin (height on the skin cross section at each position), a displacement curve on a straight line corresponding to the cross-sectional height position of the three-dimensional skin, with distance on the horizontal axis, is obtained as surface roughness information, assuming that a predetermined distance is scanned along a straight line on a three-dimensional image of the surface of human skin or a skin model.

Specifically, from the three-dimensional image of human skin gel A in Fig. 4(b), assuming that the horizontal center line indicated by the arrow in Fig. 4(a) is scanned for a distance of 24 mm, the horizontal width of the shooting range, which is the length of the arrow, each skin surface position (cross-sectional height) in the horizontal direction becomes the displacement curve. Here, the displacement curve is the height at each position on the cross section based on the horizontal center line, that is, the roughness of the line obtained on the horizontal center line, and therefore the displacement curve can be said to show the surface roughness information of the skin on a straight line of a predetermined length.

In addition, the extraction of the height on the skin cross section at each position from the three-dimensional image and the acquisition of the displacement curve in S21 and S22 were analyzed and processed in a software environment (VR-3000 G2 APPLICATION) set up in the information processing device 2.

The displacement curve in Fig. 5(b) shows the respective skin surface positions (cross-sectional heights) in the horizontal direction when the horizontal center line indicated by the arrow in the three-dimensional image of human skin gel B in Fig. 5(a) is scanned a distance of 24 mm in width. The displacement curve in Fig. 6(b) shows the respective skin surface positions (cross-sectional heights) in the horizontal direction when the horizontal center line indicated by the arrow in the three-dimensional image of human skin gel C in Fig. 6(a) is scanned a distance of 24 mm in width.

In Figures 4(c), 5(b), and 6(b), the horizontal axis indicates the horizontal position on a straight line in the image, and the vertical axis indicates the height on the skin cross section, i.e., the cross-sectional position of each position relative to the average cross-sectional position of the skin surface.

Comparing Figures 4, 5, and 6, it can be seen that as the color unevenness on the three-dimensional image becomes larger and the surface roughness increases (human skin gel A < human skin gel B < human skin gel C), the displacement curve, which is the line roughness profile, shows a large vibration amplitude.

Here, since the vibration for presenting (reproducing) the tactile sensation of the skin is based on acceleration, the obtained displacement curve (line roughness curve) needs to be converted into an acceleration curve.

Therefore, in step S23, the moving speed when touching human skin or a skin model is assumed, and the horizontal axis of the linear displacement curve, which is the surface roughness information, is converted from distance to time to create a spectrum of the displacement curve.

As mentioned above, the displacement curve indicates the height of the skin surface (height of the skin cross section) at each position on the selected line. Therefore, for example, if this curve is placed on a time axis with a length of about 1 second, it can be considered as a displacement curve of the surface height when the probe horizontally scans a distance of 24 mm in 1 s (second), and can be converted into a spectrum of the displacement curve.

Figure 7(d) shows the spectrum of the displacement curve converted based on the displacement curves of human skin gels A, B, and C of each age shown in Figures 7(a), (b), and (c). In Figure 7(d), the horizontal axis shows frequency (Hz) with 1 second set to frequencies from 0 to 500 Hz, and the vertical axis shows amplification (dB). In other words, the spectrum of the displacement curve shows the amplitude characteristics at each frequency.

The conversion of this displacement curve into a displacement curve spectrum was performed using audio software (e.g., Audacity (registered trademark)) installed in the information processing device 2.

Then, in step S24, the spectrum of the displacement curve is converted into a spectrum of acceleration.

In more detail, when the displacement curve is Fourier transformed, the original curve is decomposed into sine waves of each frequency, and the equation for the sine wave with frequency i becomes equation (1) below. Since the Fourier transform serves to decompose a time series waveform into sine waves of each frequency, the resulting equation (1) contains the equation for a sine wave with frequency i (i = 1 to 500 Hz) within the sine wave. By taking the second-order derivative of this equation (1), we obtain equation (2) for acceleration.

The negative sign in equation (2) can be removed by adding a phase difference of π (180 degrees) to the sine curve, and in the case of haptic rendering, this phase change has a very small effect on perception, so equation (3) is obtained by simply multiplying equation (1) by the ^ω2 element. In other words, it can be converted into an acceleration waveform by multiplying each frequency component by ^ω2 . When performing this processing using a software equalizer (e.g., Audacity), it is necessary to consider processing on a logarithmic graph because the amplitude is expressed in logarithm, but this can be done by taking the logarithm of equation (3), and when expressed in dB, it becomes equation (4).

The conversion from the spectrum of the displacement curve to the acceleration spectrum was performed by the equalizer function of audio software (e.g., Audacity) installed in the information processing device 2. When the above formulas (1) to (4) were calculated by the equalizer and applied to the displacement curve shown in the lower part of Figure 8, an acceleration curve (acceleration spectrum) like that shown in the upper part of Figure 8 was obtained.

Here, in the acceleration spectrum shown in the upper part of Figure 8 and in Figure 9 (a), it can be seen that the amplitude increases significantly above 150 Hz.

Therefore, in step S25, the acceleration spectrum is corrected by applying a low-pass filter. Figure 9(b) is a graph showing an example of corrected acceleration information converted in S25 by applying a low-pass filter to the acceleration spectrum in the first embodiment.

Here, the area of the image of the gel taken to obtain the above displacement curve is larger than the area of the actual contact area of the finger with the gel, and the contact area between the finger and the gel acts as a high-frequency blocking (low-pass) filter, resulting in a deviation in vibrations above 150 Hz. Therefore, a low-pass filter was applied to bring the acceleration spectrum closer to the acceleration spectrum actually measured.

For example, in S25, the amplitude of vibration in the range of 150 Hz or higher in the acceleration spectrum shown in Figure 9(a) is reduced by -12 dB using a low-pass filter function of audio software (e.g., Audacity) set in the information processing device 2, and finally a spectrum such as that shown in Figure 9(b) is obtained as corrected acceleration information.

The corrected acceleration information calculated in this manner (vibration data in Figure 9 (b)) is set as the human skin tactile sensation data as the result of haptic rendering for reproducing the actual human skin tactile sensation (step S26).

The human skin feel data generated in this way is used to present and evaluate the human skin feel as described below.

<Method for presenting human skin sensation>
FIG. 10 is a schematic flow diagram showing a method for presenting a human skin tactile sensation using the human skin tactile sensation data generated by the human skin tactile sensation data generating method of the present invention.

The human skin tactile sensation presentation method shown in Figure 10 includes a process of presenting as vibration on a human skin model in step S3 using human skin tactile sensation data (acceleration information) generated by the human skin tactile sensation data generation method S1 and S2 shown in Figure 1.

In the tactile sensation presentation process of step S3, the human skin tactile sensation data (acceleration information) generated in the data generation process is used to present the tactile sensation. When presenting the tactile sensation, evaluation criteria may be established in advance, and the tactile sensation may be presented based on the criteria. The establishment of the evaluation criteria will be described later together with the tactile sensation evaluation method (see FIG. 22).

In addition, in Figure 10, in the human skin tactile sensation presentation method, a step of executing a human skin tactile sensation data generation method is provided prior to the presentation step. However, if pre-stored human skin tactile sensation data (e.g., human skin tactile sensation data for a reference skin) is called up to present the sensation, S1 and S2 can be omitted and only the tactile sensation presentation step S3 is required.

<Human skin sensation presentation device>
Here, an example of a human skin tactile sensation presentation device that realizes the tactile sensation presentation process of Fig. 10 will be described with reference to Fig. 11 and Fig. 12. Fig. 11 is an external view showing an example of a human skin tactile sensation presentation device that realizes the tactile sensation presentation process of the present invention.

As shown in FIG. 11, the human skin tactile sensation presentation device 300 comprises a human skin gel 31, a slide volume 32, a PC 33, an amplifier 34, a speaker 35, a transmission board 36, and a device base 37.

The human skin gel 31 is the object that the user's finger or artificial finger comes into contact with, and is a skin model (artificial skin) that mimics a specific state of human skin, and is made of ultra-soft urethane resin (polyurethane skin plate) that expresses the same softness as human skin. For example, Bioskin (registered trademark) (manufactured by Bealux) is used as the human skin gel 31.

The skin gel 31 is attached and fixed to the transmission plate (transmission member) 36. In FIG. 10, an example of the skin gel 31 being circular is shown, but the skin gel 31 may also be elongated in the left-right direction.

The slide volume 32 is provided above the human skin gel 31, and serves as a finger moving unit that moves the user's finger or artificial finger at a predetermined moving speed on the human skin gel 31. Alternatively, when the user's finger moves independently, the slide volume 32 also functions as a moving speed acquiring unit that acquires the moving speed of the finger.

The slide volume 32 has a slide rail 321, left and right supports 322 that support the slide rail, a variable resistor 323 (see Figure 12), a finger slider 324, and a motor 325.

The finger slider 324 is a handle part composed of a pair of clamping support parts, and when presenting a tactile sensation, the user's finger or artificial finger can be inserted between the clamping support parts, and the finger slider 324 slides left and right to move the finger or artificial finger on the human skin gel 31. Alternatively, the finger slider 324 may move together with the movement of the person's finger.

The resistance value of the variable resistor 323 changes depending on the position of the finger slider 324 relative to the slide rail 321. In other words, when no force is applied from the finger, for example when the person sandwiching the finger slider 324 relaxes his/her finger or when the finger slider 324 sandwiches an artificial finger, the resistance value of the variable resistor 323 can be changed to move the left/right position of the finger slider 324.

Alternatively, when a person's finger actively moves, the position and moving speed of the finger can be calculated by having the downstream PC 33 read the change in resistance value that occurs when the finger slider 324 moves left and right along with the finger movement. The slide rail 321, finger slider 324, and variable resistor 323 form a moving speed acquisition unit that acquires the moving speed of the finger when the user's finger moves independently.

When no force is applied from the finger, the motor 325 controls the drive to adjust the resistance value of the variable resistor 323 in order to move the finger slider 324 at a predetermined speed along the slide rail 321. The motor 325, the variable resistor 323, and the finger slider 324 form a finger moving unit when the device moves the artificial finger or the user's finger.

PC 33 is an example of an information processing device that is connected to slide volume 32 and amplifier 34 by wire or wirelessly. PC 33 controls the speed at which motor 325 of slide volume 32 is driven to set the finger movement speed on human skin gel 31 in the left-right direction and output the vibration strength according to the movement speed. Alternatively, when the user actively moves his/her finger, PC 33 calculates and determines the finger movement speed based on the finger position acquired by slide volume 32, and outputs the vibration strength.

Furthermore, PC 33 has the function of an original signal storage unit that stores original vibration signals (human skin texture data) associated with human skin texture (touch). In the following, the original signal is a signal that is the source of vibration for generating vibrations by speakers 35A and 35B, and is a signal in the form of an audio signal associated with skin texture.

Therefore, PC 33 outputs to amplifier 34 the original signal (audio signal) selected according to the skin type to be expressed and information on the signal strength associated with the finger movement speed. In other words, PC 33 also functions as a vibration adjustment unit that adjusts the strength (amplitude value) of vibration when outputting vibration according to the original signal in accordance with the finger movement speed. The hardware configuration of PC 33 will be described in detail with reference to FIG. 12.

The amplifier (acoustic amplifier) 34 amplifies the original signal (human skin tactile sensation data) output from the PC 33 according to the intensity information.

The speaker 35 is connected to the amplifier 34 and is configured as a pair of left and right speakers 35A and 35B. The speakers 35A and 35B are vibration output units that convert the adjusted audio signal into physical vibrations and output them, and impart vibrations to the human skin gel 31 via the transmission plate 36.

The speakers 35A and 35B are connected to the amplifier 34 by wire or wirelessly. The speakers 35A and 35B are in contact with the transmission plate 36 provided with the human skin gel 31, and support the transmission plate 36 by sandwiching the speakers 35A and 35B so that vibrations are transmitted.

In detail, speakers 35A and 35B have a circular diaphragm 351 that generates physical vibrations, a frame 352 that surrounds the diaphragm 351, and a support 353 to which the frame 352 is attached.

In addition, a slit 354 is formed in the vertical center of the opposing faces of the speakers 35A and 35B, extending from the surface of the diaphragm 351 where vibrations are generated toward the back. When the propagation plate 36 with the human skin gel 31 attached is inserted into the left and right slits 354, the speakers 35A and 35B sandwich the diaphragm 351 from the left and right, so that the vibrations generated by the diaphragm 351 are propagated to the propagation plate 36.

The transmission plate 36 is a plate to which the human skin gel 31 is attached on its upper surface, supporting the human skin gel 31 and transmitting vibrations. The transmission plate 36 is made of, for example, an acrylic plate.

The device base 37 supporting the slide volume 32 and speakers 35A, 35B includes a fixed plate 371, which is an acrylic plate below the propagation plate 36, a base 372, and a cushioning material 373.

The lower ends of the left and right frames 352 of the speakers 35A and 35B are fixed to the upper surface of the fixing plate 371. The lower surface of the fixing plate 371 is attached to the upper surface of the base 372 (apparatus base, device base) via a cushioning material 373. The cushioning material 373 prevents vibrations from the speakers 35A and 35B from propagating to the base 372, so that even if the human skin tactile sensation presentation device 300 is placed on a desk 9 (see FIG. 23), vibrations are not propagated to the surface of the desk 9, making it less likely to be damaged.

In the above, an example was described in which the human skin gel 31 is the same as room temperature, but in order to make the feel of the human skin gel closer to that of real human skin, a heating section that heats the human skin gel 31 to 15°C to 37°C may be provided on the back side of the transmission plate 36.

Figure 12 is a hardware block diagram of the human skin tactile sensation presentation device 300 of Figure 11.

The PC 33 includes a USB interface 301, a memory 302, an operation system 303, a sound controller 304, an audio device connection unit 305, a system bus 306, an input unit 307, a display unit 308, etc. The input unit 307 is a mouse, a keyboard, a touchpad, a touch panel input surface, etc., and the display unit 308 is an LCD (liquid crystal display), an organic EL display, etc.

The USB interface 301, memory 302, operation system 303, sound controller 304, audio device connection unit 305, input unit 307, and display unit 308 are connected via a system bus 306. Alternatively, if PC 33 is a desktop personal computer, the input unit 307 and display unit 308 may be provided as separate units so as to be connected to the main body via the USB interface 301.

The USB interface 301 receives finger position information obtained by the variable resistor 323 of the slide volume 32.

The memory 302, which serves as an original signal storage section, stores in advance an original signal (human skin tactile data), which is an audio signal that is the source of vibrations corresponding to human skin. The human skin tactile data is acquired by a communication section (not shown) of the PC 33 transmitting and receiving data to and from the human skin tactile data generation device 100. Alternatively, data may be transferred from the human skin tactile data generation device 100 to the PC 33 via a storage medium such as a USB memory or a memory card.

The operation system (OS) 303 has, for example, a CPU (Central Processing Unit) 331, an FPGA (Field-Programmable Gate Array) 332, a RAM (Random Access Memory) 333, a ROM (Read Only Memory) 333, etc.

The CPU 331 and FPGA 332 use the RAM 333 as a working area to execute various control programs stored in the ROM 334 and output control commands to control various operations in the PC 33.

In OS 303, for example, CPU 331 changes the resistance value of variable resistor 323 of slide volume 32 by generating a signal for driving and controlling motor 325.

In the slide volume 32, the motor 325 is set to move the finger slider 324 along a constant trajectory at a predetermined moving speed by changing the resistance value of the variable resistor 323, for example, by PMW (Pulse Width Modulation) and PID (Proportional Integral Differential) control. This sets the moving speed of the finger or artificial finger on the human skin gel 31 during tactile sensation presentation.

Alternatively, if the user actively moves his/her finger, the OS 303, for example the CPU 331, reads the change in the resistance value of the variable resistor 323 of the slide volume 32 and calculates the moving speed of the finger on the human skin gel 31 during the presentation of the tactile sensation from the time derivative of the position.

Also, in OS 303, FPGA 332 calls the original signal stored in memory 302.

The sound controller 304 adjusts the volume of the called original signal (human skin tactile data) according to the movement of the finger. For example, it adjusts the volume so that the finger does not vibrate while it is stationary, but vibrates only after it starts to move, and adjusts the volume so that the faster the movement, the larger the amplitude of the sound signal is output.

The audio equipment connection unit 305 is a terminal that connects to the amplifier 34, and outputs the audio signal adjusted by the sound controller 304 to the amplifier 34.

The sound controller 304 and the audio device connection unit 305 function as a vibration adjustment unit that outputs the finger movement speed and the original signal (human skin tactile sensation data), which is an audio signal, and the adjustment amount of the amplitude of the waveform of the original signal.

In this way, in the human skin tactile sensation presentation device 300 of the present invention, the PC 33 outputs a waveform (audio signal) that has been adjusted from the human skin tactile sensation data according to the finger movement speed to the amplifier 34, and the adjusted audio signal is converted into physical vibrations and output by the speakers 35A, 35B, and the vibrations are applied as a tactile sensation to the human skin gel 31 via the transmission plate 36, thereby reproducing the virtual texture.

<Experimental Example>
Here, in order to confirm the validity of the reproduction of the human skin tactile data generated by the method of the present application, the inventors of the present application conducted an experiment to see whether the three types of skin types could be distinguished by reproducing the human skin tactile data with the human skin tactile sensation presentation device 300 and by other methods. The subjects were six right-handed people aged 24±1 year old with no functional problems in their sense of touch.

(Experiment 1)
The artificial skin samples used in the experiment had similar textures, so subjects may have difficulty distinguishing between the samples. Therefore, in Experiment 1, we conducted an experiment to confirm whether subjects could distinguish between the three different samples by directly touching actual skin samples, as shown in Figure 13. Figure 13 shows the state in which multiple types of human skin gels are directly touched with a finger to distinguish them.

First, the experimenter presented one sample in front of the subject, who was instructed to explore it for three seconds with their eyes closed. Three samples were then presented, and the subject was asked to explore each sample within five seconds, and finally to choose the sample they had explored first by answering A, B, or C. The three samples were A, a human skin gel that imitated the skin of a person in their 20s, B, a human skin gel that imitated the skin of a person in their 40s, and C, a human skin gel that imitated the skin of a person in their 70s.

The results are shown in Figure 14. Figure 14 shows the skin type discrimination results when the human skin gel was directly touched with a finger. The accuracy rate was over 80% for all samples A, B, and C, which means that the textures (feel to touch) of the three skin samples A, B, and C were different enough for the subjects to distinguish them.

(Experiment 2)
In experiment 1, the subjects directly touched the skin samples to be discriminated, but in experiments 2 and 3, experiments were conducted to verify whether a virtual texture reproduced using vibration data generated in advance on the human skin gel 31 of the human skin tactile sensation presentation device 300 could be correctly identified. Note that the human skin gel 31 is a human skin gel with a different feel from the samples A, B, and C.

In experiment 2, the experimenter acquired three-dimensional images of skin samples A, B, and C in advance in accordance with the human skin tactile data generation method of the present invention, and generated data based on the three-dimensional images to obtain three pieces of human skin tactile data (vibration data) that respectively imitate skin samples A, B, and C. The vibration data was stored in the human skin tactile sensation presentation device 300. Figure 15(a) is a diagram explaining how vibration data is created by acquiring an image of a skin sample using the human skin tactile data generation method of the present invention.

The subject is then instructed to insert a finger F into the finger slider 324 of the human skin tactile sensation presentation device 300 and touch the smooth human skin gel 31 with a pressure of about 0.5 N. The subject then moves his or her finger at a predetermined speed on the human skin gel 31, which is reproducing one of the three vibration data simulating skin samples A, B, and C generated by the human skin tactile sensation reproduction method of the present invention, and searches for a virtual texture on the human skin gel 31 within 10 seconds under the same search conditions. Figure 15(b) is a diagram showing the state in which the vibration data generated based on the image acquired as in Figure 15(a) is presented as a tactile sensation by the human skin tactile sensation presentation device 300.

After the exploration, the subject was instructed to explore three actual skin samples A, B, and C as shown in Figure 13 and to select the sample that was closest to the virtual texture reproduced by the human skin tactile sensation presentation device 300 immediately before by answering either A, B, or C.

This experiment was conducted twice, with different samples presented to each subject so that the number of samples presented was approximately equal.

The results are shown in Figure 16. Figure 16 shows the skin type discrimination results when vibration data generated by the method of the present invention was used to present a tactile sensation using a human skin tactile sensation presentation device. As shown in Figure 16, the accuracy rate of each sample exceeded 70% when a virtual texture was presented by acquiring a skin image and generating data for the human skin tactile sensation using the method of the present invention.

(Experiment 3)
In experiment 3, as a method for generating data for human skin tactile sensation according to a comparative example, an artificial finger AF is clamped on the finger slider 324 in the human skin tactile sensation presentation device 300 in advance, and an accelerometer α is attached to the upper surface of the artificial finger AF, and skin samples A, B, and C are set on the propagation plate 36 as human skin gel. Then, acceleration data is measured when the artificial finger AF is moved at a predetermined moving speed by the slide volume 32 while touching the skin samples A, B, and C. Then, data is generated based on the acceleration data to obtain three pieces of data for human skin tactile sensation (vibration data) that imitate each of the skin samples A, B, and C, and the vibration data is stored in the human skin tactile sensation presentation device 300. FIG. 17(a) is a diagram explaining a state in which vibration data is created by touching the artificial finger AF, to which the accelerometer α is attached, to the skin sample in the human skin tactile sensation presentation device 300 as a comparative example.

As in Experiment 2, the subject was instructed to insert the finger F into the finger slider 324 of the human skin tactile sensation presentation device 300 and touch the smooth human skin gel 31 with a pressure of about 0.5 N. Then, in the human skin tactile sensation presentation device 300, the subject moved his/her finger at a predetermined moving speed on the human skin gel 31, which was reproducing one of the three vibration data simulating the skin samples A, B, and C generated by the human skin tactile sensation reproduction method of the comparative example, so that the subject searched for a virtual texture on the human skin gel 31 within 10 seconds under the same search conditions. Figure 17(b) is a diagram showing a state in which the vibration data generated based on the acceleration data acquired as in Figure 17(a) is presented as a tactile sensation by the human skin tactile sensation presentation device 300.

After the exploration, as in Experiment 2, the subjects were instructed to explore three actual skin samples A, B, and C as shown in Figure 13 and to select the sample that was closest to the virtual texture reproduced by the human skin tactile sensation presentation device 300 immediately before by answering either A, B, or C.

In addition, this experiment was also conducted twice, with different samples presented to each subject, so that the number of samples presented was approximately equal.

The results are shown in Figure 18. Figure 18 shows the skin type discrimination results when the vibration data generated by the method of the comparative example was used to reproduce the tactile sensation with a human skin tactile sensation presentation device. As shown in Figure 18, the correct answer rate for each sample was about 50% for the virtual texture presented by acquiring a skin image and generating data for the human skin tactile sensation using the method of the comparative example. The correct answer rate decreased as the skin sample became rougher. In particular, the correct answer rate for sample C decreased to 45%, and the correct answer rates for samples B and C showed that subjects tended to perceive the virtual texture as smoother than the sample corresponding to the presented vibration data.

Comparing the experimental results of Figures 16 and 18, the correct answer rate for each sample of the presented virtual texture is higher in Figure 16. In Figure 16, the difference in correct answer rate between sample A, which had the highest correct answer rate, and sample B, which had the lowest correct answer rate, is 10%, and the error rate of sample B is higher than that of sample C, which had the roughest skin. Therefore, it can be said that the method of the present invention not only raised the overall reproduction rate of virtual textures, but also reduced the tendency to mistake rough skin for smooth skin.

Based on these experimental results, it can be said that the human skin tactile data generation method of the present invention can more accurately measure differences in tactile sensation by acquiring images of the skin, minimize errors due to the characteristics of the person touching it, and generate human skin tactile data that can be shared by multiple people, is highly reproducible, and allows for high readability of differences in tactile sensation.

Furthermore, comparing Experiments 2 and 3, the human skin tactile data generation method of the present invention can collect human skin information by acquiring an image of the skin, without touching the skin or using a special tactile sensor. Due to measures against the new coronavirus since 2020, it has become difficult to touch other people's skin with your fingers to acquire samples or to touch other people's skin with your fingers to evaluate the texture of the skin, but the present invention acquires an image and generates data based on that image, making it possible to acquire human skin tactile data with high reproducibility without actually touching the skin.

<Second embodiment of device and method for generating data for human skin texture>
FIG. 19 is a functional block diagram of a human skin tactile data generating device according to a second embodiment of the present invention.

As shown in FIG. 19, the human skin tactile data generation device 400 of the second embodiment includes a two-dimensional imaging means 4 which is an image acquisition unit, and an information processing device 5 which is a data generation unit.

The two-dimensional imaging means 4 captures an image of human skin or skin that is a skin model. In this embodiment, the two-dimensional imaging means 4 is a means capable of capturing two-dimensional images such as color or UV photographs of skin, and is, for example, a dedicated imaging terminal for capturing skin images with adjustable light intensity, a microscope (a skin surface texture analysis imaging device, a pore shape analysis imaging device), or a camera mounted on a smartphone or the like.

The information processing device 5 uses the acquired skin image to generate human skin tactile data estimated as vibrations that occur when touching human skin or a skin model. The information processing device is, for example, a PC capable of running a specific software or a dedicated two-dimensional skin image analysis device.

In this embodiment, the information processing device 5 is capable of executing a brightness information calculation unit 51, a surface roughness information acquisition unit 52, an acceleration information conversion unit 53, and a human skin tactile data output unit (output unit) 54.

The brightness information calculation unit 51 calculates the brightness of each position on the skin surface in a two-dimensional image of human skin or a skin model using brightness extraction software or the like.

The surface roughness information acquisition unit 52 has a brightness/cross-sectional height conversion unit 521 and a displacement curve acquisition unit 522.

The brightness/cross-section height conversion unit 521 calculates the skin surface height (skin cross-section height) at each position in the two-dimensional image based on the brightness.

The displacement curve acquisition unit 522 assumes that a certain distance is scanned along a straight line in a two-dimensional image, and acquires a displacement curve along the straight line as surface roughness information by converting the brightness of the skin surface along that straight line in the two-dimensional image into the height of the cross section of the skin at each position along the line, with the horizontal axis being distance.

2, and converts the skin surface roughness information acquired by the surface roughness information acquisition unit 52 into acceleration information. The acceleration information conversion unit 53 has a displacement spectrum conversion unit 531, an acceleration spectrum conversion unit 532, and an acceleration information correction unit 533.

The output unit 54 sets the acceleration information converted by the acceleration information conversion unit 53 as human skin tactile data (human skin expression data) and outputs it.

Note that Figure 19 shows a configuration in which the functions of the luminance information calculation unit 51, surface roughness information acquisition unit 52, acceleration information conversion unit 53, and output unit 54 are realized by a single information processing device 5, but these calculation functions may be executed using two or more information processing devices.

(Data Generation Process in the Second Example)
Details of the data generation process executable by the information processing device shown in Fig. 19 will be described with reference to Fig. 20 and Fig. 21. Fig. 20 is a detailed flow of the data generation process according to the second embodiment. Only the differences from the flow in Fig. 3 will be described.

In S201, brightness information of the skin surface is calculated from the acquired two-dimensional skin image.

In S202, surface roughness information is estimated based on skin brightness information.

In S203, a displacement curve is obtained from the skin surface roughness information calculated based on the luminance information, assuming scanning in a direction along a straight line. More specifically, assuming scanning a predetermined distance along a straight line in a two-dimensional image, the luminance of the skin surface along the straight line in the two-dimensional image is converted into the height of the cross section of the skin at each position along the straight line with the horizontal axis being the distance, and a displacement curve along the straight line is obtained as surface roughness information.

Here, in Figure 21, (a) is an example of a luminance image of human skin, and (b) and (c) are examples of displacement curves using luminance calculated in the second embodiment.

In S202, the luminance information of each position on the straight line of the two-dimensional image is replaced with the cross-sectional height of the skin surface. For example, the acquired two-dimensional image data provides luminance information for each pixel as position information, as shown in Figure 21 (b), so the luminance for each pixel is replaced with the cross-sectional height of the skin surface. In the graph of Figure 21 (b), the horizontal axis indicates pixels and the vertical axis indicates luminance.

In this case, the conversion from the brightness of each pixel to the cross-sectional height of the skin surface was carried out in this example by subtracting the average brightness value of the analysis area from the brightness value of each pixel to take the difference, and then multiplying each brightness value by 4, assuming that brightness 1 corresponds to a cross-sectional height of 4 μm.

In the conversion from Fig. 21(b) to Fig. 21(c), the change in brightness from the average value is converted as luminance 1 being 4 μm, but the relationship between the actual brightness value and roughness is thought to change depending on how the lighting is applied and the distance from the camera. Therefore, when using the same conversion formula, it is preferable to set the lighting method and the distance and angle between the subject and the camera to be constant.

In this example, the average brightness value is set to a cross-sectional height of 0 on the skin surface. However, in the case of an image in which there is a deviation in the spread of brightness, the median or most frequent value of the spread of brightness may be set to a cross-sectional height of 0 on the skin surface.

In this way, any suitable conversion formula may be used to convert luminance information into skin surface height, as long as the conditions are consistent between the skins being compared. Furthermore, the conversion formula may be changed each time the shooting conditions are changed.

Figure 21(c) shows the brightness graph for each pixel in Figure 21(b) with the horizontal axis replaced with length. In this example, the conversion from Figure 21(b) to Figure 21(c) was made based on the relationship that the horizontal axis is 640 pixels = 6.7 mm.

In this way, using the displacement curve calculated via brightness based on the two-dimensional image, in steps S203 to S206, similar to steps S23 to S26 in Figure 3, the displacement curve is converted into a spectrum, and then converted into an acceleration spectrum, and corrected with a low-pass filter to obtain data for human skin tactile sensation.

In this embodiment, the two-dimensional image is converted into the height of the skin surface via the brightness to obtain data for human skin tactile sensation. Therefore, if the data generating unit of the device for generating data for human skin tactile sensation has software that can obtain the brightness of the image, it is possible to obtain an image using a more general-purpose camera, such as a smartphone camera, without using expensive three-dimensional imaging means, and generate data for human skin tactile sensation based on that image.

As a result, human skin tactile data can be generated even when a skin information provider such as a subject is not present and a high-performance camera is not available, making it possible to realize a human skin tactile data generation device that reduces the burden on the subject, is easier to put into practical use, and is more familiar to users.

(Variation 1)
In addition, in Figs. 20 and 21, an example of calculating skin roughness information via luminance information has been described as a process for acquiring skin roughness information from a two-dimensional image in the human skin tactile data generation method of the present invention. However, data may be generated without using luminance by acquiring a special two-dimensional image, for example.

For example, a two-dimensional UV camera (e.g., Visio scan (registered trademark): two-dimensional skin surface texture analysis device) is used to acquire images, and the state of "texture and fine wrinkles" is extracted based on clear images of the skin surface morphology captured using UV on a processing terminal attached to the UV camera. Then, based on the extracted state of texture and fine wrinkles, a displacement curve may be calculated on an information processing terminal other than the dedicated calculation terminal.

(Variation 2)
Furthermore, as other methods for obtaining skin roughness information from a two-dimensional image in the human skin tactile data generation method of the present invention, skin unevenness information may be obtained by interfering light when capturing a two-dimensional image, by moving the focus when capturing a two-dimensional image, or by taking a confocal lens when capturing a two-dimensional image.

<Human skin texture evaluation method>
Next, a method for evaluating human skin feel including the method for generating data for human skin feel of the present invention will be described with reference to Figures 22 and 23. Figure 22 is a schematic flow chart showing a method for evaluating human skin feel including the method for generating data for human skin feel of the present invention.

In step S101, a skin image is acquired. During evaluation, the skin that is the subject of the image is the actual skin of the subject, customer, etc.

In step S102, similar to S2 in Figure 1, acceleration information is obtained as human skin tactile data based on the skin surface roughness information.

In step S103, the previously set evaluation criteria are called up (evaluation criteria reference process).

Then, in S104, the generated human skin feel data is compared with the evaluation criteria to evaluate/estimate the feel of the acquired image (evaluation process).

As shown in Fig. 22, in the evaluation step, the data (acceleration information) generated in the data generation step is used to evaluate the feel of the skin. Therefore, it is preferable to establish evaluation criteria in advance as a prerequisite for the evaluation, and to perform the evaluation based on the criteria.

(Example of creating evaluation criteria for tactile evaluation)
Here, the evaluation criterion is a relationship between the frequency spectrum of acceleration (vibration information) calculated from the surface roughness information of the skin image or skin model image and a sensory evaluation of the tactile feel of the skin or skin model, which has been constructed and stored in advance.

Specifically, acceleration information is calculated from the surface roughness information of each of images of reference skin (or human skin models) with different surface conditions, or images of multiple human skin gels with different characteristics. Reference skin refers to skin whose characteristics have already been identified and is not unknown.

On the other hand, it is possible to record the results of a sensory evaluation of the tactile sensation a human perceives when running a finger sideways across the "reference skin" (or human skin gel) from which surface roughness information has been obtained, and to create a database of the relationship between this data and the sensory evaluation.

When evaluating the feel of unknown skin, the sensory evaluation value of the feel of the unknown skin can be estimated by comparing the acceleration information calculated from the surface roughness information detected in the same way for the unknown skin with the above-mentioned relationship (database) that has been constructed in advance.

The relationship (evaluation criteria) between the values related to the vibration magnitude and characteristics of the human skin tactile data and the sensory evaluation can be constructed, for example, as follows.

In this case, for known skin (skin model) with multiple understood characteristics, human skin tactile data (acceleration spectrum when scanning a certain distance) is obtained in accordance with the human skin tactile data generation method of the present invention, and the relationship with the sensory evaluation is examined.

In more detail, images (three-dimensional or two-dimensional images) are acquired for multiple known skin models for standard setting according to the human skin tactile data generation method of the present invention, and data is generated based on the images to acquire multiple human skin tactile data (acceleration information) that mimic each of the known skin models. A fast Fourier transform (FFT) is then performed on the acquired acceleration information to calculate an amplitude spectrum from 0 Hz to 500 Hz. The calculated amplitude spectrum is then averaged for every 50 Hz frequency, and a statistical model is estimated using the average value of this spectrum as the independent variable (vibration feature).

It is preferable that the number of skin or skin samples used to set multiple standards for constructing the evaluation criteria is at least 10 or more, and the more samples the better.

Then, a statistical model is calculated from the vibration feature values obtained above to estimate the moistness and smoothness of the skin. "Moist" skin refers to skin that is neither sticky (excessive oil) nor dry, and has an appropriate amount of moisture (a moisturized state), and "smooth" skin refers to skin that has less open pores and less noticeable acne to the touch, and has a smooth surface texture. Note that the sensory evaluation of skin as "moist" or "smooth" is just one example, and as another example, a statistical model that numerically evaluates the skin condition based on a skin moisture meter or skin oil meter may be calculated.

On the other hand, a sensory evaluation by humans (an expert panel) may be performed on the evaluation items of the virtual texture presented by the above-mentioned human skin tactile sensation presentation device 300 for each of the above-mentioned multiple reference skins. The sensory evaluation by the above-mentioned expert panel may also be an evaluation of the tactile sensation of the skin felt when experiencing the virtual texture presented by the human skin tactile sensation presentation device 300. The sensory evaluation value by this panel is preferably the calculated average value of 10 or more people, and the more people who evaluate in order to take the average of the calculated average values, the more preferable.

The statistical model and sensory evaluation obtained as described above become the evaluation criteria for evaluating the reproduced tactile sensation of skin. In other words, if an evaluation criterion is constructed by detecting and analyzing the human skin tactile data (acceleration information) for each of a number of known skins (reference skins) and conducting a sensory evaluation by a specialist panel, data based on the human skin tactile data (acceleration information) detected for an unknown skin can be obtained and applied to the constructed evaluation criterion to estimate the sensory evaluation score of the human skin tactile data for the unknown skin. In this case, the human skin tactile data for the unknown skin is generated under the same conditions as the human skin tactile data generation of the present invention performed when determining the evaluation criterion.

(Human skin texture evaluation device)
FIG. 23 is a block diagram of a human skin texture evaluation device for implementing the human skin texture evaluation method of the present invention.

As shown in Figure 23, the human skin tactile evaluation device 600 comprises a three-dimensional imaging means 1 which is an image acquisition unit, and an information processing device 6 which is a data generation and evaluation unit.

Note that Figure 23 shows an example in which the human skin tactile evaluation device 600 realizes processes from image acquisition to generation of data for human skin tactile sensation using the method of the first embodiment, but the human skin tactile evaluation device of the present invention may also realize processes from image acquisition to generation of data for human skin tactile sensation using the method of the second embodiment or the method of the comparative example.

The three-dimensional imaging means 1 acquires a three-dimensional image of human skin or skin that is a skin model.

The information processing device 6 uses the acquired skin images to generate and evaluate human skin tactile data estimated as vibrations that occur when touching human skin or a skin model. The information processing device 6 is, for example, a PC capable of running specific software, a dedicated three-dimensional skin image analysis device, etc.

The information processing device 6 is executable to include a surface roughness information acquisition unit 61, an acceleration information conversion unit 62, a reference data storage unit 63, an evaluation unit 64, and an evaluation result output unit 65.

The surface roughness information acquisition unit 61 acquires skin surface roughness information from an image of the skin, similar to the surface roughness information acquisition unit 21 shown in Fig. 2. In detail, the surface roughness information acquisition unit 61 has a surface roughness extraction unit 611 that extracts the skin surface roughness from a three-dimensional image of the skin, and a displacement curve acquisition unit 612 that acquires a displacement curve.

The acceleration information conversion unit 62 converts the skin surface roughness information into acceleration information. The acceleration information conversion unit 62 has a displacement spectrum conversion unit 621, an acceleration spectrum conversion unit 622, and an acceleration information correction unit 623.

The reference data storage unit 63, which is a storage unit, pre-stores acceleration information of multiple types of skin that correspond to multiple types of human skin or skin models with set characteristics as a standard for skin feel, as a sensory evaluation standard.

The evaluation unit 64 evaluates the current skin feel based on the relationship between the acceleration information calculated from the current skin image calculated by the acceleration information conversion unit 62 and the sensory evaluation that serves as the standard for the skin feel stored in the reference data storage unit 63.

The evaluation result output unit (evaluation output unit) 65 is a display unit or notification unit, and outputs the current skin evaluation result by displaying or transmitting it.

23 shows a configuration in which the functions of the surface roughness information acquisition unit 61, the acceleration information conversion unit 62, the reference data storage unit 63, the evaluation unit 64, and the evaluation result output unit 65 are realized by one information processing device 6, but these calculation functions may be executed by two or more information processing devices. Also, the reference data storage unit 63 may be realized by a recording medium separate from the information processing device that performs the calculations.

In the human skin tactile sensation device of the present invention, the human skin tactile sensation data generated as described above is used for evaluation, making it possible to evaluate the skin tactile sensation that can be shared by multiple people with minimal error and high readability, without actually touching the skin.

<Application Example 1>
The above describes a method and configuration for applying the human skin tactile data generated using the skin image of the present invention to both presentation and evaluation, but the human skin tactile data may also be used in a human skin tactile evaluation and presentation device for both presentation and evaluation.

The human skin tactile sensation evaluation and presentation device will be described with reference to Figure 24. Figure 24 is a schematic diagram showing a beauty advisor presenting skin type to a customer during counseling using the human skin tactile sensation evaluation and presentation device 700 of the present invention. Note that while Figure 24 shows an example in which each component of the human skin tactile sensation evaluation and presentation device 700 is connected by wire, each component may also be connected wirelessly via short-range communication. Note that in this example, either a three-dimensional image or a two-dimensional image may be obtained as the skin image, and the human skin tactile sensation data may be generated by any of the methods of the first embodiment, the second embodiment, and the modified example.

For example, in the human skin tactile evaluation and presentation device 700, in order to reproduce normal skin conditions, desired skin conditions, etc., multiple pieces of human skin tactile presentation data (e.g., waveforms like those in Figure 9 (b)) equivalent to the original vibration signals corresponding to multiple skin types are acquired in advance and stored in PC 33.

Here, the beauty advisor BA inputs the selection of the skin type to be presented to the subject S (customer) so as to reproduce the feel of bare skin or the feel of skin after the application of cosmetics, beauty products, oils, etc. Then, in the presentation mechanism 30 of the human skin feel evaluation and presentation device 700, human skin feel presentation data, which serves as the original signal for vibration, is set according to the selected skin type, and vibration is presented to the subject's (customer's) fingers via the transmission plate 36 and human skin gel 31.

As an example, the beauty advisor BA acquires an image of the skin of the customer subject, and reproduces the current skin condition of the subject using the presentation mechanism 30 of the human skin feel evaluation and presentation device 700, and presents it on the subject's finger.

Furthermore, the beauty advisor BA acquires an image of the skin of the customer subject using the image acquisition unit 1, and evaluates the current skin condition of the subject using the evaluation mechanism 60 of the human skin tactile evaluation and presentation device 700, and displays the evaluation result to the subject. Alternatively, if the beauty advisor's tactile judgment level is immature or if the beauty advisor is unable to directly touch the customer's skin due to COVID-19 countermeasures, it is also effective to display a tactile score using the evaluation criteria along with the tactile presentation as a form of support.

Furthermore, the ideal skin or the desired skin condition may be reproduced by the presentation mechanism 30 and presented to the subject's finger. Specifically, a state that can be achieved by using a specific cosmetic product is reproduced by the presentation mechanism 30 by, for example, turning the reference liquid R into drops of human skin gel and presenting it to the subject's finger. If the surface of the cosmetic product changes over time, for example, when the cosmetic product is used, such as when it dissolves over time due to body temperature, the skin after 10 minutes, 20 minutes, or 30 minutes can be reproduced and presented by the human skin tactile sensation presentation device even if the time has not yet arrived. Alternatively, if it is desired to explain the effects of the skin that appear when the cosmetic product is used continuously, the skin after 1 day, 3 days, 1 week, 2 weeks, etc., can be reproduced and presented by the presentation mechanism 30 by continuously using the cosmetic product, for example, twice a day, even if the time has not yet arrived.

In this way, it becomes possible to use the human skin tactile sensation presentation device to provide a concrete tactile suggestion of the future skin condition of the subject when the cosmetic product is applied, so that by stimulating the sense of touch in addition to the visual aspect of a pamphlet, it is possible to appeal to the sensibilities of the subject customer in a multifaceted way. This allows the customer to gain a deeper understanding of their skin condition. Meanwhile, the beauty advisor can provide advice that corresponds to the reproduced skin quality, which can actively encourage purchases.

24 shows an example in which a beauty advisor proposes to a customer the human skin feel evaluation and presentation device 700 of the present invention, but the human skin feel evaluation and presentation device 700 may also be used for training beauty advisors. For example, in the human skin feel evaluation and presentation device 700, the beauty advisor can practice taking pictures using the image acquisition unit 1, and then be presented with the feel of a reference skin type, answer the feel, and have an evaluation score showing the degree of agreement displayed, thereby training the beauty advisor on the feel of the fingertips.

<Application Example 2>
In FIG. 24 above, the human skin tactile sensation evaluation and presentation device 700 is realized in one location by a wired connection or a short-distance connection, but the functions included in the human skin tactile sensation evaluation and presentation device 700 may be realized remotely as a system via a network.

Figure 25 is a schematic diagram of a human skin feel evaluation and presentation system using the human skin feel evaluation method of the present invention.

In one example of a human skin texture evaluation and presentation system 800, a subject takes pictures using a camera mounted on a general smartphone, tablet, mobile phone, etc. as an image acquisition and communication terminal 81, transmits the acquired images via a network N, and is connected to an advice PC 82 and/or an analysis PC 83. In this example, since the subject has a general camera in hand, human skin texture data is generated using the method of the second embodiment.

The advising PC 82 calculates human skin texture data based on the skin image information sent from the subject's image acquisition and communication terminal 81, and sends comments such as advice to the image acquisition and communication terminal 81 of the subject S by referring to the correlation between the stored evaluation criteria and the vibration texture. The advising PC 82 may be operated manually by the beauty advisor BA, or it may be programmed to automatically respond by incorporating advice software inside the advising PC 82.

In addition, when the beauty advisor BA performs remote counseling, the tactile sensation can be understood by outputting unknown skin vibration information from the presentation mechanism 30B, so there is no need to refer to the evaluation criteria.

The analysis PC 83 is equipped with tabulation software that collects and analyzes the responses from the subjects. Note that the advice PC 82 and the analysis PC 83 may be used together.

In this way, by subject S sending an image taken using a smartphone or the like, it becomes possible to share the skin image tactilely with multiple people in different spaces.

Alternatively, the input terminal 84 of the human skin texture evaluation and presentation system 800 of the present invention may be installed in a public space such as a shopping center or a rest area so that it can be used without a beauty advisor. When a dedicated input terminal 84 is installed, either a three-dimensional image or a two-dimensional image may be acquired by the image acquisition unit 1, and the human skin texture data may be generated by any of the methods of the first embodiment, the second embodiment, and the modified example.

The input terminal 84, which is placed in a public space and connected to the image acquisition unit 1 and presentation mechanism 30A, incorporates the functions of the information processing device 6 of the human skin tactile sensation evaluation device 600 and the PC 33 of the human skin tactile sensation presentation device 300, and can immediately calculate vibrations (human skin tactile sensation data) based on the currently acquired image. This makes it possible to present the current state of one's own skin with the presentation mechanism 30A, present reference human skin tactile sensation reproduction data as an evaluation standard with the presentation mechanism 30A, and output evaluation results that have been compared with the evaluation standard using the current human skin tactile sensation data.

Furthermore, it is preferable that the input terminal 84 has a built-in guidance program or advice program so that it can be used comfortably without a beauty advisor.

In addition, when an input terminal 84 is provided, the accumulated data on the skin condition of multiple subjects stored in the input terminal 84 can be transferred to the analysis PC 83 via the network N at predetermined intervals and used for analysis by the cosmetics developer CR.

Although a preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment described above, and various modifications and variations are possible within the scope of the gist of the present invention as described in the claims.

This international application claims priority to Japanese Patent Application No. 2020-200573, filed on December 2, 2020, and the entire contents of No. 2020-200573 are incorporated by reference into this international application.

1. Three-dimensional imaging means (image acquisition unit)
2. Information processing device (data generation unit)
4 Two-dimensional imaging means 5 Information processing device (data generation unit)
6. Information processing device (data generation and evaluation unit)
21, 52 Surface roughness information acquisition unit 22, 54 3 Acceleration information conversion unit 23, 54 Human skin tactile data output unit (output unit)
51 Luminance information calculation unit 30 Presentation mechanism 31 Human skin gel 32 Slide volume (finger movement unit, movement speed acquisition unit)
33 PC (original signal storage unit, vibration adjustment unit)
34 Amplifier (vibration output section)
35 (35A, 35B) Speaker (vibration output unit)
36 Propagation plate (propagation member)
61: Surface roughness information acquisition unit 62: Acceleration information conversion unit 63: Reference data storage unit (storage unit)
64 Evaluation unit 65 Evaluation result output unit (evaluation output unit)
100: Human skin tactile data generating device 300: Human skin tactile sensation presentation device 400: Human skin tactile sensation data generating device 600: Human skin tactile sensation evaluation device 700: Human skin tactile sensation evaluation and presentation device 800: Human skin tactile sensation evaluation and presentation system

Claims (15)

acquiring an image of skin, the image being human skin or a skin model;
and generating human skin tactile data using the acquired skin image, the human skin tactile data being estimated as vibrations that occur when touching the human skin or a skin model. The step of generating the human skin tactile data includes:
obtaining skin surface roughness information from the skin image;
converting the skin surface roughness information into acceleration information;
The human skin tactile data generating method according to claim 1 , further comprising a step of setting the acceleration information as the human skin tactile data. In the step of acquiring the skin image, a three-dimensional image of a surface of human skin or a skin model is acquired as the skin image by a laser microscope;
In the step of acquiring skin surface roughness information from the skin image,
3. The human skin tactile data generating method according to claim 2, further comprising: assuming a case where a predetermined distance is scanned along a straight line of the three-dimensional image of the surface of the human skin or skin model, a displacement curve on the straight line, which corresponds to the cross-sectional height position of the three-dimensional skin with the horizontal axis being distance, is obtained as the surface roughness information. In the step of acquiring the skin image, a color or black and white two-dimensional image of a surface of human skin or a skin model is acquired as the skin image by photography;
In the step of acquiring skin surface roughness information from the skin image,
calculating a luminance of a skin surface in the two-dimensional image of the human skin or skin model;
and acquiring, as the surface roughness information, a displacement curve on a straight line, the brightness of the skin surface on the straight line in the two-dimensional image being converted into the height of the cross section of the skin at each position on the straight line, with the horizontal axis being distance, assuming a case where a straight line on the two-dimensional image is scanned a predetermined distance. In the step of converting the skin surface roughness information into acceleration information,
A moving speed when touching the human skin or skin model is assumed, and the horizontal axis of the displacement curve on the straight line, which is the surface roughness information, is converted from distance to time to obtain a spectrum of the displacement curve;
5. The human skin tactile data generating method according to claim 3, further comprising converting the spectrum of the displacement curve into a spectrum of acceleration. In the step of converting the skin surface roughness information into acceleration information,
The method further includes a step of correcting the acceleration spectrum by applying a low-pass filter to obtain the acceleration information;
In the step of setting the acceleration information as human skin tactile data,
6. The method for generating data for human skin tactile sensation according to claim 5, further comprising the step of applying a low-pass filter to the acceleration spectrum to correct the acceleration information, and using the corrected acceleration information as the data for human skin tactile sensation. The human skin tactile data generating method according to claim 1 , wherein the human skin is bare skin or skin to which a cosmetic material has been applied. an image acquisition unit for acquiring an image of human skin or skin that is a skin model;
a roughness information acquisition unit that acquires surface roughness information from the skin image;
an acceleration information conversion unit that converts the skin surface roughness information into acceleration information;
and an output unit that outputs the acceleration information as human skin tactile data. A step of generating human skin tactile data, which is acceleration information, by the human skin tactile data generating method according to any one of claims 1 to 7;
and evaluating the tactile feel of the human skin or skin model using the acceleration information, which is the human skin tactile feel data. 10. The method for evaluating the texture of skin according to claim 9, wherein in the step of evaluating the texture of the human skin or skin model, the texture of the skin is evaluated based on a relationship between acceleration information obtained in advance and a sensory evaluation that serves as a reference for the texture of the skin. The human skin tactile data generating device according to claim 8 ;
an evaluation unit that evaluates the tactile sensation of the human skin or a skin model using the acceleration information, which is the human skin tactile data generated by the human skin tactile data generation device;
and an evaluation output unit that outputs the evaluation of the tactile sensation. the evaluation unit evaluates the tactile sensation of the skin based on a relationship between the acceleration information and a sensory evaluation that is a reference for the tactile sensation of the skin, the relationship being determined in advance;
The human skin texture evaluation device comprises:
The human skin tactile evaluation device according to claim 11, further comprising a storage unit that stores in advance acceleration information of a plurality of types of skin associated with a plurality of types of human skin or skin models having set characteristics as a reference for the skin tactile feel. 9. A human skin tactile sensation presentation device comprising: a presentation mechanism that acquires the acceleration information, which is the human skin tactile data generated by the human skin tactile sensation data generation device according to claim 8, and presents the acceleration information as vibration on a human skin model. The presentation mechanism includes:
A propagation member through which vibration propagates;
a human skin model that is provided on the propagation member and that comes into contact with a user's finger or an artificial finger;
a finger moving unit that moves the finger or artificial finger on the human skin model at a predetermined moving speed;
a movement speed acquisition unit that acquires a movement speed of the user's finger on the human skin model;
an original signal storage unit that stores the acceleration information associated with the acquired skin or skin model as an original signal;
A vibration adjustment unit that adjusts the amplitude of the original signal according to the moving speed of the finger or artificial finger;
The human-skin tactile sensation presentation device according to claim 13 , further comprising: a vibration output unit that converts the adjusted original signal into a physical vibration and outputs the physical vibration. A step of generating human skin tactile data, which is acceleration information, by the human skin tactile data generating method according to any one of claims 1 to 7;
and using the acceleration information, which is the human skin tactile data, to present the acceleration information as vibration on a human skin model.

Images