JP7740905B2 - Information processing device, method, program, and system - Google Patents

Description

The present invention relates to an information processing device, method, program, and system.

Conventionally, beauty treatments have been known in beauty salons and the like, where practitioners such as estheticians perform beauty-related procedures on customers. Customers can gain satisfaction from receiving beauty-related procedures from others.

On the other hand, there is also a known beauty technique known as "self-beauty treatment," in which the customer, rather than the therapist, performs the beauty treatments themselves. Furthermore, customers may also perform beauty treatments themselves at home, imitating the actions of a professional (Patent Document 1).

JP 2012-37626 A

However, even when performing the treatment on myself, it was difficult to feel as if I was receiving treatment from someone else.

The present invention aims to enhance the sense of reality of others even when performing treatment on oneself.

An information processing device according to one embodiment of the present invention includes an acquisition unit that acquires video of at least a first person and a second person, and a video editing unit that generates video that displays the first person in an area including the center of the image and also displays a portion of the second person according to a preset display ratio.

This invention can enhance the sense of presence of others even when performing treatment on oneself.

FIG. 1 is a diagram for explaining an overview of the present invention. 1 is a diagram showing an overall configuration according to an embodiment of the present invention; 1 is a diagram showing functional blocks of an information processing apparatus according to an embodiment of the present invention; 1 is a diagram illustrating an information processing system according to an embodiment of the present invention. 1 is a flowchart showing the overall processing flow according to an embodiment of the present invention. 1 is a flowchart showing a flow of video editing processing according to an embodiment of the present invention. 1 is a diagram illustrating a hardware configuration of an information processing apparatus according to an embodiment of the present invention.

Each embodiment will be described below with reference to the accompanying drawings. Note that in this specification and drawings, components that have substantially the same functional configuration will be assigned the same reference numerals, and redundant explanations will be omitted.

<Terminology>
- "Beauty-related activities" are any beauty-related activities performed using hands, cosmetic tools, etc. (for example, applying cosmetics, etc.).

<Summary of the Invention>
FIG. 1 is a diagram for explaining an overview of the present invention. In one embodiment of the present invention, a video is captured of a practitioner, such as an esthetician, performing a beauty treatment on a subject. This video is edited to generate a video in which the subject is displayed at the center of the image and a portion of the practitioner is displayed according to a preset display ratio. By performing a beauty treatment on themselves while watching the video generated in this way, the subject can feel as if they are receiving a beauty treatment from another person (i.e., the practitioner). Steps S1 to S4 are described below.

In step 1 (S1), a practitioner such as an esthetician (an example of a second person described below) performs a beauty treatment on a subject (an example of a first person described below). At this time, the practitioner and the subject are photographed.

In step 2 (S2), the information processing device 10 (described in detail below) acquires the video of the practitioner and subject captured in S1.

In step 3 (S3), the information processing device 10 edits the video acquired in S2 to generate a video in which the subject is displayed at the center of the image and a portion of the practitioner is displayed according to a pre-set display ratio.

In step 4 (S4), the subject performs a beauty treatment on himself or herself while watching the video generated by the information processing device 10 in S3 (a video in which the subject is displayed at the center of the image and a portion of the practitioner is displayed according to a pre-set display ratio).

<Overall structure>
FIG. 2 is a diagram showing the overall configuration according to one embodiment of the present invention.

The information processing device 10 generates and plays back a video based on a video of people 20 (at least a first person and a second person), in which the first person is displayed at the center of the image and a portion of the second person is displayed according to a preset display ratio. For example, the information processing device 10 is a smartphone, tablet, personal computer, or the like that has a camera function (which may be a 3D camera). The information processing device 10 will be described in detail later with reference to Figure 3.

In this specification, we will explain the case where the information processing device 10 is a single device (for example, a smartphone with a camera function), but the information processing device 10 may also be made up of multiple devices (for example, a device without a camera function and a digital camera). Note that a device for shooting videos, a device for generating videos based on the shot videos, and a device for playing the generated videos may be implemented in a single device, or at least one of them may be implemented in separate devices.

<Functional blocks of information processing device 10>
3 is a diagram showing functional blocks of an information processing device 10 according to an embodiment of the present invention. The information processing device 10 can include a shooting unit 101, an acquisition unit 102, an identification unit 103, a video editing unit 104, and a playback unit 105. Furthermore, by executing a program, the information processing device 10 can function as the shooting unit 101, the acquisition unit 102, the identification unit 103, the video editing unit 104, and the playback unit 105. Each of these units will be described below.

The imaging unit 101 captures images of at least a first person and a second person. The second person is performing a treatment (procedure) on the first person (for example, a beauty treatment).

The acquisition unit 102 acquires video footage of at least a first person and a second person. Note that the acquisition unit 102 may acquire video footage captured by the information processing device 10, or video footage captured by a device other than the information processing device 10.

The identification unit 103 identifies the first person in a video in which at least a first person and a second person are captured. For example, the identification unit 103 can identify a person registered in advance as the first person (for example, by using facial recognition, iris recognition, etc.). For example, the identification unit 103 can identify the person who moves the least in a video in which at least the first person and the second person are captured (i.e., the person who moves the least for the longest period of time throughout the entire video) as the first person. For example, the identification unit 103 can identify the person who appears most centrally in a video in which at least the first person and the second person are captured (i.e., the person who appears most centrally for the longest period of time throughout the entire video) as the first person.

The identification unit 103 identifies the second person in a video in which at least a first person and a second person are captured. For example, the identification unit 103 can identify a person other than the first person (e.g., a person closest to the first person) as the second person.

The video editing unit 104 generates a video that displays a first person (e.g., the first person's face) in an area including the center of the image, and also displays a portion of a second person (e.g., either the second person's face, whole body, or upper body) according to a pre-set display ratio (e.g., a display ratio at which a cut-off (i.e., partially cut-off) person can be recognized as a different person).

For example, the video editing unit 104 can generate a video in which the face of a first person is displayed at the center and the part of the face where the hand or cosmetic tool of a second person is touching the first person's face is displayed. While watching the video generated in this way, the first person can feel as if they are being touched by the second person by touching their own face with their own hand or cosmetic tool in the same way as the second person's hand or cosmetic tool.

<Display rate>
Here, the display ratio will be described. The display ratio indicates the ratio of the area in which the second person is displayed in the video to be generated. For example, the display ratio is the ratio of the area in which the second person is displayed to the area in which the first person is displayed. For example, the display ratio is the ratio of the area in which the second person is displayed when the size of the entire body of the second person estimated from the image is 100%. For example, the display ratio is the ratio of the number of pixels in the area in which the second person is displayed to the total number of pixels in the image. Note that the display ratio may be calculated based on the ratio of the area in which the first person is displayed to the area in which the second person is displayed.

For example, the subject may be able to specify the display rate itself, or the ratio of the area in which the first person is displayed to the area in which the second person is displayed (the display rate is calculated based on the ratio of the area in which the first person is displayed to the area in which the second person is displayed).

<Delete unnecessary videos>
For example, the video editing unit 104 can delete a part of a video in which at least the first person and the second person are shot, based on the movement of the first person. Specifically, the video editing unit 104 can generate a video after deleting a video in which the magnitude of the movement of the first person is greater than a threshold (for example, a video taken when the treatment is not being performed).

<First Person Display>
The video editing unit 104 can display the first person (for example, face) so that the position of the center (for example, nose) of the first person coincides with the position of the center of the image or a position within a predetermined range from the center of the image. For example, the subject or the like may be able to specify a predetermined range (i.e., an allowable range of deviation from the center).

<Displaying part of the second person>
Below, two examples of displaying a portion of the second person will be described: <Trimming the second person> and <Zooming in/out of the image>. The subject may be able to specify either <Trimming the second person> or <Zooming in/out of the image>. The subject may also be able to specify the portion of the second person to be displayed (for example, the face, the whole body, or the upper body of the second person).

<Cropping of the second person (no zoom in/out)>
The video editing unit 104 displays a part of the second person according to a preset display ratio by trimming the second person (i.e., parts other than the part of the second person are not displayed).

<Zooming in and out of the video>
The video editing unit 104 displays a part of the second person in accordance with a preset display ratio by enlarging or reducing the video in which at least the first person and the second person are shot.

For example, when the difference between a preset display rate and the (actual) display rate of the second person is greater than a threshold, the video editing unit 104 can enlarge or reduce the video of at least the first and second people. In other words, by enlarging or reducing the video (e.g., 9%, 31%) when the display rate exceeds an allowable range (e.g., 10-30%) including a preset display rate (e.g., 20%), the display range of the first person can be ensured, and visibility can be improved to clearly show that the first person is receiving treatment.

For example, the video editing unit 104 can prevent viewers from feeling sick by gradually enlarging or reducing the size of the video rather than instantly enlarging or reducing it.

<Remove background>
The video editing unit 104 may trim the background from a video in which at least the first person and the second person are shot.

The playback unit 105 plays the video generated by the video editing unit 104. Note that the video generated by the video editing unit 104 may be played on the information processing device 10, or on a device other than the information processing device 10.

<Information Processing System>
4 is a diagram showing an information processing system 1 according to an embodiment of the present invention. The information processing system 1 can include an information processing device 10, an image capturing device 30, and a playback device 40.

As described above, at least one of the device that shoots videos, the device that generates videos based on the shot videos, and the device that plays back the generated videos may be implemented as separate devices. Figure 4 shows an embodiment in which the device that shoots videos (shooting device 30), the device that generates videos based on the shot videos (information processing device 10 (e.g., a server)), and the device that plays back the generated videos (playback device 40) are implemented as separate devices. Note that the shooting device 30 and the playback device 40 may be the same device.

The image capturing device 30 has the functionality of the image capturing unit 101. The image capturing device 30 transmits the captured original image (a video capturing at least a first person and a second person) to the information processing device 10.

The information processing device 10 includes an acquisition unit 102, an identification unit 103, and a video editing unit 104. The information processing device 10 acquires original images (videos capturing at least a first person and a second person) from the image capture device 30 and performs video editing processing. The information processing device 10 also distributes processed images (processed video) to the playback device 40.

The playback device 40 has the functionality of the playback unit 105. The playback device 40 plays back videos acquired from the information processing device 10.

<Processing method>
An overall processing method and a video editing processing method according to one embodiment of the present invention will be described below.

Figure 5 is a flowchart showing the overall processing flow for one embodiment of the present invention.

In step 11 (S11), the photographing unit 101 (or the photographing device 30) photographs an original image. Specifically, the photographing unit 101 (or the photographing device 30) photographs at least a first person and a second person. The second person is performing an optional treatment (procedure) on the first person (for example, a beauty treatment).

In step 12 (S12), the video editing unit 104 performs video editing processing. Specifically, the video editing unit 104 generates a video based on the original images captured in S11.

In step 13 (S13), the playback unit 105 (or the playback device 40) plays the processed image. Specifically, the playback unit 105 (or the playback device 40) plays the video processed in S12.

Figure 6 is a flowchart showing the flow of video editing processing (S12 in Figure 5) according to one embodiment of the present invention.

In step 101 (S101), the acquisition unit 102 acquires an original image (a video capturing at least a first person and a second person).

In step 102 (S102), the identification unit 103 identifies the first person in the still images (frames) that make up the original image of S101 (a video in which at least the first person and the second person are captured).

In step 103 (S103), the video editing unit 104 moves the still image (frame) so that the first person identified in S102 is placed at the center of the image to be generated.

In step 104 (S104), the video editing unit 104 trims the second person or enlarges or reduces the original image (more specifically, the still images (frames) that make up the original image) according to a pre-set display ratio.

In step 105 (S105), the video editing unit 104 determines whether processing has been completed for all frames constituting the original image acquired in S101. If so, the video editing process ends. If not, the process returns to step 102 and processes the next still image (e.g., the frame at the next time among the frames constituting the video).

<Beauty method>
One embodiment of the present invention is a beauty treatment method for performing a beauty treatment while viewing the image described above. That is, one embodiment of the present invention is a beauty treatment method for performing a beauty treatment while viewing a video of at least a first person and a second person, in which the video of at least the first person and the second person is displayed in an area including the center of the image and a portion of the second person is displayed according to a preset display ratio.

<Image with CG image superimposed on real-time image>
In one embodiment of the present invention, a video of at least a first person and a second person may be a video in which a CG (computer graphics) image of the second person is superimposed on an image of the first person captured in real time. Specifically, the information processing device 10 acquires an image of the first person captured in real time (referred to as a real-time image). The information processing device 10 then determines the position and posture of the second person based on the first person in the real-time image (e.g., based on the position and posture of the first person's hands or the contact points between the first person's hands or makeup tools and the face), and superimposes the real-time image with the CG image of the second person at the determined position and posture. The information processing device 10 then generates a video in which the first person is displayed in an area including the center of the real-time image and a portion of the CG image of the second person is displayed according to a preset display ratio.

<Effects>
Thus, in one embodiment of the present invention, the subject (the subject) is displayed in the center as he or she receives a beauty treatment from a practitioner, and by watching a video in which a portion of the practitioner is displayed while the beauty treatment is performed on the subject, the subject can feel as if they are receiving a beauty treatment from another person (i.e., the practitioner).

<Hardware configuration>
FIG. 7 is a diagram showing the hardware configuration of an information processing apparatus according to an embodiment of the present invention.

The information processing device 10 has a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003. The CPU 1001, ROM 1002, and RAM 1003 form a so-called computer.

The information processing device 10 may also have an auxiliary storage device 1004, a display device 1005, an operation device 1006, an I/F (Interface) device 1007, and a drive device 1008.

The hardware components of the information processing device 10 are connected to each other via bus B.

The CPU 1001 is a computing device that executes various programs installed in the auxiliary storage device 1004.

ROM 1002 is non-volatile memory. ROM 1002 functions as a primary storage device that stores various programs, data, etc. required for CPU 1001 to execute various programs installed in auxiliary storage device 1004. Specifically, ROM 1002 functions as a primary storage device that stores boot programs such as BIOS (Basic Input/Output System) and EFI (Extensible Firmware Interface).

RAM 1003 is a volatile memory such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). RAM 1003 functions as a primary storage device, providing a working area into which various programs installed in the auxiliary storage device 1004 are expanded when executed by the CPU 1001.

The auxiliary storage device 1004 is an auxiliary storage device that stores various programs and information used when various programs are executed.

The display device 1005 is a display device that displays the internal state of the information processing device 10, etc.

The operation device 1006 is an input device through which the person operating the information processing device 10 inputs various instructions to the information processing device 10.

The I/F device 1007 is a communication device that connects to a network and communicates with other devices.

The drive device 1008 is a device for loading the storage medium 1009. The storage medium 1009 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The storage medium 1009 may also include semiconductor memory that records information electrically, such as EPROM (Erasable Programmable Read Only Memory) and flash memory.

The various programs installed in the auxiliary storage device 1004 are installed, for example, by setting the distributed storage medium 1009 in the drive device 1008 and reading the various programs recorded on the storage medium 1009 by the drive device 1008. Alternatively, the various programs installed in the auxiliary storage device 1004 may be installed by downloading them from a network via the I/F device 1007.

The information processing device 10 has a photographing device 1010. The photographing device 1010 photographs a person 20.

Although examples of the present invention have been described in detail above, the present invention is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present invention as set forth in the claims.

REFERENCE SIGNS LIST 1 Information processing system 10 Information processing device 20 Person 30 Shooting device 40 Playback device 101 Shooting unit 102 Acquisition unit 103 Identification unit 104 Video editing unit 105 Playback unit 1001 CPU
1002 ROM
1003 RAM
1004 Auxiliary storage device 1005 Display device 1006 Operation device 1007 I/F device 1008 Drive device 1009 Storage medium 1010 Imaging device

Claims (17)

a video editing unit that generates a video that displays a first person in an area including the center of the image and displays a part of a second person according to a preset display ratio;
The display rate is
indicating a proportion of an area in which the second person is displayed in the generated video;
indicating a ratio of an area in which the second person is displayed to an area in which the first person is displayed;
indicating a ratio of an area in which the first person is displayed to an area in which the second person is displayed;
The size of the entire body of the second person estimated from the image is taken as 100 percent, and the size indicates the proportion of the area in which the second person is displayed; or
a ratio of the number of pixels in the area where the second person is displayed to the total number of pixels in the image;
Information processing device. The information processing device according to claim 1 , further comprising: a playback unit that plays back the generated video. The information processing device of claim 1, wherein the video editing unit displays a portion of the second person according to the preset display ratio by trimming the second person, and deletes a portion of the acquired video based on the movement of the first person. The information processing device of claim 1, wherein the video editing unit displays a portion of the second person according to the preset display ratio by enlarging or reducing the acquired video, and enlarges or reduces the acquired video when the difference between the preset display ratio and the display ratio of the second person is greater than a threshold value. an identification unit that identifies the first person,
5. The information processing device according to claim 1, wherein the identification unit identifies a person who has been registered in advance as the first person, or identifies a person who moves the least in the acquired video as the first person, or identifies a person who is displayed most centrally in the acquired video as the first person. An information processing device according to any one of claims 1 to 5, wherein the video editing unit displays the first person so that the center position of the first person coincides with the center position of the image or a position within a predetermined range from the center of the image. The information processing device described in any one of claims 1 to 6, wherein the video editing unit displays the face of the first person in an area including the center of the image. The information processing device described in any one of claims 1 to 6, wherein the generated video displays either the face, the entire body, or the upper body of the second person. An information processing device according to any one of claims 1 to 8, wherein the first person is a person who receives a beauty treatment, and the second person is a practitioner who performs the beauty treatment on the first person. a video editing unit that displays a first person in an area including the center of an image and generates a video that displays a part of a second person according to a preset display ratio;
The video editing unit displays a portion of the second person according to the preset display ratio by trimming the second person, and deletes a portion of the acquired video based on the movement of the first person. a video editing unit that displays a first person in an area including the center of an image and generates a video that displays a part of a second person according to a preset display ratio;
The video editing unit displays a portion of the second person according to the preset display ratio by enlarging or reducing the acquired video, and enlarges or reduces the acquired video when the difference between the preset display ratio and the display ratio of the second person is greater than a threshold value. an imaging unit that images at least the first person and the second person;
a playback unit that plays the video created by the video editing unit,
The information processing device according to any one of claims 1 to 11. The video capturing device further includes an acquisition unit that acquires a video of at least the first person and the second person.
The information processing device according to any one of claims 1 to 12. Computer,
A program for causing the information processing device according to any one of claims 1 to 13 to function as the information processing device. A beauty method in which at least a first person and a second person perform beauty-related activities while viewing a video that has been shot,
a step of generating a video by a computer in which the first person is displayed in an area including the center of an image and a part of the second person is displayed according to a preset display ratio;
A step in which a viewer of the video performs a beauty-related activity while viewing the video;
Including,
The display rate is
indicating a proportion of an area in which the second person is displayed in the generated video;
indicating a ratio of an area in which the second person is displayed to an area in which the first person is displayed;
indicating a ratio of an area in which the first person is displayed to an area in which the second person is displayed;
The size of the entire body of the second person estimated from the image is taken as 100 percent, and the size indicates the proportion of the area in which the second person is displayed; or
and indicating a ratio of the number of pixels in the area in which the second person is displayed to the total number of pixels in the image. A beauty method in which at least a first person and a second person perform beauty-related activities while viewing a video that has been shot,
a step of generating a video by a computer in which the first person is displayed in an area including the center of an image and a part of the second person is displayed according to a preset display ratio;
A step in which a viewer of the video performs a beauty-related activity while viewing the video;
and
The step of generating the video includes:
the computer crops the second person to display a portion of the second person according to the preset display ratio, and deletes a portion of the captured video based on a movement of the first person; or
a step in which the computer displays a portion of the second person according to the preset display ratio by enlarging or reducing the acquired video, and enlarging or reducing the acquired video when a difference between the preset display ratio and the display ratio of the second person is greater than a threshold value. The beauty treatment method described in claim 15 or claim 16, wherein the first person is a viewer of the video.