JP7677900B2 - Information processing system, information processing method, information processing device, and program - Google Patents

Description

Applicable under Article 30, Paragraph 2 of the Patent Act Exhibition date: April 2, 2019 Exhibition name and location: Media Preview Global Innovation Center (1-2-11 Takashima, Nishi-ku, Yokohama, Kanagawa Prefecture) Presenter: Uotani Masahiko Exhibits: At the media preview, Uotani Masahiko unveiled the information processing system, information processing method, information processing device, and program invented by Maruyama Takashi.

The present invention relates to an information processing system, an information processing method, an information processing device, and a program.

For example, technology for generating an estimated face by estimating a facial image at a given point in time from a facial image at another point in time has been known for some time (see, for example, Patent Document 1).

Patent document 1 discloses an age change data generation means for generating age change data indicative of changes in a person's face due to age, a facial image data input acceptance means for accepting input of facial image data, and an estimated face generation means for applying the age change data to the facial image data accepted by the facial image data input acceptance means to generate estimated facial data at a specific point in time.

JP 2014-170488 A

Patent document 1 describes a technology for generating an estimated face by estimating a facial image at one point in time from a facial image at another point in time, and for matching or external output, but does not describe other uses of the estimated face generated by estimating a facial image at one point in time from a facial image at another point in time.

One embodiment of the present invention aims to provide an information processing system that enables communication using facial image data for each user's predicted facial age.

In order to solve the above-mentioned problems, one embodiment of the present invention is an information processing system in which a plurality of users communicate using a plurality of display devices , comprising: a facial image simulation means for predicting the facial features of each of the plurality of users at a facial age that is increased or decreased from their current age based on data relating to the faces of the plurality of users, and generating a facial image at the predicted facial age for each of the users; and a communication means for allowing the plurality of users to communicate among themselves by using the display device on which the facial image of the communication partner at the facial age is displayed, while showing each of the plurality of users the facial image of their communication partner at the facial age displayed on the display device.

According to one embodiment of the present invention, communication can be carried out using facial image data for each user's predicted facial age.

1 is a configuration diagram of an example of an information processing system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to the present embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of the imaging and measuring device according to the present embodiment. FIG. 2 is a functional block diagram of an example of an information processing device according to the present embodiment. 1 is a cross-sectional view of an example of an information processing system according to an embodiment of the present invention. 10 is a flowchart of an example of processing of the information processing system according to the present embodiment. FIG. 13 is an image diagram of an example of a screen for accepting selection of facial features and age. FIG. 13 is an image diagram of an example of a screen for accepting selection of facial features and age. 11A to 11C are diagrams illustrating an example of a facial feature image simulation process. 13 is a flowchart of an example of a communication process. FIG. 13 is an image diagram of an example of a screen before a facial feature image simulation result is displayed. FIG. 13 is an image diagram of an example of a screen before a facial feature image simulation result is displayed. FIG. 11 is an image diagram of an example of a screen after a facial feature image simulation result is displayed. FIG. 11 is an image diagram of an example of a screen after a facial feature image simulation result is displayed. 1 is a configuration diagram of an example of an information processing system according to an embodiment of the present invention. FIG. 4 is a functional block diagram of an example of a client terminal according to the embodiment; 10 is a flowchart of an example of processing of the information processing system according to the present embodiment. 10 is a flowchart of an example of processing of the information processing system according to the present embodiment.

Next, an embodiment of the present invention will be described in detail.

[First embodiment]
<System Configuration>
Fig. 1 is a configuration diagram of an example of an information processing system according to the present embodiment. In the information processing system 1 of Fig. 1, an information processing device 10, a monitor 12A, a monitor 12B, a photographing and measuring device 14A, a photographing and measuring device 14B, an operation reception device 16A, and an operation reception device 16B are connected via a network 18 such as the Internet or a LAN (local area network). The information processing system 1 of Fig. 1 is connected via the network 18, but may be connected by a connection interface such as a USB (universal serial bus) connection.

In the information processing system 1 in FIG. 1, an example of two monitors 12A and 12B is shown, but one or three or more monitors may be used. In the following, monitors 12A and 12B will be collectively referred to as monitor 12 when either is acceptable. In the case of one monitor 12, multiple users will share the screen area for use.

In addition, in the information processing system 1 of Figure 1, an example of two photographing and measuring devices 14A and 14B is shown, but three or more may be used. In the following, when either the photographing and measuring device 14A or 14B is acceptable, they will be collectively referred to as the photographing and measuring device 14.

In addition, while the information processing system 1 in Figure 1 shows an example of two operation reception devices 16A and 16B, one or three or more may be used. In the following, both operation reception devices 16A and 16B will be collectively referred to as operation reception device 16. In the case of one operation reception device 16, multiple users take turns using it.

The monitor 12 is also called a display, and is an example of a device that displays still image data and video data output from the information processing device 10. The photographing and measuring device 14 is an example of a device that has a function of photographing a face using an RGB camera and a function of measuring three-dimensional shape data of the face using a depth sensor. The photographing and measuring device 14 is realized, for example, by Kinect (registered trademark). The photographing and measuring device 14 transmits the photographed video data and the measured three-dimensional shape data of the face to the information processing device 10. Note that when using a technology that estimates three-dimensional shape data from a two-dimensional image, the photographing and measuring device 14 may be configured to omit the function of measuring three-dimensional shape data of the face using a depth sensor.

The operation reception device 16 is an example of a device that receives a selection operation from a user, and is realized by a switch or a touch panel. The operation reception device 16 transmits the operation content received from the user to the information processing device 10. The information processing device 10 is realized by a computer such as a PC. Based on the video data of the user's face and the three-dimensional shape data of the face received from the photographing and measuring device 14, and the user's operation content received from the operation reception device 16, the information processing device 10 generates video data such as still image data and video data to be displayed on the monitor 12, audio data to be played on the monitor 12, etc., as described below, and transmits them to the monitor 12.

Note that the information processing system 1 in FIG. 1 is just one example, and it goes without saying that there are various system configuration examples depending on the application and purpose. For example, the information processing device 10 may be configured to be distributed across multiple computers. Furthermore, the information processing device 10 may be realized by using a cloud service.

<Hardware Configuration>
"Information processing device"
The information processing apparatus 10 in Fig. 1 is realized by, for example, a computer having a hardware configuration as shown in Fig. 2. Fig. 2 is a diagram showing an example of the hardware configuration of a computer according to the present embodiment.

2 includes an input device 501, an output device 502, an external I/F 503, a RAM 504, a ROM 505, a CPU 506, a communication I/F 507, and a HDD 508, all of which are interconnected by a bus B. Note that the input device 501 and the output device 502 may be connected and used when necessary.

The input device 501 is a touch panel, operation keys or buttons, a keyboard, a mouse, a microphone, etc. used for input. The output device 502 is composed of a display such as an LCD or an OLED display for displaying a screen, and a speaker for outputting sound data such as voice and music. The communication I/F 507 is an interface that connects the computer to the network 18. The HDD 508 is an example of a non-volatile storage device that stores programs and data. The computer may also use a drive device that uses flash memory as a storage medium (e.g., a solid state drive: SSD).

The external I/F 503 is an interface with an external device. The external device may be a recording medium 503a. This allows the computer to read and/or write to the recording medium 503a via the external I/F 503. The recording medium 503a may be a flexible disk, a CD, a DVD, an SD memory card, a USB memory, etc.

ROM 505 is an example of a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. RAM 504 is an example of a volatile semiconductor memory (storage device) that temporarily retains programs and data. CPU 506 is a calculation device that realizes the control and functions of the entire computer by reading programs and data from storage devices such as ROM 505 and HDD 508 onto RAM 504 and executing processing. The information processing device 10 of this embodiment can realize various functions by executing programs on a computer with the hardware configuration described above. Note that the hardware configuration in FIG. 2 is one example, and it goes without saying that there are various configuration examples depending on the application and purpose.

"Photographic Measuring Device"
The photographing and measuring device 14 in Fig. 1 is realized by, for example, a device having a hardware configuration as shown in Fig. 3. Fig. 3 is a diagram showing the hardware configuration of an example of the photographing and measuring device according to this embodiment.

The photographing and measuring device 14 in Fig. 3 is configured to include a camera 601, a light source 602, a depth sensor 603, a microphone 604, a processor 605, an external I/F 606, and a communication I/F 607. The camera 601 is, for example, an RGB camera, which photographs the user's face and outputs the user's face image data. The light source 602 projects infrared rays onto the user's face, which is the subject of measurement of three-dimensional shape data. The depth sensor 603 photographs the user's face with infrared rays (projection pattern) projected by the light source 602, measures the depth of each point on the image, and outputs three-dimensional shape data of the user's face.

The microphone 604 captures the user's voice and generates and outputs audio data. The processor 605 controls processing by the camera 601, the light source 602, the depth sensor 603, and the microphone 604. The processor 605 transmits the user's face image data output from the camera 601, the three-dimensional shape data of the user's face output from the depth sensor 603, and the audio data output from the microphone 604 to the information processing device 10 via the external I/F 606 or the communication I/F 607.

<Software configuration>
The software configuration of the information processing device 10 according to this embodiment will be described. Fig. 4 is a functional block diagram of an example of the information processing device according to this embodiment. The information processing device 10 executes a program to realize an operation determination unit 20, a facial feature image simulation unit 22, a communication unit 24, a display control unit 26, a question DB 28, an average face model DB 30, and an initial setting DB 32.

The operation judgment unit 20 judges the operation content accepted by the operation reception device 16. The facial feature image simulation unit 22 performs a facial feature image simulation process described below according to the operation content judged by the operation judgment unit 20. The communication unit 24 also performs a communication process described below in which the facial feature image simulation results described below are displayed on the display control unit 26 and communication is performed between the users. Note that in this embodiment, an example is described in which the facial feature image simulation results of two people are displayed on the display control unit 26 and two users communicate with each other, but three or more users may also be involved. The display control unit 26 controls the display of the output device 502 and the playback of audio data.

The question DB 28 stores questions for a user communicating through the information processing system 1 according to this embodiment, in association with the relationship with the communication partner (friend, family, etc.). It is desirable for the questions to have content that corresponds to the relationship with the communication partner.

The average face model DB30 categorizes ages into teens, twenties, etc., and stores average face models for each age group created from three-dimensional shape data of the faces of men and women in each age group. The average face models represent aging changes in facial features, and are used to predict future facial features from current facial features, as described below, and to predict past facial features. The creation of average face models will be described later.

The initial setting DB 32 stores various information (nickname, relationship with the communication partner, gender, age, etc.) initially set by a user who communicates through the information processing system 1 according to this embodiment. The initial setting by the user may be accepted from the operation acceptance device 16, or may be accepted from a PC, smartphone, tablet terminal, etc. other than the operation acceptance device 16 that is capable of data communication with the information processing device 10.

<Appearance>
The external appearance of the information processing system 1 according to this embodiment will be described. Fig. 5 is a cross-sectional view of an example of the information processing system 1 according to this embodiment. The information processing system 1 in Fig. 5 is provided with two private rooms 100A and 100B. Users who wish to communicate enter the separate private rooms 100A or 100B. Note that the private rooms 100A and 100B may be surrounded by transparent or highly transparent glass, for example, so that the users who wish to communicate can feel each other's presence even after entering the separate private rooms 100A or 100B.

For example, Figure 5 shows an example in which user A enters private room 100A and user B enters private room 100B. Private room 100A is equipped with monitor 12A, photographing and measuring device 14A, and operation reception device 16A. Private room 100B is equipped with monitor 12B, photographing and measuring device 14B, and operation reception device 16B.

Although Fig. 5 shows an example in which the information processing device 10 is installed between private rooms 100A and 100B, it does not necessarily have to be installed in the position shown in Fig. 5. The information processing device 10 is connected to the monitors 12A and 12B, the photographing and measuring devices 14A and 14B, and the operation reception devices 16A and 16B via a connection interface such as a USB connection or a network 18 such as a LAN.

User A can see the facial image of user B after the facial image simulation process displayed on monitor 12A. Similarly, user B can see the facial image of user A after the facial image simulation process displayed on monitor 12B. In this way, user A and user B communicate with each other while looking at each other's facial image after the facial image simulation process displayed on monitor 12.

An operation reception device 16A such as a switch is installed in a position where user A can operate it while sitting facing the monitor 12A. User A can select facial features and age from the operation reception device 16A. The photographing and measuring device 14A photographs and measures user A's face, and transmits image data of the photographed face of user A and three-dimensional shape data of the measured face of user A to the information processing device 10.

When estimating three-dimensional shape data from a two-dimensional image, for example, image data of user A's face captured is sent to the information processing device 10, and the information processing device 10 estimates the three-dimensional shape data of user A's face. When estimating three-dimensional shape data from a two-dimensional image, for example, the photographed image data of user A's face captured by the photographing and measuring device 14A may be estimated as three-dimensional shape data of user A's face, and the three-dimensional shape data of user A's face may be transmitted to the information processing device 10.

An operation reception device 16B such as a switch is placed in a position where user B can operate it while sitting facing the monitor 12B. User B can select facial features and age from the operation reception device 16B. The photographing and measuring device 14B photographs and measures the face of user B, and transmits the photographed image data of user B's face and the measured three-dimensional shape data of user B's face to the information processing device 10. Note that the process for estimating three-dimensional shape data from a two-dimensional image is the same as the process for the image data of user A's face described above.

In this way, user A and user B can face each other via monitors 12A and 12B, which are an example of an interface, and communicate through conversation while looking at the face of the communication partner, whose facial features and age have changed. The conversation may be carried out using voice data via the photographic measurement devices 14A and 14B, or may be carried out via a chat system or the like.

In FIG. 5, an example is shown in which user A and user B enter separate private rooms 100A and 100B and sit facing each other, but this is not limited to this. For example, in FIG. 5, the monitors 12A and 12B also function as partitions that block the line of sight so that the face of the communication partner is not directly visible, so they may enter the same private room together and sit facing each other. Also, the transparency of the glass between the two private rooms 100A and 100B may be reduced to serve as a partition that blocks the line of sight so that the face of the communication partner is not directly visible. Also, it is not necessary to sit facing each other, and they may sit facing the same direction or back to back. Furthermore, the private rooms 100A and 100B may be installed in separate locations (remotely) so that the face of the communication partner is not directly visible.

<Processing>
The information processing system 1 according to this embodiment allows two or more users to communicate with each other by changing the facial age of the user who is the communication partner and displayed on the monitor 12, for example, in a procedure as shown in Fig. 6. Fig. 6 is a flowchart of an example of the processing of the information processing system according to this embodiment. Note that an example in which user A and user B shown in Fig. 5 communicate with each other will be described here.

In step S10, the information processing device 10 accepts initial settings of various information (nickname, relationship with the communication partner, gender, age, etc.) by the users who communicate. The initial settings in step S10 may be set from a tablet terminal or smartphone before users A and B enter the private rooms 100A and 100B, or may be set from operation reception devices 16A and 16B after users A and B enter the private rooms 100A and 100B. The various information initially set by users A and B is stored in the initial setting DB 32.

In step S12, the display control unit 26 of the information processing device 10 uses various initially set information and image data of the faces of user A and user B captured by the photographing measurement devices 14A and 14B to display a screen such as that shown in Figures 7A and 7B, and accepts selection of the facial ages of user A and user B from at least one of user A and user B.

Figures 7A and 7B are image diagrams of an example of a screen that accepts the selection of facial age. Figure 7A is an example of a screen displayed on monitor 12A. Figure 7B is an example of a screen displayed on monitor 12B. Figures 7A and 7B show example screens in which user A selects the facial ages of users A and B, but user B may select the facial ages of users A and B, or users A and B may each select the facial age of the person they are communicating with.

Screen 1000A in Figure 7A is an example of a screen viewed by user A, and displays a facial image 1002A of user B, who is a communication partner. Also displayed at the bottom of screen 1000A is a selection field 1004A that user A uses to select his or her own facial age and the facial age of user B, who is a communication partner. User A operates operation reception device 16A while viewing selection field 1004A, and selects the facial ages of user A and user B.

Screen 1000B in Figure 7B is an example of a screen seen by user B, and displays an image 1002B of the face of user A, who is the communication partner. In addition, at the bottom of screen 1000B, the facial ages of user A and user B that user A is selecting or has selected from selection field 1004A are displayed in selection field 1004B. By looking at selection field 1004B, user B can confirm the facial ages of user A and user B that user A is selecting or has selected.

The operation judgment unit 20 of the information processing device 10 judges the facial ages of user A and user B selected by user A, and notifies the facial image simulation unit 22. The facial ages of user A and user B may be judged on the spot, or may be judged using age data recorded in the initial setting DB 32. When the facial ages of user A and user B are notified, the facial image simulation unit 22 proceeds to step S14 and performs a facial image simulation process as shown in FIG. 8.

Figure 8 is an explanatory diagram of an example of a facial feature image simulation process. The average face model 1100 is a model of average face images for people in their 20s, 30s, 40s, 50s, and 60s, which are examples of age categories.

The average face model 1100 in Figure 8 is created by collecting three-dimensional shape data of the faces of, for example, Japanese men and women in their 20s, 30s, 40s, 50s, and 60s, and averaging the three-dimensional shape data for each age group. Note that the age groups are just an example.

The skin texture library 1102 is also a library of skin textures for each age group that correspond to the age groups of the average face model 1100. A skin texture is an image used to represent the skin of the user's face.

The user's face image 1104 is video data or still image data captured by the photographing and measuring device 14. The three-dimensional shape data 1106 of the user's face is three-dimensional shape data obtained by measuring the user's face by the photographing and measuring device 14. The three-dimensional shape data 1106 of the user's face may be created from the user's face image 1104.

The facial feature image simulation unit 22 reads out from the average face model DB 30 an average face model 1100 of the age of the user (hereinafter referred to as the target user) who is the subject of the facial feature image simulation processing initially set in step S10, and an average face model 1100 of the facial feature age of the target user selected in step S12.

Then, the facial feature image simulation unit 22 calculates the difference data of the read average face model 1100. The difference data of the average face model 1100 may be calculated in advance and recorded in the average face model DB 30. For example, if the age of the target user, user B, is in his 30s and the facial feature age of the selected target user is in his 50s, the difference data between the average face model 1100 in his 30s and the average face model 1100 in his 50s is calculated.

Then, the facial feature image simulation unit 22 applies (synthesizes) the difference data between the average face model 1100 for people in their 30s and the average face model 1100 for people in their 50s to the three-dimensional shape data 1106 of the face of the target user. The facial feature image simulation unit 22 also reads out the skin texture of the selected facial feature age from the skin texture library 1102.

Then, the facial feature image simulation unit 22 can generate a facial feature image 1110 of the target user at the selected facial feature age by CG synthesis of the three-dimensional shape data of the target user's face to which the differential data has been applied with skin texture read out from the skin texture library 1102.

In this way, the facial feature image simulation unit 22 can output the facial feature image 1110 of the target user at the facial feature age selected by the user as the facial feature image simulation result. By utilizing the average face model 1100 and the skin texture library 1102, the facial feature image 1110 of the target user generated by the facial feature image simulation unit 22 is a past or future facial feature image predicted from the target user's current facial features.

Returning to step S16 in FIG. 6, the communication unit 24 of the information processing device 10 displays a screen including the facial image simulation results on the monitor 12 as described below, and performs the communication process of FIG. 9 to allow communication between users.

Figure 9 is a flowchart of an example of communication processing. In step S30, the communication unit 24 uses the facial image simulation results to transition the display on the monitor 12, for example, from the screens of Figures 10A and 10B to the screens of Figures 11A and 11B.

Figures 10A and 10B are image diagrams of an example of a screen before the facial image simulation results are displayed. Figure 10A is an example of a screen displayed on monitor 12A. Figure 10B is an example of a screen displayed on monitor 12B. Figures 10A and 10B show screens 1200A and 1200B, which display messages 1202A and 1202B for the facial ages of users A and B selected by user A.

Figures 11A and 11B are image diagrams of an example of a screen after the facial feature image simulation results are displayed. Figure 11A is an example of a screen displayed on monitor 12A. Figure 11B is an example of a screen displayed on monitor 12B.

In Fig. 11A, a facial image 1302A of user B with the selected facial features and age is displayed as the result of the facial features image simulation. In Fig. 11B, a facial image 1302B of user A with the selected facial features and age is displayed as the result of the facial features image simulation.

In step S32, the communication unit 24 reads out a question from the question DB 28 according to the relationship with the communication partner (the relationship between user A and user B) initially set in step S10 and the selected facial features and age, and displays it as in message 1304A in Fig. 11A and message 1304B in Fig. 11B. It is desirable that messages 1304A and 1304B have content that will start a conversation between user A and user B or that will stimulate conversation in the conversation process in the next step S34.

In step S34, the communication unit 24 causes user A and user B to communicate through conversation while the screens 1300A and 1300B in Fig. 11A and Fig. 11B are displayed on the monitors 12A and 12B, respectively. This allows user A and user B to communicate through conversation while looking at the face of the communication partner whose facial features and age have been aged or decreased from their current age.

According to the information processing system 1 of this embodiment, various effects can be expected, such as giving people an opportunity to remember forgotten feelings or making them aware of feelings they were not aware of, by having a conversation while seeing what they will look like in the future, such as 20 years from now, having a conversation while seeing what they looked like in the past, such as 20 years ago, or having a conversation while seeing the age-swapped appearance of parents and children.

Second Embodiment
The information processing system 1 according to the first embodiment is just one example, and the monitor 12, the photographing and measuring device 14, and the operation reception device 16 may be replaced with a client terminal such as a PC, smartphone, or tablet terminal having a facial photographing function using an RGB camera and a function for measuring three-dimensional facial shape data using a depth sensor.

In this way, in the configuration of the second embodiment, a service similar to that of the first embodiment can be realized by the client terminal and the information processing device 10.

[Third embodiment]
In the information processing system 1 according to the first and second embodiments, the information processing device 10 is configured to have an operation judgment unit 20, a facial image simulation unit 22, a communication unit 24, a display control unit 26, a question DB 28, an average face model DB 30, and an initial setting DB 32, but these configurations may also be configured to be included in, for example, the client terminals 200A and 200B in FIG. 12.

Figure 12 is a configuration diagram of an example of an information processing system according to this embodiment. In the information processing system 1a of Figure 12, a client terminal 200A, a client terminal 200B, and a server device 202 are connected via a network 104. Note that, as in the first embodiment, the information processing system 1a of Figure 12 shows an example of two client terminals 200A and 200B, but three or more client terminals may be used. In the information processing system 1a of Figure 12, the server device 202 mediates data communication between the client terminals 200A and 200B.

Figure 13 is a functional block diagram of an example of a client terminal according to this embodiment. The client terminal 200 in Figure 13 executes an application program to realize an operation judgment unit 220, a facial image simulation unit 222, a communication unit 224, a display control unit 226, a communication unit 228, a question DB 230, an average face model DB 232, and an initial setting DB 234.

The information processing system 1a of this embodiment changes the facial features and ages of the users displayed on the client terminals 200A and 200B, for example, in a procedure such as that shown in FIG. 14, thereby allowing user A operating client terminal 200A and user B operating client terminal 200B to communicate with each other.

Figure 14 is a flowchart of an example of processing of the information processing system according to this embodiment. In step S50, client terminals 200A and 200B are connected in a state in which data communication is possible. In step S52, client terminals 200A and 200B accept initial settings of various information (nickname, relationship with communication partner, gender, age, etc.) by users A and B. In addition, client terminals 200A and 200B accept a selection of facial features and age by at least one of users A and B.

In step S54, the client terminals 200A and 200B perform the facial image simulation process of step S14 described above. In step S56, the client terminals 200A and 200B transmit and receive the facial image simulation results via the server device 202.

In step S58, client terminals 200A and 200B start displaying a screen (e.g., screens 1300A and 1300B) containing the received facial feature image simulation results. In step S60, client terminals 200A and 200B display a question message (e.g., messages 1304A and 1304B on screens 1300A and 1300B) on the screen containing the received facial feature image simulation results.

In step S62, client terminals 200A and 200B continue to display the screen including the received facial image simulation results while transmitting and receiving conversation (audio data), thereby allowing users A and B to communicate through conversation.

In the processing of Figure 14, an example is shown in which the facial image simulation results are transmitted and received by the client terminals 200A and 200B, but as shown in Figure 15, the client terminals 200A and 200B may transmit and receive captured video data and measured three-dimensional shape data of the face as the capture results.

In the process of Figure 15, the client terminals 200A and 200B perform a facial image simulation process in step S108 using the photographic results received from the communication partner. The process from step S110 onwards is the same as the process of Figure 14, and therefore will not be described.

As shown in Figures 14 and 15, the data that client terminals 200A and 200B send to the communication partner may be the shooting results before the facial features image simulation processing, or the facial features image simulation results after the facial features image simulation processing.

Although the present invention has been described above based on the embodiments, the present invention is not limited to the above embodiments and various modifications are possible within the scope of the claims. This application claims priority to basic application No. 2019-203589 filed on November 8, 2019 with the Japan Patent Office, the entire contents of which are incorporated herein by reference.

REFERENCE SIGNS LIST 1, 1a Information processing system 10 Information processing device 12A, 12B Monitor 14A, 14B Photographing and measuring device 16A, 16B Operation reception device 18 Network 20, 220 Operation determination unit 22, 222 Facial image simulation unit 24, 224 Communication unit 26, 226 Display control unit 228 Communication unit 28, 230 Question DB
30,232 Average Face Model DB
32, 234 Initial setting DB
100A, 100B Private room 200A, 200B Client terminal 202 Server device 1000A, 1000B Screen 1002A, 1002B Face image 1004A, 1004B Selection field 1100 Average face model 1102 Skin texture library 1104 User's face image 1106 Three-dimensional shape data of user's face 1110 Facial image of target user at selected facial age 1304A, 1304B Message

Claims (14)

An information processing system in which a plurality of users communicate with each other using a plurality of display devices ,
a facial feature image simulation means for predicting the facial features of each of the users at a facial feature age that is increased or decreased from a current age based on face data of the plurality of users, and generating a facial feature image of each of the users at the predicted facial feature age;
a communication means for allowing the plurality of users to communicate with each other while showing the facial feature images of the communication partners at the facial feature ages displayed on the display device to each of the plurality of users , using the display device on which the facial feature images of the communication partners at the facial feature ages are displayed;
An information processing system having the above configuration. The face data includes:
(i) face image data captured by one or more imaging means for capturing faces of the plurality of users; and
2. The information processing system according to claim 1, wherein the information is at least one of (ii) three-dimensional shape data of faces measured by one or more measuring means for measuring the faces of the plurality of users. an operation receiving means for receiving from the user a selection operation of the facial features and age of the communication partner;
The information processing system according to claim 1 , further comprising: The information processing system according to claim 1, characterized in that the facial image simulation means divides the age into a plurality of generations and predicts the facial features of each of the users at a facial age that is older or younger than their current age based on an average face model for each generation created from learning face data for the ages included in the generation. The information processing system according to claim 4, characterized in that the facial feature image simulation means predicts the facial features of each of the users at a facial feature age that is older or younger than the current age based on difference data between an average face model for the age group that includes the current age and an average face model for the age group that includes the facial feature age. 4. The information processing system according to claim 3, wherein the operation receiving means receives, from the user, a selection operation of a relationship with the communication partner. 7. The information processing system according to claim 6, wherein said communication means asks questions based on relationships with said communication partners to said plurality of users who are communicating with said plurality of users. a partition means for blocking the line of sight so that the face of the communication partner cannot be directly seen;
The information processing system according to claim 1 , further comprising: 3. The information processing system according to claim 2, wherein a plurality of said image capturing means and said communication means are provided remotely so that the face of said communication partner cannot be directly seen. An information processing method executed by an information processing system in which a plurality of users communicate with each other using a plurality of display devices , comprising:
A step of predicting facial features of each of the users at a facial age that is increased or decreased from a current age based on face data of the multiple users, and generating a facial feature image at the predicted facial age of each of the users;
a step of using the display device on which a facial feature image of a communication partner at the facial feature age is displayed, causing the plurality of users to communicate with each other while showing the facial feature image of the communication partner at the facial feature age displayed on the display device to each of the plurality of users;
An information processing method comprising the steps of: An information processing device for communicating with a communication partner user,
a facial features image simulation means for receiving a selection operation of a facial features age obtained by increasing or decreasing the current age of an operating user who operates the information processing device or a user who is a communication partner , predicting the facial features of the operating user at the facial features age based on data related to the face of the operating user, and generating a facial features image of the operating user at the predicted facial features age;
a communication means for transmitting the generated facial image of the operating user at the facial age to a device operated by the communication partner user, and receiving the facial image of the communication partner user at the facial age from the device operated by the communication partner user;
a display means for displaying a facial image of the user who is a communication partner at the facial age;
a communication means for allowing the operating user and the communication partner user to communicate with each other while showing each other's facial features images at the facial features ages displayed on the display means, by utilizing a facial features image at the facial features ages of the communication partner user displayed on the display means;
having
The facial feature image simulation means is an information processing device that generates the facial feature image of the operating user based on data regarding the face of the operating user and the facial feature age of the operating user who has accepted the selection operation . An information processing device for communicating with a communication partner user,
a communication means for transmitting data relating to a face of an operating user who operates the information processing device to a device operated by a user of the communication partner, and receiving data relating to the face of the user of the communication partner from the device operated by the user of the communication partner;
a facial feature image simulation means for receiving a selection operation of a facial feature age obtained by increasing or decreasing the current age of an operating user who operates the information processing device or a communication partner user, predicting the facial features of the communication partner user at the facial feature age based on the received data on the face of the communication partner user, and generating a facial feature image of the predicted facial feature age of the communication partner user;
a display means for displaying a facial image of the user who is a communication partner at the facial age;
a communication means for allowing the operating user and the communication partner user to communicate with each other while showing each other's facial features images at the facial features ages displayed on the display means, by utilizing a facial features image at the facial features ages of the communication partner user displayed on the display means;
having
The facial image simulation means is an information processing device that generates the facial image of the communication partner user based on data regarding the face of the communication partner user and the facial age of the communication partner user who accepted the selection operation . An information processing device for communicating with a communication partner user,
a facial features image simulation means for receiving a selection operation of a facial features age obtained by increasing or decreasing the current age of an operating user who operates the information processing device or a user who is a communication partner , predicting the facial features of the operating user at the facial features age based on data related to the face of the operating user, and generating a facial features image of the operating user at the predicted facial features age;
a communication means for transmitting the generated facial image of the operating user at the facial age to a device operated by the communication partner user, and receiving the facial image of the communication partner user at the facial age from the device operated by the communication partner user;
A display means for displaying a facial image of the user at the facial age of the communication partner;
a communication means for allowing the operating user and the communication partner user to communicate with each other while showing each other the facial feature images of the communication partner at the facial feature age displayed on the display means, by utilizing a facial feature image of the communication partner user at the facial feature age displayed on the display means;
Function as a
The facial feature image simulation means is a program that generates the facial feature image of the operating user based on data relating to the face of the operating user and the facial feature age of the operating user who has accepted the selection operation . An information processing device for communicating with a communication partner user,
a communication means for transmitting data relating to a face of an operating user who operates the information processing device to a device operated by a user of the communication partner, and receiving the data relating to the face of the user of the communication partner from the device operated by the user of the communication partner;
a facial features image simulation means for receiving a selection operation of a facial features age obtained by increasing or decreasing the current age of an operating user who operates the information processing device or the communication partner user, predicting the facial features of the communication partner user at the facial features age based on the received data relating to the face of the communication partner user, and generating a facial features image of the predicted facial features age of the communication partner user;
A display means for displaying a facial image of the user at the facial age of the communication partner;
a communication means for allowing the operating user and the communication partner user to communicate with each other while showing each other the facial feature images of the communication partner at the facial feature age displayed on the display means, by utilizing a facial feature image of the communication partner user at the facial feature age displayed on the display means;
Function as a
The facial image simulation means is a program that generates the facial image of the communication partner user based on data regarding the face of the communication partner user and the facial age of the communication partner user who accepted the selection operation .

Images