JP6902127B2 - Video output system - Google Patents

Description

The present invention relates to a moving image output system.

While it is possible to actually communicate with a person close to you, it is difficult to actually communicate with a deceased person or a celebrity. For example, in Patent Document 1 below, information on a target person such as a deceased person or a celebrity who has difficulty in actual communication is registered in a homepage server, and information on the target person can be browsed on the Internet by browsing the homepage of each person. The technical idea to do is described.

JP-A-2009-187514

However, for example, if the target person is a deceased person, only the information obtained during his lifetime can be obtained, and if it is a celebrity, it is difficult to obtain information other than general-purpose information such as photographs that are open to the public. There is a problem that it is difficult to feel close to the target person because only limited information is available.

In view of the above circumstances, the purpose of this proposal is to provide a moving image output system that can express a person whose information is difficult to obtain so that the viewer can feel closer to him / her.

The moving image output system of the present invention can process a voice input via a sound collecting microphone, convert it into a voice having at least one of a different voice color and speaking style, and output the voice. Further, this moving image output system can generate and output a moving image based on a still image in which a human face is captured.

According to the moving image output system of the present invention, it is possible to output audio having various meanings depending on the input audio. Therefore, according to this moving image output system, even a person who can obtain only limited information can output audio including various information together with the moving image. By outputting the audio together with the video in this way, the viewer can feel the person closer.

As described above, the moving image output system of the present invention is a moving image output system having excellent characteristics that allows a person whose information is difficult to obtain to be expressed so that the viewer can feel closer to him / her.

The perspective view of the digital portrait (video output system) in Example 1. FIG. The explanatory view of the operation state of the digital portrait in Example 1. FIG. The block diagram which shows the electrical structure of the digital portrait in Example 1. FIG. The explanatory view of the feature point extraction process in Example 1. FIG. The flow chart which shows the procedure of modeling a deceased person in Example 1. FIG. The flow chart which shows the flow of the speaker information extraction processing in Example 1. FIG. The flow chart which shows the flow of moving image generation processing, audio conversion processing in Example 1. The flow chart which shows the flow of the speaker information extraction processing in Example 2. FIG. The flow chart which shows the flow of the voice conversion processing in Example 2. The explanatory view of the moving image output system in Example 4. The explanatory view of the moving image output system in Example 5. The flow chart which shows the flow of the moving image generation processing in Example 6. The explanatory view of the stereoscopic display part in Example 7. Explanatory drawing which illustrates the stereoscopic image in Example 7. FIG. Explanatory drawing of another stereoscopic display part in Example 7. Explanatory drawing which illustrates the stereoscopic image in Example 7. FIG. The figure which shows the digital portrait in Example 8. The explanatory view of the structure of the stereoscopic display part in Example 8. The figure which shows the other digital portrait in Example 8.

Embodiments of the present invention will be specifically described with reference to the following examples.
(Example 1)
This example is an example relating to a digital portrait 1 capable of outputting a moving image together with audio. This content will be described with reference to FIGS. 1 to 7.
Digital portrait 1, which is an example of a moving image output system, is used as a deceased's deceased by moving images with audio. The digital portrait 1 functions as a high-performance output device in which the deceased deceased responds to the questions of attendees.

Digital portrait 1 (FIG. 1) has a lacquered black frame 1F, and a display screen 210 of a liquid crystal display 21 (see FIG. 3) is arranged inside the frame 1F. On the back side of the digital portrait 1, a housing 13 that is slightly smaller than the frame 1F is provided, and a storage space for an electrical configuration is secured.

On the outer periphery of the housing 13 (FIGS. 1 and 2), built-in speakers 223 are provided on both left and right side surfaces, and a power switch 131, external terminals 132 to 135, and the like are arranged. Examples of the external terminal include a speaker terminal 134, a microphone terminal 135, a USB terminal 133, and an external monitor terminal 132. Further, a built-in camera 221 and a built-in microphone 222 are embedded in the front surface of the frame 1F. The built-in microphone 222 is, for example, a sound collecting microphone for acquiring a voice asking an individual by a funeral attendee or the like. The built-in camera 221 is an imaging camera for imaging attendees asking the deceased.

A USB standard keyboard, mouse, or the like can be connected to the USB terminal 133. If a keyboard or mouse is connected, the digital portrait 1 can be operated in the same manner as a general PC, and various setting operations can be performed. At the time of setting operation, it is also possible to use the display screen 210 surrounded by the black-painted frame 1F as a monitor. Further, an external memory 39 (FIG. 3) such as a general USB memory can be attached to the USB terminal 133. By using a storage medium such as a USB memory, it is possible to transfer image data of a still image of the deceased, which is the source of a moving image, audio data of the deceased, software for enhancing the function of digital portrait 1, and the like.

An external microphone (sound collecting microphone) 31 can be connected to the microphone terminal 135. If the external microphone 31 is used, the operator 300 in the backyard can input voice. It is also possible to provide a plurality of microphone terminals 135. In this case, an external microphone for the attendees to input voice can be connected. Even if the attendees cannot get close to the deceased on the altar, if there is an external microphone for the attendees, the question voice can be input with high certainty.

An external speaker 32 can be connected to the speaker terminal 134. For example, it is preferable to install the external speaker 32 in the back yard, which is reserved by the operator 300 corresponding to the questioning voice of the attendees. If the external speaker 32 that outputs the questioning voice is set, the degree of freedom in installing the backyard that the operator 300 corresponding to the questioning voice refrains from increases.

A general-purpose PC monitor can be connected to the external monitor terminal 132. For example, it is also possible to install an external monitor 33 in the backyard reserved by the operator 300 and display an image captured by the attendees by the built-in camera 221. If an external monitor 33 that displays attendees is installed in the backyard, the operator 300 can more accurately respond to the questions of attendees while checking the facial expressions and gestures of attendees. Further, for example, when performing the setting operation of the digital portrait 1, for example, if the keyboard, mouse, and external monitor are provided on the desk on which the operator 300 works, various setting operations and function switching operations of the digital portrait 1 are performed. Etc. can be carried out from the backyard. It is also good to install a second external monitor in the backyard. In parallel with the work of responding to the voice of the attendees, the digital portrait 1 setting operation and the like can be performed.

All the functions of the external terminals can be replaced by wireless technology such as Bluetooth (registered trademark) and wireless display. According to the wireless technology, it is possible to significantly improve the degree of freedom in setting the backyard, which is reserved by the operator 300 who responds to the question voice and the setting operation. It is also possible to provide a backyard for the operator 300 to refrain from at a remote location. For example, if the Internet is used, an operator 300 at a remote location can respond to a question voice.

As shown in FIG. 3, the housing 13 includes a main board 2 that realizes functions as a moving image generation unit 202, a sound conversion unit 204, and the like, a display control board 211 that controls a liquid crystal display 21, a hard disk (HD) 24, and the like. The storage device, the amplifier board 22 of the built-in speaker 223 and the built-in microphone 222, the built-in camera 221 and the power supply circuit (not shown) are housed in the storage device.

The main board 2 is an electronic board on which a CPU (Central Processing Unit) 20 that executes various arithmetic processes, storage elements such as ROM 207 and RAM 208, and an I / O controller 209 that controls input / output are mounted. The digital portrait 1 has a hardware configuration similar to that of a personal computer (PC) with an integrated display. A general-purpose OS (Operating System) such as Windows (registered trademark) or Linux (registered trademark) may be incorporated in the digital portrait 1, or an original OS may be incorporated.

The hard disk 24 has a storage area that can be read by the CPU 20 of the main board 2, and has a function as a deceased information storage unit 240. The deceased information storage unit 240 serves as a storage area for storing various information related to the deceased in addition to the still image and audio data of the deceased.

The CPU 20 executes a software program stored in the hard disk 24 to generate a three-dimensional model of the deceased, a deceased modeling unit 201, a moving image generating unit 202 that generates a moving image of the deceased, and voice information of the deceased (speaker information). Various functions are realized as a speaker information extraction unit 203 for extracting (one example), a voice conversion unit 204 for converting the voice color of the operator's response voice, a voice recognition unit 205, and the like.

Next, (1) deceased information storage unit 240, (2) deceased modeling unit 201, (3) video generation unit 202, (4) speaker information extraction unit 203, (5) voice conversion unit 204, (6) Each function of the voice recognition unit 205 will be described.

(1) Deceased information storage department (speaker information storage department)
The deceased information storage unit 240 stores, for example, the following information related to the deceased.
(1.1) Still image of the deceased: A still image of the deceased taken in advance.
(1.2) Voice data of the deceased: Pre-recorded voice data of the deceased.
(1.3) Voice information of the deceased: Voice information that identifies the voice of the deceased.
(1.4) Three-dimensional model data of the deceased: Data of a three-dimensional model (wire frame model) based on a still image of the deceased.
(1.5) Information on the deceased: Information on the deceased's hobbies, favorite foods, friendships, and relatives. It is also possible to include the face image and personal information of a person who has a friendship or relative relationship with the deceased in the deceased's information. It is preferable that the information of the deceased person can be referred to as appropriate according to the operation by the operator. With the information of the deceased, the operator can respond more accurately to the questions of the attendees.

(2) Deceased Modeling Unit The deceased modeling unit 201 is composed of a feature point extraction unit that extracts facial feature points and a three-dimensional model generation unit.
(2.1) Feature point extraction unit The feature point extraction unit extracts feature point FPs (see FIG. 4) such as eyes, nose, lips, and eyebrows from the face area of the still image of the deceased (see FIG. 4). Execute an example of image processing). For example, for the eyes, the inner corners of the eyes, the outer corners of the eyes, the black eyes, the points forming the contour of the eyes, and the like are extracted as feature points FP. For example, for the mouth, the corners of the mouth, the contoured points of the upper lip, the contoured points of the lower lip, and the like are extracted as feature points FP.
(2.2) 3D model generation unit The 3D model generation unit executes a 3D model generation process for generating a 3D model of the deceased. The 3D model generation process, which is an example of image processing, is a process of transforming a standard 3D model, which is a wire frame model representing a 3D shape of a standard face (standard face), to fit the deceased. This is a process to generate a three-dimensional model of the deceased.

The standard three-dimensional model is a wire frame model in which feature points corresponding to eyes, nose, lips, eyebrows, etc. are defined as vertices or intersections, for example. In the 3D model generation process, the feature points extracted from the still image of the deceased and the feature points of the standard 3D model are associated with each other. A 3D model of the deceased can be generated by modifying the standard 3D model so as to match the positional relationship of the feature points related to the deceased. It is advisable to assign the color and texture of the corresponding area of the still image to each surface of the wire frame model that forms the three-dimensional model of the deceased.

(3) Movie Generation Unit The movie generation unit 202 executes a movie generation process of transforming the deceased's three-dimensional model to generate motion. In this moving image generation process, a process of locally or totally deforming the wire frame model forming the three-dimensional model is performed. For example, when realizing the operation of closing the eyelids, it is preferable to displace the vertices or intersections of the wire frame model such as the feature points constituting the eyes. For example, when the operation of opening the mouth is realized, a process of displacing the feature points constituting the mouth is performed. Furthermore, for example, when realizing a nodding motion, instead of the local deformation of the wire frame model forming the deceased's three-dimensional model, the entire wire frame model is rotationally displaced so as to lightly move forward, and then immediately. The process of returning to the original position is performed. Further, when the deceased person is made to express anger, the wire frame model is deformed so as to lift the eyebrows and the outer corners of the eyes. For example, during the deceased's utterance, the deceased's natural utterance can be reproduced by appropriately combining blinking eyes, opening and closing the mouth, and nodding.

(4) Speaker Information Extraction Unit The speaker information extraction unit 203 executes a speaker information extraction process for separating and extracting voice color information from pre-recorded voice data of the deceased. When signal processing is applied to the voice data, it can be decomposed into sound source information derived from the vibration of the vocal cords and voice color information reflecting the skeleton of the speaker's jaw and the shape of the oral cavity. The sound source information is information representing the characteristics of the sound generated from the sound source, such as the strength and pitch of the sound. Voice information is information that represents the characteristics of a path through which sound generated from a sound source is transmitted. The voice color information is information indicating the voice quality, and is information that reflects the shape of the oral cavity through which the vibration of the vocal cords is transmitted. The voice color information extracted by the speaker information extraction unit 203 is stored in the hard disk 24 as the deceased information storage unit 240 as information representing the characteristics of the voice color of the deceased person.

(5) Voice conversion unit The voice conversion unit 204 executes a voice conversion process for converting (processing) the voice color of the operator's response voice in the backyard. The voice conversion unit 204 includes a sound source information extraction unit that extracts sound source information from the voice data of the response voice, and a voice synthesis unit that combines the voice color information related to the deceased with the sound source information.
(5.1) Sound Source Information Extraction Unit The sound source information extraction unit executes a sound source information extraction process that separates and extracts sound source information from the operator's voice data (response voice) input using the external microphone 31. The structure of this sound source information extraction unit is similar to that of the speaker information extraction unit 203 described above. The sound source information extraction unit has the same technical configuration as the speaker information extraction unit 203 in that the voice data is decomposed into sound source information and voice color information. The speaker information extraction unit 203 separates and extracts voice color information, whereas the sound source information extraction unit separates and extracts sound source information.
(5.2) Voice synthesis unit The voice synthesis unit executes a voice synthesis process for generating a new voice by combining the voice color information related to the deceased with the sound source information extracted by the sound source information extraction unit. The voice data synthesized by the voice synthesis unit is output from the built-in speaker 223.

(6) Voice recognition unit The voice recognition unit 205 executes a voice recognition process for specifying the meaning and content of the questioning voice of the attendees and the response voice of the operator. The voice recognition unit 205 of this example does not strictly specify the meaning and content of the voice, but specifies whether it is calm, humorous, serious, sad, or the like. .. In the voice recognition process of this example, for example, the number of phonemes per unit time (speech speed), which is the smallest phonological unit analyzed by phonology, the pitch of the voice, the volume of the voice, the presence or absence of laughter, etc. , The meaning and content as described above are specified.

The digital portrait 1 having the above configuration is used, for example, in a festival hall where the funeral of the deceased is held, as a ghost to remember the deceased, in addition to the ghost of the altar. The digital portrait 1 is attached to, for example, the wall surface of a space provided to remember the deceased, and attendees can interact with the deceased in the digital portrait 1. The voice of the digital portrait 1 is the voice of the operator 300 (see FIG. 2) in the backyard converted into the voice of the deceased. For attendees, it feels like they are interacting with the deceased in their lifetime, and it is possible to miss the deceased.

Next, (a) preparatory work and (b) operation for enabling such operation of the digital portrait 1 will be described in order.
(A) Preparatory work When operating the digital portrait 1, it is necessary to save the still image and audio data of the deceased in advance on the hard disk 24 as the deceased information storage unit 240. For example, if an external memory 39 such as a USB memory that stores the still image and audio data of the deceased is used, the data such as the still image can be easily transferred to the digital portrait 1. In order to extract highly accurate voice information, it is advisable to read aloud a predetermined pattern of conversational sentences and record them during the life of the deceased. It is advisable to set a conversation pattern that can extract voice color information with high accuracy in the conversation sentence of a predetermined pattern.

Further, the hard disk 24 as the deceased information storage unit 240 may store the above-mentioned (1.5) deceased information. The information of the deceased is useful as reference information when the operator responds to the question voice of the attendees. As described above, the information related to the deceased (still image, audio data, deceased information) stored in the deceased information storage unit 240 can be transferred to the digital portrait 1 using, for example, a USB memory. Instead of a USB memory or the like, an SD card with a built-in flash ROM may be used. Further, if the digital portrait 1 has a wireless communication function such as WiFi (registered trademark), data transfer may be performed wirelessly from another PC device. Further, if the digital portrait 1 can be connected to the Internet via WiFi (registered trademark) or the like, it is also possible to transfer data from another PC device or server device that can connect to the digital portrait 1 via the Internet. ..

In order to set the digital portrait 1 in an operable state, it is necessary to execute a predetermined standby process by the standby program stored in the hard disk 24. This standby process is a process including the above-mentioned feature point extraction process, three-dimensional model generation process, and speaker information extraction process. The contents of the standby process will be described with reference to FIGS. 5 and 6.

In the standby process, as shown in FIG. 5, the CPU 20 of the main board 2 first reads the image data of the still image of the deceased from the hard disk 24 as the deceased information storage unit 240 (S101). Then, a feature point extraction process for extracting the feature point FP from the still image of the deceased is executed (see S102, FIG. 4).

The CPU 20 of the main board 2 executes a process of transforming the standard three-dimensional model by the extracted feature point FP of the deceased, and generates a three-dimensional model of the deceased (S103, three-dimensional model generation process). The CPU 20 writes a three-dimensional model of the deceased on the hard disk 24 as the deceased information storage unit 240 (S104). This allows the CPU 20 to use the deceased's 3D model during the operation of Digital Portrait 1.

Further, in the standby process, as shown in FIG. 6, the CPU 20 of the main board 2 reads the voice data of the deceased from the hard disk 24 as the deceased information storage unit 240 (S201). Then, the CPU 20 executes a speaker information extraction process that separates and extracts voice color information from the voice data of the deceased person (S202). The CPU 20 writes the voice information of the deceased person to the hard disk 24 as the deceased person information storage unit 240 (S203). As a result, the CPU 20 can use the voice information of the deceased during the operation of the digital portrait 1.

(B) Operation Digital portrait 1 is operated by executing a predetermined operation program. During the operation of Digital Portrait 1, a moving image of the deceased is displayed on the display screen 210, and dialogue is possible. The deceased in Digital Portrait 1 blinks, occasionally tilts his head, and turns his face. In particular, the deceased in conversation performs various actions such as opening and closing his mouth according to the utterance and nodding lightly according to the content of the conversation.

Attendees facing the deceased in Digital Portrait 1 are imaged by the built-in camera 221 embedded in the frame 1F and displayed on the external monitor 33 in the backyard. Further, the voice that the attendee asks the deceased is converted into an electric signal by the built-in microphone 222 embedded in the frame 1F, and is output from the external speaker 32 in the backyard via the digital portrait 1.

As described above, in the backyard where the operator who responds to the attendees refrains, in addition to the external speaker 32, an external microphone 31 that converts the operator's voice into an electric signal, an external monitor 33 that displays the captured image of the attendees, etc. Is installed. The operator can appropriately respond to the question voice while visually recognizing the attendees displayed on the external monitor 33. The response voice by the operator is converted into the voice of the deceased by voice synthesis and output from the built-in speaker 223.

The operation flow of the digital portrait 1 as described above will be described with reference to the flow chart of FIG. When the operation of the digital portrait 1 is started, the CPU 20 of the main board 2 first reads the three-dimensional model and voice information of the deceased from the hard disk 24 as the deceased information storage unit 240 (S301).

Subsequently, the CPU 20 determines whether or not there is a voice of the attendee or a voice of the operator, that is, whether or not the voice is in the input state (S302). The voice includes a questioning voice of attendees, an operator's response voice to the questioning voice, an operator's greeting voice, and the like. The greeting voice is a reading voice of a fixed phrase or the like. As a fixed phrase, for example, there is a greeting such as "Thank you for attending my funeral today ....".

When any of the voices is present (S302: Yes), the CPU 20 executes the acquisition of the voice data of the voice (S303). The CPU 20 determines whether the voice source is an operator or an attendee (S304). When the operator in the backyard is the utterance source, the CPU 20 applies the above sound source information extraction process to the voice data to separate and extract the sound source information from the operator's voice (S304: YES → S305). Then, the CPU 20 executes voice synthesis that combines the voice color information of the deceased with the sound source information extracted from the operator's voice (S306). On the other hand, when the voice source is not the operator but the attendee (S304: NO), the CPU 20 bypasses the above-mentioned processes S305 and S306 for reproducing the voice of the deceased by voice synthesis.

The CPU 20 applies voice recognition processing to the voice captured in S303, and specifies the meaning and content of the voice, regardless of whether the voice is uttered by an operator or an attendee (S307). As described above, the voice recognition at this time does not strictly specify the meaning and content of the voice. For example, depending on the number of phonemes per unit time (speech speed), the pitch of the voice, the volume of the voice, the presence or absence of laughter, etc., the meaning of the voice is calm, humorous, serious, or includes sadness. It only specifies the meaning and content of, etc.

Subsequently, the CPU 20 appropriately deforms the three-dimensional model of the deceased read in S301 in order to give motion to the deceased displayed in the digital portrait 1 (S308). As described above, the deformation modes of the three-dimensional model include blinking eyes, opening and closing the mouth, nodding, changing the direction of the face, and changing the facial expression.

When neither the attendee's voice nor the operator's voice is input (S302: None), the deceased's movements are mainly blinking eyes and changing the direction of the face to look around. The movement of the deceased while inputting the questioning voice of the attendees (S302: Yes → S304: NO) is a nod according to the time interval or division of the questioning voice, and the deceased according to the meaning and content of the questioning voice. There is a change in facial expression. In addition, the movement of the deceased during the operator's utterance (S302: Yes → ... → S304: YES) includes opening and closing of the mouth according to the utterance of the reply voice, and a facial expression change according to the meaning and content of the reply voice. ..

For example, when opening and closing the mouth according to the utterance, for example, the deceased performs a transformation process for opening and closing the mouth according to the strength of the sound source data constituting the response voice and the type of sound source information (vowel or consonant, etc.). It is good to apply it to the 3D model of. The deformation process is a process of displacing a part or all of the vertices or intersections of the wire frame model forming the three-dimensional model. For example, when opening and closing the mouth, the CPU 20 realizes the opening and closing operation of the mouth by appropriately displacing the vertices or intersections of the wire frame model forming the contour of the mouth.

The CPU 20 converts the three-dimensional model of the deceased into a two-dimensional image viewed from the front at any time, and updates the image of the deceased on the display screen 210. As a result, the moving image of the deceased can be displayed in the digital portrait 1 (S309). When the operator is speaking, the CPU 20 synchronizes with the moving image of the deceased person and outputs the voice of the deceased person, which is voice-synthesized in the above S306, from the built-in speaker 223. The voice of the deceased includes a response voice to the question voice of the attendees and a reading voice of a standard greeting sentence.

The digital portrait 1 configured as described above is a highly functional remnant capable of displaying the deceased as a moving image using digital technology and interacting with the deceased. By utilizing this digital portrait 1, it is possible to remember and miss the deceased through dialogue with the deceased. During the dialogue with the deceased, the deceased may nod in response to a question from the attendees, or the deceased's facial expression may change depending on the content of the story. For the attendees on the questioning side, the deceased feels as if they are responding to their own questions, and the deceased can be felt closer to them.

In addition, attendees can receive a response in the voice of the deceased when asking the deceased. Furthermore, the deceased person of Digital Portrait 1 shows various facial expressions such as opening and closing his mouth when replying, and laughing or getting angry depending on the content of the reply. For attendees, it feels like the deceased responds to their questions in the same way as they did in their lifetime, and they can feel familiar with the deceased.

It is also possible to perform image processing on the image captured by the built-in camera 221 to detect attendees passing in front of the digital portrait 1 and attendees who stop. The deceased person in Digital Portrait 1 may be made to follow the passing attendees with his eyes, or the deceased person may be made to give a bow to the attendees passing in front of him.

As a function of the CPU 20 of the main board 2, it is also possible to add a function as a person identification unit that executes a person identification process for identifying attendees. For example, the person identification unit cuts out a face area from an image captured by an attendee by the built-in camera 221 and refers to a storage area of a hard disk 24 that stores a face image of a person who is related to the deceased as information on the deceased. , Identify attendees. When the attendees are specified, the CPU 20 of the main board 2 may read the information about the attendees from the hard disk 24 and present it to the operator. As a presentation method, it is also possible to provide a display window that occupies a part of the display screen of the external monitor 33 that displays the captured image of the attendees, and display information about the attendees. In this case, the backyard operator can visually recognize the attendees, grasp the information about the attendees, and appropriately respond to the questions of the attendees. If the relationship between the attendee and the deceased and the information of the attendee are known, the operator can accurately respond to the question voice. In addition, it is also possible to switch so that the display content of the display window is displayed larger on the display screen of the external monitor 33 according to the operation of the operator. When the attendees can be identified, it is also possible to provide a means for notifying the operator that there is information about the attendees. In this case, it is preferable to switch the display screen of the external monitor 33 to the display screen of the information about the attendees in response to the switching operation of the operator who requests the information of the attendees.

In this example, the voice data of the deceased is stored in the hard disk 24 or the like in advance, and the voice information of the deceased (an example of speaker information) is generated by executing the standby process and stored in the hard disk 24. Instead of this, when the voice data of the deceased is acquired, the voice color information may be immediately separated and extracted. Then, the voice information of the deceased may be stored in the hard disk 24 as the deceased information storage unit 240.

In this example, an example of collecting voice data by reading aloud a predetermined pattern of conversational sentences during the life of the deceased is described. Instead of this, it is also possible to cut out the voice data of the deceased from the video image or the like taken by the deceased in his lifetime and extract the voice color information. Alternatively, it is also possible to have relatives with similar voice qualities, such as siblings and parents and children, read aloud a predetermined pattern of conversational sentences to extract voice information. In the case of close relatives, the shape of the oral cavity and the shape of the skeleton, which affect the voice quality, are often similar, so it may be used as voice information for the deceased.

Further, in this example, an example is described in which a still image of the deceased is stored in the hard disk 24 or the like in advance, and a three-dimensional model of the deceased is generated and stored in the hard disk 24 by executing the standby process. Instead of this, it is also possible to generate a three-dimensional model of the deceased based on the still image in advance and save it on the hard disk 24.

In this example, the configuration in which the operator thinks and responds to the question voice by the attendee is illustrated. Further, as a function of assisting the operator, a function of displaying a captured image of an attendee on an external monitor 33, a function of displaying a text display such as information on attendees and information on a deceased person on an external monitor 33, and the like are exemplified. There is. Instead of this, a configuration may be adopted in which the CPU 20 of the main board 2 generates the response text and displays the text on the external monitor 33. As a configuration for the CPU 20 of the main board 2 to generate a response text, for example, there is a configuration in which the meaning and content of the question voice are specified by voice recognition and a response example is generated according to the recognition result. The meaning and content specified by voice recognition at this time is preferably at a level at which dialogue with the question voice is possible. When the meaning and content of the question voice can be specified to a level that allows dialogue, the CPU 20 of the main board 2 selectively selects the information corresponding to the meaning and content of the question voice from the information of the deceased stored in the hard disk 24. Can be obtained. If the information of the deceased corresponding to the meaning and content of the question voice can be obtained, the text of the response example reflecting the information can be generated. For each process such as specifying the meaning and content of the question voice (speech recognition), selectively acquiring the information of the deceased, and generating the text of the response example based on the information of the deceased, for example, artificial intelligence using machine learning. Approach can be applied. By utilizing artificial intelligence technology, these processes can be executed efficiently and with high accuracy.

It is also possible to cut out the phonemes necessary for voice synthesis from the voice data of the deceased and save them on the hard disk 24 as the phoneme data of the deceased. In this case, texts such as greetings prepared in advance and response texts generated by the CPU 20 of the main board 2 can be read aloud in the voice of the deceased by voice synthesis based on the phoneme data of the deceased. It is also possible to adopt a mode in which the operator types in the response text and the CPU 20 synthesizes the text by voice.

In this example, the digital portrait 1 in which the deceased responds to the question voice of the attendee is illustrated, but the deceased in the digital portrait 1 does not respond and does not output the voice such as just smiling. However, it may be a mode in which only a standard greeting is given. The action of making a smiling expression of the deceased person may be executed in response to the surrounding sound, may be executed at regular intervals, may be executed with the detection of a person passing in front of the person, and attendees. It may be executed with the input of the question voice of.

This example is an example of using the digital portrait 1 as a ghost. Digital portrait 1, which is an example of the video output system 1S, is a portrait of a celebrity such as an entertainer or a cultural person, a portrait of a company or store owner, or a portrait of a department store or a receptionist of a company. It can be widely used as such. For example, when it is used as a portrait of a receptionist of a company or the like, it is possible to set up a backyard in a remote place using the Internet so that one operator can concurrently perform reception work of a plurality of companies.

In this example, a configuration for generating a moving image according to the voice is illustrated, but for example, by specifying a facial expression pattern such as a laughing facial expression or a sad facial expression, the moving image can be generated with the specified facial expression pattern. Well, in this case, it may be a moving image with a facial expression specified according to the sound.
Further, in this example, for example, in application to an iei, a configuration in which a moving image is generated based on one still image that becomes an iei is illustrated. In this case, as the still image to be the deceased, a captured image of the deceased person shortly before his death is often selected so that he / she can feel familiarity with the deceased person. In addition to video generation based on the deceased's still image shortly before his death, for example, based on the deceased's teenage still image, 20's still image, 30's still image, etc. It is also good to generate a video of the deceased when he was young. In this case, it is possible to look back not only on the deceased but also on the life of the deceased by age group, and to remember the deceased in a memorable age. Such an operation is useful not only for the ghost but also for other portraits such as the celebrity mentioned above.

(Example 2)
This example is an example in which the speaking style of the deceased is included in the speaker information based on the digital portrait of Example 1. Among the configurations of this example, the configurations of the speaker information extraction unit that extracts speaker information from the voice data of the deceased and the voice conversion unit that converts the operator's voice are different from those of the first embodiment. The contents of this example will be described with reference to FIGS. 8 and 9. FIG. 8 is a flow chart showing a processing flow by the speaker information extraction unit. FIG. 9 is a flow chart showing a processing flow by the voice conversion unit.

In the first embodiment, voice information that can identify the voice of the deceased is illustrated as the speaker information of the deceased. On the other hand, this example is a configuration example in which, in addition to the voice information, the speaker information includes the speaking style information that can identify the speaking style of the deceased person. The speaking style information in this example is the type information of speaking styles such as Akita dialect, Kumamoto dialect, and Nagoya dialect, including standard language.

Next, a procedure for the speaker information extraction unit to extract speaker information will be described with reference to FIG. The speaker information extraction unit first reads the voice data of the deceased by reading aloud a conversational sentence having a predetermined pattern (S401). Then, by applying the speaker information extraction process to the voice data of the deceased, the voice color information and the speaking style information are extracted (S402).

The speaker information extraction unit decomposes the voice data into phonemes and performs voice recognition processing to convert the voice into text. Then, a phoneme string forming a specific word or a phoneme string forming a specific phrase is cut out, and the type of dialect is specified from the intonation. For example, regarding the phoneme string corresponding to the place name "Nagoya", the first letter "na" has an accent, or the third letter "ta" in the phoneme string of "You are what you want". Voice data such as the accent on the third character "to" from the end can be classified as Nagoya dialect.

Similar to the first embodiment, the speaker information extraction unit writes the voice color information and the speaking style information to the hard disk as speaker information (S403). For example, for the voice data classified into the Nagoya dialect, in addition to the voice color information, speaker information including the type information of the speaking style representing the Nagoya dialect is stored in the hard disk.

Next, a procedure in which the voice conversion unit of this example converts voice will be described with reference to FIG. Similar to the first embodiment, the voice conversion unit first captures the operator's voice (S501), separates and extracts the sound source information (S502). Then, a voice synthesis that combines the voice color information of the deceased is executed with respect to the extracted sound source information (S503), and a speaking style conversion process is executed (S504).

In the speech conversion process of S504, the operator's voice is decomposed into phonemes and voice recognition processing is performed to convert the voice into text. Then, when a specific word, a specific phrase, or the like appears, a predetermined pattern of intonation is assigned to the word. For example, for the phrase "you, what are you talking about", it is advisable to add an accent to the third letter "ta" and the third letter "te" from the end. Furthermore, if you extend the sound of "ta" to "ta" and change the pronunciation of "te" from (te) to (to), it becomes more like a Nagoya dialect. The above-mentioned specific word or phrase whose intonation is to be changed differs depending on the type of speaking style (type of dialect) represented by the speaking style information. This is because characteristic words and phrases differ depending on the type of speech.

The digital portrait (video output system) of this example converts the voice color of the operator's voice as well as the speaking style. According to this digital portrait, the reproducibility of the deceased in life can be improved by bringing the output voice closer to the voice of the deceased.

As speaker information, it is also possible to extract a characteristic intonation pattern of the deceased's speaking style from the voice data and save it on the hard disk as speaker information. Furthermore, it is advisable to associate text with the intonation pattern. The voice conversion unit may convert the operator's voice into the deceased's voice by voice synthesis based on the text obtained by processing the operator's voice. For the part of the text that does not correspond to the intonation pattern, it is advisable to assign an intonation close to the standard language. On the other hand, for the part corresponding to the text related to the intonation pattern, it is preferable to assign the intonation pattern that forms speaker information.
The other configurations and effects are the same as in Example 1.

(Example 3)
This example is a configuration example that does not require a backyard operator based on the digital portrait (video output system) of the first embodiment.
In the configuration of this example, the digital portrait hard disk has a function as a voice pattern storage unit for storing voice patterns. The hard disk as the voice pattern storage unit records voice patterns such as a pattern in which the deceased speaks spontaneously (spontaneous pattern) and a response pattern to a question. The digital portrait generates the voice of the deceased by combining (processing) the voice information of the deceased with the sound source information of the voice pattern. The voice pattern may be a voice recorded in advance by an announcer or the like, or may be sound source information separated and extracted from the voice.

Spontaneous patterns include, for example, greeting patterns such as "Good morning" and "Welcome to you." The voice of the deceased based on the spontaneous pattern may be output when a predetermined time is reached, or when someone crosses in front of the digital portrait. The greeting pattern when someone crosses in front should be changed according to the time of day, such as "Good morning" in the morning and "Hello" in the afternoon. The digital portrait generates and outputs the voice of the deceased by combining the voice information of the deceased with the sound source information of the selected spontaneous pattern.

As a response pattern, a pattern corresponding to a question and answer expected with attendees is prepared. The digital portrait identifies the meaning and content of the attendee's question voice (voice recognition processing) and selects the corresponding response pattern. Then, the response voice of the deceased in the response pattern is generated and output by combining the voice color information of the deceased with the sound source information of the response pattern.

In the digital portrait of this example, the operator can speak and talk with the voice of the deceased without inputting voice.
The spontaneous pattern and the response pattern may be text. The voice of the deceased can be generated by reading the text in the voice of the deceased using the voice information of the deceased.
The voice of the deceased related to the spontaneous pattern and the response pattern may be generated by voice synthesis or the like when speaking, but each pattern is converted into the voice of the deceased in advance and saved in the hard disk as the voice pattern storage unit. It is also good to keep it. In this case, when each pattern is needed, it can be read from the hard disk and output immediately. The conversion process to the voice of the deceased may be performed by the digital portrait main body, but the voice converted by another PC device may be transferred to the digital portrait. The voice transfer may be performed using a recording medium such as a USB memory, or may be transferred using wireless communication such as WiFi (registered trademark). The voice of the deceased may be transferred to a digital portrait from a remote server device or the like via the Internet.
The other configurations and effects are the same as in Example 1.

(Example 4)
This example is an example of the moving image output system 1S configured so that the digital portrait 1 displays the moving image generated by the separate PC device 100 based on the first embodiment. The contents of this example will be described with reference to FIG.
The digital portrait 1 of this example is a rental device provided and operated by a trader to a user. Once the user of Digital Portrait 1 sends or hands over the USB memory 391 or the like that stores the still image of the deceased to a trader, the USB memory 391 or the like that stores the moving image or the like generated by the separate PC device 100 is stored. Can be returned. The information exchanged using the USB memory 391 may include audio in addition to the moving image.

In using the digital portrait 1, the user only needs to attach the USB memory 391 returned from the vendor to the USB terminal of the digital portrait 1. The moving image or the like stored in the USB memory 391 is automatically read by the CPU of the digital portrait 1 as an output device, and is ready for output.

There are various motion patterns such as blinking eyes, opening and closing mouth patterns, smiling patterns, and nodding patterns as videos prepared in advance by the vendor. Digital portrait 1 displays the deceased as a moving image by combining operation patterns according to the contents of the question voice, the voice uttered by the deceased, and the surrounding situation.

This example is an example in which the moving image output system 1S is configured by combining the digital portrait 1 and the separate PC device 100. In the configuration of this example, the external PC device 100 is provided with functions such as a deceased modeling unit that generates a three-dimensional model from a still image of the deceased and a moving image generating unit that generates a moving image of the deceased. In the moving image output system 1S of this example, the processing load of the digital portrait 1 can be reduced, and the product cost can be reduced.

In the configuration of this example, both the digital portrait 1 and the external PC device 100 need to have a configuration for realizing data exchange. When data is exchanged using a recording medium such as a USB memory 391, both sides need a configuration for directly or indirectly reading the data from the recording medium such as a USB memory.

In this example, the USB memory 391 is illustrated as a recording medium for data such as still images and moving images, but the recording medium is not limited to the USB memory 391, and various recording media such as an SD card and a CD-R can be used. ..

For at least one of the data supply of still images and the like from the user to the trader and the data supply of the moving image and the like from the trader to the user, instead of using a recording medium such as a USB memory 391, It may be realized by the mode of communication using the Internet.

The digital portrait 1 may have a communication function such as WiFi or a wired LAN and may be connected to the Internet. In this case, when the USB memory 391 that stores the still image or audio data is attached to the USB terminal, the still image or the like is automatically read out, and the data such as the still image is sent to the dedicated site operated by the trader via the Internet. Should be sent. It is preferable to generate a moving image based on a still image on a dedicated site and reply to the digital portrait 1 of the sender. Digital portrait 1 can output video and audio received from a dedicated site. In this case, the digital portrait 1 can output the video and audio of the deceased only by attaching the USB memory 391 for storing the still image of the deceased and the audio data of the deceased to the USB terminal.

It is also possible to provide the user with application software that generates a moving image based on a still image, or a PC device on which the application software is installed, by selling, renting, or the like. In this case, the user of Digital Portrait 1 can generate a moving image or the like by using his / her own PC device or the provided PC device. The generated moving image can be output in digital portrait 1.

In this example, the USB memory 391 that stores various video patterns is returned to the user, but it is also possible to return the USB memory that stores the 3D model of the deceased required for video generation to the user. .. In this case, a moving image of the deceased can be generated by appropriately deforming the three-dimensional model on the digital portrait 1 side.
The other configurations and effects are the same as in Example 1.

(Example 5)
This example is an example of a moving image output system 1S that operates the digital portrait 1 via the Internet 101 based on the digital portrait of the first embodiment. This content will be described with reference to FIG.

In the video output system 1S of this example, the digital portrait 1 which is an output device for outputting the video and audio of the deceased and the server device 100 for generating the video and audio are installed in different places, and both are installed on the Internet 101. It is connected so that it can communicate via. The operator 300 can respond to the question voice of the attendees by using a PC device (not shown) communicatively connected to the server device 100.

The PC device of the operator 300 may be installed in a place different from that of the server device 100. If the Internet 101 is used, data communication between the server device 100 and the PC device becomes possible. It is also possible to omit the PC device of the operator 300 and provide the server device 100 with an external microphone 31, an external monitor 33, an external speaker 32, or the like.

In the moving image output system 1S, a dedicated site managed by the server device 100 is registered in the digital portrait 1. Digital portrait 1 is provided with an access means for automatically accessing the above-mentioned dedicated site when a still image of the deceased, voice data of the deceased, information of the deceased, or the like is read from a recording medium such as a USB memory 391. There is. The still image of the deceased, the voice data of the deceased, the information of the deceased, etc. saved in the recording medium such as the USB memory 391 are uploaded to the server device 100 in response to the automatic access to the dedicated site managed by the server device 100. Will be done.

When the attendee asks the deceased person of Digital Portrait 1, the questioning voice, the captured image of the attendee, and the like are uploaded to the server device 100 and transferred to the PC device of the operator 300. Further, the operator's voice is transmitted to the server device 100, converted into the voice of the deceased person, and then transmitted to the digital portrait 1. The digital portrait 1 outputs the audio and the moving image received from the server device 100 in this way. The digital portrait 1 in the moving image output system 1S of this example is a terminal that only uploads the questioning voice of the attendees and the captured image, and outputs the moving image received via the Internet 101, the voice of the deceased, and the like.

The server device 100 has functions as a deceased information storage unit, a deceased modeling unit, a moving image generation unit, a speaker information extraction unit, a voice conversion unit, and a voice recognition unit. The server device 100 executes processing such as specifying the meaning and content of the questioning voice of the attendees, generating a three-dimensional model of the deceased, generating a moving image of the deceased, and converting the response voice of the operator. In particular, when generating a moving image of the deceased, the moving image of the deceased is generated so that an operation or a facial expression according to the meaning and content of the voice can be realized.

In the configuration of this example, for example, when a recording medium such as a USB memory 391 that stores a still image of the deceased or audio or information is attached to the digital portrait 1, the still image of the deceased is transmitted to the server device via the Internet. It is good. In this configuration, a USB terminal, a CPU that reads data from a USB memory, and the like constitute a still image acquisition unit for acquiring a still image of the deceased. The still image or the like acquired by the still image acquisition unit may be uploaded to the server device 100 as the moving image generation unit in association with the identification information of the digital portrait 1. On the server device 100 side, it is preferable to associate the moving image generated by the moving image generation unit with the identification information related to the still image which is the original data. If the identification information of the digital portrait 1 which is the output device is associated with the moving image, the moving image can be reliably transmitted to the corresponding digital portrait 1.

Instead of the configuration in which the still image of the deceased is automatically or semi-automatically uploaded from the digital portrait 1 to the server device 100, the dedicated site managed by the server device is accessed by the operation of the digital portrait 1. A configuration for uploading a still image or audio of the deceased may be adopted. When uploading a still image of the deceased, it is preferable to associate the identification information of the corresponding digital portrait 1.
The other configurations and effects are the same as in Example 1.

(Example 6)
This example is a configuration example in which the method of generating a moving image is changed based on the first embodiment. This content will be described with reference to FIG.

An external camera that captures an operator in the backyard is connected to the digital portrait that forms the moving image output system of this example. The digital portrait has a configuration in which a moving image of the deceased is generated by using an image captured by the operator.

The operation of generating a moving image by the digital portrait of this example will be described with reference to the flow chart of FIG. The figure is a flow chart for explaining the flow of processing on the premise that the three-dimensional data of the deceased has been read. Here, the content of the processing will be described mainly by the CPU (reference numeral 20 in FIG. 3) of the main board included in the digital portrait.

The CPU of the main board captures the captured image of the operator in the backyard (S601) and cuts out the face region from the captured image (S602). Then, the CPU extracts the feature points of the operator's face (S603) and executes the association with the feature points of the deceased person (S604). The feature points of the corresponding parts are associated with each other, for example, the association between the operator's mouth and the deceased's mouth, the association between the operator's eyes and the deceased's eyes, and the like.

In the state where the feature points are associated with each other, the CPU detects the movement of the feature points in the captured image (moving image) of the operator (S605). Then, the CPU deforms the three-dimensional model of the deceased so that the same movement occurs at each feature point related to the deceased based on the information of the movement of each feature point related to the operator (S606). Then, a moving image based on this three-dimensional model is displayed (S607).

According to the configuration of this example, it is possible to make the deceased person in the digital portrait perform the same facial expressions and movements as the operator. For example, if the operator smiles during a conversation with an attendee, the deceased in the digital portrait will also smile. Also, for example, if the operator nods during a conversation with the attendees, the deceased in the digital portrait will nod as well.
The other configurations and effects are the same as in Example 1.

(Example 7)
This example is a configuration example that enables stereoscopic display of a moving image based on the digital portrait of the first embodiment. This content will be described with reference to FIGS. 13 to 16.
This example is an example of a digital portrait in which a three-dimensional display unit 5 for three-dimensionally displaying the deceased is incorporated.
The three-dimensional display unit 5 (FIG. 13) is configured to display the deceased 511 three-dimensionally by using the half mirror 50. Further, in the three-dimensional display unit 5, the image 531 is displayed so as to be superimposed on the deceased 511 that is three-dimensionally displayed.

The three-dimensional display unit 5 has a space in which the half mirror 50 is diagonally arranged inside the transparent glass panel 500 arranged so as to face the spectators such as attendees. A liquid crystal display 51 is arranged on the bottom surface of this space so as to face the half mirror 50 at an angle. The viewer's line of sight is bent by the half mirror 50 toward the liquid crystal display 51.

Various images 531 are drawn on the wall surface 53 on the back side of the space facing the viewer through the half mirror 50. In this example, statues 531 of lotus, birds, etc., which are often drawn on the background of the Buddha, are drawn on the wall surface 53. The viewer can see the image 531 through the half mirror 50.

In the stereoscopic display unit 5 of this example, a part of the line of sight of the viewer looking inside the glass panel 500 is bent by the half mirror 50 and heads toward the deceased 511 of the liquid crystal display 51 arranged along the bottom surface of the space. .. On the other hand, a part of the line of sight of the viewer passes through the half mirror 50 and heads toward the wall surface 53. At this time, the stereoscopic image 511A and the image 531 of the deceased are visually recognized by the viewer facing the glass panel 500 in an overlapping manner (FIG. 14).

By adjusting the position of the wall surface 53 in the depth direction, it is possible to give a sense of depth to the image visually recognized by the viewer as shown in FIG. For example, if the wall surface 53 is moved to the back side, the viewer can feel that the deceased 511 is protruding from the image 531.
Instead of the liquid crystal display 51 arranged on the bottom surface of the space, a combination of a liquid crystal projector and a screen for projecting an image can be adopted. In this case, the screen may be arranged instead of the liquid crystal display 51, and the liquid crystal projector may be arranged on the ceiling of the space facing the screen.

In this example, the wall surface 53 on which the image 531 is drawn is illustrated, but instead, a liquid crystal display may be provided on the wall surface. If a liquid crystal display is provided on the wall surface, it is possible to change and display the background image of the deceased stereoscopic image 511A.

Instead of this example, the stereoscopic display unit 5 of FIG. 15 may be adopted. In the three-dimensional display unit 5 of the figure, the oblique angle of the half mirror 50 is changed, and a liquid crystal display 51 displaying the deceased 511 is arranged along the ceiling of the space. Further, a display panel 53 is arranged on the wall surface facing the viewer. The display panel 53 in the figure is a light source panel in which LEDs serving as a light source are studded. In this display panel 53, some or all of the LEDs can be selectively turned on.

Further, in the stereoscopic display unit 5 of FIG. 15, the space in which the half mirror 50 is arranged is extended in the depth direction. In the stereoscopic display unit 5, an LED serving as a light source is located on the back side of the stereoscopic image 511A of the deceased 511. Therefore, in the stereoscopic display unit 5, the stereoscopic image 511A of the deceased 511 is imaged on the front side of the display panel 53.

In the stereoscopic display unit 5 of FIG. 15, the viewer can visually recognize the stereoscopic image 511A of the deceased 511 in a state where the LED 538, which is a light source, is located on the back side (FIG. 16). The LED 538 that lights up at a position farther than the stereoscopic image 511A when viewed from the viewer side has the effect of giving the stereoscopic image 511A a further sense of depth. For example, it is also possible to switch the lit LEDs so that the size of the polygons gradually decreases while maintaining the similar figures of the polygons connecting the LEDs 538 of FIG. In this case, the viewer can feel that the stereoscopic image 511A protrudes toward the front side as the background moves away. A liquid crystal display may be used instead of the display panel 53 studded with LED 538.
The other configurations and effects are the same as in Example 1.

(Example 8)
This example is an example in which a moving image of the whole body of the deceased can be displayed three-dimensionally based on the digital portrait of Example 7. This content will be described with reference to FIGS. 17 and 18. The digital portrait 1 of this example can display the deceased as a whole body as shown in FIG. 17, and can also display the deceased in three dimensions.

The structure of the three-dimensional display unit 5 included in the digital portrait 1 will be described with reference to the cross-sectional view of FIG. 18 showing the structure of the cross-sectional view taken along the line AA in FIG. The three-dimensional display unit 5 has a space in which the half mirror 50 is diagonally arranged inside the transparent glass panel 500 arranged so as to face the spectators such as attendees. On the side surface of this space, a liquid crystal display 51 for displaying a moving image of the whole body of the deceased 511 is arranged so as to face the half mirror 50 diagonally.

According to the stereoscopic display unit 5, the stereoscopic image 511A formed on the back side of the half mirror 50 can be visually recognized by the viewer. This digital portrait 1 is suitable for installation at the entrance of a funeral hall or the like, for example.

Instead of this example, it is also possible to adopt the digital portrait 1 of FIG. 19 including the transparent screen 52 and the projector 521 that projects an image onto the transparent screen 52 obliquely upward from below. In this digital portrait 1, an image of the deceased 521A can be formed on the transparent screen 52. For the viewer, the image of the deceased 521A can be visually recognized as a three-dimensional image floating in space.
The other configurations and effects are the same as in Example 7.

Although specific examples of the present invention have been described in detail as in the examples, these specific examples merely disclose an example of the technology included in the claims. Needless to say, the scope of claims should not be construed in a limited manner depending on the composition of specific examples, numerical values, and the like. The scope of claims includes technologies that are variously modified, modified, or appropriately combined with the above-mentioned specific examples by utilizing known technologies, knowledge of those skilled in the art, and the like.

1 Digital portrait (output device)
1S video output system 13 housing 133 USB terminal 201 deceased modeling unit 202 video generation unit 203 speaker information extraction unit 204 voice conversion unit 205 voice recognition unit 21 liquid crystal display 210 display screen 221 built-in camera 222 built-in microphone 223 built-in speaker 24 hard disk (HD)
240 Deceased information storage unit 300 Operator 31 External microphone (sound collecting microphone)
32 External speaker 33 External monitor 39 External memory 391 USB memory 5 Stereoscopic display 50 Half mirror 511A, 521A Stereoscopic image

Claims (4)

A voice conversion unit that processes the operator's voice input via a sound source sound collecting microphone that converts sound into an electric signal and converts it into a voice in which at least one of the voice color and the way of speaking is different.
A feature point extraction unit that actual face of a person extracts a feature point of a face definitive for still images that are captured,
By displacement of the feature points extracted by the feature point extraction unit , changes in facial expressions, blinking of eyes, movement of mouth, and changes in face orientation synchronized with the voice converted by the voice conversion unit can be performed. A video generator that generates a video that includes at least one of them,
An information storage unit that stores personal information indicating a hobby, favorite food, friendship, or relative relationship of a person whose video is to be generated by the video generation unit.
Includes a moving image generated by the moving image generation unit and an output device for outputting the sound converted by the audio conversion unit.
When the questioning voice is input via the questioning sound collecting microphone that converts the sound into an electric signal , the moving image generation unit operates a person with a still image in synchronization with the voice converted by the voice conversion unit. first moving picture and, producible der and second video, the operating static image of a person without the sound to be converted by the voice conversion unit which is,
Said output device, said to the operator for inputting a voice to be converted in the sound conversion unit, video output system has been configured to present the personal information as reference information when replying in response to Question seat .. The moving image output system according to claim 1, wherein the moving image generation unit can generate the second moving image, which is a moving image in which a person in a still image operates even when the question voice is not input. Including a voice recognition unit that performs voice recognition
The moving image output according to claim 1 or 2, wherein the second moving image when the questioning voice is input is a moving image in which a person in a still image operates in response to the recognition of the questioning voice specified by the voice recognition unit. system. The moving image generation unit is configured to generate a moving image in which a person with a still image operates in synchronization with the sound output by the output device with a facial expression pattern specified from a plurality of facial expression patterns provided in advance. The moving image output system according to any one of claims 1 to 3.

Images