JP7706092B2 - Terminal device control program, terminal device, terminal device control method, server device control program, server device, and server device control method - Google Patents

Description

The embodiments disclosed in this specification relate to a control program for a terminal device, a terminal device, a control method for a terminal device, a control program for a server device, a server device, and a control method for a server device.

Conventionally, there has been known an information system in which multiple users communicate with each other by using a terminal device such as a PC (Personal Computer) to transmit various information such as images showing each user and/or the voice of each user to the terminal device of other users. For example, Patent Literature 1 discloses a technology for an information system in which each user uses a character object such as an avatar that acts as the user's alter-ego in a virtual space to conduct voice chat with character objects of other users.

JP 2009-199198 A

However, in conventional information systems, there was a lack of diversity in the output from the user's terminal device when the user was communicating with other users, which led to the problem that the user would become bored with communicating with other users.

The disclosed terminal device control program, terminal device, terminal device control method, server device control program, server device, and server device control method make it possible to increase a user's motivation to continue communicating with other users.

The control program of the terminal device is a control program of the terminal device of a first user, and causes the terminal device of the first user to at least display a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user, transmit information related to the display of the first image to the terminal device of the second user, transmit the first voice to the terminal device of the second user when the first voice of the first user is acquired, and display a first image including a first object that changes in response to the action or first voice of the first user, output the second voice when the second voice of the second user is received, and display a second image including a second object that changes in response to the action or second voice of the second user, and display a first image including a first object at least a part of which has been changed based on at least one third object selected from among a plurality of third objects in response to an instruction from the first user.

In addition, in the control program of the terminal device, when two or more third objects are selected in response to an instruction from the first user, it is preferable to execute the following to display on the terminal device of the first user a plurality of first selection objects for the first user to select a third object to be used to change the first object from among the two or more third objects, and a second selection object for collectively selecting all of the two or more third objects to be used to change the first object.

In addition, it is preferable that the control program of the terminal device causes the terminal device of the first user to receive information about at least one fourth object selected from a plurality of fourth objects in response to an instruction from a second user, and when an instruction is given to apply the fourth object selected by the second user, to display a second image including a second object at least a portion of which has been changed based on the information about the at least one fourth object, and to display a first image including a first object at least a portion of which has been changed based on the information about the at least one fourth object in response to an instruction from the first user.

In addition, it is preferable that the control program of the terminal device causes the terminal device of the first user to receive information about at least one fourth object selected from among the plurality of fourth objects in response to an instruction from the second user, and when an instruction is given to apply the fourth object selected by the second user, to display a second image including a second object at least a portion of which has been changed based on the information about the at least one fourth object, and to select at least one fourth object from among the plurality of fourth objects in response to an instruction from the first user, and to display a first image including a first object at least a portion of which has been changed based on the selected at least one fourth object.

In addition, it is preferable that the control program for the terminal device causes the terminal device of the first user to receive information on at least one fourth object selected from among the plurality of fourth objects in response to an instruction from the second user, display a second image including a second object at least a portion of which has been changed based on the information on the at least one fourth object in response to the instruction from the second user, select at least one fourth object from among the plurality of fourth objects in response to an instruction from the first user for a second object included in the displayed second image, and display a first image including a first object at least a portion of which has been changed based on the selected at least one fourth object.

In addition, the control program of the terminal device causes the terminal device of the first user to execute, when a predetermined first condition is satisfied, a process of returning a first object at least partially changed based on at least one fourth object to the first object before the change and displaying a first image including the first object returned to the state before the change, and it is preferable that during the period from when the first object is changed based on at least one fourth object to when it is returned to the first object before the change, the first user is able to select at least one fourth object from among the multiple fourth objects.

In addition, in the control program of the terminal device, the predetermined first condition preferably includes: obtaining a specific instruction from the first user; completing transmission of information related to the display of the first image and the first sound and receiving the second sound; completing transmission of information related to the display of the second image and the second sound and receiving the first sound; a predetermined time has elapsed since at least a portion of the first object has been changed based on at least one fourth object; or, when at least a portion of the first object is being changed based on at least one fourth object, the first user has instructed in response to an instruction to select at least one third object from among the multiple third objects; or, when at least a portion of the first object is being changed based on at least one fourth object, the second user has instructed in response to an instruction to select at least one third object from among the multiple third objects.

In addition, in the control program of the terminal device, it is preferable that the display of the first image including the first object at least partially modified based on at least one fourth object is executed when a predetermined second condition is satisfied.

In addition, in the control program of the terminal device, it is preferable that the predetermined second condition includes that the number of times a first image including a first object at least partially modified based on at least one fourth object is displayed is equal to or less than a predetermined number of times.

In addition, it is preferable that the control program of the terminal device causes the terminal device of the first user to execute the following: when at least one fourth object is selected from among the plurality of fourth objects in response to the operation of the first instruction object by the first user, the first benefit is granted to the second user.

In addition, it is preferable that the control program of the terminal device causes the terminal device of the first user to execute the following: when at least one fourth object is selected from among the plurality of fourth objects in response to the operation of the first instruction object by the first user, a second benefit is granted to the first user.

In addition, it is preferable that the control program of the terminal device executes the following on the terminal device of the first user: if the first instruction object is not operated by the first user and at least one fourth object is not selected from the plurality of fourth objects, the terminal device of the first user is caused to grant a third benefit to the first user.

In addition, it is preferable that the control program of the terminal device causes the terminal device of the first user to, when a predetermined second condition is satisfied, display a second instruction object for selecting at least one fifth object from among the plurality of fifth objects and information relating to the selection, select at least one fifth object from among the plurality of fifth objects in response to an operation of the second instruction object by the first user, and display a first image including a first object at least a portion of which has been changed based on the selected at least one fifth object.

In addition, in the control program of the terminal device, it is preferable that the predetermined second condition is that a predetermined time has elapsed since the first object was changed based on at least one third object.

Furthermore, in the control program of the terminal device, it is preferable that the plurality of fifth objects include an object that is applied to and displayed as a second object representing the second user.

The control program of the server device is a control program of the server device capable of communicating with each of a terminal device of a first user and a terminal device of a second user different from the first user, and receives information regarding the display of a first image including a first object representing the first user from the terminal device of the first user, receives information regarding the display of a second image including a second object representing the second user from the terminal device of the second user, transmits information for at least displaying the first image to at least the terminal device of the second user, transmits information for at least displaying the second image to at least the terminal device of the first user, and when a first voice of the first user is received, transmits the first voice to the terminal device of the second user, and outputs a command to the terminal device of the second user to notify the user of the first user of the action or the first voice of the first user. The method executes the following: transmitting information for displaying a first image including a first object that changes in response to voice to at least a terminal device of a second user; when a second voice of the second user is received, transmitting the second voice to the terminal device of a first user; transmitting information for displaying a second image including a second object that changes in response to an action or the second voice of the second user to at least a terminal device of the first user; selecting at least one third object from among a plurality of third objects in response to an instruction from the first user; and transmitting information for displaying a first image including a first object at least a portion of which has been changed based on the selected at least one third object to at least a terminal device of the second user.

The terminal device is a terminal device of a first user, and includes a processing unit that displays at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user, transmits information related to the display of the first image to the terminal device of the second user, transmits the first voice to the terminal device of the second user when the first voice of the first user is acquired, displays a first image including a first object that changes in response to the action or first voice of the first user, outputs the second voice when the second voice of the second user is received, displays a second image including a second object that changes in response to the action or second voice of the second user, and displays a first image including a first object at least a portion of which has been changed based on at least one third object selected from a plurality of third objects in response to an instruction from the first user.

The server device is a server device capable of communicating with each of a terminal device of a first user and a terminal device of a second user different from the first user, and receives information regarding the display of a first image including a first object representing the first user from the terminal device of the first user, receives information regarding the display of a second image including a second object representing the second user from the terminal device of the second user, transmits information for at least displaying the first image to at least the terminal device of the second user, transmits information for at least displaying the second image to at least the terminal device of the first user, and when a first voice of the first user is received, transmits the first voice to the terminal device of the second user and converts the first voice according to the action or the first voice of the first user. The processing unit transmits information for displaying a first image including a first object that changes in response to the action or second voice of the second user to at least the terminal device of the second user, transmits the second voice to the terminal device of the first user when the second voice of the second user is received, transmits information for displaying a second image including a second object that changes in response to the action or second voice of the second user to at least the terminal device of the first user, selects at least one third object from among a plurality of third objects in response to an instruction from the first user, and transmits information for displaying a first image including a first object at least a portion of which has been changed based on the selected at least one third object to at least the terminal device of the second user.

The control method of the terminal device is a control method of the terminal device of a first user, and includes the steps of: the terminal device of the first user displays at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user; transmitting information related to the display of the first image to the terminal device of the second user; when acquiring a first voice of the first user, transmitting the first voice to the terminal device of the second user and displaying a first image including a first object that changes in response to the action or the first voice of the first user; when receiving a second voice of the second user, outputting the second voice and displaying a second image including a second object that changes in response to the action or the second voice of the second user; and displaying a first image including a first object at least a part of which has been changed based on at least one third object selected from among a plurality of third objects in response to an instruction by the first user.

The control method of the server device is a control method of a server device capable of communicating with each of a terminal device of a first user and a terminal device of a second user different from the first user, and includes receiving information regarding the display of a first image including a first object representing the first user from the terminal device of the first user, receiving information regarding the display of a second image including a second object representing the second user from the terminal device of the second user, transmitting information for at least displaying the first image to at least the terminal device of the second user, transmitting information for at least displaying the second image to at least the terminal device of the first user, and when a first voice of the first user is received, transmitting the first voice to the terminal device of the second user, and transmitting an action or a first sound of the first user. transmitting, to at least the terminal device of the second user, information for displaying a first image including a first object that changes in response to the voice of the second user; when a second voice of the second user is received, transmitting the second voice to the terminal device of the first user; transmitting, to at least the terminal device of the first user, information for displaying a second image including a second object that changes in response to the action or second voice of the second user; selecting at least one third object from among a plurality of third objects in response to an instruction from the first user; and transmitting, to at least the terminal device of the second user, information for displaying a first image including a first object at least a portion of which has been changed based on the selected at least one third object.

The control program for the terminal device, the terminal device, the control method for the terminal device, the control program for the server device, the server device, and the control method for the server device make it possible to increase a user's motivation to continue communicating with other users.

FIG. 1 is a schematic diagram for explaining an example of an overview of an information system. FIG. 1 is a diagram illustrating an example of a schematic configuration of an information system. FIG. 2 is a diagram illustrating an example of a schematic configuration of a terminal device. 3A to 3C are diagrams illustrating examples of data structures of various tables. FIG. 2 illustrates an example of a schematic configuration of a server device. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 11 is a diagram illustrating an example of an operation sequence of the information system. FIG. 13 is a diagram showing an example of a screen displayed on a display device of the terminal device. FIG. 2 is a schematic diagram for explaining a virtual space.

Various embodiments of the present invention will be described below with reference to the drawings. However, please note that the technical scope of the present invention is not limited to these embodiments, but extends to the inventions described in the claims and their equivalents.

(Outline of the information system)
1 is a schematic diagram for explaining an example of an overview of an information system. The information system includes a terminal device operated by each of a plurality of users and a server device. For example, the terminal device is an information processing device such as a multi-function mobile phone (a so-called "smartphone") owned by the user. For example, the server device is a computer for providing a communication service between the terminal devices via a communication network.

The terminal device stores a control program such as an application program, loads the control program into memory in response to a start operation from the user, and starts a communication service by executing instructions contained in the loaded control program. When the communication service starts, the terminal device realizes multiple functions by executing instructions contained in the control program.

For example, the user's terminal device realizes a generation function that generates output information including character video data including motion data based on various input data entered by the user. For example, the input data is a plurality of pieces of imaging data acquired at predetermined sampling time intervals by an imaging device provided in the user's terminal device, and in this case, the user is depicted in each piece of imaging data. The character video data is an example of information related to the display of a user output image including a character object representing the user. During the communication service, the output information is generated at predetermined time intervals.

The character object is, for example, a three-dimensional object generated based on three-dimensional model data having the appearance of a humanoid animation character. The character object may also be a three-dimensional object generated based on three-dimensional model data having the appearance of various shapes, such as an animal such as a dog or a cat, or an artificial object such as a robot. In this case, the user output image includes an image of the character object seen from a predetermined viewpoint. The character object may be a two-dimensional object generated based on two-dimensional model data. Furthermore, image information showing an animal such as a dog or a cat, or an artificial object such as a robot, is applied to the surface of the character object, thereby expressing the appearance of an animal such as a dog or a cat, or an artificial object such as a robot. Hereinafter, three-dimensional objects and two-dimensional objects may be collectively referred to simply as objects.

When a character object has the appearance of a humanoid animation character, the character object is composed of a plurality of partial objects. For example, the character object is composed of a partial object corresponding to the head, a partial object corresponding to the torso, a partial object corresponding to the arms, a partial object corresponding to the legs, and the like. The partial objects may also correspond to each part of the face (eyes, mouth, flowers, eyebrows, beard, hair, and the like). Furthermore, a plurality of types of partial objects may be prepared for the same part (head, torso, and the like). For example, as partial objects corresponding to the hair part, partial objects corresponding to black hair and partial objects corresponding to brown hair may be prepared, and as partial objects corresponding to the eyes part, partial objects corresponding to black eyes with droopy eyes, partial objects corresponding to blue eyes with droopy eyes, partial objects corresponding to black eyes with slanted eyes, and partial objects corresponding to blue eyes with slanted eyes may be prepared. Furthermore, the character object may be composed of a partial object corresponding to a hat that covers at least a part of the head, partial objects corresponding to clothing that covers at least a part of the torso and arms, and the like. In this way, a character object is an object composed of multiple partial objects, and a partial object is an object that is a part of the constituent elements of a character object. Furthermore, the character object and partial objects may be configured so that their sizes can be changed. For example, when a user instructs a change in height, the sizes of various partial objects such as the head, torso, legs, and arms are changed to match the changed height. Furthermore, for example, when a user instructs a change in the size of a partial object, the size of the corresponding partial object may be changed (for example, when a user instructs a change in leg length, the length of the legs may be changed).

In response to an instruction from the user, one partial object constituting a character object can be changed to a new partial object of the same type as the partial object at that position. For example, by changing a partial object corresponding to long pants constituting a character object to a new partial object corresponding to shorts, the character object's appearance is changed from one in which the character object is wearing long pants to one in which the character object is wearing shorts. In addition, in response to an instruction from the user, a new partial object can be set as a partial object constituting a character object. For example, by applying a new partial object corresponding to a hat to a character object not wearing a hat, the character object's appearance is changed from one in which the character object is not wearing anything on its head to one in which the character object is wearing a hat. In this way, by applying a new partial object to a character object, the appearance of the new partial object of the character object is changed. Note that an additional object that can be attached to a character object, which will be described later, is a type of partial object.

For example, the user's terminal device displays a user output image based on character video data, and transmits the generated output information to the terminal devices of other users via the server device, thereby realizing an output function of transmitting the output information to the server device at predetermined time intervals. Also, for example, the user's terminal device realizes a function of displaying other user output images including character objects representing other users based on the output information of other users transmitted from the server device at predetermined time intervals.

When a communication service is started, as shown in FIG. 1(a), for example, a user output image (1) and another user output image (2) are displayed by the user's terminal device. If another user is not participating in the communication service when the communication service is started, only the user output image (1) is displayed by the user's terminal device. If a user is not participating in the communication service when the communication service is started, only the other user output image (2) is displayed by the other user's terminal device.

The user output image (1) includes a character object of the user that moves according to the user's motion data included in the character video data. The other user output image (2) includes a character object of another user that moves according to the motion data of the other user included in the received output information of the other user. In this way, the user's terminal device displays the user output image (1) that includes the user's character object that changes according to the user's movements, and displays the other user output image (2) that includes the character object of the other user that changes according to the movements of the other user.

Because both the user's terminal device and the other user's terminal device are equipped with microphones, the user's terminal device acquires the voice uttered by the user, and the other user's terminal device acquires the voice uttered by the other user. The user's terminal device then includes the acquired user's voice data in output information, and transmits the output information including the user's voice data to the other user's terminal device via the server device, thereby transmitting the output information. The user's terminal device also receives from the server device the output information including the other user's voice data transmitted to the server device by the other user's terminal device.

For example, when a user produces a sound, the user's motion data included in the character video data includes face motion data showing the movement of the user's lips. In this case, a user output image (1) including a character object of the user whose lips move in approximate synchronization with the sound produced by the user is displayed on the user's terminal device. The output information of other users received by the user's terminal device may include the voice data of other users along with the motion data of other users. In this case, the voice of the other users is output on the user's terminal device, and another user output image (2) including a character object of the other users whose lips move in approximate synchronization with the voice of the other users is displayed.

In addition to the above-mentioned functions, the information system may realize a function of providing a selection event for selecting information on at least one specific object from among a plurality of specific objects in response to an instruction from a user. In addition, the user's terminal device may realize a function of displaying a user output image (1) including the user's character object, at least a part of which has been changed based on at least one selected specific object, when the specific object can be used as a partial object of a character object. Any object may be selected in the selection event as long as it changes the display mode of the user's character object and/or changes the audio data to be output. For example, in the selection event, a background image that becomes the background of the character object in the user output image may be selected. In addition, in the selection event, an image or partial object for changing the facial expression of the character object in the user output image may be selected. For example, when the image or partial object for changing the facial expression is used (applied) by the user, at least a part of the face of the animation of the character object, which is generated so that the facial expression of the character object changes based on the user's facial motion data, is changed to an image or partial object for changing the facial expression. Since the image or partial object does not change based on the facial motion data, when this image or partial object is used, the facial expression is changed to a different expression from the user's facial motion data. In addition, in the selection event, auto-motion data for automatically moving the character object (waving, making a peace sign, etc.) may be selected. In addition, in the selection event, sound source data for outputting a predetermined sound effect or music may be selected. In addition, in the selection event, rights data indicating the right to execute a voice conversion process for changing a voice element such as the pitch and/or formant of the voice data when input voice data is output may be selected. When the rights data related to the voice conversion process is used (applied) by the user, the voice data input by the user is changed by the voice conversion process, and output information including the changed voice data is transmitted. In addition, in the selection event, a character object may be selected. For example, when a character object is selected in the selection event, the current user character object may be changed to the selected character object temporarily (for example, for one minute) or until the end of the communication service. In addition, rights data indicating the right to implement a predetermined event may be selected in the selection event. For example, the rights data is rights data indicating the right to implement a specific selection event, etc.

For example, when a command to display a selection instruction screen is input by a user during a communication service, the user's terminal device displays the selection instruction screen (FIG. 1(b)). The selection instruction screen includes, for example, a selection instruction button. When a selection instruction button included in the displayed selection instruction screen is selected in response to a user's operation, the user's terminal device transmits information indicating a selection instruction to the server device. The server device stores information regarding a plurality of specific objects. The information regarding the specific objects includes, for example, identification information (object ID (identification)) for identifying the specific objects. The information regarding the specific objects may include the name, attributes, etc. of the specific objects.

When the server device receives information indicating a selection instruction from the user's terminal device, it extracts information related to the stored multiple specific objects. The server device selects information related to at least one specific object from the extracted information related to the multiple specific objects according to a predetermined selection method. The predetermined selection method is, for example, a method of selecting information related to at least one specific object from the information related to the multiple specific objects based on a predetermined selection probability associated with the information related to each specific object. The predetermined selection probability is an example of a selection probability for selecting information related to a specific object. For example, when information related to each specific object is selected with equal probability from information related to the multiple specific objects, the same selection probability is associated with each piece of information related to the multiple specific objects. Note that the predetermined selection method may be a method of selecting information related to at least one specific object from information related to the multiple specific objects according to the order associated with the information related to each specific object.

The server device transmits information about the selected at least one specific object to the user's terminal device (the terminal device that transmitted the information indicating the selection instruction). The user's terminal device displays a selection result screen based on the received information about the at least one specific object (FIG. 1(c)). The selection result screen includes, for example, information about the at least one specific object and a use button. When a use button included in the displayed selection result screen is selected in response to a user's operation, a use instruction is input to the user's terminal device. The user's terminal device displays a user output image (1) including the user's character object, at least a portion of which has been changed based on the information about the at least one specific object in response to the use instruction (FIG. 1(d)).

For example, the information about the at least one specific object is an object ID for identifying an attachment object that changes the appearance of a portion of the user's character object. The information about the at least one specific object may be an object ID for identifying a wallpaper object that is placed behind the user's character object. In the example shown in FIG. 1(c), the information about the specific object used to change at least a portion of the user's character object is an object ID for identifying a "hat" attachment object that changes the appearance of the character object's "head".

In the example shown in Figures 1(c) and (d), in response to a usage instruction from the user, the user's terminal device acquires image information showing the wearable object "hat" based on information about the wearable object "hat." For example, the user's terminal device acquires image information of the "hat" associated with an object ID for identifying the wearable object "hat" received from the server device from a storage device of the terminal device. Then, based on the acquired image information, the user's terminal device generates an animation of the user's character object with the wearable object "hat" worn on its "head." The user's terminal device displays a user output image (1) including the user's character object wearing the "hat" based on the generated animation, as shown in Figure 1(d).

As described above with reference to FIG. 1, the terminal device control program, terminal device, terminal device control method, server device control program, server device, and server device control method display a user output image (1) including a user's character object at least a portion of which has been changed based on a specific object selected from among a plurality of specific objects in response to an instruction from the user. In this way, the terminal device control program, terminal device, terminal device control method, server device control program, server device, and server device control method allow the output from the terminal device to be changed by the user or another user during a communication service, thereby improving the user's motivation to continue using the communication service.

In addition, in a conventional information system, in order to start another service or event while a user is communicating with another user through a character object such as an avatar representing the user, the user must interrupt or end the communication service. After that, the user and other users can receive the provision of the other service by instructing the server device or the like to start the other service or event. In this way, in a conventional information system, in order for the user and other users to receive the provision of another service while receiving the provision of a service, multiple communications related to the interruption or termination of the service and the start of the other service occur, which can cause a problem of an increase in the processing load of the terminal device and/or the communication load between the terminal device and the server device. In contrast, in the control program for the user's terminal device, the user's terminal device, the control method for the user's terminal device, the control program for the server device, the server device, and the control method for the server device disclosed in the embodiments, the user and other users can receive the provision of a selection event of a specific object simply by communicating about the start of the selection event of a specific object, which can reduce the processing load of the terminal device and/or the communication load between the terminal device and the server device. In addition, the control program for the user's terminal device, the user's terminal device, the control method for the user's terminal device, the control program for the server device, the server device, and the control method for the server device disclosed in the embodiments make it possible to realize a seamless transition to other services or other events, such as a selected event, with a simple interface while continuing the communication service.

In the example shown in FIG. 1, the user and other users are users who belong to a specific communication group among multiple users who can use the communication service. The specific communication group is a group created by either the user or the other users. In the example shown in FIG. 1, character objects of two users are displayed, but character objects of three or more users who belong to the specific communication group may be displayed, and the system may be configured so that three or more users can communicate.

In the example shown in FIG. 1, the user and the other users may be users who have a predetermined relationship. For example, the other users who have a predetermined relationship with the user are mutual followers of the user. In addition, when the user follows the other users and the other users also follow the user (the user and the other users are mutually following each other), the user and the other users become mutual followers. In addition, the other users who have a predetermined relationship with the user may be other users who have a mutual follow relationship with a specific user who is a mutual follower of the user. In addition, the other users who have a predetermined relationship with the user may be other users who have a friend relationship with the user, or other users whose specific information (such as a phone number, an email address, or a predetermined ID (identification)) is stored in the user's terminal device. In addition, when the user's terminal device has a function of distributing a user output image including the user's character object to one or more terminal devices of other users via a server device, the other users who have a predetermined relationship with the user are other users who have viewed the user output image distributed by the user. In this case, when a user and another user who have a certain relationship communicate as shown in FIG. 1, a certain communication group consisting of the user and the other user may be temporarily created by the terminal device of the user or the other user or by the server device.

The above description of FIG. 1 is merely provided to deepen understanding of the contents of the present invention. The present invention is specifically embodied in the following embodiments, and may be embodied in various modified forms without substantially departing from the principles of the present invention. All such modified forms are included within the scope of the present invention and the disclosure of this specification.

(Information System 1)
FIG. 2 is a diagram showing an example of a schematic configuration of the information system 1. The information system 1 includes a terminal device 2 operated by each of a plurality of users, and a server device 3. The terminal device 2 and the server device 3 are connected to each other via a communication network such as a base station 4, a mobile communication network 5, a gateway 6, and the Internet 7. Between the terminal device 2 and the server device 3, communication is performed based on a communication protocol such as Hypertext Transfer Protocol (HTTP). In addition, a connection is first established between the terminal device 2 and the server device 3 by HTTP communication, and thereafter, communication may be performed based on WebSocket, which performs bidirectional communication with lower cost (lower communication load and processing load) than HTTP communication. Note that the communication method between the terminal device 2 and the server device 3 is not limited to the above-mentioned method, and any communication method technology may be used between the terminal device 2 and the server device 3 as long as it can realize this embodiment.

The terminal device 2 is an information processing device such as a smartphone. The terminal device 2 may be a mobile phone, a notebook PC, a tablet terminal, a tablet PC, a head mounted display (HMD), an e-book reader, a wearable computer, or the like. The terminal device 2 may be a portable game device, a game console, or the like. The terminal device 2 may be any information processing device as long as it is capable of displaying and outputting the character objects of the user and other users, and outputting the voices of the user and other users.

In the example shown in FIG. 2, one server device 3 is illustrated as a component of the information system 1, but the server device 3 may be a collection of multiple physically separate server devices 3. In this case, each of the multiple server devices 3 may have the same function, or may have the functions of a single server device 3 in a distributed manner. For example, a server device 3a having a function of receiving and transmitting the user's output information and output information of other users, and a server device 3b having a function of providing the user with a selected event may be components of the information system 1.

(Terminal device 2)
3 is a diagram showing an example of a schematic configuration of the terminal device 2. The terminal device 2 is connected to the server device 3 via a communication network such as a base station 4, a mobile communication network 5, a gateway 6, and the Internet 7, and communicates with the server device 3. The terminal device 2 generates character video data including motion data according to various data (image data, etc.) input by a user, and transmits output information including the generated character video data and/or the user's voice data to the server device 3. The terminal device 2 also receives output information of other users transmitted from the server device 3, and displays character objects of other users and/or outputs voices of other users based on the received output information. To realize such functions, the terminal device 2 includes a terminal communication I/F 21, a terminal storage device 22, a display device 23, an input device 24, an image capture device 25, a microphone 26, and a terminal processing device 27.

The terminal communication I/F 21 is implemented as hardware, firmware, or communication software such as a TCP/IP (Transmission Control Protocol/Internet Protocol) driver or a PPP (Point-to-Point Protocol) driver, or a combination of these. The terminal device 2 can transmit data to other devices such as the server device 3 and receive data from other devices via the terminal communication I/F 21.

The terminal storage device 22 is, for example, a semiconductor memory device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The terminal storage device 22 stores an operating system program, a driver program, a control program, data, etc. used for processing in the terminal processing device 27. The driver programs stored in the terminal storage device 22 include an output device driver program that controls the display device 23 and an input device driver program that controls the input device 24. The control programs stored in the terminal storage device 22 are, for example, application programs for realizing various functions related to the communication service. The control programs may be those transmitted from the server device 3 or other devices.

The terminal storage device 22 stores, as data, identification information (e.g., a user ID (identification)) for uniquely identifying a user who operates the terminal device 2. The terminal storage device 22 also stores, as data, background data and model data. The terminal storage device 22 also stores, as data, a user table T1, an object table T2, and a group table T3. The terminal storage device 22 may also temporarily store data related to a specified process.

The background data is asset data for constructing a virtual space in which the user's character object exists in the user output image. The background data includes data for drawing the background of the virtual space, data for drawing various objects included in the user output image, and data for drawing various other background objects displayed in the user output image. The background data may also include object position information that indicates the positions of various background objects in the virtual space.

The display device 23 is a liquid crystal display. The display device 23 may be an organic EL (Electro-Luminescence) display or the like. The display device 23 displays on a display screen moving images corresponding to moving image data and/or still images corresponding to still image data supplied from the terminal processing device 27. The display device 23 does not have to be a component of the terminal device 2. In this case, the display device 23 may be a display of an HMD capable of communicating with the server device 3, a projection device for projection mapping capable of communicating with the terminal device 2 via wired or wireless communication, a retinal projection type projection device, or the like.

The input device 24 is a pointing device such as a touch panel. When the input device 24 is a touch panel, the input device 24 can detect various touch operations such as tapping, double tapping, and dragging by the user. The touch panel may be equipped with a capacitance-type proximity sensor and configured to be capable of detecting non-contact operations by the user. The input device 24 may be an input key or the like. The user can use the input device 24 to input letters, numbers, and symbols, or positions on the display screen of the display device 23, etc. When operated by the user, the input device 24 generates a signal corresponding to the operation. The input device 24 then supplies the generated signal to the terminal processing device 27 as an instruction from the user.

The imaging device 25 is a camera having an imaging optical system, an imaging element, an image processing unit, etc. The imaging optical system is, for example, an optical lens, and focuses a light beam from a subject on the imaging surface of the imaging element. The imaging element is, for example, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and outputs an image of the subject imaged on the imaging surface. The image processing unit creates moving image data in a specified file format from images continuously generated by the imaging element at specified intervals, and outputs the data as imaging data. Alternatively, the image processing unit creates still image data in a specified file format from images generated by the imaging element, and outputs the data as imaging data.

The microphone 26 is a sound collection device configured to acquire voice uttered by the user and convert it into voice data. The microphone 26 is configured to acquire voice input from the user, converts the user's voice input acquired by the microphone 26 into voice data, and outputs the voice data to the terminal processing device 27.

The terminal processing device 27 is a processor that loads the operating system program, driver program, and control program stored in the terminal storage device 22 into memory and executes the instructions contained in the loaded programs. The terminal processing device 27 is, for example, an electronic circuit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit), or a combination of various electronic circuits. The terminal processing device 27 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field Programmable Gate Array), or an MCU (Micro Controller Unit). Although the terminal processing device 27 is illustrated as a single component in FIG. 3, the terminal processing device 27 may be a collection of multiple physically separate processors.

The terminal processing device 27 executes various commands included in the control program to function as a generation unit 271, a transmission unit 272, a reception unit 273, a display processing unit 274, and an audio output unit 275. The functions of the generation unit 271, the transmission unit 272, the reception unit 273, the display processing unit 274, and the audio output unit 275 will be described later.

(Various tables)
4 is a diagram showing an example of the data structures of the user table T1, object table T2, and group table T3 stored in the terminal storage device 22. At least one of the user table T1, object table T2, and group table T3 may be stored in the server storage device 32 of the server device 3. In this case, the terminal device 2 may receive the latest table stored in the server storage device 32 at a predetermined timing and store it in the terminal storage device 22.

(User Table T1)
FIG. 4(a) is an example of the data structure of a user table T1 that manages users participating in the communication service. In the user table T1, for each user, a user ID, the user's name, the user's character object, the user's owned objects, and the objects used are stored in association with each other. In the user table T1, the user's owned coins (information indicating the total amount of "coins", a virtual monetary value, held by the user) and/or user points may be stored in association with the user's user ID. The user ID is an example of identification data for uniquely identifying each user. The name is an example of data indicating the name of each user.

For example, the character object is a model ID for identifying model data for generating animation of the character object. The model data is stored in the terminal storage device 22 in association with the model ID. The model data may be three-dimensional model data for generating three-dimensional animation, or two-dimensional model data for generating two-dimensional model data. The model data includes, for example, rig data (so-called "skeleton data") indicating the skeleton of the face and parts other than the face of the character object, and surface data indicating the shape and texture of the surface of the character object. The model data may include multiple model data that are different from each other. Each of the multiple model data may have different rig data from each other, or may have the same rig data. Each of the multiple model data may have different surface data from each other, or may have the same surface data. Note that multiple object IDs associated with each of the multiple partial objects constituting the character object may be associated with the model ID. In this case, three-dimensional model data for generating three-dimensional animation may be stored in the terminal storage device 22 and the server storage device 32 for each object ID of the multiple partial objects. In this case, in the display process of the character object, the object IDs of the multiple partial objects associated with the model ID may be extracted, the three-dimensional model data (of each partial object) corresponding to each object ID may be identified, and the character object may be displayed based on the identified three-dimensional model data (of each partial object). Also, image information corresponding to each three-dimensional model data may be stored.

The owned objects are object IDs that indicate objects owned by each user. Owned objects include wearable objects that can be associated with a specific part of a character object, and wallpaper objects that are placed behind the user's character object in the user output image. Details of wearable objects and wallpaper objects will be described later. Owned objects may include specific objects selected by a selection event provided by the server device 3 through a user's operation. Owned objects may also be objects that the user has come to own by spending "coins", which have a virtual monetary value, in the purchase function of the information system 1.

The used object is an object ID indicating an object used in the user output image being displayed on the user's terminal device 2. For example, when a user's change instruction is input during a communication service, an attachment object held as a user's owned object is attached to the user's character object included in the user output image being displayed. In this case, the object ID of the attachment object attached to the user's character object is stored in the user table T1 in association with the user ID as a used object. Also, for example, when a user's change instruction is input during a communication service, a wallpaper object held as a user's owned object is placed behind the user's character object in the user output image being displayed. In this case, the object ID of the wallpaper object placed in the user output image is stored in the user table T1 in association with the user ID as a used object.

(Object table T2)
4B shows an example of the data structure of an object table T2 that manages objects selected as additional objects. An additional object is, for example, a gift object that a user gives to another user. In the object table T2, the object ID, name, image information, rarity, placement location, etc. of each additional object are stored in association with each other.

The object ID is an example of identification data for uniquely identifying each additional object. The image information is one or more still images corresponding to each additional object. The image information may be one or more types of moving images corresponding to each additional object. The rarity is information indicating the rarity of each additional object. The placement location is information indicating the location where the image information of each additional object is displayed. Note that if the display position and display range of a character object are determined, information indicating the relative position of the character object may be stored as the placement location.

The additional objects are classified into a number of types (categories). For example, the additional objects include effect objects used as effect gifts, normal objects used as normal gifts, attachment objects used as attachment gifts, and message objects used as message gifts. The additional objects may also include wallpaper objects that are placed behind the character object in the user output image. Information indicating the type (category) of the additional object may be stored in object table T2 in association with the object ID of the additional object. Below, image information and placement locations are explained according to the type of additional object.

An effect object is an object that affects the overall impression of a user output image and/or other user output images. An effect object is, for example, an object that resembles confetti. When the effect object is an object that resembles confetti, an image showing multiple pieces of paper is stored as the image information.

Information indicating "space" as the placement location of the effect object is stored. For example, an object resembling confetti is displayed in the entirety of the user output image and/or the other user output image. The effect object may be displayed in the entirety of a screen (a communication screen described later) including the user output image and the other user output image in response to a user instruction. The effect object may be displayed only in an output image selected by the user from among the user output image and the other user output image. The effect object may be displayed so as to overlap with a character object included in the user output image and/or the other user output image. Unlike an attachment object described later, the effect object is displayed without being associated with a specific part of the character. In this way, the terminal device 2 of the information system 1 can change the overall impression of the user output image and/or the other user output image by displaying an effect object in response to a user instruction, compared to before the effect object was displayed.

A normal object is, for example, an object that imitates a stuffed animal, a bouquet of flowers, an accessory, or an item suitable for a gift or present. Information indicating a "space" is stored as the placement location of the normal object. For example, information regarding a predetermined movement route is associated with the normal object, and the normal object is displayed moving along the predetermined movement route in the user output image or the other user output image. The information indicating the "space" stored as the placement location of the "normal object" may be classified into multiple types. The multiple types of "space" are, for example, a "center space" (the area in the center of the image when the user output image or the other user output image is divided into three parts horizontally), a "left space" (the area on the left side of the image when the user output image or the other user output image is divided into three parts horizontally), and a "right space" (the area on the right side of the image when the user output image or the other user output image is divided into three parts horizontally). The multiple types of "spaces" may be "center space" (a central area of the image when the user output image or the other user output image is divided into three parts vertically), "upper space" (an upper area of the image when the user output image or the other user output image is divided into three parts vertically), and "lower space" (a lower area of the image when the user output image or the other user output image is divided into three parts vertically). In this case, the movement route associated with the normal object is set within the type of "space" associated as the placement location of the normal object. For example, when information indicating "right space" is stored in the normal object, information regarding the movement route moving within the right area of the image when the user output image or the other user output image is divided into three parts horizontally is associated with the normal object. In addition, the normal object may be displayed as a normal object that moves according to a predetermined movement rule from a position set for each normal object in the user output image or the other user output image or a position automatically and randomly determined. The predetermined movement rule is, for example, a movement rule set so that the normal object moves as if it is falling freely. The normal object may be displayed so as to overlap with the character object. In this case, the normal object is displayed without being associated with a specific part of the character object, unlike the attached object described below. The normal object may also be displayed to move in a bouncing manner when it comes into contact with the character object.

When a normal object is displayed overlapping with a character object, for example, the normal object may be displayed so as to overlap with "parts other than the 'head', including the 'face'" of the character object. In this case, the normal object is displayed so as not to overlap with the "head" of the character object. Also, the normal object may be displayed so as to overlap with "parts other than the 'upper body', including the 'face'" of the character object. In this case, the normal object is displayed so as not to overlap with the "upper body" of the character object.

An attachment object is an object that is displayed in a user output image or another user output image in association with a specific part (attachment location) of a character object. For example, specific parts of the character object's head include the front left, front right, rear left, rear right, front center, rear center, left eye, right eye, left ear, right ear, and all of the hair. In addition, specific parts may be the thumb, ring finger, wrist, elbow, shoulder, upper arm, entire hand, entire arm, etc.

An attachment object associated with a specific part of a character object is displayed in the user output image or other user output image so as to be in contact with the specific part of the character object. In addition, an attachment object associated with a specific part of a character object may be displayed in the user output image or other user output image so as to cover a part or all of the specific part of the character object. The specific part may be identified by three-dimensional position information indicating a position in a three-dimensional coordinate space, or may be associated with position information in the three-dimensional coordinate space.

The image information of the wearing object is, for example, image information that imitates accessories (headbands, necklaces, earrings, etc.) worn by the character object, clothing (T-shirts, dresses, etc.), costumes, and other items that can be worn by the character object.

As the placement location of the wearing object, information is stored that indicates the wearing part, which indicates which part of the character object the wearing object is associated with. For example, if the wearing object is a "headband", information is stored that indicates the "head" of the character object as the placement location of the wearing object. Also, if the wearing object is a "T-shirt", information is stored that indicates the "torso" of the character object as the placement location of the wearing object.

Information indicating multiple attachment positions in three-dimensional coordinate space may be stored as the placement location of the attachment object. For example, if the attachment object is a "headband," information indicating two positions on the character object, the "left rear of the head" and the "right rear of the head," may be stored as the placement location of the attachment object. In this way, an attachment object modeled after a "headband" is displayed as being attached to both the "left rear of the head" and the "right rear of the head" of the character object.

When multiple types of attachment objects that are attached to the same attachment location are attached to a character object, the multiple types of attachment objects are attached to the character object at different times. As a result, multiple types of attachment objects that are attached to the same attachment location are not attached to the character object at the same time. For example, if the attachment location of an attachment object indicating a "headband" and an attachment object indicating a "hat" are both stored as "head," the attachment object indicating a "headband" and the attachment object indicating a "hat" will not be displayed on the head of the character object at the same time.

In object table T2, a display time according to the type of additional object may be stored in association with the object ID of the additional object. For example, the display time of an attached object may be stored as a time longer than the display time of an effect object and the display time of a normal object. For example, if "60 seconds" is stored as the display time of an attached object, "5 seconds" may be stored as the display time of an effect object, and "10 seconds" may be stored as the display time of a normal object.

(Group table T3)
4(c) is an example of the data structure of a group table T3 that manages groups to which a user belongs (such as a group of users who are friends with each other, a group with other users who are mutual followers, or a group arbitrarily created by the user or other users). The group table T3 is provided individually for each user, and the group table T3 shown in FIG. 4(c) is the group table T3 of the user who operates the terminal device 2 that stores the group table T3.

In group table T3, for each group, the group ID of the group, the name of the group, the users belonging to the group, etc. are stored in association with each other. The group ID is an example of identification data for uniquely identifying each group. The name is an example of data indicating the name of each group. The users belonging to the group are the user IDs of each user belonging to the group.

Returning to FIG. 3, the functions of the generation unit 271, transmission unit 272, reception unit 273, display processing unit 274, and audio output unit 275 will be described.

(Generation unit 271)
The generating unit 271 acquires the imaging data continuously output by the imaging device 25 from the imaging device 25. For example, the imaging device 25 of the terminal device 2 held by the user is provided on the terminal device 2 so that the face of the user looking at the display screen of the display device 23 is in the imaging direction. The imaging device 25 continuously captures the user's face in the imaging direction, acquires imaging data of the user's face, and outputs the data to the generating unit 271 of the terminal processing device 27. The imaging device 25 may acquire imaging data by capturing an image of a part other than the user's face, for example, the head, arms, hands (including fingers), chest, torso, legs, or other parts. The imaging device 25 may also be a 3D camera capable of detecting the depth of a person's face.

The generation unit 271 generates face motion data, which is a digital representation of the user's facial movements, as time passes, based on the continuously acquired imaging data. The face motion data may be generated at predetermined sampling time intervals. In this way, the face motion data generated by the generation unit 271 can digitally represent the user's facial movements (changes in facial expressions) in a time series.

The generation unit 271 may generate body motion data, which is a digital representation of the position and orientation of each part of the user (head, arms, hands (including fingers), chest, torso, legs, or other parts), together with or separately from the facial motion data.

The body motion data may be generated based on detection information from a known motion sensor worn by the user. In this case, the terminal communication I/F 21 of the terminal device 2 has a predetermined communication circuit for acquiring the detection information from the motion sensor worn by the user via wireless communication. The generation unit 271 then generates the body motion data based on the detection information acquired by the terminal communication I/F 21. The body motion data may be generated at predetermined sampling time intervals. In this way, the body motion data generated by the generation unit 271 can digitally represent the movement of the user's body in a time series.

The generation of body motion data based on detection information from a motion sensor worn by a user may be performed, for example, in a photography studio. In this case, the photography studio may be equipped with a base station, a tracking sensor, and a display. The base station may be, for example, a multi-axis laser emitter. The motion sensor worn by the user may be, for example, a Vive Tracker provided by HTC CORPORATION, and the base station may be, for example, a base station provided by HTC CORPORATION.

The supporter computer may also be installed in a separate room adjacent to the photography studio. The display of the photography studio may be configured to display information received from the supporter computer. The server device 3 may be installed in the same room as the room in which the supporter computer is installed. The room in which the supporter computer is installed and the photography studio may be separated by a glass window. In this case, the operator of the supporter computer can see the user. The supporter computer may be configured to be able to change the settings of various devices provided in the photography studio in response to the operation of the supporter. The supporter computer can, for example, set the scanning interval by the base station, set the tracking sensor, and change various settings of various other devices. The operator may input a message to the supporter computer, and the input message may be displayed on the display of the photography studio.

The generation unit 271 generates character video data including facial motion data and/or body motion data generated at any time, and outputs the generated character video data to the display processing unit 274. Hereinafter, facial motion data and body motion data may be collectively referred to as motion data. The generation unit 271 also generates output information including the generated character video data and a user ID stored in the terminal storage device 22, and outputs the generated output information to the transmission unit 272. When the generation unit 271 acquires user voice data output from the microphone 26, it generates output information including the generated character video data, the acquired voice data, and the user ID stored in the terminal storage device 22, and outputs the generated output information to the transmission unit 272.

The generation unit 271 generates game image data showing a game screen of the game being played by the user at any time during the progress of the game, generates output information including the generated game image data and a user ID stored in the terminal storage device 22, and outputs the generated output information to the transmission unit 272. Note that the output information may include the generated character video data and/or acquired audio data in addition to the generated game image data.

(Transmitter 272)
The transmitting unit 272 transmits the output information output from the generating unit 271 together with the destination information to the server device 3 via the terminal communication I/F 21. The destination information includes the user IDs of one or more other users participating in the communication service with the user. The user IDs included in the output information may also be used as the destination information. For example, the server device 3 stores the user IDs of one or more other users participating in the communication service with the user, and upon acquiring the output information, identifies the user IDs of one or more other users participating in the communication service with the user of the user ID included in the output information, and transmits the received output information to the terminal device 2 of the other user indicated by the identified user ID.

In addition, the transmission unit 272 transmits information indicating various instructions input by the user by operating the input device 24, together with the user ID and destination information stored in the terminal storage device 22, to the server device 3 via the terminal communication I/F 21. Various instructions by the user include, for example, a selection instruction, a use instruction, and an addition instruction. Information indicating an addition instruction is transmitted to the server device 3 via the terminal communication I/F 21 together with the object ID of an additional object to be included in the other user output image and the user ID stored in the terminal storage device 22. Information indicating a selection instruction is transmitted to the server device 3 via the terminal communication I/F 21 together with the user ID, etc. stored in the terminal storage device 22. Information indicating a use instruction is transmitted to the server device 3 via the terminal communication I/F 21 together with the object ID of at least one specific object set as a use object and the user ID, etc. stored in the terminal storage device 22. In addition, when a user selects a partial object and issues an instruction to create a character object composed of the selected partial objects, a model ID for identifying model data of the character object and object IDs of the multiple partial objects constituting the character object may be transmitted to the server device 3 via the terminal communication I/F 21 together with the user ID and the like stored in the terminal storage device 22. In this case, the server device 3 may store the user ID, the model ID, and the object IDs of the multiple partial objects constituting the character object in the server storage device 32 in association with each other. In addition, the terminal device 2 of the other user that has received the model ID for identifying model data of the user's character object, the object IDs of the multiple partial objects constituting the user's character object, and the user ID of the user via the server device 3 may store the user ID, the model ID, and the object IDs of the multiple partial objects constituting the character object in the terminal storage device 22 in association with each other. In this way, the terminal device 2 of the user only needs to transmit the user ID (or model ID) to the terminal device of the other user via the server device 3, and the terminal device of the other user can extract the object IDs of the multiple partial objects associated with the received user ID (or model ID). The terminal devices of the other users can then display the user's character objects based on the three-dimensional model data (of each partial object) that corresponds to each extracted object ID.

(Receiving unit 273)
The receiving unit 273 receives the output information of the other user and information indicating various instructions of the other user transmitted from the server device 3 via the terminal communication I/F 21. The receiving unit 273 outputs the received information to the terminal processing device 27. The output information of the other user transmitted from the server device 3 is generated by the generating unit 271 of the terminal device 2 of the other user and transmitted to the server device 3 by the transmitting unit 272 of the terminal device 2 of the other user. The output information of the other user includes character video data of the other user, voice data of the other user, and the user ID of the other user. The output information of the other user may not include voice data of the other user, and the output information of the other user may not include character video data of the other user.

The receiving unit 273 also receives information indicating the selection result transmitted from the server device 3 via the terminal communication I/F 21. The information indicating the selection result is information indicating the result of the selection process executed by the server device 3 in response to a selection instruction by the user or another user. The receiving unit 273 receives the user ID of the user who issued the selection instruction along with the information indicating the selection result. When the user ID of the user operating the terminal device 2 is the same as the received user ID, the receiving unit 273 stores the object ID of at least one specific object received as the information indicating the selection result in the user table T1 as an object owned by the user, in association with the user ID of the user.

(Display Processing Unit 274)
The display processing unit 274 displays an image drawn based on the motion data (character moving image data) generated by the generation unit 271 as a user output image on the display device 23. The display processing unit 274 also displays an image drawn based on the motion data (character moving image data) included in the output information of other users received by the reception unit 273 as a other user output image on the display device 23. The image drawing process based on the motion data will be described below. In the image drawing process based on the motion data of the user (the user operating the terminal device 2), the user ID stored in the terminal storage device 22 is used, and in the image drawing process based on the motion data included in the output information of other users received by the reception unit 273, the user ID included in the output information is used.

First, the display processing unit 274 extracts the model ID of the character object associated with the user ID and the object ID of the used object from the user table T1. Next, the display processing unit 274 reads out the model data associated with the extracted model ID and the image information and placement location associated with the extracted object ID of the used object from the terminal storage device 22. Next, the display processing unit 274 generates an animation of the character object in which the used object is attached to the placement location based on the read model data and the image information, placement location, and motion data of the used object. Note that, if the object IDs of multiple partial objects constituting the character object are associated with the model ID of the character object, the display processing unit 274 extracts model data corresponding to the object IDs of the multiple partial objects associated with the extracted model ID. Next, the display processing unit 274 generates an animation of the character object in which the used object is attached to the placement location based on the read model data of the multiple partial objects and the image information, placement location, and motion data of the used object. Note that the used object may be included in multiple partial objects.

When the motion data includes face motion data, the display processing unit 274 generates animation of the character object so that the facial expression of the character object changes based on the face motion data. As a result, the display processing unit 274 can generate animation of the character object that moves in synchronization with the facial expression of the user based on the read model data and face motion data. When the motion data includes face motion data and body motion data, the display processing unit 274 generates animation of the character object that moves in synchronization with the facial expression and body movement of the user based on the read model data, face motion data, and body motion data. Next, the display processing unit 274 draws an image including the animation (moving image) of the generated character object and a background image showing the background generated using the background data stored in the terminal storage device 22. Note that the display processing unit 274 may use a wallpaper object that is associated with the communication group in advance and stored in the terminal storage device 22 as a background image. In this case, the same wallpaper object may be applied as the background image of all members of the communication group. Then, the display processing unit 274 outputs drawing data showing the drawn image and ends the drawing process of the image based on the motion data. Note that an image drawn based on a character object associated with a user ID stored in the terminal storage device 22 is the user output image of the user. Also, an image drawn based on a character object associated with a user ID included in the received output information of another user is the other user output image of that other user.

In addition, the display processing unit 274 may display on the display device 23 a character object newly wearing an attachment object owned by the user as a user's owned object in response to a change instruction from the user. For example, if the attachment object instructed to be changed by the user is a "headband", the display processing unit 274 extracts the object ID of the attachment object ("headband") from the owned objects associated with the user's user ID in the user table T1, and stores it as a use object. The display processing unit 274 extracts image information and a placement location associated with the object ID stored as a use object from the object table T2. Next, the display processing unit 274 generates a user output image including a character object wearing an attachment object ("headband") on the head (the placement location associated with the object ID of the "headband") based on the extracted image information and placement location. The display processing unit 274 may generate a user output image including a character object wearing an attachment object that moves in conjunction with the movement of the attachment part (placement location) of the character object.

The display processing unit 274 also reads out image information associated with the object ID of the specific object received as information indicating the selection result from the terminal storage device 22, and displays a selection result screen on the display device 23 based on the read image information.

The display processing unit 274 may also display on the display device 23 a user output image including a character object equipped with a new attachment object, based on information indicating the selection result received from the server device 3. The information indicating the selection result is information indicating the result of a selection process executed by the server device 3 in response to a selection instruction from the user. The information indicating the selection result includes, for example, the object ID of a specific object selected by the selection process. The number of object IDs of specific objects included in the information indicating the selection result may be one or two or more. An example of a use process in response to a use instruction is described below.

First, when a use instruction is input by the user operating the input device 24, the display processing unit 274 of the user's terminal device 2 executes a process of storing the object ID of a specific object included in the information indicating the selection result. The process of storing a specific object is a process in which the object ID of the specific object included in the information indicating the selection result is stored in the user table T1 as a used object in association with the user ID of the user. Note that when the information indicating the selection result includes object IDs of multiple specific objects, the object ID of at least one specific object instructed to be used by the user from among the multiple specific objects is stored in the user table T1 as a used object in association with the user ID of the user. In this way, the object ID of at least one specific object set as a used object is stored.

In addition, when the specific object stored as the use object is an attached object, the display processing unit 274 executes a display process of the attached object. An example of the display process of the attached object is described below. In addition, in the description of the example of the display process of the attached object, the specific object stored as the use object is called the "attached object". First, the display processing unit 274 extracts image information and a placement location associated with the object ID of the attached object (specific object) stored as the use object from the object table T2. Based on the extracted image information and placement location, the display processing unit 274 generates an animation of the user's character object in which the attached object is attached to the placement location. Then, based on the generated animation, the display processing unit 274 displays a user output image including the user's character object in which the attached object is attached to the placement location. With the above, the display process of the attached object is completed, and the use process according to the use instruction is also completed.

In addition, in the display process of the wearing object, the display processing unit 274 may display a user output image including a character object wearing a wearing object that moves in conjunction with the movement of the wearing part (placement location) of the character object. For example, if the specific object stored as the use object is the wearing object "headband", a user output image is displayed including the user's character object with the wearing object "headband" worn on the head. Also, in the user output image, the wearing object "headband" is displayed to move in conjunction with the movement of the character object's head.

Furthermore, when the specific object stored as the use object is a wallpaper object, the display processing unit 274 executes a display process for the wallpaper object. An example of the display process for the wallpaper object is described below. Note that, in the description of the example of the display process for the wallpaper object, the specific object stored as the use object is referred to as a "wallpaper object." First, the display processing unit 274 extracts image information associated with the object ID of the wallpaper object (specific object) stored as the use object from the object table T2. The display processing unit 274 displays a user output image using the extracted image information as a background image for the user's character object. This completes the display process for the wallpaper object, and also completes the use process in response to the use instruction.

In addition, the display processing unit 274 displays a user output image including an additional object in response to an additional instruction from another user to add the additional object. The display processing unit 274 displays a other user output image including an additional object in response to an additional instruction from a user to display an additional object in the other user output image.

When the object ID included in the instruction to add an additional object from another user is the object ID of one of the effect objects, the display processing unit 274 refers to the object table T2, extracts a still image or a moving image (image information) of the effect object associated with the object ID included in the instruction to add from the terminal storage device 22, and generates a user output image including the extracted still image or moving image. For example, when an instruction to add an effect object showing confetti or fireworks is given, the display processing unit 274 generates a user output image including a moving image of an effect object (not shown) that imitates confetti or fireworks. When the object ID included in the instruction to add an additional object from a user is the object ID of one of the effect objects, the display processing unit 274 similarly refers to the object table T2, extracts a still image or a moving image (image information) of the effect object associated with the object ID included in the instruction to add from the terminal storage device 22, and generates a user output image including the extracted still image or moving image.

When the object ID included in the instruction to add an additional object from another user is the object ID of a normal object, the display processing unit 274 refers to the object table T2 and extracts a still image or video (image information) of the normal object associated with the object ID of the normal object and the placement location. Next, the display processing unit 274 generates a user output image including a still image or video of the normal object placed at the position indicated by the placement location. Similarly, when the object ID included in the instruction to add an additional object from a user is the object ID of a normal object, the display processing unit 274 similarly refers to the object table T2 and extracts a still image or video (image information) of the normal object associated with the object ID of the normal object and the placement location, and generates an other user output image including a still image or video of the normal object placed at the position indicated by the placement location.

The display processing unit 274 may generate a user output image and another user output image including a moving image of a normal object moving within the display area of the user output image and the other user output image. Hereinafter, the user output image and the other user output image may be collectively referred to as an output image. For example, the display processing unit 274 may generate an output image including a moving image of a normal object falling from the top to the bottom of the output image. In this case, the normal object may be displayed within the display area of the output image from the start of the fall until it falls to the bottom end of the output image, and may disappear from the output image after it falls to the bottom end of the output image. The movement route of the normal object may be a route from left to right, right to left, top left to bottom left, or any other direction in the output image, and may also be a route along a straight orbit, a circular orbit, an elliptical orbit, a spiral orbit, or any other orbit.

When the object ID included in an instruction to add an additional object from another user is the object ID of a worn object, the display processing unit 274 displays notification information regarding the wearing instruction in the user output image. In response to the wearing instruction from the user, the display processing unit 274 refers to the object table T2 and extracts image information and a placement location of the wearing object associated with the object ID included in the addition instruction. Next, based on the extracted image information and placement location, the display processing unit 274 generates a user output image including a character object wearing the wearing object at the wearing part indicated by the placement location.

The display processing unit 274 also starts the game played by the user in response to the user's instructions, and displays the game screen of the game on the display device 23. When the game starts, the display processing unit 274 displays the game screen of the game that progresses in response to game progress instructions by the user's game operation on the display device 23, and ends the display of the game screen when a game end condition is met. The game end condition is determined in advance according to the rules of the game, and is, for example, when the user's game medium (character, etc.) wins or loses against an opponent, when the user's game medium reaches a predetermined destination, or when a game object operated by the user reaches a predetermined state (for example, when multiple pieces (game objects) are lined up in squares on the game field), etc.

(Audio output unit 275)
When the output information of another user includes voice data of the other user, the voice output unit 275 outputs the voice of the other user generated based on the voice data from a speaker (not shown) while displaying the other user output image by the display processing unit 274. As a result, the voice of the other user is output during the movement of the lips of the character object of the other user included in the other user output image, so that the user can see the other user output image including the character object of the other user that appears to be making a sound.

(Server device 3)
5 is a diagram showing an example of a schematic configuration of the server device 3. The server device 3, which provides communication services to each of the terminal devices 2 of a plurality of users, transmits output information and information indicating various instructions transmitted from the terminal device 2 of one user to the terminal device 2 of another user different from the one user. To this end, the server device 3 includes a server communication I/F 31, a server storage device 32, and a server processing device 33. For example, the terminal device 2 to which information is transmitted by the terminal device 2 of one user is the terminal device 2 of the user indicated by the user ID included in the received destination information.

The server communication I/F 31 is implemented as hardware, firmware, or communication software such as a TCP/IP driver or a PPP driver, or a combination of these. The server device 3 can send information to other devices and receive information from other devices via the server communication I/F 31.

The server storage device 32 is, for example, a semiconductor memory device such as a ROM or a RAM. The server storage device 32 may be, for example, a magnetic disk, an optical disk, or any other storage device capable of storing data. The server storage device 32 stores an operating system program, a driver program, an application program, data, and the like used in the processing in the server processing device 33. The server storage device 32 may store a user table T1, an object table T2, and a group table T3 as data. The server storage device 32 also stores a selection event ID for identifying a selection event and an object group ID for identifying an object group used in the selection event in association with each other. The server storage device 32 stores each object ID of a plurality of specific objects in association with the object group ID. For example, when the server device 3 can provide a plurality of types of selection events, a plurality of selection event IDs for identifying each of the plurality of types of selection events are stored. A plurality of object group IDs associated with each of the plurality of selection event IDs are also stored, and each object group ID stores each object ID of a plurality of specific objects.

The server processing device 33 is a processor that loads the operating system program, driver program, and control program stored in the server storage device 32 into memory and executes the instructions contained in the loaded programs. The server processing device 33 is, for example, an electronic circuit such as a CPU, MPU, DSP, or GPU, or a combination of various electronic circuits. The server processing device 33 may be realized by an integrated circuit such as an ASIC, PLD, FPGA, or MCU. In FIG. 5, the server processing device 33 is illustrated as a single component, but the server processing device 33 may be a collection of multiple physically separate processors. The server processing device 33 functions as a server receiving unit 331, a server transmitting unit 332, and a selection unit 333 by executing various instructions contained in the control program.

(Server Receiving Unit 331)
The server receiving unit 331 receives output information, information indicating various instructions, and the like transmitted from the terminal device 2 of any one of the multiple users via the server communication I/F 31. Note that the server receiving unit 331 may receive other information, such as destination information, together with the output information and information indicating various instructions transmitted from the terminal device 2.

For example, the server receiving unit 331 receives information indicating a selection instruction, a selection event ID, user A's user ID, and destination information from the user's terminal device 2 via the server communication I/F 31. The server receiving unit 331 inputs the received information indicating a selection instruction and the selection event ID to the selection unit 333. If information indicating the number of selections is transmitted together with the information indicating a selection instruction, the server receiving unit 331 receives the information indicating the number of selections together with the information indicating the selection instruction, and inputs the received information indicating the number of selections to the selection unit 333.

(Server transmission unit 332)
The server transmitting unit 332 transmits the output information and information indicating various instructions received by the server receiving unit 331 to the terminal device 2 of another user indicated by the user ID included in the destination information received at the same time, via the server communication I/F 31. The server transmitting unit 332 may also transmit the information indicating various instructions received by the server receiving unit 331 to one or more terminal devices 2 of other users designated by the user who transmitted the information, via the server communication I/F 31.

In addition, when the object ID of at least one specific object is selected by the selection unit 333 (described later) in response to a selection instruction by the user, the server transmission unit 332 has a function of transmitting the object ID of the selected specific object. For example, the server transmission unit 332 acquires the object ID of at least one specific object selected by the selection unit 333 in a selection event specified by the user from the selection unit 333 as information indicating the selection result. In addition, the server transmission unit 332 acquires the user ID of the user and the destination information received from the server reception unit 331 together with the information indicating the selection instruction. Next, the server transmission unit 332 transmits the information indicating the selection result and the user ID of the user who issued the selection instruction via the server communication I/F 31 to the user's terminal device 2 and the user's terminal device 2 indicated by the user ID included in the destination information.

(Selection unit 333)
The selection unit 333 acquires information indicating a selection instruction and a selection event ID from the server receiving unit 331. When information indicating the number of selections is transmitted together with the information indicating the selection instruction, the selection unit 333 also acquires information indicating the number of selections from the server receiving unit 331. The selection unit 333 extracts an object group ID associated with the input selection event ID from among a plurality of object group IDs stored in the server storage device 32.

Next, the selection unit 333 selects the object ID of at least one specific object from among the object IDs of the multiple specific objects associated with the extracted object group ID according to a predetermined selection method. Note that, when the selection unit 333 acquires information indicating the number of selections, it selects the object IDs of the specific objects from among the object IDs of the multiple specific objects associated with the extracted object ID by the number of selections. For example, when the selection unit 333 acquires information indicating the number of selections "1", it selects the object ID of one specific object from among the object IDs of the multiple specific objects associated with the extracted object ID. Also, for example, when the selection unit 333 acquires information indicating the number of selections "10", it selects the object IDs of ten specific objects from among the object IDs of the multiple specific objects associated with the extracted object ID. Then, the selection unit 333 inputs the object IDs of the selected specific objects to the server transmission unit 332.

(Examples of various screens)
6 to 15, examples of various screens displayed on the display device 23 of the user's terminal device 2 will be described below. In Fig. 6 to 15, multiple components with the same reference numerals are components having equivalent functions.

FIG. 6(a) is a diagram showing an example of a group creation screen 600 displayed on the display device 23 of the user's terminal device 2. The group creation screen 600 includes a other user display area 601, a selection object 602, and a create button 603. For example, the group creation screen 600 is displayed when a group creation object or the like on a home screen (not shown) displayed in response to execution of a control program stored in the terminal storage device 22 is selected by the user operating the input device 24.

The other user display area 601 displays other user information indicating each of the other users who have a specific relationship with the user. In the example shown in FIG. 6(a), the display processing unit 274 of the user's terminal device 2 displays thumbnail images of the character objects of the other users and the names of the other users as other user information.

The selection object 602 is an operation object for selecting each other user indicated by each other user information. In the example shown in FIG. 6(a), the display processing unit 274 of the user's terminal device 2 displays a check box object corresponding to each other user information as the selection object 602. For example, when the selection object 602 is selected by the user's operation of the input device 24, the selection object 602 is displayed including a "check" mark. In this case, the other user of the other user information corresponding to the selection object 602 displayed including the "check" mark is selected. Also, when the selection object 602 displayed including the "check" mark is selected by the user's operation of the input device 24, the selection object 602 is displayed without a "check" mark. In this case, the other user of the other user information corresponding to the selection object 602 displayed without a "check" mark is not selected. Each of the selection objects 602 is associated with the user ID of the other user related to the corresponding other user information.

The create button 603 is a button object or the like for the user to create a new communication group. When the create button 603 is selected by the user operating the input device 24, a new group is created with the user operating the terminal device 2 and the other user selected in the selection object 602 as belonging users. For example, the display processing unit 274 of the user's terminal device 2 identifies the user ID of the user stored in the terminal storage device 22 and the user ID of the other user associated with the selection object 602 related to the other user selected in the selection object 602. Then, the display processing unit 274 associates the user ID of the identified user and the user ID of the other user with the newly created group ID as belonging users and stores them in the group table T3. At this time, the display processing unit 274 may automatically associate the name of the created group with the newly created group ID, or may associate the name of the group input by the user.

When a new group is created in response to the selection of the create button 603, the display processing unit 274 displays an information exchange screen 700 for users belonging to the new group. FIG. 7(a) is a diagram showing an example of the information exchange screen 700 displayed on the display device 23 of the user's terminal device 2. The information exchange screen 700 includes a display area 701 that displays information such as text or images entered by each user belonging to the new group, information (such as a thumbnail image of a character object) 702 indicating the user who entered the information displayed in the display area 701, an input object 703 for the user to input the text or image to be displayed in the new display area 701, and a start object 704 for starting the communication service.

When the information exchange screen 700 is displayed, the transmitting unit 272 of the user's terminal device 2 transmits an instruction to display the information exchange screen together with information about the new group and destination information to the server device 3 via the terminal communication I/F 21. The information about the new group is the group ID, name, and user IDs of the new group's users. The destination information includes the user IDs of the other users who belong to the new group, excluding the user who operates the terminal device 2. The server receiving unit 331 of the server device 3 receives the instruction to display the information exchange screen, information about the new group, and destination information via the server communication I/F 31. The server receiving unit 331 may store the received destination information in the server storage device 32. The server transmitting unit 332 of the server device 3 transmits the instruction to display the information exchange screen and information about the new group to the terminal device 2 of the other user indicated by the user ID included in the destination information via the server communication I/F 31. When the terminal device 2 of the other user receives an instruction to display the information exchange screen sent from the user's terminal device 2 via the server device 3, it can display the information exchange screen 700 where information can be exchanged with users belonging to the new group.

Returning to FIG. 7(a), for example, when new information is input to the input object 703 by an operation of one of the users, a display area 701 displaying the new information is added to the information exchange screen 700 displayed by the terminal device 2 of the user who input the information. In addition, the transmission unit 272 of the terminal device 2 of the user who input the information transmits the inputted new information via the server device 3 to the terminal devices 2 of users who belong to the group other than the user who input the information. As a result, a display area 701 displaying the new information is also added to the information exchange screen 700 displayed on the terminal devices 2 of other users who belong to the group other than the user who input the information.

The start object 704 is a button object or the like for starting a communication service in which each user belonging to the new group can participate. When the start object 704 is selected by the user operating the input device 24, a start process for a communication service is executed in which groups capable of exchanging information via the information exchange screen 700 are treated as communication groups. An example of the start process in response to the user's selection of the start object 704 is described below.

First, the display processing unit 274 of the user's terminal device 2 displays a communication screen 810 (FIG. 8(b)) that can be displayed by the terminal device 2 of each user belonging to the communication group. The display processing unit 274 also stores the group ID of the communication group in the terminal storage device 22 as the group ID of the group that has started the communication service. Furthermore, the display processing unit 274 stores the user's user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. Note that if all of the other users belonging to the communication group have not yet participated in the communication service, only the user output image of the user is displayed on the communication screen 810.

Next, the transmitting unit 272 of the user's terminal device 2 transmits a start instruction for the communication service to the server device 3 via the terminal communication I/F 21 together with the user's user ID, information about the communication group, and/or destination information stored in the terminal storage device 22. The information about the communication group is the group ID, name, and user ID of each of the users belonging to the communication group. The destination information includes each of the user IDs of other users belonging to the communication group, excluding the user who operates the terminal device 2. Note that if the destination information is stored in the server storage device 32, the destination information does not need to be transmitted. The server receiving unit 331 of the server device 3 receives the start instruction, the user ID, information about the communication group, and the destination information via the server communication I/F 31. The server transmitting unit 332 of the server device 3 transmits the start instruction, the user ID, and information about the communication group via the server communication I/F 31 to the terminal device 2 of the other user indicated by the user ID included in the destination information.

The receiving unit 237 of the terminal device 2 of the other user receives from the server device 3 the start instruction, the user ID, and information about the communication group transmitted from the terminal device 2 of the user. The receiving unit 237 stores the group ID of the information about the communication group in the terminal storage device 22 as the group ID of the group that has started the communication service. Furthermore, the display processing unit 274 stores the transmitted user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. The display processing unit 274 of the terminal device 2 of the other user displays a notification screen (not shown) based on the received information about the communication group. In response to the user's selection of a specific object in the notification screen, the display processing unit 274 displays a group selection screen 710 (FIG. 7(b)) for participating in the communication service on the display device 23. This completes the start process. The display processing unit 274 of the terminal device 2 of the other user may include a participation button 805, described later, in the notification screen. This allows other users to immediately participate in the communication service without displaying the group selection screen 710.

The start process may be executed when the user operates the input device 24 to select the create button 603 on the group creation screen 600. That is, in response to the user's selection of the create button 603, a start process for a communication service in which a new group created on the group creation screen 600 is treated as a communication group may be executed. In this case, the information exchange screen 700 is not displayed.

The start process is not limited to the start process of a communication service in which a new group created on the group creation screen 600 is used as a communication group. For example, the start process of a communication service in which an already created group is used as a communication group may be executed. For example, FIG. 6(b) is a diagram showing an example of a group selection screen 610 for selecting a communication group created by any user. For example, the group selection screen 610 is displayed when a group selection object or the like on a home screen (not shown) displayed in response to execution of a control program stored in the terminal storage device 22 is selected by the user operating the input device 24.

The group selection screen 610 displayed on the user's terminal device 2 includes a group display area 611 for selecting one of multiple communication groups. The group display area 611 displays group information indicating each of the communication groups that have already been created and to which the user belongs. In the example shown in FIG. 6(b), the display processing unit 274 of the user's terminal device 2 displays thumbnail images of each of the users belonging to the communication group and the name of the group as group information. In the example shown in FIG. 6(b), thumbnail images of each of the other users among the belonging users are displayed, but thumbnail images of each of the belonging users may also be displayed. Note that if the communication group includes only one other user other than the user, the name of the one other user may be displayed instead of the name of the group, as shown in FIG. 6(b).

When a piece of group information included in the group display area 611 is selected by the user operating the input device 24, an information exchange screen 700 by users belonging to the communication group corresponding to the selected group information is displayed. The information exchange screen 700 includes information such as text or images entered by each of the users belonging to the communication group. Even in this case, an instruction to display the information exchange screen and information related to the communication group are transmitted via the server device 3 to the terminal device 2 of the other users belonging to the communication group, and the information exchange screen 700 is also displayed on the terminal device 2 of the other users.

When the user selects the start object 704 on the information exchange screen 700, a start process of a communication service by a communication group that exchanges information on the information exchange screen 700 is executed. When the start process is executed, the display processing unit 274 of the user's terminal device 2 displays a communication screen 810 (FIG. 8(b)) that can be displayed by the terminal device 2 of each user belonging to the communication group. In addition, the display processing unit 274 of the terminal device 2 of the other user displays a notification screen (not shown), and displays a group selection screen 710 (FIG. 7(b)) for participating in the communication service on the display device 23 in response to an operation of the other user on the notification screen. Note that even in this case, the display processing unit 274 of the terminal device 2 of the other user may include a join button 805 (described later) on the notification screen.

In this case, the start process may also be executed when a piece of group information is selected on the group selection screen 610 by the user operating the input device 24. That is, in response to the user's selection of a piece of group information, a start process for a communication service by the communication group corresponding to the selected group information may be executed. In this case, the information exchange screen 700 is not displayed.

Next, a method for a user who has not yet joined a communication service that has already started to join will be described. FIG. 7(b) is a diagram showing an example of a group selection screen 710 displayed on the display device 23 of the user's terminal device 2. For example, the group selection screen 710 is displayed in response to the selection of a specific object in the displayed notification screen. The group selection screen 710 may also be displayed when a group selection object or the like on a home screen displayed in response to the execution of a control program stored in the terminal storage device 22 is selected by the user operating the input device 24.

In the group display area 711 of the group selection screen 710, group information indicating a communication group that has started a communication service is displayed so as to be distinguishable from other group information. In the example shown in FIG. 7(b), the group information displayed at the top indicates the communication group that has started a communication service. For example, information indicating the number of users who are participating in the communication service is displayed superimposed on or near each thumbnail image of the users who belong to the communication group. In addition, a mark image 712 indicating the communication service is displayed near the name of the communication group.

When a user operates the input device 24 to select group information included in the group display area 711 that indicates a communication group that has started a communication service, an information exchange screen 800 for users belonging to the communication group that corresponds to the selected group information is displayed.

Figure 8 (a) is a diagram showing an example of an information exchange screen 800 displayed on the display device 23 of the user's terminal device 2. The information exchange screen 800 includes display areas 801a-801b that display information such as text or images entered by each of the users belonging to the communication group who have started a communication service, information (such as thumbnail images of character objects) 802a-802b indicating the users who entered the information displayed in each of the display areas 801a-801b, an input object 803 that allows the user to enter text or images to be displayed in the new display area 801, a service display area 804 that indicates a communication service that has already started, and a participation button 805 for participating in a communication service that has already started.

Thumbnail images of the character objects of three users participating in the communication service are displayed in the service display area 804 shown in FIG. 8(a). Information indicating that the communication service has started is displayed in the display area 801b shown in FIG. 8(a). The participation process is executed when the user selects the participation button 805. An example of the participation process is described below.

When the user selects the join button 805 by operating the input device 24, the display processing unit 274 of the user's terminal device 2 first stores the user ID of the user stored in the terminal storage device 22 in the terminal storage device 22 as the user ID of the user participating in the communication service related to the service display area 804. Next, the display processing unit 274 displays a communication screen 810 (FIG. 8(b)) including user output images of all users participating in the communication service.

Next, the transmitting unit 272 of the user's terminal device 2 transmits a participation instruction for the communication service to the server device 3 via the terminal communication I/F 21 together with the user's user ID, information about the communication group, and/or destination information stored in the terminal storage device 22. The information about the communication group is the group ID, name, and user ID of each of the users belonging to the communication group. The destination information includes each of the user IDs of other users belonging to the communication group, excluding the user operating the terminal device 2. The server receiving unit 331 of the server device 3 receives the participation instruction, the user ID, information about the communication group, and/or destination information via the server communication I/F 31. The server transmitting unit 332 of the server device 3 transmits the participation instruction, the user ID, and information about the communication group via the server communication I/F 31 to the terminal device 2 of the other user indicated by the user ID included in the destination information.

The receiving unit 237 of the other user's terminal device 2 receives from the server device 3 the participation instruction, user ID, and information related to the communication group transmitted from the user's terminal device 2. The receiving unit 237 stores the received user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. When the communication screen 810 is displayed on the terminal device, the display processing unit 274 of the other user's terminal device 2 displays the communication screen 810 including the user output images of all users participating in the communication service. This completes the participation process.

The participation process may be executed when group information is selected on the group selection screen 710 by the user operating the input device 24. In other words, in response to the user's selection of group information, a participation process may be executed for a communication service in which the communication group corresponding to the group information can participate. In this case, the information exchange screen 800 is not displayed.

The embodiment of starting or participating in a communication service is not limited to the above-mentioned example. For example, other starting processes different from the starting process in response to the user's selection of the starting object 704 may be executed. The other starting process is executed when a predetermined display object for specifying the start of a communication service is selected by the user on a predetermined screen such as a home screen. The predetermined display object may be displayed when a thumbnail image of the user's character object is selected, or may be included in a menu item on a predetermined screen. The other starting process may be executed automatically when a predetermined starting condition is satisfied. For example, the predetermined starting condition is that the current time is a predetermined time, that a predetermined time has passed since the end of the previous communication service, that the number of mutual followers of the users exceeds a predetermined number, etc. An example of the other starting process is described below.

First, the transmission unit 272 of the user's terminal device 2 transmits an instruction to start the communication service to the server device 3 via the terminal communication I/F 21 together with the user ID of the user stored in the terminal storage device 22 and/or the destination information. The transmission unit 272 also stores the user ID of the user in the terminal storage device 22 as the user ID of the user participating in the communication service. The destination information includes the user IDs of one or more other users who have a predetermined relationship with the user. For example, the destination information may include the user IDs of one or more other users who are mutual followers of the user. The destination information may also include the user IDs of other users who have a mutual follow relationship with a predetermined user who is a mutual follower of the user, instead of or in addition to the user IDs of one or more other users who are mutual followers of the user.

The server receiving unit 331 of the server device 3 receives the start instruction, the user ID, and/or the destination information via the server communication I/F 31. The server transmitting unit 332 of the server device 3 transmits the start instruction, the user ID, and the destination information to the terminal device 2 of the other user indicated by the user ID included in the destination information via the server communication I/F 31. The receiving unit 237 of the terminal device 2 of the other user receives the start instruction, the user ID, and the destination information transmitted from the terminal device 2 of the user from the server device 3. The receiving unit 237 stores the transmitted user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. The display processing unit 274 of the terminal device 2 of the other user displays a notification screen (not shown) on the display device 23 based on the received user ID. The notification screen includes information encouraging the user indicated by the received user ID to participate in the communication service, a participation button, etc.

When the other user operates the input device 24 to select the join button in the notification screen, the display processing unit 274 of the other user's terminal device 2 stores the user ID of the other user stored in the terminal storage device 22 in the terminal storage device 22 as the user ID of the user participating in the communication service related to the service display area 804. Next, the display processing unit 274 displays a communication screen 810 (FIG. 8(b)) including user output images of all users participating in the communication service.

Next, the transmitting unit 272 of the terminal device 2 of the other user transmits a participation instruction for the communication service to the server device 3 via the terminal communication I/F 21 together with the user ID and/or destination information of the other user stored in the terminal storage device 22. The destination information is obtained by deleting the user ID of the other user of the terminal device 2 transmitting the destination information from the user ID included in the destination information previously received by the receiving unit 237, and adding the user ID received by the receiving unit 237 at the same time. The server receiving unit 331 of the server device 3 receives the participation instruction, the user ID, and/or the destination information via the server communication I/F 31. The server transmitting unit 332 of the server device 3 transmits the participation instruction, the user ID, and the destination information via the server communication I/F 31 to the terminal device 2 of the other user identified by the user ID included in the destination information.

The receiving unit 237 of the terminal device 2 of the user to whom the participation instruction is to be sent receives from the server device 3 the participation instruction, user ID, and destination information sent from the user's terminal device 2. The receiving unit 237 stores the received user ID in the terminal storage device 22 as the user ID of the user currently participating in the communication service. When the communication screen 810 is displayed on the terminal device 2, the display processing unit 274 of the terminal device 2 displays the communication screen 810 including user output images of all users currently participating in the communication service. This completes the series of participation processes.

Note that the embodiment of starting or participating in a communication service with other users who have a predetermined relationship with the user is not limited to the above-mentioned example. For example, when a user selects a predetermined display object for specifying the start of a communication service, a selection screen including information indicating other users who are mutual followers of one or more users may be displayed. In this case, when the user selects information indicating any one of the other users, another start process based on a start instruction to start communication with the selected other user may be executed. Furthermore, in this case, an information exchange screen with the selected other user may be displayed, and a start button in the information exchange screen may be selected to execute another start process based on a start instruction to start communication with the other user.

In conventional information systems, a user cannot use a simple interface to initiate and/or instruct participation in communication with other users who have a specific relationship with the user, such as the user's mutual followers. In the information system 1 disclosed in this embodiment, the above-mentioned other initiation process is executed, thereby improving the user interface related to initiating and/or instructing participation in communication, and reducing the communication load between the server device 3 and the terminal device 2.

Figure 8 (b) is a diagram showing an example of a communication screen 810 displayed on the display device 23 of the terminal device 2. The communication screen 810 is a communication screen of a communication group that includes the user operating the terminal device 2 as a member user. Hereinafter, the user operating the terminal device 2 will be referred to as user A, and users other than user A who are participating in the communication service corresponding to the communication screen 810 will be referred to as user B1, user B2, user B3, .... Note that user B1, user B2, user B3, ... may be collectively referred to as user B.

The communication screen 810 includes at least a user output image 812a including a character object 811a of user A. In the example shown in FIG. 8B, there are three users (user B1, user B2, and user B3) other than user A who are participating in the communication service. That is, the communication screen 810 includes user output images 812b1, 812b2, and 812b3 including character objects 811b1, 811b2, and 811b3 of user B1, user B2, and user B3, respectively. Hereinafter, the character objects 811b1, 811b2, and 811b3 may be collectively referred to as character objects 811b, and the user output images 812b1, 812b2, and 812b3 may be collectively referred to as user output images 812b. Note that user B is another user for user A, and the user output image 812b of user B displayed on user A's terminal device 2 is another user output image.

When there are no users other than user A participating in the communication service, only user output image 812a is included in the communication screen 810, and each time user B participates in the communication service, user output image 812b of user B is newly included in the communication screen 810. There is no upper limit to the number of users who can participate in the communication service, and for example, when nine users B other than user A are participating in the communication service, user output images 812b1, 812b2, 812b3, ... 812b9 of each of the nine users B are included in the communication screen 810 along with user output image 812a of user A.

The character object 811a is an animation (moving image) of the character object of user A generated by the display processing unit 274 based on the motion data of user A generated by the generation unit 271 and the character object and the use object associated with the user ID of user A in the user table T1. The motion data of user A generated by the generation unit 271 is the same as the motion data included in the output information transmitted to the terminal device 2 of user B. The output information of user A includes the voice data of user A together with the motion data of user A. When the motion data of user A generated by the generation unit 271 includes the face motion data of user A, a user output image 812a including a character object 811a whose facial expression changes in synchronization with the movement of the facial expression of user A is displayed. For example, when user A is making a sound, a user output image 812a including a character object 811a whose lips are moving is displayed. Furthermore, if the motion data of user A generated by the generation unit 271 includes body motion data of user A, a user output image 812a including a character object 811a whose corresponding body part moves in synchronization with the body movement of user A is displayed.

The transmitting unit 272 of the terminal device 2 of the user A transmits the output information of the user A together with the destination information (the user ID of each user B (user B1 to user B3) included in the communication group) to the server device 3 via the terminal communication I/F 21. Next, the server receiving unit 331 of the server device 3 receives the output information of the user A transmitted from the terminal device 2 of the user A via the server communication I/F 31. Next, the server transmitting unit 332 of the server device 3 refers to the received user ID of the user B (user B1 to user B3) and transmits the output information of the user A to the terminal device 2 of the user B (user B1 to user B3) via the server communication I/F 31. Then, when the output information is received by the receiving unit 273 of the terminal device 2 of the user B via the terminal communication I/F 21, the display processing unit 274 of the terminal device 2 of the user B displays the user output image 812a including the character object 811a based on the motion data of the user A indicated by the user ID included in the output information.

The character object 811b is an animation (moving image) of the character object of user B generated by the display processing unit 274 based on the motion data included in the output information of user B received by the receiving unit 273 and the character object associated with the user ID of user B in the user table T1. When the motion data of user B received by the receiving unit 273 includes face motion data of user B, a user output image 812b including a character object 811b whose facial expression changes in synchronization with the movement of the facial expression of user B is displayed. For example, when user B utters a sound, a user output image 812b including a character object 811b whose lips are moving is displayed. Also, when the motion data of user B generated by the generating unit 271 includes body motion data of user B, a user output image 812b including a character object 811b whose corresponding part moves in synchronization with the movement of the body of user B is displayed. If the output information of user B received by the receiving unit 273 includes voice data of user B, the display processing unit 274 displays the user output image 812b, and the voice of user B generated based on the voice data is output by the voice output unit 275.

In this way, by displaying the communication screen 810 including the user output image 812a and the user output image 812b, user A can hear the voice of user B while looking at the character object 811b of user B, which appears to be making a sound. In addition, the character object and voice data of user A are output by the terminal device 2 of user B. This allows users A and B participating in the communication service to communicate via the character objects.

The communication screen 810 includes a selection event button 813 and a change button 814. In the example shown in FIG. 8B, the selection event button 813 and the change button 814 are arranged in the display area of the user output image 812a, but may be displayed outside the display area of the user output image 812a. In addition, the display of the selection event button 813 and the change button 814 may be controlled by the display processing unit 274 so that they are displayed for a predetermined time when a predetermined display condition is satisfied. The predetermined display condition is, for example, that a predetermined time has elapsed since the communication service related to the communication screen 810 was started. In addition, the predetermined display condition may be that the number of users participating in the communication service related to the communication screen 810 exceeds a predetermined number (for example, two people). In addition, the predetermined display condition may be that a specific user has participated in the communication service related to the communication screen 810. A specific user is a user whose specific parameter value is equal to or greater than a predetermined value. When the specific parameter value owned by a user is the number of followers of the user, a specific user is a user whose specific number (for example, 30 people) is equal to or greater than a predetermined number. Furthermore, in the case where the specific parameter value of the user is the number of times the selected event has been performed by the user, the specific user is a user who has performed the selected event a predetermined number of times or more. In the case where the specific parameter value of the user is the number of times the user has participated in a communication service performed by the user, the specific user is a user who has participated in a communication service a predetermined number of times or more. In the case where the specific parameter value of the user is the total time that the communication service in which the user participated was used, the specific user is a user who has used the communication service in which the user participated for a total time of a predetermined number of times or more. In the case where the specific parameter value of the user is a parameter value such as a level related to a communication service, the specific user is a user whose parameter value such as a level related to a communication service is a predetermined value or more. The level related to a communication service increases in response to the user's participation in the communication service. In the case where the specific parameter value of the user is a parameter value such as a level related to a distribution event, the specific user is a user whose parameter value such as a level related to a distribution event is a predetermined value or more. The level related to a distribution event increases every time the user becomes a distribution user and distributes a distribution image. The level related to a distribution event may also increase every time the user becomes a viewing user and views a distribution image. The specified display condition may also be that two or more of the above conditions are satisfied.

The selected event button 813 is an object image such as a button object for instructing the display of a selection screen 900 for selecting a selected event. When the selected event button 813 is selected by user A operating the input device 24, the selection screen 900 is displayed. Details of the selection screen 900 will be described later.

The change button 814 is an object image such as a button object for attaching a new attachment object to the character object 811a. If an attachment object of the same type or in the same location as the new attachment object is already attached to the character object 811a, the already attached attachment object is changed to the new attachment object. When the change button 814 is selected by user A operating the input device 24, a communication screen 1100 (FIG. 11(b)) is displayed. The display of the communication screen 1100 may be executed when the character object 811a or the user output image 812a is selected by user A operating the input device 24. Details of the communication screen 1100 will be described later.

FIG. 9(a) is a diagram showing an example of a selection screen 900 displayed on the display device 23 of the terminal device 2. The selection screen 900 includes event start buttons 901 corresponding to multiple types of selected events. A selected event is an event in which at least one specific object is selected from an object group consisting of multiple specific objects according to a predetermined selection method in response to an instruction from the user, and presented to the user. A selected event corresponding to the event start button 901 selected by user A from the multiple event start buttons 901 is provided to user A. The event start button 901 is associated with a selected event ID of the corresponding selected event.

The object groups used in each of the multiple types of selection events are different from each other. For example, the multiple specific objects that make up the object group used in one selection event are not included in the object group used in the other selection events. Some of the multiple specific objects that make up the object group used in one selection event may be included in the object group used in the other selection event. Also, the multiple specific objects that make up the object group used in one selection event may be the same as the multiple specific objects that make up the object group used in the other selection event. The number of specific objects selected in one selection event may be different from the number of specific objects selected in the other selection event.

In the example shown in FIG. 9(a), four event start buttons 901a to 901d corresponding to four types of selected events are included in the selection screen 900. The number of event start buttons 901 displayed on the selection screen 900 is not limited to four. If the display processing unit 274 of the terminal device 2 cannot include all types of event start buttons 901 on the selection screen 900, the event start buttons 901 may be displayed in a scrollable manner. In this case, in response to a swipe operation by user A (for example, an operation in which a finger is moved from the bottom to the top of the screen while touching the screen), the event start buttons 901 that are not currently displayed on the selection screen 900 are scrolled and displayed on the selection screen 900.

When user A operates the input device 24 to select one of the multiple event start buttons 901, the input device 24 inputs an instruction to display a selection instruction screen to the terminal processing device 27. The input device 24 displays the selection instruction screen and inputs a selected event ID associated with the selected event start button 901 to the terminal processing device 27. When user A operates the input device 24 to select one of the event start buttons 901, the display of the selection screen 900 ends, and a communication screen 910 and a selection instruction screen 911 are displayed.

During the period when the selection screen 900 is displayed, the transmission unit 272 of the terminal device 2 of the user A may include information for displaying the selection screen 900 in the output information transmitted to the terminal device 2 of the user B (users B1 to B3) via the server device 3. The information for displaying the selection screen 900 is video information for displaying a video image of the selection screen 900 displayed on the display device 23 of the terminal device 2 of the user A. The information for displaying the selection screen 900 may be image information for displaying a still image of the selection screen 900 displayed on the display device 23 of the terminal device 2 of the user A. In this case, the output information may not include character video data including the motion data of the user A. As a result, when the terminal device 2 of the user B (users B1 to B3) receives the output information of the user A via the server device 3, it becomes possible to display the selection screen 900 according to the operation of the user A, and the user B can visually confirm the operation status of the user A on the selection screen 900. The information for displaying the selection screen 900 may be an object ID for identifying the event start buttons 901a to 901d. In this case, when the terminal device 2 of user B (users B1 to B3) receives the output information of user A via the server device 3, it may extract the drawing data of the event start buttons 901a to 901d corresponding to the object ID included in the output information from the terminal storage device 22, and draw the selection screen 900.

Figure 9 (b) is a diagram showing an example of a communication screen 910 and a selection instruction screen 911 displayed on the display device 23 of the terminal device 2. As shown in Figure 9 (b), the selection instruction screen 911 is displayed superimposed on the communication screen 910. The selection instruction screen 911 may be displayed on the entire screen of the display device 23, in which case the communication screen 910 is not displayed.

A user output image 812a including user A's character object 811a is displayed enlarged on the communication screen 910. A user output image 913 including character objects 811b1, 811b2, and 811b3 of users B1, B2, and B3 is displayed superimposed on the user output image 812a. When the communication screen 910 is displayed, voice data of users B1, B2, and B3 is output. Note that the user output image 913 does not have to be displayed on the communication screen 910. Even in this case, voice data of users B1, B2, and B3 may be output.

The selection instruction screen 911 includes information about the selected event indicated by the selected event ID associated with the event start button 901 selected on the selection screen 900 (e.g., the name of the selected event, etc.), and a selection instruction button 912. The selection instruction screen 911 shown in FIG. 9(b) includes two types of selection instruction buttons 912a and 912b. When the selection instruction button 912a is selected by user A, the number of selected object IDs of specific objects selected in the selected event is "1". When the selection instruction button 912b is selected by user A, the number of selected object IDs of specific objects selected in the selected event is "10". The number of selection instruction buttons 912 included in the selection instruction screen 911 may be one or three or more, and the number of selected object IDs of specific objects selected in the selected event corresponding to the selection instruction button 912 is not limited to "1" or "10", and may be any number.

During the period in which the communication screen 910 and the selection instruction screen 911 are displayed, the transmission unit 272 of the terminal device 2 of user A may include information for displaying the selection instruction screen 911 in the output information transmitted to the terminal device 2 of user B (users B1 to B3) via the server device 3. The information for displaying the selection instruction screen 911 is video information for displaying a video image of the selection instruction screen 911 displayed on the display device 23 of the terminal device 2 of user A. The information for displaying the selection instruction screen 911 may be image information for displaying a still image of the selection instruction screen 911 displayed on the display device 23 of the terminal device 2 of user A. As a result, when the terminal device 2 of user B (users B1 to B3) receives the output information of user A via the server device 3, it becomes possible to display the communication screen 910 and the selection instruction screen 911 according to the operation of user A, and user B can visually confirm the operation status of user A on the selection instruction screen 911. The information for displaying the selection instruction screen 911 may be a selected event ID associated with the event start button 901 selected by user A and an object ID for identifying the two types of selection instruction buttons 912a and 912b. In this case, when the terminal device 2 of user B (users B1 to B3) receives the output information of user A via the server device 3, the terminal device 2 of user B may extract information on the selected event corresponding to the selected event ID included in the output information (e.g., the name of the selected event, etc.) and drawing data of the selection instruction buttons 912a and 912b corresponding to the object ID included in the output information from the terminal storage device 22 to draw the selection instruction screen 911. Note that, during the period when the selection result screens 1000 and 1010 are displayed on the terminal device 2 of user A, information for displaying the selection result screens 1000 and 1010 may be transmitted to the terminal device 2 of user B (users B1 to B3) via the server device 3. As a result, the selection result screens 1000 and 1010 are displayed on the terminal device 2 of user B (users B1 to B3).

Hereinafter, the two types of selection instruction buttons 912a and 912b included in the selection instruction screen 911 displayed on the terminal device 2 of user A will be collectively referred to as selection instruction buttons 912, and the functions common to both will be described. First, when the selection instruction button 912 is selected by user A operating the input device 24, the input device 24 inputs a selection instruction to the terminal processing device 27 together with the selection event ID input together with the instruction to display the selection instruction screen. The transmission unit 272 of the terminal device 2 of user A transmits information indicating the input selection instruction and the selection event ID together with user A's user ID and destination information to the server device 3 via the terminal communication I/F 21. In the example shown in FIG. 9(b), when the selection instruction button 912a or the selection instruction button 912b is selected, information indicating the number of selections corresponding to the selected selection instruction button 912 is transmitted together with information indicating the selection instruction.

The server receiving unit 331 of the server device 3 receives information indicating a selection instruction, the selected event ID, user A's user ID, and destination information via the server communication I/F 31. The server receiving unit 331 inputs the received information indicating a selection instruction and the selected event ID to the selection unit 333. If information indicating the number of selections is transmitted together with the information indicating a selection instruction, the server receiving unit 331 receives the information indicating the number of selections together with the information indicating the selection instruction, and inputs the received information indicating the number of selections to the selection unit 333.

The selection unit 333 of the server device 3 extracts an object group ID associated with the input selected event ID from among the multiple object group IDs stored in the server storage device 32. Next, the selection unit 333 selects at least one object ID of a specific object from among the multiple object IDs of specific objects associated with the extracted object group ID according to a predetermined selection method. Note that, when information indicating the number of selections is input, the selection unit 333 selects the object IDs of specific objects from among the object IDs of the multiple specific objects associated with the extracted object ID by the number of selections. For example, when information indicating the number of selections "1" is input, the selection unit 333 selects the object ID of one specific object from among the object IDs of the multiple specific objects associated with the extracted object ID. Also, for example, when information indicating the number of selections "10" is input, the selection unit 333 selects the object IDs of ten specific objects from among the object IDs of the multiple specific objects associated with the extracted object ID. Then, the selection unit 333 inputs the object IDs of the selected specific objects to the server transmission unit 332 as information indicating the selection result.

The server transmitting unit 332 of the server device 3 obtains information indicating the selection result from the selecting unit 333. The server transmitting unit 332 also obtains the user ID of user A and the destination information received from the server receiving unit 331 along with the information indicating the selection instruction. Next, the server transmitting unit 332 transmits the information indicating the selection result and the user ID of user A via the server communication I/F 31 to the terminal device 2 of user A and the terminal devices 2 of users B1 to B3 indicated by the user IDs included in the destination information.

The receiving unit 237 of user A's terminal device 2 receives the information indicating the selection result and user A's user ID transmitted from the server device 3 via the terminal communication I/F 21. The receiving unit 237 of user A's terminal device 2 stores the object ID of the specific object received as the information indicating the selection result in user table T1 as an object owned by user A, in association with user A's user ID.

The display processing unit 274 of the terminal device 2 of user A also refers to the object table T2 to extract image information and/or names associated with the object IDs of the specific objects received as information indicating the selection results. The display processing unit 274 of the terminal device 2 of user A displays the selection result screen 1000 or the selection result screen 1010 on the display device 23 based on the read image information and/or names.

Fig. 10(a) is a diagram showing an example of a selection result screen 1000 displayed on the display device 23 of user A's terminal device 2, and Fig. 10(b) is a diagram showing an example of a selection result screen 1010 displayed on the display device 23 of user A's terminal device 2. The selection result screen 1000 is displayed when the object ID of one specific object is received as information indicating the selection result, and the selection result screen 1010 is displayed when the object IDs of multiple specific objects are received as information indicating the selection result.

The selection result screen 1000 shown in FIG. 10(a) includes information 1001 indicating one specific object, a use button 1002, and a cancel button 1003. The information 1001 indicating one specific object is image information and/or a name associated with the object ID of one specific object received as information indicating the selection result. The use button 1002 is an object image such as a button object for instructing the use of one specific object that is the selection result. When the use button 1002 is selected by user A operating the input device 24, the above-mentioned use process is executed. The cancel button 1003 is an object image such as a button object for terminating the display of the selection result screen 1000 and redisplaying the communication screen 810 (FIG. 8(b)).

When the use button 1002 is selected by user A operating the input device 24, the display processing unit 274 of the terminal device 2 of user A displays a communication screen 1100 on the display device 23. FIG. 11(a) is a diagram showing an example of the communication screen 1100 displayed on the display device 23 of the user's terminal device 2. The communication screen 1100 includes a user output image 812a including a character object 811a of user A, and user output images 812b1, 812b2, and 812b3 including character objects 811b1, 811b2, and 811b3 of users B1, B2, and B3, respectively, similar to the communication screen 810. The communication screen 1100 also includes a select event button 813 and a change button 814, similar to the communication screen 810.

When user A operates the input device 24 to select the use button 1002, the above-mentioned use process is executed based on the object ID of one specific object received as information indicating the selection result. Execution of the use process results in a user output image 812a being displayed, including user A's character object 811a modified by image information associated with the object ID of one specific object received as information indicating the selection result. In the example shown in FIG. 11(a), the character object 811a is modified to a character object 811a in which the wearable object "hat" 1101 is worn on the head, and a user output image 812a being displayed including the modified character object 811a.

The selection result screen 1010 shown in FIG. 10(b) includes information 1011 indicating multiple specific objects, a selected object 1012, a use button 1013a, a use button 1013b, and a cancel button 1014. The information 1011 indicating multiple specific objects is image information and/or names associated with each of the object IDs of the multiple specific objects received as information indicating the selection result.

The selection object 1012 is an operation object for selecting the object ID of a plurality of specific objects. In the example shown in FIG. 10(b), the display processing unit 274 of the user's terminal device 2 displays check box objects corresponding to information indicating each specific object as the selection object 1012. For example, when the selection object 1012 is selected by the user's operation of the input device 24, the selection object 1012 is displayed with a "check" mark. In this case, the object ID of the specific object corresponding to the selection object 1012 displayed with a "check" mark is selected. Also, when the selection object 1012 displayed with a "check" mark is selected by the user's operation of the input device 24, the selection object 1012 is displayed without a "check" mark. In this case, the object ID of the specific object corresponding to the selection object 1012 displayed without a "check" mark is not selected. Each of the selection objects 1012 is associated with the object ID of the corresponding specific object.

The use button 1013a and the use button 1013b are object images such as button objects for instructing the use of a specific object that is a selection result. When the use button 1013a is selected by the user A operating the input device 24, a use process based on the object ID of the specific object selected by the user A's operation on the selected object 1012 is executed. By executing the use process, the character object 811a is changed based on image information associated with each of the object IDs selected by the user A among the object IDs of the multiple specific objects received as information indicating the selection result. Then, on the communication screen 1100, a user output image 812a including the changed character object 811a is displayed. When the use button 1013b is selected by the user A operating the input device 24, a use process based on the object IDs of all the specific objects received as information indicating the selection result is executed. By executing the use process, the character object 811a is changed based on image information associated with each of the object IDs of all the specific objects received as information indicating the selection result. Then, on the communication screen 1100, a user output image 812a including the changed character object 811a is displayed. The cancel button 1014 is an object image such as a button object for terminating the display of the selection result screen 1010 and redisplaying the communication screen 810 (Figure 8 (b)).

The use instruction for executing the use process is not limited to being input in response to a user's operation on the selection result screens 1000 and 1010. For example, the use instruction may be input in response to a user's operation on the selection screen 1110 for selecting an object to be used from among the objects owned by the user.

FIG. 11(b) is a diagram showing an example of a selection screen 1110 displayed on the display device 23 of the terminal device 2. The selection screen 1110 shown in FIG. 11(b) is displayed superimposed on the communication screen 910. The selection screen 1110 may be displayed after the display of the communication screen 910 is terminated.

The selection screen 1110 includes a plurality of type objects 1111 indicating the type of the attachment object, and a plurality of attachment candidate objects 1112 indicating the attachment object. When user A operates the input device 24 to select one of the plurality of type objects 1111, the type of the attachment object corresponding to the selected type object 1111 is identified. When the type of the attachment object is identified, attachment candidate objects 1112 indicating the identified type of attachment object among the objects owned by user A are displayed on the selection screen 1110. Here, when user A's terminal device 2 receives information indicating the selection result, the object ID of the specific object received as the information indicating the selection result is stored in the user table T1 as an object owned by user A in association with user A's user ID. Therefore, the attachment candidate objects 1112 include attachment candidate objects 1112 indicating the specific object selected by the selection event.

When user A operates the input device 24 to select from among the multiple attachment candidate objects 1112 a candidate attachment object 1112 that corresponds to the object ID of the specific object selected by the selection event, the input device 24 inputs a use instruction to the terminal processing device 27, and the above-mentioned use process based on the object ID of the selected specific object is executed. As a result, the display processing unit 274 displays the communication screen 1100 including a user output image 812a including a character object 811a wearing the specific object selected by the selection event.

Information indicating the selection result of the selected event in response to the selection instruction by user A is transmitted to the terminal device 2 of user A, and is also transmitted to the terminal device 2 of each of user B (users B1 to B3). User B's character object 811b may be changed based on the information indicating the selection result of the selected event in response to the selection instruction by user A. Various processes related to the change of user B1's character object 811b1, which are executed in user B1's terminal device 2, are described below. The various processes executed in user B1's terminal device 2 are also executed in the same way in user B2's terminal device 2 and user B3's terminal device 2.

The receiving unit 237 of the terminal device 2 of user B1 receives, via the terminal communication I/F 21, information indicating the selection result of the selected event in response to the selection instruction by user A and the user ID of user A, which are sent from the server device 3. The receiving unit 237 of the terminal device 2 of user B1 stores the object ID of the specific object received as information indicating the selection result in the terminal storage device 22 until the communication service in which user B1 is currently participating ends. Note that the object ID of the specific object is not stored as an object owned by user B1. This allows user B1 to temporarily use the selection result of the selected event in response to the selection instruction by user A.

Next, the display processing unit 274 of the terminal device 2 of user B1 refers to the object table T2 to extract image information and/or name associated with the object ID of the specific object received as information indicating the selection result of the selection event in response to the selection instruction by user A. The display processing unit 274 of the terminal device 2 of user B1 displays the selection result screen 1200 or the selection result screen 1210 on the display device 23 based on the read image information and/or name.

Fig. 12(a) is a diagram showing an example of a selection result screen 1200 displayed on the display device 23 of user B1's terminal device 2, and Fig. 12(b) is a diagram showing an example of a selection result screen 1210 displayed on the display device 23 of user B1's terminal device 2. The selection result screen 1200 is displayed when the object ID of one specific object is received as information indicating the selection result of a selection event in response to a selection instruction by user A. Also, the selection result screen 1210 is displayed when the object IDs of multiple specific objects are received as information indicating the selection result of a selection event in response to a selection instruction by user A.

The selection result screen 1200 shown in FIG. 12(a) includes information 1201 indicating one specific object, a temporary use button 1202, and a cancel button 1203. The information 1201 indicating one specific object is image information and/or name associated with the object ID of one specific object received as information indicating the selection result of a selection event in response to a selection instruction by user A. The temporary use button 1202 is an object image such as a button object for instructing temporary use of one specific object that is the selection result of a selection event in response to a selection instruction by user A. Temporary use of a specific object means using a specific object that is not stored as an object owned by user B1 until a specified end condition is met.

The predetermined termination condition may be, for example, that user A performs an operation to instruct to terminate temporary use and obtains a temporary use termination instruction in response to the operation. The predetermined termination condition may also be that user A's participation in the communication service is terminated in response to an instruction from user A. For example, the termination of user A's participation in the communication service terminates the transmission of output information from user A and the reception of output information from each of users B1 to B3, and the predetermined condition is satisfied. The predetermined termination condition may also be that any of users B1 to B3's participation in the communication service is terminated in response to an instruction from any of users B1 to B3. For example, the termination of user B1's participation in the communication service terminates the transmission of output information from user B1 and the reception of output information from each of users A, B2, and B3, and the predetermined condition is satisfied. The predetermined termination condition may also be that any of users B1 to B3's participation in the communication service is forcibly terminated in response to an instruction from user A. The predetermined termination condition may also be that user A's participation in the communication service is forcibly terminated in response to an instruction from any of users B1 to B3.

The predetermined end condition may be that a predetermined time has elapsed since the provisional use of a specific object. The predetermined end condition may also be that a selection event has been started by user A during the provisional use of a specific object. The predetermined end condition may also be that a selection event has been started by any of users B1 to B3 during the provisional use of a specific object.

When the temporary use button 1202 is selected by user B1 operating the input device 24, the same process as the use process described above is executed. The cancel button 1203 is an object image such as a button object for ending the display of the selection result screen 1200 and redisplaying the communication screen 810.

The provisional use button 1202 may be controlled so that selection of the provisional use button 1202 is permitted only when a predetermined use condition is satisfied. The predetermined use condition is, for example, that the number of times that a process for using a specific object has been executed (the number of provisional uses) in response to the selection of the provisional use button 1213a by the operation of the user of the terminal device displaying the selection result screen 1200 (user B1 in the example shown in FIG. 12(a)) is less than a predetermined number of times. The predetermined use condition may be that a predetermined time (e.g., 10 minutes) has elapsed since the start of a communication service in which the user of the terminal device displaying the selection result screen 1200 (user B1 in the example shown in FIG. 12(a)) is participating. The predetermined use condition may be a period from the start of a communication service in which the user of the terminal device displaying the selection result screen 1200 (user B1 in the example shown in FIG. 12(a)) is participating until a predetermined time (e.g., 10 minutes) has elapsed. The predetermined trial condition may be a period until a selection event is started again by user A.

For example, when user B1 operates the input device 24 to select the temporary use button 1202, the display processing unit 274 of user B1's terminal device 2 displays the communication screen 1100 on the display device 23. Like the communication screen 810, the communication screen 1100 includes a user output image 812a including a character object 811a of user A, and user output images 812b1, 812b2, and 812b3 including character objects 811b1, 811b2, and 811b3 of users B1, B2, and B3, respectively. Also, like the communication screen 810, the communication screen 1100 includes a select event button 813 and a change button 814.

When user B1 operates the input device 24 to select the temporary use button 1202, the above-mentioned use process is executed based on the object ID of one specific object received as information indicating the selection result of the selection event in response to the selection instruction by user A. Execution of the use process results in displaying a user output image 812b1 including user B1's character object 811b1 modified by image information associated with the object ID of one specific object received as information indicating the selection result in response to the selection instruction by user A.

The selection result screen 1210 shown in FIG. 12(b) includes information 1211 indicating multiple specific objects, a selected object 1212, a temporary use button 1213a, a temporary use button 1213b, and a cancel button 1214. The information 1211 indicating multiple specific objects is image information and/or names associated with each of the object IDs of the multiple specific objects received as information indicating the selection result of a selection event in response to a selection instruction by user A.

The selection object 1212 is an operation object for selecting the object ID of a plurality of specific objects. In the example shown in FIG. 12(b), the display processing unit 274 of the user's terminal device 2 displays check box objects corresponding to information indicating each specific object as the selection object 1212. For example, when the selection object 1212 is selected by the user's operation of the input device 24, the selection object 1212 is displayed with a "check" mark. In this case, the object ID of the specific object corresponding to the selection object 1212 displayed with a "check" mark is selected. Also, when the selection object 1212 displayed with a "check" mark is selected by the user's operation of the input device 24, the selection object 1212 is displayed without a "check" mark. In this case, the object ID of the specific object corresponding to the selection object 1212 displayed without a "check" mark is not selected. Each of the selection objects 1212 is associated with the object ID of the corresponding specific object.

The temporary use button 1213a and the temporary use button 1213b are object images such as button objects for instructing temporary use of a specific object that is a selection result of a selection event in response to a selection instruction by user A. When the temporary use button 1213a is selected by user B1 operating the input device 24, a use process is executed based on the object ID of the specific object that is result information of the selection event in response to the selection instruction by user A. By executing the use process in response to the selection of the temporary use button 1213a, the character object 811b1 is changed based on image information associated with each of the object IDs selected by user B1 from the object IDs of multiple specific objects received as information indicating the selection result by user A. Then, a user output image 812b1 including the changed character object 811b1 is displayed on the communication screen 1100.

When the temporary use button 1213b is selected by user A operating the input device 24, a use process is executed based on the object IDs of all specific objects received as information indicating the selection result of the selection event in response to the selection instruction by user A. By executing the use process in response to the selection of the temporary use button 1213b, the character object 811b1 is changed based on the image information associated with each of the object IDs of all specific objects received as information indicating the selection result of the selection event in response to the selection instruction by user A. Then, on the communication screen 1100, a user output image 812b1 including the changed character object 811b1 is displayed. The cancel button 1214 is an object image such as a button object for ending the display of the selection result screen 1210 and redisplaying the communication screen 810 (FIG. 8(b)).

When information indicating the selection result of a selected event in response to a selection instruction by user A is received in the terminal device 2 of user B (users B1 to B3), the same selected event as the selected event in response to the selection instruction by user A may be provided to user B (users B1 to B3). Below, various processes related to providing the same selected event, which are executed in the terminal device 2 of user B1, are described. The various processes executed in the terminal device 2 of user B1 are also executed in the terminal device 2 of user B2 and the terminal device 2 of user B3 in the same way.

When object IDs of multiple specific objects are received as information indicating the selection result of a selection event in response to a selection instruction by user A, multiple selection result screens 1200 may be displayed sequentially instead of displaying the selection result screen 1210. The selection result screen 1200 displayed first includes information 1201 indicating one specific object among the multiple specific objects, a temporary use button 1202, and a cancel button 1203. When the temporary use button 1202 is selected by user B1 operating the input device 24, the same process as the use process described above is executed, and the selection result screen 1200 related to the next specific object among the multiple specific objects is displayed. The cancel button 1203 is an object image such as a button object for terminating the display of the selection result screen 1200 and displaying the selection result screen 1200 related to the next specific object among the multiple specific objects. When the selection result screens 1200 related to all of the multiple specific objects are displayed, when the temporary use button 1202 of the last displayed selection result screen 1200 is selected, the same process as the use process is executed, and the selection result screen 1300 is displayed. Furthermore, when the selection result screen 1200 relating to all of the specified objects is displayed, if the cancel button 1203 on the last displayed selection result screen 1200 is selected, the selection result screen 1300 is displayed.

Fig. 13(a) is a diagram showing an example of a selection result screen 1300 displayed on the display device 23 of user B1's terminal device 2, and Fig. 13(b) is a diagram showing an example of a selection result screen 1310 displayed on the display device 23 of user B1's terminal device 2. The selection result screen 1300 is displayed when the object ID of one specific object is received as information indicating the selection result of a selection event in response to a selection instruction by user A. Also, the selection result screen 1310 is displayed when the object IDs of multiple specific objects are received as information indicating the selection result of a selection event in response to a selection instruction by user A.

The selection result screen 1300 is a screen in which two types of selection instruction buttons 1301a and 1301b are further added to the selection result screen 1200 (FIG. 12(a)). The selection result screen 1310 is a screen in which two types of selection instruction buttons 1301a and 1301b are further added to the selection result screen 1210 (FIG. 12(b)). The two types of selection instruction buttons 1301a and 1301b have the same functions as the two types of selection instruction buttons 912a and 912b of the selection instruction screen 911 (FIG. 9(b)). Note that the terminal device 2 or the server device 3 may control the selection instruction buttons 912a and 912b to be selectable when a specific object (e.g., a "ticket object") owned by the user is consumed or when the user pays a fee.

For example, when user B1 operates the input device 24 to select the selection instruction button 1301a, the terminal device 2 of user B1 transmits information indicating the selection instruction, the selected event ID (the selected event ID of the selected event corresponding to the selection instruction by user A), the user ID of user B1, and the destination information (the user IDs of users A, B2, and B3) to the server device 3. In the server device 3, the object ID of one specific object is selected according to a predetermined selection method from the object IDs of multiple specific objects associated with the received selected event ID, and the object ID of the selected specific object is transmitted as information indicating the selection result to the terminal device 2 of user B1 and the terminal device 2 of the user indicated by the user ID included in the destination information.

When user B1 operates the input device 24 to select the selection instruction button 1301b, the terminal device 2 of user B1 transmits information indicating the selection instruction, the selected event ID (the selected event ID of the selected event corresponding to the selection instruction by user A), the user ID of user B1, and the destination information (the user IDs of users A, B2, and B3) to the server device 3. In the server device 3, the object IDs of 10 specific objects are selected according to a predetermined selection method from the object IDs of multiple specific objects associated with the received selected event ID, and the object IDs of the selected specific objects are transmitted as information indicating the selection result to the terminal device 2 of user B1 and the terminal device 2 of the user indicated by the user ID included in the destination information.

In this way, when a selection event is implemented in response to a selection instruction from one of multiple users belonging to a communication group who participate in a communication service and communicate with each other, other users belonging to the communication group can also implement the same selection event as the user implemented. Furthermore, when other users wish to implement a selection event that was implemented earlier, they have to perform multiple operations, such as searching for the desired selection event from among multiple selection events, before they can reach a screen for implementing the desired selection event, which is a problem. With the information system 1 of this embodiment, a user can reach a screen for implementing the desired selection event with simple user operations, making it possible to significantly improve usability.

In Figures 13(a) and (b), when information indicating the selection result of a selection event in response to a selection instruction by another user is received, a selection result screen is displayed to provide the user with the same selection event as the selection event performed by the other user. During the display period of the character object of the other user in which a specific object corresponding to the selection result of a selection event in response to a selection instruction by the other user is used, an event provision button may be displayed to allow the user to execute the same selection event as the selection event performed by the other user.

Fig. 14(a) is a diagram showing an example of a communication screen 1400 displayed on the display device 23 of user A's terminal device 2. User B1's character object 811b1 in user output image 812b1 included in communication screen 1400 has been changed based on a specific object corresponding to the selection result of a selected event in response to a selection instruction by user B1. During the period in which user output image 812b1 including the changed character object 811b1 is displayed, an event provision button 1402 is displayed in the display area of user A's user output image 812a.

When the event provision button 1402 is selected by user A operating the input device 24, a selection instruction screen 911 (FIG. 14(b)) is displayed. The selection instruction screen 911 shown in FIG. 14(b) has the same function as the selection instruction screen 911 shown in FIG. 9(b). When the selection instruction button 912 (selection instruction buttons 912a and 912b) included in the selection instruction screen 911 shown in FIG. 14(b) is selected, a selection process of a selection event that is the same as the selection event according to the selection instruction by user B1 is executed. When two or more users among users B1 to B3 have already performed a selection event, the character objects 811b of the two or more users are changed based on a specific object corresponding to the selection result of the selection event. User A selects one of the user output images 812b (or a specified button in the user output image 812b) in which the character object 811b has been changed before or after the event provision button 1402 is selected. As a result, a selection instruction screen 911 (FIG. 14B) relating to the selected event previously performed by user B corresponding to the selected user output image 812b is displayed. In this case, when user A selects the event provision button 1402, a screen for selecting one of two or more users B who previously performed the selected event may be displayed. User A operates this screen to display a selection instruction screen 911 (FIG. 14B) relating to the selected event previously performed by the selected user B. In addition, during the period when the user output image 812b1 including the changed character object 811b1 is displayed, the event provision button 1402 may not be displayed in the display area of the user output image 812a of user A. In this case, when the character object 811b1 or the user output image 812b1 is selected by user A operating the input device 24, the selection instruction screen 911 (FIG. 14B) is displayed.

In this way, when a selection event is implemented in response to a selection instruction by one of multiple users belonging to a communication group who participate in a communication service and communicate with each other, during the period in which the character object or user output image of the one user is changed based on the specific object selected by the selection event, other users belonging to the communication group can also implement the same selection event as the one implemented by the user.

The event provision button 1402 shown in FIG. 14(a) that can be operated by user A may be displayed during a period in which user A's character object is provisionally using a specific object received as information indicating the selection result of a selected event in response to a selection instruction from any of users B1 to B3.

FIG. 15(a) is a diagram showing an example of a communication screen 1500 displayed on the display device 23 of user A's terminal device 2. User B1's character object 811b1 in user output image 812b1 included in communication screen 1500 has been changed based on a specific object corresponding to the selection result of a selection event in response to a selection instruction by user B1. In this case, an event provision button 1402 may be displayed in the display area of user A's user output image 812a during a period in which user A's character object 811a in user output image 812a included in communication screen 1500 is provisionally using a specific object corresponding to the selection result of a selection event in response to a selection instruction by user B1.

The event provision button 1402 shown in FIG. 15(a) that can be operated by user A may be displayed when the number of character objects provisionally using a specific object received as information indicating the selection result of a selected event in response to a selection instruction by any of users B1 to B3 reaches a predetermined number (e.g., "2") or more. When the event provision button 1402 is selected by user A operating the input device 24, a selection instruction screen 911 is displayed. The displayed selection instruction screen 911 has the same function as the selection instruction screen 911 shown in FIG. 9(b). When a selection instruction button 912 (selection instruction buttons 912a and 912b) included in the selection instruction screen 911 is selected, a selection process is executed for a selected event that is the same as the selected event in response to the selection instruction by user B1.

15B is a diagram showing an example of a communication screen 1510 displayed on the display device 23 of the terminal device 2 of user A. The character object 811b1 of user B1 in the user output image 812b1 included in the communication screen 1500 is changed based on a specific object corresponding to the selection result of the selection event in response to the selection instruction by user B1. In this case, when two characters (= a predetermined number or more) of the character object 811b2 of user B2 and the character object 811b3 of user B3 in the user output image 812a included in the communication screen 1500 are provisionally using a specific object corresponding to the selection result of the selection event in response to the selection instruction by user B1, the event provision button 1402 may be displayed in the display area of the user output image 812a of user A. When the event provision button 1402 is selected by the operation of the input device 24 by user A, a selection instruction screen 911 is displayed. The displayed selection instruction screen 911 has the same function as the selection instruction screen 911 shown in FIG. 9B. When a selection instruction button 912 (selection instruction buttons 912a and 912b) included in the selection instruction screen 911 is selected, a selection process is executed for a selection event that is the same as the selection event corresponding to the selection instruction by user B1.

In addition, in the conventional information system, after starting communication with another user via a character object, the user had to interrupt the communication service in order to change the output image including the character object. Then, after the communication service was interrupted, it was necessary to activate a function for changing the appearance of the character object and/or an image such as the background in the output image including the character object. Then, the user had to perform a cumbersome procedure of rejoining the communication service after changing the character object and/or the output image using the change function. In contrast, the information system 1 of the present embodiment makes it possible to display information related to the selected event without interrupting communication via the character object. As a result, in the information system 1 of the present embodiment, the user interface for changing the user's character object during communication is not complicated, so that it is possible to improve usability compared to the conventional information system. In addition, in the information system 1 of the present embodiment, since it is possible to change the user's character object without interrupting communication, it is possible to reduce the number of communications and the amount of communication between the server device 3 and the terminal device 2. In this way, in the information system 1 of the present embodiment, it is possible to improve communication between the server device 3 and the terminal device 2. Furthermore, if a user wants to try using an object provided by a selection event, the user has no choice but to perform the selection event, and some users may hesitate to perform the selection event. With the information system 1 of this embodiment, a user can provisionally use objects, etc. acquired by other users through a selection event performed by the other users with a simple operation. Furthermore, the user can perform the selection event after visually checking the usage status of objects acquired by other users through selection events. In this way, the information system 1 of this embodiment has improved usability compared to conventional information systems, and also makes it possible to realize trial use of objects from selection events.

(Operation sequence of information system 1)
16 is a diagram showing an example of an operation sequence of the information system 1. This operation sequence is executed mainly by the terminal processing device 27 and the server processing device 33 in cooperation with each element of the terminal device 2 and the server device 3, based on a control program previously stored in the terminal storage device 22 and the server storage device 32. In the following, it is assumed that user A operates terminal device 2a, user B1 operates terminal device 2b1, and user B2 operates terminal device 2b2. Note that the terminal devices of users other than user A are not limited to terminal devices 2b1-2b2 of users B1 and B2, and terminal devices 2 of three or more users may be connected to the server device 3.

First, the transmission unit 272 of the terminal device 2a transmits output information including character video data including motion data of user A generated by the generation unit 271 based on the imaging data continuously output by the imaging device 25, voice data of user A output from the microphone 26, and the user ID of user A to the server device 3 via the terminal communication I/F 21 (step S101). Note that the output information may include game image data showing a game screen of the game played by the user and the user ID of user A. In addition, destination information may be included in the transmission process of the transmission unit 272. The process of step S101 is executed continuously at predetermined time intervals (for example, every 2 seconds) until the distribution event ends, so the processes related to steps S101 to S110 are executed intermittently.

Next, the server transmission unit 332 of the server device 3 transmits the output information received from the terminal device 2a via the server communication I/F 31 to the terminal device 2b1 (step S102) and also to the terminal device 2b2 (step S103).

The transmitting unit 272 of the terminal device 2b1 transmits output information including the character video data of user B1, the voice data of user B1, and the user ID of user B1 to the server device 3 via the terminal communication I/F 21 (step S104). Next, the server transmitting unit 332 of the server device 3 transmits the output information received from the terminal device 2b1 to the terminal device 2a (step S105) and also to the terminal device 2b2 via the server communication I/F 31.

The transmitting unit 272 of the terminal device 2b2 transmits output information including the character video data of user B2, the voice data of user B2, and the user ID of user B2 to the server device 3 via the terminal communication I/F 21 (step S106). Next, the server transmitting unit 332 of the server device 3 transmits the output information received from the terminal device 2b2 to the terminal device 2a (step S107) and also to the terminal device 2b1 via the server communication I/F 31.

The display processing unit 274 of the terminal device 2a displays a communication screen including a user output image including a character object of user A, a user output image including a character object of user B1, and a user output image including a character object of user B2 on the display device 23 of the terminal device 2a based on the output information of each of user A, user B1, and user B2, and outputs the voices of user B1 and user B2 (step S108).

Similar to step S108, the display processing unit 274 of the terminal device 2b1 displays a communication screen including a user output image including a character object of user A, a user output image including a character object of user B1, and a user output image including a character object of user B2 based on the output information of each of user A, user B1, and user B2 on the display device 23 of the terminal device 2b1, and outputs the voices of user A and user B2 (step S109).

Similar to step S108, the display processing unit 274 of terminal device 2b2 displays and outputs a communication screen including a user output image including a character object of user A, a user output image including a character object of user B1, and a user output image including a character object of user B2 based on the output information of each of user A, user B1, and user B2 on the display device 23 of terminal device 2b2, and outputs the voices of user A and user B1 (step S110).

Next, the transmission unit 272 of the terminal device 2a transmits information indicating the selection instruction input in response to user A's operation of the input device 24 to the server device 3 via the terminal communication I/F 21 (step S111).

Next, the selection unit 333 of the server device 3 executes a selection process based on the information indicating the selection instruction received from the terminal device 2a (step S112), and inputs information indicating the result of the selection process (information indicating the selection result) to the server transmission unit 332.

Next, the server transmission unit 332 of the server device 3 transmits information indicating the selection result obtained from the selection unit 333 via the server communication I/F 31 to the terminal device 2a (step S113), to the terminal device 2b1 (step S114), and to the terminal device 2b2 (step S115).

The display processing unit 274 of the terminal device 2a displays and outputs the selection result screens 1200 and 1210 on the display device 23 of the terminal device 2b1 based on the information indicating the selection result transmitted from the server device 3 (step S116). The display processing unit 274 of the terminal device 2b1 displays and outputs the selection result screens 1300 and 1310 on the display device 23 of the terminal device 2b1 based on the information indicating the selection result transmitted from the server device 3 (step S117). The display processing unit 274 of the terminal device 2b2 displays and outputs the selection result screens 1300 and 1310 on the display device 23 of the terminal device 2b1 based on the information indicating the selection result transmitted from the server device 3 (step S118).

As described above in detail, in the information system 1 of this embodiment, a user output image is displayed that includes a character object of user A that has been at least partially modified based on a specific object selected from among a plurality of specific objects in response to an instruction from user A. In this way, the information system 1 of this embodiment diversifies the output from the terminal device 2 in response to an instruction from user A during a communication service, thereby making it possible to increase the user's motivation to continue using the communication service.

(Variation 1)
Note that the present invention is not limited to the present embodiment. For example, a privilege may be given to the user or another user depending on the user's selection of the selection instruction buttons 1301 a and 1301 b included in the selection result screens 1300 and 1310.

For example, when any of users B1 to B3 selects selection instruction buttons 1301a and 1301b included in selection result screens 1300 and 1310 related to information indicating the selection result of a selection event in response to a selection instruction by user A, a bonus W is granted to user A. For example, information indicating the bonus W associated with user A's user ID is stored in the terminal storage device 22 of user A's terminal device 2 and/or the server storage device 32 of the server device 3, thereby granting bonus W to user A. Bonus W is, for example, a specified object (e.g., a "ticket object") that is consumed to select selection instruction buttons 912a and 912b. Bonus W may be, for example, points that can be exchanged for a specified object that is consumed to select selection instruction buttons 912a and 912b. When any of users B1 to B3 selects the selection instruction buttons 1301a and 1301b included in the selection result screens 1300 and 1310 related to information showing the selection result of the selection event in response to the selection instruction by user A, if a predetermined time condition is satisfied, a privilege W may be granted to user A. The predetermined time condition is, for example, that the time from the selection instruction by user A to the selection of the selection instruction buttons 1301a and 1301b by any of users B1 to B3 is within a predetermined grant condition period. The predetermined time condition may also be that the time from the display of the selection result screens 1300 and 1310 related to information showing the selection result of the selection event in response to the selection instruction by user A to the selection of the selection instruction buttons 1301a and 1301b by any of users B1 to B3 is within a predetermined grant condition period.

In addition, when any of users B1 to B3 selects the selection instruction buttons 1301a and 1301b included in the selection result screens 1300 and 1310 related to information indicating the selection result of the selection event in response to the selection instruction by user A, a privilege X may be granted to the user who selected the selection instruction buttons 1301a and 1301b. In addition, when any of users B1 to B3 selects the selection instruction buttons 1301a and 1301b included in the selection result screens 1300 and 1310 related to information indicating the selection result of the selection event in response to the selection instruction by user A, a privilege Y may be granted to the user who did not select the selection instruction buttons 1301a and 1301b. In addition, when none of users B1 to B3 selects the selection instruction buttons 1301a and 1301b included in the selection result screens 1300 and 1310 related to information indicating the selection result of the selection event in response to the selection instruction by user A within a predetermined time, a privilege Y may be granted to users B1 to B3. In addition, when any of users B1 to B3 selects the selection instruction buttons 1301a and 1301b included in the selection result screens 1300 and 1310 related to information showing the selection result of the selection event in response to the selection instruction by user A, the privilege Z may be granted to user A and all users B1 to B3. In addition, when any of users B1 to B3 selects the selection instruction buttons 1301a and 1301b included in the selection result screens 1300 and 1310 related to information showing the selection result of the selection event in response to the selection instruction by user A, the privilege may be granted to all or some of users A and users B1 to B3 who have not yet been granted a privilege.

Note that privilege W is a privilege that is more valuable than privileges X and Y. Also, privilege X may be a privilege that is more valuable than privilege Y. If the privilege is an additional object that is selected in a selection event, a privilege with a higher value is an additional object that is less likely to be selected in a selection event. Also, if the privilege is a ticket object, the more times the selection instruction button 912 can be selected, the more valuable the ticket object is.

Furthermore, when user B1 selects and then user B2 selects selection instruction buttons 1301a and 1301b included in selection result screens 1300 and 1310 related to information showing the selection results of a selection event in response to a selection instruction by user A, the bonus X granted to user B1 may be a bonus of higher value than the bonus X granted by user B2. In this way, the user who selects selection instruction buttons 1301a and 1301b first is granted a higher bonus X.

Furthermore, when user B1 selects and then user B2 selects selection instruction buttons 1301a and 1301b included in selection result screens 1300 and 1310 related to information showing the selection result of a selection event in response to a selection instruction by user A, the bonus X granted to user B1 may be a bonus of lower value than the bonus X granted by user B2. In this way, a user who is tempted by the selection of selection instruction buttons 1301a and 1301b by another user and selects selection instruction buttons 1301a and 1301b is granted a bonus X of higher value than the user who selected earlier.

(Variation 2)
Furthermore, when user A and all users B1 to B3 who belong to the same communication group and participate in the communication service execute a selection process of the same type of selected event during the communication service, user A and users B1 to B3 may be assigned one of a plurality of specific objects related to the same type of selected event. Furthermore, when user A and users B1 to B3 execute a selection process of the selected event in a predetermined order during the communication service, user A and users B1 to B3 may be assigned one of a plurality of specific objects related to a predetermined selected event.

(Variation 3)
In addition, when any of user A and users B1 to B3 who belong to the same communication group and participate in the communication service consumes a specified object during the communication service and executes a selection process for a selected event by selecting the selection instruction buttons 912a and 912b, other users of user A and users B1 to B3 may be able to select the selection instruction buttons 912a and 912b and execute a selection process for a selected event without consuming the specified object until the communication service ends.

(Variation 4)
In addition, when any of user A and users B1 to B3 who belong to the same communication group and participate in the communication service consumes a specified object during the communication service and executes a selection process for a selected event by selecting the selection instruction buttons 912a and 912b, other users of user A and users B1 to B3 may be able to select the selection instruction buttons 912a and 912b and execute a selection process for a selected event without consuming the specified object until the communication service ends.

(Variation 5)
Also, for example, when a predetermined time has elapsed since a selection event corresponding to a selection instruction by user A was performed, the display processing unit 274 of the terminal device 2 of user A may display information 1700 prompting the selection of the selection instruction buttons 912a and 912b ( FIG. 17(a) ). Also, when a predetermined time has elapsed since a character object of user A used a specific object received as information indicating a selection result of a selection event corresponding to a selection instruction by user A, the display processing unit 274 of the terminal device 2 of user A may display information 1700 prompting the selection of the selection instruction buttons 912a and 912b. Also, when a predetermined time has elapsed during which the number of character objects provisionally using a specific object received as information indicating a selection result of a selection event corresponding to a selection instruction by user A is less than a predetermined number (for example, "1"), the display processing unit 274 of the terminal device 2 of user A may display information 1700 prompting the selection of the selection instruction buttons 912a and 912b. The display of the information 1700 by the terminal device 2 of the user A, which prompts the selection of the selection instruction buttons 912a and 912b, may be executed when the character object of the target user, among the users B1 to B3, for whom the user A performed a predetermined operation, uses a specific object. For example, in a communication screen, when the size of the user output image of another user can be changed by the operation of the user, the target user for whom the user A performed a predetermined operation is the user of the user output image whose size has been greatly changed by the operation of the user A. In this case, the selection event prompted by the information 1700 is a selection event in which the specific object used by the target user for whom the user A performed a predetermined operation can be selected. Since the communication screen 1100 includes the selection event button 813, when the motivation to perform the selection event prompted by the information 1700 increases, the user A can reach a screen for starting the selection event by selecting the selection event button 813. In this case, when the selection event button 813 is selected, the selection instruction screen 911 related to the selection event prompted by the information 1700 may be displayed. In addition, the information 1700 may be displayed so that it can be selected by the user. When the information 1700 is selected by user A, a selection instruction screen 911 relating to the selection event prompted by the information 1700 may be displayed.

The display of information 1710 by the terminal device 2 of user A to prompt the selection of the selection instruction buttons 912a and 912b may be executed when the character object uses a specific object received as information indicating the selection result of the selection event in response to the selection instruction by any of users B1 to B3 (FIG. 17(b)). The display of information 1710 by the terminal device 2 of user A to prompt the selection of the selection instruction buttons 912a and 912b may be executed when the number of users B participating in the communication service in which user A is participating is less than a predetermined number. The display of information 1710 by the terminal device 2 of user A to prompt the selection of the selection instruction buttons 912a and 912b may be executed when the number of users B participating in the communication service in which user A is participating is equal to or greater than a predetermined number. The display of information 1710 by the terminal device 2 of user A to prompt the selection of the selection instruction buttons 912a and 912b may be executed when a predetermined time has elapsed since the number of users B participating in the communication service in which user A is participating has stopped changing. Since communication screen 1100 includes selected event button 813, when user A's motivation to perform the selected event prompted by information 1700 increases, user A can reach a screen for starting the selected event by selecting selected event button 813. In this case, when selected event button 813 is selected, selection instruction screen 911 related to the selected event prompted by information 1700 (selected event performed in response to a selection instruction by any of users B1 to B3) may be displayed. Information 1700 may also be displayed so that it can be selected by the user. When information 1700 is selected by user A, selection instruction screen 911 related to the selected event prompted by information 1700 may be displayed.

(Variation 6)
Also, for example, each time a character object provisionally uses a specific object received as information indicating the selection result of a selection event in response to a selection instruction by user A, the display processing unit 274 of user A's terminal device 2 may display the username of the character object provisionally used this time and information that it has been provisionally used.

(Variation 7)
When the character object is generated based on three-dimensional model data for generating three-dimensional animation, the user output image of each user may be an image of the character object, which is a solid object (three-dimensional object) arranged in a virtual space determined for each user, viewed from a predetermined viewpoint in the virtual space. For example, the display processing unit 274 arranges a two-dimensional screen in the line of sight of the predetermined viewpoint, projects the three-dimensional coordinates of various solid objects arranged in the virtual space onto the two-dimensional screen, and displays an image of the two-dimensional screen surface onto which the various objects are projected as a user output image. Note that, for example, the line of sight of the predetermined viewpoint for displaying the user output image of user A displayed on the terminal device 2 of user A is a direction toward the character object of user A. Then, the two-dimensional screen for displaying the user output image of user A displayed on the terminal device 2 of user A is controlled to move and rotate in accordance with the movement of the predetermined viewpoint and the rotation of the line of sight.

(Variation 8)
When a character object is generated based on three-dimensional model data for generating three-dimensional animation, the user output image of each user may be an image viewed in the virtual space from a user viewpoint located at a predetermined position (e.g., the eye position of the character object) of the character object, which is a solid object (three-dimensional object) located in a virtual space common to each user. Note that the user viewpoint may move from the predetermined position of the character object in response to an operation by the user and/or automatically to a position behind, above, or in front of the character object.

An example of a method of using a communication service in a virtual space common to each user will be described. For example, as shown in FIG. 18, the display processing unit 274 makes a device object S, such as a virtual tablet PC, appear within the field of view of the character object C of user A from the user's viewpoint in the virtual space. The various screens described in FIGS. 6 to 15 and 17 are displayed on the display device portion of this device object S. In this case, the user output image of user A displayed on the display device portion of the device object S is a two-dimensional image of the virtual space as viewed from a virtual camera provided at a predetermined position on the device object S. The voice data of user A is voice data acquired by the microphone 26 of the terminal device 2.

The transmission unit 272 of the terminal device 2 transmits the user output image of user A, the voice data of user A, and the user ID of user A to all of the terminal devices 2 of user B via the server device 3. Note that the transmission unit 272 may transmit the position and line of sight direction of the virtual camera in the virtual space, the position and body orientation of the character object of user A in the virtual space, and the motion data of user A, instead of the user output image of user A. Furthermore, the terminal device 2 of user A receives the user output image of each user B, the voice data of each user B, and the user ID of each user B from the terminal device 2 of each user B via the server device 3. Then, the display processing unit 274 of the terminal device 2 of user A displays the user output image of each user B on the display device portion of the device object S, and outputs the voice data of each user B from the terminal device 2. This makes it possible to realize a communication service using the virtual device object S.

(Variation 9)
At least a part of the functions realized by the terminal processing device 27 of the terminal device 2 may be realized by a processor other than the terminal processing device 27 of the terminal device 2. At least a part of the functions realized by the terminal processing device 27 may be realized, for example, by the server processing device 33 of the server device 3. Specifically, a part of the functions of the generation unit 271 and the display processing unit 274 may be executed in the server device 3. For example, the terminal device 2 transmits continuously acquired imaging data and audio data to the server device 3 together with the user ID of the user who operates the terminal device 2. Then, the server device 3 generates character video data using the functions of the generation unit 271 and the display processing unit 274, and generates display information for displaying a communication screen including the user output images of all users on the display device 23. Then, the server device 3 transmits the display information together with audio information (audio data) to each terminal device 2 of all users. Each terminal device 2 outputs each piece of information received from the server device 3 (so-called "server rendering").

(Variation 10)
The terminal device 2 has a function of displaying a communication screen or the like by executing various commands included in a control program stored in the terminal device 2. In order to realize a communication service, the terminal device 2 may call up a browser function incorporated in a web application, receive a web page (a document written in a markup language such as HTML (Hyper Text Markup Language)) from a server device using this browser function, and execute the control program incorporated in this web page (so-called "browser rendering"). For example, by using HTML 5 as a markup language, the terminal device 2 can easily execute new information processing. By adopting such a web application to realize a communication service in the terminal device, the creator of the program can provide the client side (terminal device side) with new information processing simply by incorporating a new program in a web page transmitted by the server device, and the man-hours required for creating a new program can be significantly reduced. In addition, the client side can receive a new service simply by receiving a web page without downloading a new control program, which makes it possible to reduce the load on the communication network, communication costs and/or communication time compared to downloading a control program, and to simplify the user interface.

(Modification 11)
The generating unit 271 of the terminal device 2 of the user A may generate output information including face motion data without using imaging data. For example, the terminal storage device 22 of the terminal device 2 stores face motion data corresponding to voice data, and the generating unit 271 may extract face motion data corresponding to the acquired voice data of the user A from the terminal storage device 22 and generate output information including the extracted face motion data, the voice data, and the user ID of the user A. The generating unit 271 of the terminal device 2 of the user A may generate output information including the voice data of the user A output from the microphone 26 and the user ID stored in the terminal storage device 22 without including the face motion data. In this case, the display processing unit 274 of the terminal device 2 of the user B may extract face motion data corresponding to the voice data of the user A included in the output information of the user A received via the server device 3 and generate animation of the character object of the user A. The generation process of the face motion data corresponding to the voice data of the user may be a process using a known lip sync algorithm.

In this way, on the terminal device 2 of user A, which uses an HMD as the display device 23, it is possible to communicate with user B via a character object without acquiring image data of user A.

1 Information system 2 (2a, 2b1, 2b2) Terminal device 21 Terminal communication I/F
22 Terminal storage device 23 Display device 24 Input device 25 Imaging device 26 Microphone 27 Terminal processing device 271 Generation unit 272 Transmission unit 273 Reception unit 274 Display processing unit 275 Audio output unit 3 Server device 31 Server communication I/F
32 Server storage device 33 Server processing device 331 Server receiving unit 332 Server transmitting unit 333 Selection unit 4 Base station 5 Mobile communication network 6 Gateway 7 Internet

Claims (18)

A control program for a terminal device of a first user,
A terminal device of the first user
Displaying at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user;
Transmitting information regarding the display of the first image to a terminal device of the second user;
When a first voice of the first user is acquired, the first voice is transmitted to a terminal device of the second user;
displaying the first image including the first object that changes in response to a motion or the first voice of the first user;
When a second voice of the second user is received, the second voice is output.
displaying the second image including the second object that changes in response to the action or the second voice of the second user;
displaying the first image including the first object at least a part of which has been changed based on at least one third object selected from a plurality of third objects in response to a first instruction by the first user;
Run the command ,
A terminal device of the first user
providing an interface for the first user to provide the first instruction when a first condition is satisfied with respect to the first or second user;
A control program for executing the above . A terminal device of the first user
receiving information about at least one fourth object selected from a plurality of fourth objects in response to a second instruction by the second user;
displaying the second image including the second object at least a part of which has been changed based on information related to the at least one fourth object in response to an application instruction of the selected fourth object by the second user;
selecting at least one of the plurality of fourth objects in response to a third instruction by the first user, and displaying the first image including the first object at least a portion of which has been changed based on the selected at least one fourth object;
The control program according to claim 1 , 3. The control program of claim 2, wherein when the second object included in the displayed second image is selected by the first user in the third instruction by the first user, the multiple fourth objects in the selection responsive to the third instruction are the multiple fourth objects selected by the second user indicated by the second object selected by the first user. A terminal device of the first user
if a second condition is satisfied, restoring the first object, at least a part of which has been changed based on the at least one fourth object, to the first object before the change, and displaying the first image including the first object before the change;
Run the command,
4. The control program according to claim 2 or 3, wherein the first user is capable of selecting at least one fourth object from among the plurality of fourth objects during a period from when the first object is changed based on the at least one fourth object to when the first object returns to the first object before the change. The second condition is:
obtaining a specific instruction from the first user;
the transmission of the information regarding the display of the first image and the first sound, and the reception of the second sound have been completed;
the transmission of the information relating to the display of the second image and the second sound, and the reception of the first sound have been completed;
a predetermined time has elapsed since at least a portion of the first object was modified based on the at least one fourth object;
When at least a part of the first object is changed based on the at least one fourth object, the first user starts selecting at least one third object from among a plurality of third objects in response to an instruction from the first user; or
the second user starts selecting at least one third object from among a plurality of third objects in response to an instruction while at least a part of the first object is being changed based on the at least one fourth object;
The control program according to claim 4 . A control program according to any one of claims 2 to 5, wherein displaying the first image including the first object at least partially changed based on the at least one fourth object is executed when a third condition is satisfied. The third condition is:
the number of times that the first image including the first object at least a part of which has been changed based on the at least one fourth object has been displayed is equal to or less than a predetermined number of times;
The control program according to claim 6 . A terminal device of the first user
when at least one fourth object is selected from the plurality of fourth objects in response to an operation of a first instruction object by the first user, a first benefit is granted to the second user;
The control program according to any one of claims 2 to 7 , A terminal device of the first user
When at least one of the plurality of fourth objects is selected in response to an operation of a first instruction object by the first user, a second benefit is granted to the first user.
The control program according to any one of claims 2 to 8 , A terminal device of the first user
when a first instruction object is not operated by the first user and at least one fourth object is not selected from among the plurality of fourth objects, granting a third benefit to the first user;
The control program according to any one of claims 2 to 9 , A terminal device of the first user
when the third condition is satisfied, displaying a second instruction object for executing a selection of at least one fifth object from among a plurality of fifth objects, and information regarding the execution of the selection;
selecting at least one fifth object from among the plurality of fifth objects in response to an operation of the second instruction object by the first user, and displaying the first image including the first object at least a portion of which has been changed based on the selected at least one fifth object;
The control program according to any one of claims 1 to 10 , The control program according to claim 11 , wherein the third condition is that a predetermined time has elapsed since the first object was changed based on the at least one third object. The control program according to claim 11 , wherein the plurality of fifth objects includes an object that is applied to the second object representing the second user and is displayed. A control program for a server device capable of communicating with a terminal device of a first user and a terminal device of a second user different from the first user,
The server device,
receiving information regarding a display of a first image including a first object representing the first user from a terminal device of the first user, and receiving information regarding a display of a second image including a second object representing the second user from a terminal device of the second user;
Transmitting information for displaying at least the first image to at least a terminal device of the second user, and transmitting information for displaying at least the second image to at least a terminal device of the first user;
When receiving a first voice of the first user, transmitting the first voice to a terminal device of the second user;
Transmitting information for displaying the first image including the first object that changes in response to the action or the first voice of the first user to at least a terminal device of the second user;
When a second voice of the second user is received, the second voice is transmitted to a terminal device of the first user;
Transmitting information for displaying the second image including the second object that changes in response to the action or the second voice of the second user to at least a terminal device of the first user;
transmitting, to at least a terminal device of the second user, information for displaying the first image including the first object at least a portion of which has been changed based on at least one third object selected from a plurality of third objects in response to a first instruction by the first user;
Run the command ,
In the terminal device of the first user,
providing an interface for the first user to provide the first instruction when a first condition is satisfied with respect to the first or second user;
A control program comprising : A terminal device of a first user,
Displaying at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user;
Transmitting information regarding the display of the first image to a terminal device of the second user;
When a first voice of the first user is acquired, the first voice is transmitted to a terminal device of the second user;
displaying the first image including the first object that changes in response to a motion or the first voice of the first user;
When a second voice of the second user is received, the second voice is output.
displaying the second image including the second object that changes in response to the action or the second voice of the second user;
a processing unit that displays the first image including the first object at least a part of which has been changed based on at least one third object selected from a plurality of third objects in response to a first instruction by the first user ,
The processing unit includes:
providing an interface for the first user to provide the first instruction when a first condition is satisfied with respect to the first or second user;
A terminal device which executes the above . A server device capable of communicating with a terminal device of a first user and a terminal device of a second user different from the first user,
receiving information regarding a display of a first image including a first object representing the first user from a terminal device of the first user, and receiving information regarding a display of a second image including a second object representing the second user from a terminal device of the second user;
Transmitting information for displaying at least the first image to at least a terminal device of the second user, and transmitting information for displaying at least the second image to at least a terminal device of the first user;
When receiving a first voice of the first user, transmitting the first voice to a terminal device of the second user;
Transmitting information for displaying the first image including the first object that changes in response to the action or the first voice of the first user to at least a terminal device of the second user;
When a second voice of the second user is received, the second voice is transmitted to a terminal device of the first user;
Transmitting information for displaying the second image including the second object that changes in response to the action or the second voice of the second user to at least a terminal device of the first user;
a processing unit that transmits, to at least a terminal device of the second user, information for displaying the first image including the first object at least a part of which has been changed based on at least one third object selected from a plurality of third objects in response to a first instruction by the first user ;
In the terminal device of the first user,
providing an interface for the first user to provide the first instruction when a first condition is satisfied with respect to the first or second user;
A server device comprising : A method for controlling a terminal device of a first user, comprising:
The terminal device of the first user,
Displaying at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user;
Transmitting information regarding the display of the first image to a terminal device of the second user;
When a first voice of the first user is acquired, the first voice is transmitted to a terminal device of the second user;
displaying the first image including the first object that changes in response to a motion or the first voice of the first user;
When a second voice of the second user is received, the second voice is output.
displaying the second image including the second object that changes in response to the action or the second voice of the second user;
displaying the first image including the first object at least a part of which has been changed based on at least one third object selected from a plurality of third objects in response to a first instruction by the first user;
Including,
The terminal device of the first user,
providing an interface for the first user to provide the first instruction when a first condition is satisfied with respect to the first or second user;
A control method comprising : A method for controlling a server device capable of communicating with a terminal device of a first user and a terminal device of a second user different from the first user, the method comprising:
The server device,
receiving information regarding a display of a first image including a first object representing the first user from a terminal device of the first user, and receiving information regarding a display of a second image including a second object representing the second user from a terminal device of the second user;
Transmitting information for displaying at least the first image to at least a terminal device of the second user, and transmitting information for displaying at least the second image to at least a terminal device of the first user;
When receiving a first voice of the first user, transmitting the first voice to a terminal device of the second user;
Transmitting information for displaying the first image including the first object that changes in response to the action or the first voice of the first user to at least a terminal device of the second user;
When a second voice of the second user is received, the second voice is transmitted to a terminal device of the first user;
Transmitting information for displaying the second image including the second object that changes in response to the action or the second voice of the second user to at least a terminal device of the first user;
transmitting, to at least a terminal device of the second user, information for displaying the first image including the first object at least a portion of which has been changed based on at least one third object selected from a plurality of third objects in response to a first instruction by the first user;
Including,
In the terminal device of the first user,
providing an interface for the first user to provide the first instruction when a first condition is satisfied with respect to the first or second user;
A control method comprising :

Images