JP7709706B2 - PROGRAM, INFORMATION PROCESSING METHOD, AND INFORMATION PROCESSING SYSTEM - Google Patents

Description

The present invention relates to a program, an information processing method, and an information processing system.

There has been known an information processing system that distributes videos that include animation of an avatar object based on the movements of a user (see, for example, Patent Document 1). This system generates videos that include two characters in real time. It also synthesizes the voice of a voice actor into the video in real time and distributes it over a network.

JP 2015-184689 A

In an information processing system that distributes videos, it is desirable to increase the opportunities for viewing users to participate in the videos being distributed, in order to increase the number of viewing users, or to increase the cumulative viewing time or number of views. Specifically, it is being considered to have avatar objects corresponding to viewing users appear in the videos.

However, when drawing a video that includes multiple avatar objects, there is a possibility that the action intended by the user will not be drawn. In particular, when an avatar object corresponding to a viewing user performs an action related to an avatar object corresponding to a broadcasting user, the progress of the video may be hindered if the viewing user's action is not performed smoothly. For example, when avatar objects shake hands, the timing or position of each avatar object's hand may differ, preventing the handshake action from being completed. As a result, the progress of the video may be hindered.

In one aspect, a program is provided. The program causes one or more computers to execute a first output control process for outputting a video including a first avatar object corresponding to a distribution user on a display; a reception process for receiving a selection operation by a viewing user who is viewing the video to select a gift object; a gift transmission process for transmitting gift identification information of the selected gift object and identification information of the viewing user to a server together with an output request for the gift object; and a second output control process for outputting the video based on data for outputting a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user on the display, and the second output control process renders the second avatar object using at least one of tracking data of the viewing user and selected motion data selected by the viewing user, and preset automatic motion data.

In another aspect, an information processing method is provided. The information processing method includes a first output control process in which one or more computers output a video including a first avatar object corresponding to a distribution user to a display; a reception process in which a selection operation for selecting a gift object by a viewing user who is viewing the video is received; a gift transmission process in which gift identification information of the selected gift object and identification information of the viewing user are transmitted to a server together with an output request for the gift object; and a second output control process in which the video based on data for outputting a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user is output to the display, the second output control process drawing the second avatar object using at least one of tracking data of the viewing user and selected motion data selected by the viewing user, and preset automatic motion data.

In another aspect, an information processing method is provided. The information processing method includes a first transmission process in which one or more computers transmit data for outputting a video including a first avatar object corresponding to a broadcast user to at least a broadcast user device used by a broadcast user who watches the video; a gift reception process in which gift identification information of a gift object and identification information of the first broadcast user are received from a first broadcast user device used by the first broadcast user, the broadcast user, together with an output request for the gift object; and a second transmission process in which the gift identification information and an output request for outputting a video including the gift object corresponding to the gift identification information, the first avatar object, and a second avatar object corresponding to the first broadcast user are transmitted at least to a broadcast user device used by the broadcast user and a second broadcast user device used by a second broadcast user other than the first broadcast user, the output request transmitted in the second transmission process is transmitted together with data used to draw the second avatar object or identification information of the data, and the data used to draw the second avatar object includes at least one of tracking data of the first broadcast user and selected motion data selected by the first broadcast user, and preset automatic motion data.

In another aspect, an information processing device is provided. The information processing device includes a first transmission unit that transmits data for outputting a video including a first avatar object corresponding to a broadcast user to at least a broadcast user device used by a broadcast user who watches the video; a gift reception unit that receives gift identification information of a gift object and identification information of the first broadcast user from a first broadcast user device used by the first broadcast user, the broadcast user, together with an output request for the gift object; and a second transmission unit that transmits the gift identification information and an output request for outputting a video including the gift object corresponding to the gift identification information, the first avatar object, and a second avatar object corresponding to the first broadcast user, to at least a broadcast user device used by the broadcast user and a second broadcast user device used by a second broadcast user other than the first broadcast user, the output request transmitted by the second transmission unit is transmitted together with data used to draw the second avatar object or identification information of the data, and the data used to draw the second avatar object includes at least one of tracking data of the first broadcast user and selected motion data selected by the first broadcast user, and preset automatic motion data.

The present invention increases the opportunities for viewers who watch videos to participate in video distribution and enables the video to progress smoothly.

FIG. 1 is a schematic diagram illustrating an information processing system according to a first embodiment. FIG. 11 is a diagram showing user management data in the embodiment. FIG. 2 is a diagram showing a configuration of main body part data according to the embodiment. 13 is a diagram showing a configuration of mounting parts data in the embodiment; FIG. FIG. 13 is a diagram showing the configuration of gift data in the embodiment. 10 is a flowchart showing a processing procedure when outputting a gift object in the embodiment. FIG. 2 is a diagram showing a home screen of the embodiment. FIG. 2 is a diagram showing a viewing screen of the embodiment. FIG. 2 is a diagram showing a viewing screen of the embodiment. FIG. 2 is a diagram showing a viewing screen of the embodiment. FIG. 2 is a diagram showing a viewing screen of the embodiment. 10 is a flowchart showing a processing procedure when outputting a gift object in the second embodiment. FIG. 2 is a diagram showing a viewing screen of the embodiment. 13 is a flowchart showing a processing procedure when outputting a gift object in the third embodiment. FIG. 13 is a diagram showing a distribution screen in another embodiment.

First Embodiment
Hereinafter, a first embodiment of a program, an information processing method, and an information processing device will be described.

<Information Processing System>
An information processing system 11 will be described with reference to Fig. 1. The information processing system 11 includes a user device 12 and a server 13. The user device 12 and the server 13 transmit and receive data via a network 14. The user device 12 receives data necessary for distribution and viewing of videos from the server 13 based on an input operation by a user.

A user who uses a user device 12 to distribute videos is called a distribution user, and a user who watches the distributed videos is called a viewing user. A user can be both a distribution user and a viewing user. In other words, a user is a viewing user when watching videos, and a distribution user when distributing videos. Furthermore, a user device 12 used by a distribution user is called a distribution user device 12A, and a user device 12 used by a viewing user is called a viewing user device 12B. When there is no distinction between a distribution user and a viewing user, or between a distribution user device 12A and a viewing user device 12B, they are simply referred to as a user and a user device 12. There may be one or more distribution user devices 12A. There may be one or more viewing user devices 12B.

The user device 12 is a smartphone (multi-function phone terminal), a tablet terminal, a personal computer, a console game machine, a wearable computer, or an information processing device other than these devices that can play videos. The wearable computer only needs to have a screen on which the user can watch videos, and is, for example, a head-mounted display worn on the user's head, or a glasses-type wearable terminal. The user device 12 may be a device that operates independently, or may be composed of multiple devices that are connected so that they can send and receive various data to each other. An example of the latter user device 12 is a system that performs tracking using the outside-in method.

The user device 12 includes a control unit 20, a storage 22 (storage medium), and a communication interface (I/F) 23. The control unit 20 includes one or more arithmetic circuits, such as a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and an NPU (Neural network Processing Unit). The control unit 20 also includes a memory, which is a main storage device (storage medium) that can be read and written by the arithmetic circuits. The memory is composed of a semiconductor memory or the like. The control unit 20 loads an operating system and other programs from the storage 22 or an external storage into the memory, and executes commands retrieved from the memory. The communication I/F 23 can transmit and receive data to and from the server 13 and other user devices 12 via the network 14. The network 14 includes various networks, such as a local area network and the Internet. The control unit 20 executes the first output control process and the second output control process of the claims.

Storage 22 is an auxiliary storage device (storage medium), such as a magnetic disk, an optical disk, or a semiconductor memory. Storage 22 can also be used in combination with multiple storage devices.

The control unit 20 executes various programs recorded in the storage 22 based on the user's input operation on the operation unit 27, thereby executing various processes related to the distribution of videos and various processes related to viewing. For ease of explanation, the state in which a video is distributed will be referred to as the distribution mode, and the state in which a video distributed by another distribution user is viewed will be referred to as the viewing mode.

The user device 12 also includes a sensor unit 24, a speaker 25, a microphone 26, an operation unit 27, and a display 28. At least one of the sensor unit 24, the speaker 25, the microphone 26, the operation unit 27, and the display 28 may be provided in the same device as the control unit 20, or may be provided in a manner connected to the device in which the control unit 20 is provided.

The sensor unit 24 is one or more sensors that detect face motions that indicate changes in the user's facial expression and body motions that indicate changes in the relative position of the user's body with respect to the sensor unit 24. Face motions include blinking, opening and closing of the mouth, and other movements. The sensor unit 24 can be a publicly known sensor. An example of the sensor unit 24 includes a ToF sensor that measures the time of flight (Time of Flight) until light irradiated toward the user is reflected by the user's face and returns, a camera that captures the user's face, and an image processing unit that processes the data captured by the camera. The sensor unit 24 may also include an RGB camera that captures visible light and a near-infrared camera that captures near-infrared light. The RGB camera and the near-infrared camera can be, for example, the True Depth of the iPhone X (registered trademark), or the LIDAR (Light Detection and Ranging, or Laser Imaging Detection and Ranging) of the iPad Pro (registered trademark), or other ToF sensors installed in smartphones. Specifically, this camera uses a dot projector to project tens of thousands of invisible dots onto the user's face, etc. Then, it detects and analyzes the reflected light from the dot pattern to form a depth map of the face, and captures an infrared image of the face, etc., thereby capturing accurate facial data. The arithmetic processing unit of the sensor unit 24 generates various information based on the depth map and the infrared image, and compares the information with registered reference data to calculate the depth of each point on the face (the distance between each point and the near-infrared camera) and positional deviations other than depth.

The sensor unit 24 may have a function of tracking not only the user's face but also the hands (hand tracking). The sensor unit 24 may also include a sensor that detects the position or orientation of a human body part other than the hand. The sensor unit 24 may further include a sensor such as an acceleration sensor that detects speed or acceleration, or a gyro sensor that detects direction or orientation. The sensor unit 24 may have a spatial mapping function that recognizes objects in the real space where the user exists based on the detection results of the above-mentioned ToF sensor or other known sensors, and maps the recognized objects on a spatial map. Hereinafter, when the face motion detection data and the body motion detection data are described without any particular distinction, they are simply referred to as "tracking data". The image processing unit of the sensor unit 24 may be provided in the control unit 20.

The speaker 25 converts voice data into voice and outputs it. The microphone 26 inputs voice spoken by the user and converts it into voice data. The display 28 outputs various images according to output instructions from the control unit 20.

The operation unit 27 may be one that corresponds to the type of the user device 12. An example of the operation unit 27 is a touch panel integrated with the display 28. Other examples of the operation unit 27 include operation buttons provided on the housing of the user device 12, a keyboard, a mouse, and a controller operated by the user's hand. The controller may incorporate various known sensors such as an acceleration sensor and an inertial measurement unit (IMU) such as a gyroscope. Another example of the operation unit 27 may be a tracking device that identifies the user's hand movement, eye movement, head movement, and line of sight. In this embodiment, for example, the user's instruction can be determined based on the user's hand movement, and various operations can be performed, such as starting or ending the distribution of a video, rating a message or a video, and outputting a predetermined object (for example, a gift, which will be described later). Note that if the sensor unit 24 also has an input interface function such as a hand truck function, the operation unit 27 can be omitted.

<Server>
Next, the server 13 will be described. The server 13 is used by a service provider or the like that provides a service for distributing videos. The server 13 includes a control unit 30, a communication I/F 34, and a storage 35. The control unit 30 has one or more arithmetic circuits. The control unit 30 has the same configuration as the control unit 20 of the user device 12. The communication I/F 34 and the storage 35 have the same configuration as the user device 12. The server 13 may be composed of one device, or may be composed of multiple devices. When the server 13 is composed of multiple devices, the devices have the function of the control unit 30 and various data stored in the storage 35 in a distributed manner. The control unit 30 corresponds to the first transmission unit, the gift receiving unit, and the second transmission unit in the claims.

<Video distribution method>
The video delivery method may be one of a client rendering method, a browser rendering method, a video delivery method, and a server delivery method.

In the client rendering method, the broadcasting user device 12A and the viewing user device 12B receive video configuration data required to generate a video from the server 13 and render the video. The video configuration data includes tracking data detected by the sensor unit 24 of the broadcasting user device 12A. Each user device 12 also acquires audio data based on the speech of the broadcasting user and outputs the audio in synchronization with the video. In this case, the storage 22 of each user device 12 stores application programs and various data such as rendering data for rendering avatars.

The browser rendering method is a method in which the distribution user device 12A and the viewing user device 12B use a web browser program stored in the storage 22 to render a video. At this time, the distribution user device 12A transmits tracking data, etc. to the server 13. The user device 12 downloads a web page written in a markup language such as HTML (Hyper Text Markup Language) and using CSS (Cascading Style Sheets), JavaScript (registered trademark), etc. from the server 13. The web browser program started on the user device 12 renders avatar objects and other objects using JavaScript running on the browser. In the browser rendering method, the user device 12 stores data received from the server 13 in volatile memory and stores data such as the URL of the web page for outputting the video in non-volatile memory. For this reason, the rendering data of the avatar object is not stored in the user device 12.

The video distribution method is a method in which the broadcasting user device 12A generates video data. The viewing user device 12B outputs video to the display 28 based on the video data received from the broadcasting user device 12A. Specifically, the broadcasting user device 12A generates video data using tracking data, etc. The broadcasting user device 12A also transmits encoded video data and audio data to the server 13. The server 13 transmits the video data and audio data to the viewing user device 12B. The viewing user device 12B outputs video based on the received video data and audio data.

The server distribution method is a method in which the server 13 generates video data. The server 13 receives tracking data and audio data from the broadcasting user device 12A. The server 13 then generates video data based on these data. The server 13 transmits the generated video data to the broadcasting user device 12A and the viewing user device 12B. The broadcasting user device 12A and the viewing user device 12B output videos to the display 28 based on the received video data.

In this way, either the user device 12 or the server 13 is responsible for generating the video data. The device responsible for generating the video data can be changed depending on the distribution method. The "data for outputting video" described in the claims corresponds to each piece of data transmitted and received between the user device 12 and the server 13 in each of the distribution methods described above. When the user device 12 is responsible for generating the video data, the viewing user device 12B may generate the video data, or the distribution user device 12A may generate the video data. Furthermore, if necessary, the user device 12 transmits the generated video data to another user device 12 as the transmission destination.

The broadcasting user device 12A and the viewing user device 12B may output video in different methods. For example, the broadcasting user device 12A may output video in a client rendering method and the viewing user device 12B may output video in a browser rendering method. Furthermore, each viewing user device 12B may output video based on video data distributed in different distribution methods.

In this embodiment, a method in which each user device 12 delivers video using a client rendering method will be described.
In the client rendering method, the user device 12 stores a video application program in the storage 22. The storage 22 also stores three-dimensional model data of avatar objects and other objects used in executing the video application program.

The control unit 20 of the broadcasting user device 12A transmits to the server 13 video composition data including tracking data acquired from the sensor unit 24 and audio data based on audio collected by the microphone 26. The broadcasting user can set the parts of the avatar object corresponding to the broadcasting user to his/her preferred parts. The video composition data includes identification information (parts ID) indicating the parts of the avatar object. The broadcasting user can also adjust the position of the virtual camera in the virtual space. The video composition data may include the position of the virtual camera. The virtual camera is set to a predetermined position in the virtual space. The tracking data and audio data are given a timestamp. The timestamp is generated based on reference time information based on a circuit (internal clock) having a clocking function of the control unit 20. Alternatively, the timestamp is generated based on time information transmitted from the server 13 or reference time information acquired from the server 13 providing a time distribution service. The timestamp may represent the elapsed time from a reference time, or may represent the time.

The control unit 20 of the viewing user device 12B stores drawing data and drawing programs for objects including each avatar object in the storage 22. The control unit 20 also receives identification information such as part IDs constituting the avatar object, data for operating the avatar object such as tracking data, and audio data from the server 13. The control unit 20 acquires drawing data for objects including the avatar object that has already been stored according to the received identification information for the avatar object. The control unit 20 then performs rendering using the acquired drawing data and data for operating the avatar object, and outputs a video including the avatar object to the display 28. The control unit 20 also outputs audio from the speaker 25 in synchronization with the video.

The control unit 20 of the broadcasting user device 12A and the control unit 20 of the viewing user device 12B perform rendering including avatar objects and objects other than avatar objects. Rendering here refers to drawing processing including obtaining the position of a virtual camera, perspective projection, and hidden surface removal based on depth information associated with each object. Rendering may be at least one of these processes, or may include processing such as shading and texture mapping.

The broadcasting user device 12A and the viewing user device 12B receive data such as posted messages or notifications from viewing users from the server 13. The control unit 20 outputs video data that combines images of avatar objects and other objects with the posted messages or notifications to the display 28. The control unit 20 also synchronizes audio data with the video data based on a timestamp and outputs the audio data from the speaker 25.

Furthermore, based on the operation of the viewing user, the control unit 20 sends a request for output of a gift object (gift output request), posts a message for the video, and rates the video (selecting the "Like!" button or the "favorite" button, etc.).

<Data structure used in video>
Next, each piece of data used for distribution or viewing of a moving image will be described in detail with reference to FIG. 2 to FIG.

Figure 2 shows an example of user management data 50. The user management data 50 is stored in the server 13. The user management data 50 includes data on a user ID, which is user identification information, coins, and points. Coins are a medium that a user can purchase. Points are a medium that can be obtained by actions such as watching videos and streaming videos. In addition, the user management data 50 may include a score that is accumulated by participating in an event. The coins, points, and scores can be used within the video streaming service. The user device 12 may store the user management data 50.

3 shows an example of the body part data 51 of an avatar object. The body part data 51 is stored in the storage 22 of the user device 12.
The body parts will be described. The body parts are parts that constitute the body parts of the avatar object. The data for drawing the body parts includes polygon data, skeletal data (bones) for expressing the movements of the avatar object, texture data, etc. The skeletal data and texture data are stored in advance in the user device 12 at a predetermined timing such as when an application program is installed or when an application program is started. In addition, avatar attributes are associated with the body parts. The avatar attributes are, for example, a type such as a male type such as "M type" or a female type such as "F type", height, skin color, body type, etc. The skeletal data, polygon data, etc. of the avatar object differ depending on the avatar attributes. By the user selecting the avatar attributes, body parts such as the upper body and the lower body are selected. In addition, some of the body parts can be arbitrarily set by the user. For example, the user can select parts such as the eyes, mouth, hairstyle, etc. Each user can register one or more avatar objects as avatars available to the user. In this embodiment, three avatar objects can be registered.

Body part data 51 includes information regarding the slot number, type, height, skin color, body type, and other body parts of the avatar object corresponding to the user. Body part data 51 has various information for each avatar slot number. Type is the type that is the avatar attribute selected by the user. Height, skin color, and body type are identification information for the height, skin color, and body type selected by the user. Parts are IDs of individual parts such as eyes, mouth, and hairstyle. Each body part that makes up the avatar object, such as the head, torso, arms, and legs, is assigned a part ID.

4 shows an example of the attachment part data 52. The attachment part data 52 is stored in the user device 12. The attachment part data 52 may also be stored in the server 13.
The wearing parts data 52 is data on the wearing parts that the user can use. The wearing parts are prepared in advance in the initial state when the use of the application is started, or can be acquired by the user exchanging them with a medium such as a coin, or by executing a lottery (gacha). The wearing parts will be described. The wearing parts are parts used on a predetermined part of the avatar object. The wearing parts include texture data. The wearing parts may include skeletal data for expressing the swinging caused by the movement of the avatar object. For example, the wearing parts of the clothes may include skeletal data. The texture data and skeletal data are stored in the user device 12. The wearing parts are, for example, a "top" used on the upper body, "shoes" used on the feet, and a "skirt" used on the lower body. The user can select a preferred wearing part from a plurality of wearing parts according to the avatar attributes of the avatar object. In other words, the viewing user can use the wearing parts for each avatar object with slot numbers "1" to "3".

Attached parts are associated with attribute data that indicates a body part of the avatar object. Note that attached parts can be anything that is associated with a specific body part of the avatar object, and can be not only clothing of the avatar object, but also a "cup," "mug," "umbrella," "musical instrument," etc. For example, a "cup" is associated with the hand part of the avatar object. By associating a "cup" part with the hand, it is possible to output an avatar object holding a "cup." Attached parts include standard parts that are stored when the application program is installed, and parts that are added by the server 13 when the user exchanges for coins, participates in an event, etc.

The attached part data 52 includes a part ID, an item name, a body part, an attribute, a color, and a slot number of an avatar object in which the attached part is used. The part ID is identification information (ID) of the selected attached part. The item name is the name of the attached part. For example, the name of an item such as "Beginner T-shirt" is stored. The body part is the body part of the avatar object in which the attached part is used. The attribute is the attribute of the avatar in which the attached part is used, and is set to all attributes (for example, "All"), "M type", or "F type". The color indicates the color of the attached part. The slot number indicates the slot number of the avatar object in which the attached part is used, among the avatar objects registered by the user. Note that one attached part may be usable for an avatar object in each slot.

The user device 12 may store non-attached parts data in addition to the attached parts data 52. Non-attached parts are parts other than attached parts. For example, non-attached parts are objects such as wallpaper that are output as a background. The non-attached parts data includes a part ID, item name, and color for each part. Note that non-attached parts may be stored in the user device 12 in association with the avatar object to be used.

Figure 5 shows an example of gift data 53. Gift data 53 is stored at least in server 13. Gift data 53 is data related to a gift object. A gift object is an object given from a viewing user to a broadcasting user. Note that gift objects include objects that are placed in a virtual space, as well as objects that are used for effects in the virtual space, such as fireworks and snow. Gift objects are output to the video in exchange for the viewing user's coins or points.

Gift data 53 includes a gift ID, the name of the gift object, the number of coins, and the number of points. The gift ID is identification information of the gift object. The name is the name of the gift object, such as "Rabbit Plush Toy". The number of coins indicates the number of coins to be exchanged for the gift object. The number of points indicates the number of points to be exchanged for the gift object.

(operation)
Next, the operations of the user device 12 and the server 13 will be described.
In the viewing mode, the control unit 20 of the user device 12 executes a first output control process, a reception process, a gift sending process, and an output control process.

In the first output control process, a video including a first avatar object corresponding to the broadcast user is output to a display.
In the reception process, a selection operation for selecting a gift object by a viewing user who is viewing a video is received.

In the gift sending process, the gift ID (gift identification information) of the selected gift object and the user ID of the viewing user are sent to the server 13 along with a request to output the gift object. When sending the request to output the gift object, or at a different timing, the viewing user device 12B sends information about the second avatar object corresponding to the viewing user. The information about the second avatar object includes the part ID of the main body of the second avatar object, the part ID of the attached part, etc. The viewing user corresponding to the second avatar object is referred to as the first viewing user, and a viewing user other than the first viewing user is referred to as the second viewing user. When describing the first viewing user and the second viewing user without making a particular distinction, they are simply referred to as viewing users for convenience.

In the second output control process, a video based on data for outputting a video including the selected gift object, the first avatar object, and the second avatar object is output to the display 28. The second output control process may be executed when a selection operation for selecting a gift object by the first viewing user is received in the reception process.

In addition, in the second output control process, the second avatar object is drawn using the tracking data of the first viewing user or selected motion data selected by the first viewing user from a plurality of preset motion data, and automatic motion data that is automatically used for the second avatar object.

The viewing user device 12B of the first viewing user acquires tracking data of the viewing user from the sensor unit 24. Therefore, this viewing user device 12B transmits the tracking data to the server 13, while drawing a second avatar object using the tracking data. Note that the viewing user device 12B of the first viewing user may receive the tracking data it transmitted itself or data that has been processed from the tracking data from the server 13, and draw a second avatar object using the received tracking data.

The control unit 30 of the server 13 executes a first transmission process, a gift receiving process, and a second transmission process.
In the first transmission process, data for outputting a video including a first avatar object corresponding to the broadcast user is transmitted to at least the viewing user device 12B. In other words, the server 13 does not need to transmit tracking data and audio data of the broadcast user to the broadcast user device 12A. The server 13 transmits messages, gift output requests, etc. transmitted by the viewing user device 12B to the broadcast user device 12A.

In the gift reception process, the gift ID of the gift object and the user ID of the viewing user are received from the viewing user device 12B (first viewing user device) used by the first viewing user, along with an output request for the gift object.

In the second transmission process, the gift ID and an output request for outputting a video including the first avatar object and the second avatar object corresponding to the first viewing user are transmitted to at least the distribution user device 12A and the viewing user device 12B used by a second viewing user other than the first viewing user.

The output request sent in the second transmission process is sent to at least the distribution user device 12A and the viewing user device 12B used by a second viewing user other than the first viewing user, together with the data used to draw the second avatar object or identification information of the data.

Specifically, the output request transmitted in the second transmission process may specify data expressing the movement of the second avatar object 150 and may include a request for rendering using the specified data. The data used to render the second avatar object may include tracking data of the first viewing user, at least one of selected motion data selected by the first viewing user, and preset automatic motion data. Alternatively, the data used to render the second avatar object may include tracking data of the first viewing user, at least one of identification information of the selected motion data selected by the first viewing user, and identification information of preset automatic motion data. The identification information of the data used to render the second avatar object may be identification information such as a part ID constituting the avatar object. When the first viewing user selects the selected motion data, the viewing user device 12B used by the first viewing user transmits the selected selected motion data or its identification information to the server 13.

The server 13 transmits data that is missing from each of the user devices 12. For example, the broadcasting user device 12A acquires the tracking data of the broadcasting user by the sensor unit 24. Therefore, the server 13 does not need to transmit tracking data for drawing the first avatar object to the broadcasting user device 12A. In addition, the viewing user device 12B of the first viewing user acquires the tracking data of the first viewing user by the sensor unit 24. Therefore, the server 13 does not need to transmit tracking data for drawing the second avatar object to the viewing user device 12B used by the first viewing user.

The server 13 does not need to transmit the selected motion data to the user device 12 that has stored it. When the selected motion data is to be used, the server 13 transmits identification information (ID) of the selected motion data selected by the first viewing user together with an instruction to use the selected motion to all user devices 12 that have stored the selected motion data, or to the distributing user device 12A and the viewing user device 12B of the second viewing user among the user devices 12 that have stored the selected motion data. The user device 12 that receives the identification information of the selected motion data acquires the motion data corresponding to the ID of the selected motion data from the storage 22. The user device 12 also uses the acquired motion data to draw a second avatar object.

Similarly, the server 13 does not need to transmit the automatic motion data to the user device 12 that has stored the automatic motion. When the automatic motion data is to be used, the server 13 transmits identification information (ID) of the automatic motion data together with an instruction to use the automatic motion to all user devices 12 that have stored the automatic motion data, or to the distributing user device 12A and the viewing user device 12B of the second viewing user among the user devices 12 that have stored the automatic motion data. The user device 12 that receives the identification information of the automatic motion data acquires the motion data corresponding to the ID of the automatic motion data from the storage 22. The user device 12 also uses the acquired motion data to draw a second avatar object.

In this embodiment, in a scene where a gift object is output, the user device 12 draws the second avatar object 150 using the following (1) and (2).
(1) Tracking data of the first viewing user who transmitted the output request for the gift object.
(2) Pre-set automatic motion data.
As described above, this automatic motion data may be stored in each user device 12. Alternatively, the automatic motion data may be stored in the server 13 and transmitted to each user device 12 when it is used. The automatic motion data may include motion data used for the second avatar object and motion data used for the first avatar object. Furthermore, the automatic motion data may be linked to a gift ID. Furthermore, the automatic motion data may be linked to an attribute that is the type of the gift object.

The parts (areas) of use of the automatic motion data are specified in advance. In other words, the automatic motion data is associated with the body part ID of the avatar object. For example, the automatic motion data is data that expresses the movement of the second avatar object extending an arm part forward. For example, the automatic motion data is data that expresses the movement of associating a gift object associated with the hand part of the second avatar object with the hand part of the first avatar object.

In addition, in a scene where a gift object is output, automatic motion data is also used for the first avatar object. The automatic motion data used for the first avatar object is data that expresses the motion of extending the arms. By using automatic motion data for the first avatar object and the second avatar object, it is possible to depict the motion of the first avatar object receiving a gift object held by the second avatar object.

If a scene in which a first avatar object receives a gift object held by a second avatar object is rendered using the tracking data of the first viewing user and the tracking data of the broadcasting user, the first viewing user may become confused about what to do. In this case, the movement of the second avatar object may stop. In addition, the position at which the second avatar object extends its hand may not match the position at which the first avatar object extends its hand, and the handover of the gift object may not be rendered properly. In such a case, the progress of the video will be hindered. In response to this, the video can progress more smoothly by using automatic motion data to render scenes that are difficult to render using tracking data alone, as described above.

Next, referring to FIG. 6, the processing procedure when the server 13 outputs a gift object will be described. In the following, a request to output a gift object will be referred to as a gift output request. The following procedure will mainly describe the drawing of the second avatar object. It is assumed that the tracking data of the broadcast user is used for the first avatar object in scenes other than when the gift object is handed over. Note that in this embodiment, the second avatar object is not generally drawn in the video except in scenes when the gift object is displayed.

The server 13 determines whether or not a gift output request has been received from the viewing user device 12B used by the first viewing user (step S11). At this time, the server 13 receives the user ID of the first viewing user and the gift ID of the gift object together with the gift output request. In addition, the server 13 has received, from the viewing user device 12B of the first viewing user, the part IDs of the main body parts and the part IDs of the attached parts constituting the avatar object of the first viewing user, at the same time as or prior to the gift output request. Note that the server 13 may grant a medium such as coins or points to the broadcasting user by accepting the output request of the gift object 135. In this way, a broadcasting user who is given many gift objects 135 by viewing users can obtain more coins or medium such as points.

If the server 13 has not received a gift output request (step S11: NO), the process proceeds to step S16. If the server 13 determines that a gift output request has been received (step S11: YES), the server 13 determines whether it is time to output a gift object (step S12).

Here, the server 13 determines the timing of outputting the gift object according to the attribute (type) of the gift object. In the case of a gift object such as a stuffed toy, which does not have a specific output timing restriction, the server 13 determines that the gift object can be output when a gift output request is received. In the case of a wearable object, the server 13 notifies the broadcasting user device 12A that a gift object that is a wearable object has been given. Furthermore, when the broadcasting user performs an operation to allow the output of the wearable object, the server 13 determines that the gift object can be output. In addition, there may be restrictions on the output of the gift object, such as the display position of the gift object being fixed. For example, gift objects such as "hanging scrolls" and "tower cakes" are displayed in a predetermined display position for a certain period of time, but a new gift object cannot be output unless the gift object already displayed in the display position is erased. In such a case, the server 13 determines that the gift object can be output when the display position associated with the gift object becomes available.

When the server 13 determines that it is not time to output the gift object (step S12: NO), the server 13 waits for the timing to output the gift object.
When the server 13 determines that it is time to output the gift object (step S12: YES), it determines whether or not the first viewing user who gave the gift object is currently viewing the video (step S13). At this time, the server 13 determines whether or not the first viewing user is currently viewing the video based on, for example, receiving information indicating that the connection to the video has been disconnected from the viewing user device 12B used by the first viewing user, or receiving an error code in response to the information sent by the server 13.

When the server 13 determines that the first viewing user who sent the gift object is currently viewing a video (step S13: YES), it transmits an output request for the second avatar object and the gift object together with the gift ID to the broadcasting user device 12A, the viewing user device 12B of the first viewing user, and the viewing user device 12B of the second viewing user (step S14). For example, the server 13 transmits information required to render the second avatar object to at least the broadcasting user device 12A and the viewing user device 12B of the second viewing user. The information required to render the second avatar object may include the body part ID and the attached part ID of the second avatar object.

The server 13 also transmits, together with the output request, the tracking data received from the viewing user device 12B used by the first viewing user and the automatic motion data or the ID of the automatic motion data to at least the distribution user device 12A and the viewing user device 12B of the second viewing user. At this time, the server 13 may transmit the gift ID and the output request to the viewing user device 12B of the first viewing user.

When the broadcasting user device 12A and the viewing user device 12B of the second viewing user receive the gift output request along with the gift ID, and the part IDs of the body parts and the attached parts of the second avatar object, they create a gift object corresponding to the gift ID and draw an image of the second avatar object. At this time, they draw the second avatar object using automatic motion data. They also output the gift object in association with the second avatar object.

For example, a second avatar object is drawn with a "bouquet" gift object associated with the hand. The tracking data is also used as part of the second avatar object. For example, the face of the second avatar object is drawn using face tracking data of the first viewing user, and the facial expression of the first viewing user is reflected in the second avatar object. There are no particular limitations on the timing of using the automatic motion data and the timing of using the tracking data. The tracking data may be used after the second avatar object is drawn using the automatic motion data. Or, the automatic motion data may be used after the second avatar object is drawn using the tracking data. Or, as described above, the automatic motion data may be used for a specific part of the second avatar object, while the tracking data may be used for a part of the second avatar object other than the specific part.

When the viewing user device 12B of the first viewing user receives a gift output request from the server 13 or accepts an operation to give a gift object, it draws a gift object corresponding to the gift ID selected by the first viewing user. In addition, the viewing user device 12B of the first viewing user outputs an image of the second avatar object. At this time, the viewing user device 12B of the first viewing user outputs the gift object in association with the second avatar object based on the automatic motion data.

At this time, each user device 12 also uses automatic motion data for the first avatar object. For example, the first avatar object is drawn associated with the hand part so that the first avatar object receives a "bouquet" gift object held by the second avatar object. Furthermore, the user device 12 may use the broadcasting user's tracking data as part of the first avatar object. For example, the face of the first avatar object is drawn using the broadcasting user's face tracking data, and the facial expression of the broadcasting user is reflected in the first avatar object. In this case, the server 13 receives the tracking data from the broadcasting user device 12A and transmits it to each viewing user device 12B.

On the other hand, if the server 13 determines that the first viewing user who sent the gift object is not currently viewing the video (step S13: NO), it outputs the gift object without outputting the second avatar object (step S15). Specifically, the server 13 transmits a gift ID to the user device 12. Upon receiving the gift ID, the user device 12 draws a gift object corresponding to the gift ID.

In other words, there may be some lag between the timing when the viewing user device 12B used by the first viewing user transmits the gift output request and the timing when the gift object is output. Also, depending on the type of gift object, it may take some time for the gift object to be output. And, there is a possibility that the first viewing user will finish viewing between the time when the viewing user device 12B transmits the gift output request and the time when the gift object is output.

If the second avatar object is output even though the first viewing user has finished watching the video before the gift object is output, it is difficult to use the tracking data of the first viewing user for the second avatar object. Furthermore, if there is a user who is aware that the first viewing user has finished watching, it will feel strange to have the second avatar object corresponding to that first viewing user output. However, if the gift object selected by spending points or coins is not output to the video just because the first viewing user is not watching the video, the satisfaction of the distribution user and the first viewing user will decrease. Therefore, in such a case, the second avatar object is not output, and only the gift object is output to the video.

In step S16, if the server 13 does not end the distribution (step S16: NO), the process returns to step S11 and repeatedly determines whether or not a gift output request has been received. If the server 13 has ended the distribution (step S16: YES), the process ends.

(Viewing screen)
Next, screens that the user device 12 outputs to the display 28 will be described with reference to FIGS.

The home screen 101 will be described with reference to FIG. 7. When the user launches the video application, the user device 12 receives a list of videos being distributed from the server 13. The user device 12 outputs the home screen 101 to the display 28 based on the list. A video selection section 102 indicating the videos that can be viewed is output to the home screen 101. When the user selects the video selection section 102, the user device 12 transmits the ID of the selected video together with a viewing request to the server 13. The server 13 updates the number of viewing users and transmits data for rendering the video and audio data to the user device 12.

The viewing user device 12B, which receives data for rendering a video, generates video data using the received data. It then outputs a viewing screen to the display 28 based on the video data.

The viewing screen and the distribution screen output to the distribution user device 12A have different GUIs (Graphical User Interfaces), but with some exceptions, the content of the video including the avatar object is almost the same. For this reason, the viewing screen 100 will be illustrated in the description in Figures 8 to 11.

Figure 8 shows an example of the viewing screen 100. The viewing screen 100 includes an image of a first avatar object 110 corresponding to the broadcast user. Tracking data based on the movements of the broadcast user is used for the first avatar object 110. Drawing data of worn parts 111 is used for the first avatar object 110. Non-worn parts 125, which are wallpaper, are used in the virtual space. In addition, the viewing user device 12B outputs sound from the speaker 25 in synchronization with the movements of the first avatar object 110.

The viewing screen 100 also includes a viewer count 112 and a total heart count 113. The viewer count 112 is the cumulative number of viewers who have watched the video. Alternatively, the viewer count 112 may be the number of viewers currently watching the video. Furthermore, the total heart count 113 is the number of times viewers have given the video a positive rating since the video started.

The viewing screen 100 also includes a viewing ranking button 114, a distributor ranking button 115, a share button 116, and a follow button 117. When the viewing user operates the viewing ranking button 114, the viewing user device 12B outputs a ranking of viewing users who have viewed videos distributed by that distributor, based on the number of gifts sent. The distributor ranking button 115 outputs a ranking of broadcast users, based on the number of gifts sent, among all broadcast users who distribute videos.

When the viewing user selects the share button 116, the viewing user device 12B transmits the link (URL) of the video via another social network service, copies the link, or reads the follow list of mutually following users. When the viewing user selects the follow button 117, the server 13 associates this viewing user with the distributor user who distributes this video.

The viewing screen 100 also includes a message input section 118 and a message display section 119. The message input section 118 is an input form that allows viewing users to send messages to the broadcast user. The message display section 119 is a message sent by each viewing user device 12B, or a standard message sent by the server 13 based on the action of the viewing user. The message display sections 119 are output so that the most recent message is at the top (or bottom), and when a new message display section 119 is added, it moves downward (or upward), and is erased when it reaches the bottom (or top) of the displayable area of the message display section 119.

The viewing screen 100 also includes a camera button 120, a good rating button 121, and a gift button 122. When the camera button 120 is selected, the viewing user device 12B captures the viewing screen 100 being output. The captured viewing screen 100 is stored in the storage 22. When the good rating button 121 is selected, the viewing user device 12B outputs an indicator indicating a good rating, such as a heart mark, to the viewing screen 100. The viewing user device 12B also transmits a request to the server 13 to reflect the good rating. When the server 13 receives the request to reflect the good rating, it updates the total number of hearts 113.

When the gift button 122 is selected by the first viewing user, the viewing user device 12B outputs a list of gift objects based on the gift data 53, superimposed on the viewing screen 100. The first viewing user selects a gift from the list of gift objects. The viewing user device 12B transmits a gift output request to the server 13. Note that the viewing user device 12B of the first viewing user may transmit the user ID of the first viewing user and the gift ID of the gift object selected by the first viewing user together with the gift output request.

Figure 9 shows a viewing screen 100 that is output by the viewing user device 12B after selecting a gift object. The viewing screen 100 includes a slot change unit 139. The first viewing user operates the slot change unit 139 to select an arbitrary slot number. A second avatar object 150 corresponding to the slot number is output on the viewing screen 100. The second avatar object 150 uses wearable parts 111 that have been registered in advance by the first viewing user.

In addition, an attachment part selection unit 130 is output on the viewing screen 100. A second avatar object 150 corresponding to the first viewing user who sent the gift output request is output to the video in order to deliver the gift object. Therefore, the first viewing user selects attachment parts in the attachment part selection unit 130 to make the avatar object suitable for appearing in the video. For example, in a scene to celebrate the broadcasting user, the first viewing user selects an outfit suitable for that scene. If a theme is set for the video, the first viewing user selects an outfit that matches the theme. The first viewing user may also select an outfit that matches the season in the real world.

The viewing screen 100 as illustrated in FIG. 9 may be output before the first viewing user selects a gift object. In this case, the first viewing user can change the second avatar object 150 and the attached parts before selecting a gift object.

In addition, after the first viewing user selects a gift object, a viewing screen 100 as exemplified in FIG. 9 may be output, and when the second avatar object 150 and wearable parts are determined on the viewing screen 100, the viewing user device 12B of the first viewing user may transmit a gift output request. Furthermore, the viewing screen 100 may include a button for transmitting the gift output request. When the first viewing user selects the button, the viewing user device 12B transmits the gift output request.

Similarly, the wearable parts selection section 130 may be output to the broadcast screen displayed on the broadcast user device 12A. In this case, a notification of receiving a gift object is displayed on the broadcast screen. The broadcast user confirms this notification and causes a selection section similar to the wearable parts selection section 130 on the viewing screen 100 to be displayed. The broadcast user selects wearable parts to make the avatar object suitable for receiving the gift object 135.

The attachment part selection unit 130 includes a category selection unit 131 and an attachment part display unit 132. The category selection unit 131 indicates each category of attachment parts. The attachment part display unit 132 indicates attachment parts available to the user. The first viewing user selects a category and selects attachment parts output for each category. Alternatively, the first viewing user cancels the selection of attachment parts that are already being used. When the viewing user device 12B accepts the selection of attachment parts, it uses the selected attachment parts for the second avatar object. Furthermore, when the viewing user device 12B accepts the cancellation of the selection of attachment parts, it does not use the attachment parts specified for the avatar object.

When the first viewing user has finished selecting the wearable parts, he/she selects the save button 133. When the viewing user device 12B accepts the selection of the save button 133, it transmits to the server 13 the part IDs of the main body parts of the second avatar object 150 and the part IDs of the used wearable parts.

Figure 10 shows a viewing screen 100 in a scene where a gift object is handed over. The example in Figure 10 is a viewing screen 100 that is output when a viewing user is watching a video. The viewing screen 100 in Figure 10 shows an example in which the position (viewpoint) of the virtual camera is set to an overhead viewpoint above the first avatar object 110 and the second avatar object 150. The user device 12, including the viewing user device 12B, renders a video in a virtual space as seen from the set virtual camera position. The position of the virtual camera is not particularly limited. In this embodiment, the position of the virtual camera is a common position among the distribution user device 12A, the viewing user device 12B of the first viewing user, and the viewing user device 12B of the second viewing user.

For example, the position of the virtual camera may be set to the position of the first avatar object 110, the position of the second avatar object 150, or another position. The position of the first avatar object 110 is, for example, the position of the eyes of the first avatar object 110. The position of the second avatar object 150 is, for example, the position of the eyes of the second avatar object 150. When the user device 12 determines that the virtual camera is at the position of the first avatar object 110, the user device 12 outputs a first-person perspective video of the first avatar object 110 to the display 28. When the user device 12 determines that the virtual camera is at the position of the second avatar object 150, the user device 12 outputs a first-person perspective video of the second avatar object 150 to the display 28. The center of view, which is the line of sight of the virtual camera, may be constant or may be changed based on the user's operation of the operation unit 27.

A first avatar object 110, a gift object 135, and a second avatar object 150 are displayed on the viewing screen 100. The second avatar object 150 and the gift object 135 are associated with each other. Specifically, the second avatar object 150 is displayed as if it is holding the gift object 135 by associating a hand part 150A with the gift object 135.

The viewing user device 12B draws the second avatar object 150 using automatic motion data. This automatic motion data is motion data that expresses the action of handing over the gift object 135. The automatic motion data is a motion of rotating the arm parts 150B that are lowered on both sides of the second avatar object 150 approximately 90 degrees upward. This outputs an animation of the second avatar object 150 holding the gift object 135 in its hand out in front of it to hand over the gift object 135 to the first avatar object 110.

On the other hand, while the automatic motion data is used for the second avatar object 150, the tracking data of the first viewing user is used for the facial part 150C (first part) of the second avatar object 150. As a result, the facial expression of the first viewing user is reflected in the second avatar object 150. When a non-display condition for hiding the second avatar object 150 is satisfied, the broadcasting user device 12A and the viewing user device 12B put the second avatar object 150 in a non-display state. An example of a non-display condition is that the second avatar object 150 has completed an operation. For example, the second avatar object 150 has handed over the gift object 135 to the first avatar object 110. When the operation of the second avatar object 150 is completed, the second avatar object 150 is put in a non-display state. In the non-display state, the second avatar object 150 is hidden or moves outside the display area of the screen to exit the screen. Automatic motion data is used for the exiting action of the second avatar object 150.

The determination of whether the motion of the second avatar object 150 is completed may be made not only by automatic motion data, but also by (A) collision determination, or (B) determination of whether the position of the gift object 135 and the position of the first avatar object 110 are within a predetermined distance. In the collision determination of (A), when the gift object 135 comes into contact with a predetermined part of the first avatar object 110 associated with the gift object 135, it is determined that the motion of the second avatar object 150 is completed. In other words, it is determined that the non-display condition is satisfied. In (B), when the relative distance between the position of the gift object 135 and the position of the first avatar object 110 is within a predetermined distance, it is determined that the motion of the second avatar object 150 is completed. The non-display condition may be a condition according to the gift object 135. For example, when the gift object 135 is of a type to which (A) and (B) cannot be applied, the non-display condition may be that the playback of the animation of the gift object 135 is completed. Gift objects 135 of a type to which (A) and (B) cannot be applied are animations that are expressed with one or more two-dimensional particles, such as "fireworks," "rain," and "flames."

Note that a notification such as "Your avatar will be displayed" may be output before the second avatar object 150 is output to the video. After displaying this notification, the second avatar object 150 may be output to the video. Alternatively, a notification of "Do you want to appear?" and a button to approve the appearance and a button to cancel the appearance may be output. If the first viewing user selects the approval button, the second avatar object 150 may be output to the video, and if the cancel button is selected, the second avatar object 150 may not be output. That is, if the first viewing user selects the approval button, the viewing user device 12B of the first viewing user transmits an output permission notification of the second avatar object 150 to the server 13. Upon receiving the output permission notification, the server 13 determines the timing to output the second avatar object 150. Then, the server 13 transmits an output request of the second avatar object 150 to the distribution user device 12A and each viewing user device 12B based on the timing to output the second avatar object 150. The server 13 also determines the timing for outputting the gift object, and transmits an output request for the gift object to the broadcasting user device 12A and each viewing user device 12B based on the timing. The broadcasting user device 12A and each viewing user device 12B draw the second avatar object 150 based on the output request for the second avatar object 150 and the output request for the gift object.

In addition, if the first viewing user selects the cancel button, the viewing user device 12B of the first viewing user transmits an output stop notification for the second avatar object 150 to the server 13. Upon receiving the output stop notification, the server 13 determines the timing for outputting the gift object, and transmits an output request for the gift object to the broadcasting user device 12A and each viewing user device 12B based on the timing. The broadcasting user device 12A and each viewing user device 12B draw the gift object 135 based on the drawing instruction for the gift object.

In this way, by automatically using the motion data for passing the gift object 135 to the second avatar object 150, the process of passing the gift object 135 can be smoothly carried out. In addition, by using the first viewing user's own tracking data for the face part 150C of the second avatar object 150, the first viewing user can easily get the feeling that he or she is actually appearing in the video.

Figure 11 shows the viewing screen 100 in a scene where a gift object is output. The example in Figure 11 shows the viewing screen 100 that is output when the first viewing user is not watching a video when the gift object is output.

A first avatar object 110 and a gift object 135 are output on the viewing screen 100. In this case, the second avatar object 150 is not output on the viewing screen 100. The gift object 135 is output in the virtual space. The user device 12 may place the gift object 135 at a predetermined position, such as on the floor. The user device 12 may also draw the gift object 135 so that it falls at a specific position in the virtual space.

In this way, even if the first viewing user is not watching the video, the gift object 135 sent by the first viewing user is output. This can increase the satisfaction of the distribution user and the first viewing user.

The effects of the first embodiment will be described.
(1-1) In the first embodiment, the user device 12 uses automatic motion data for the second avatar object corresponding to the first viewing user. This allows appropriate and smooth output of scenes that are difficult to render using tracking data alone. Furthermore, the user device 12 uses the tracking data of the first viewing user for the second avatar object in addition to the automatic motion data, allowing the first viewing user to feel as if he or she is actually appearing in the video. This allows the viewing user to be more satisfied with the video.

(1-2) In the first embodiment, at least some parts of the second avatar object 150 are drawn using tracking data of the first viewing user, and at least some other parts of the second avatar object are drawn using automatic motion data. For this reason, automatic motion data can be used for parts that are difficult to express using tracking data. In addition, tracking data can be used for parts of the second avatar object 150 that are more likely to reflect the intentions of the first viewing user.

(1-3) In the first embodiment, the tracking data of the first viewing user is used for the first part of the second avatar object 150, such as the face part 150C, and automatic motion data is used for the second part of the second avatar object 150, such as the hand part 150A and the arm part 150B. This makes it possible to smoothly output the action of handing over the gift object 135 while reflecting the facial expression of the first viewing user in the second avatar object 150. In addition, by using the automatic motion data in this way, the process of acquiring tracking data and the process of transmitting tracking data by the viewing user device 12B of the first viewing user can be omitted. This makes it possible to render the second avatar object 150, which reflects the intention of the first viewing user and has individuality, while suppressing the processing load on the viewing user device 12B of the first viewing user.

(1-4) In the first embodiment, when the gift object 135 is output, if the first viewing user is not continuing to watch the video, the second avatar object 150 is not output. This makes it possible to prevent the second avatar object 150 corresponding to the first viewing user from being output even though the first viewing user is not watching.

Second Embodiment
Next, a second embodiment of the information processing system will be described. The second embodiment differs from the first embodiment in that the first viewing user can select an operation. Hereinafter, the same reference numerals are used to designate the same parts as those in the first embodiment, and detailed descriptions thereof will be omitted.

In this embodiment, user device 12 renders second avatar object 150 using the following (2) and (3).
(2) Pre-set automatic motion data.
(3) Selected motion data selected by the first viewing user from among the motion data prepared in advance.
12 , a process will be described in which the viewing user device 12B of the first viewing user draws the second avatar object 150 and the gift object 135. In this embodiment, it is assumed that the first viewing user is watching a video when the gift object 135 is output.

When the first viewing user selects a gift object, the control unit 20 of the viewing user device 12B of the first viewing user transmits the gift ID of the selected gift object and the user ID of the first viewing user to the server 13. The control unit 20 outputs the gift object 135 and the second avatar object 150 to the display 28 in response to an operation of the first viewing user or based on the gift ID received from the server 13 (step S20).

Next, the control unit 20 outputs the emote selection unit to the video (step S21). The emote selection unit is a GUI for selecting an action of the second avatar object 150. For example, the emote selection unit includes a plurality of selection buttons for expressions (hereinafter referred to as emotes) of the second avatar object 150, such as "handing over with outstretched arms" and "handing over while kneeling". The first viewing user selects one of the selection buttons. Note that the order of outputting the gift object 135 and the second avatar object 150 to the display 28 (step S20) and outputting the emote selection unit to the video (step S21) is not limited. It is also possible to perform the process of step S21 before the process of step S20.

The control unit 20 receives the emote selected by the first viewing user (step S22), and transmits information indicating the selected emote to the server 13.
The control unit 20 acquires selected motion data corresponding to the emote selected by the first viewing user (step S23). The viewing user device 12B may acquire the selected motion data stored in advance in the storage 22 from the storage 22. At this time, the viewing user device 12B may receive identification information of the selected motion data and an output request from the server 13. In this case, the viewing user device 12B may use the selected motion data for the second avatar object 150 when it receives the identification information of the selected motion data and the output request. Alternatively, the viewing user device 12B may transmit identification information indicating an emote to the server 13 and then receive the selected motion data corresponding to the emote from the server 13.

The viewing user device 12B then uses the acquired selected motion data for the second avatar object 150 (step S24). For example, if the first viewing user selects the emote "hand over with both hands outstretched," the user device 12 uses the selected motion data of handing over the gift object 135 with both hands outstretched for the second avatar object 150. The viewing user device 12B and the broadcasting user device 12A used by the second viewing user receive the selected motion data from the server 13, or receive identification information of the selected motion data together with an output request. The viewing user device 12B and the broadcasting user device 12A used by the second viewing user then use the acquired selected motion data for the second avatar object 150.

When the control unit 20 finishes the operation based on the selected motion data, the control unit 20 acquires automatic motion data (step S25). The acquired automatic motion data is then used for the second avatar object 150 (step S26). For example, the automatic motion data is "shake hands". As a result, the second avatar object 150 performs the action of shaking hands by placing the hand part 150A over the hand part of the first avatar object 110. After automatically using the automatic motion data, the viewing user device 12B causes the second avatar object 150 to exit. Note that the process related to the selected motion, which includes outputting the emote selection unit to the video (step S21), accepting the emote selected by the first viewing user (step S22), acquiring selected motion data corresponding to the emote selected by the first viewing user (step S23), and using the acquired motion data for the second avatar object 150 (step S24), and the process related to the automatic motion, which includes acquiring automatic motion data (step S25) and using the acquired automatic motion data for the second avatar object 150 (step S26), may be performed in the reverse order. In other words, the process related to the automatic motion may be performed before the process related to the selected motion.

The automatic motion data may be stored in each user device 12. In this case, the user device 12 receives identification information for the automatic motion data from the server 13. Alternatively, each user device 12 may receive the automatic motion data from the server 13.

Figure 13 shows a viewing screen 100 when the viewing user device 12B outputs a gift object. The example in Figure 13 is a viewing screen 100 that is output when a first viewing user is watching a video. The viewing screen 100 includes an emote selection unit 140.

A first avatar object 110, a gift object 135, and a second avatar object 150 are displayed on the viewing screen 100. The second avatar object 150 and the gift object 135 are associated with each other.

The emote selection unit 140 is output in association with the second avatar object 150. Specifically, the emote selection unit 140 is output so as to overlap with or in the vicinity of the second avatar object 150. The emote selection unit 140 includes selection buttons 140A and 140B. The selection button 140A is a button for selecting an action such as "handing over with both hands extended." The selection button 140B is a button for selecting an action such as "handing over while kneeling." The action of the second avatar object 150 is determined depending on which of the selection buttons 140A and 140B the first viewing user selects. When the first viewing user selects the selection button 140A, the selection motion data for handing over the gift object 135 with both hands extended is used for the second avatar object 150. On the other hand, when the first viewing user selects the selection button 140B, the selection motion data for handing over the gift object 135 while kneeling is used for the second avatar object 150.

Once the viewing user device 12B and the broadcasting user device 12A have completed using the selected motion data, they apply the automatic motion data to the second avatar object 150.
According to the second embodiment, in addition to at least the effects described in (1-4) of the first embodiment, the following effects can be obtained.

(2-1) In the second embodiment, the user device 12 uses automatic motion data for the second avatar object 150 corresponding to the first viewing user. This allows appropriate and smooth output of scenes that are not easy to draw using tracking data alone. On the other hand, the user device 12 uses selected motion data selected by the first viewing user for the second avatar object 150 in addition to the automatic motion data, allowing the first viewing user to feel as if he or she is actually appearing in the video. This increases the first viewing user's satisfaction with the video. Furthermore, by using the selected motion data in this way, the viewing user device 12B of the first viewing user can omit the process of acquiring tracking data and the process of transmitting tracking data. This allows a variety of videos to be output while reducing the processing load on the viewing user device 12B of the first viewing user.

(2-2) In the second embodiment, automatic motion data may be used for the second avatar object 150, and selected motion data selected by the first viewing user may also be used. In this case, it is possible to smoothly output the action of handing over the gift object 135, while allowing the second avatar object 150 to reflect the intention of the first viewing user.

(2-3) In the second embodiment, an emote selection unit 140 is output for selecting selected motion data. This makes it easier for the first viewing user to select selected motion data.

Third Embodiment
Next, a third embodiment of the information processing system will be described. In this embodiment, user device 12 renders second avatar object 150 using the following (1) and (3).
(1) Tracking data of the first viewing user who transmitted the output request for the gift object.
(3) Selected motion data selected by the first viewing user from among the motion data prepared in advance.

Hereinafter, the same parts as those in the first embodiment will be denoted by the same reference numerals and detailed description thereof will be omitted.
14, a process in which the viewing user device 12B renders the second avatar object 150 and the gift object 135 will be described. Note that in this embodiment, it is assumed that the first viewing user is viewing a video when the gift object 135 is output.

The viewing user device 12B used by the first viewing user performs steps S30 to S34 in the same manner as steps S20 to S24 in the second embodiment. The viewing user device 12B and the broadcasting user device 12A used by the second viewing user draw the second avatar object 150 based on the selected motion data or identification information designating the selected motion data received from the server 13. Note that the order of outputting the gift object 135 and the second avatar object 150 to the display 28 (step S30) and outputting the emote selection section to the video (step S31) is not limited. It is also possible to perform the processing of step S31 before the processing of step S30.

Furthermore, the control unit 20 of the viewing user device 12B used by the first viewing user acquires tracking data of the first viewing user (step S35). The control unit 20 also draws the second avatar object 150 based on the acquired tracking data (step S36). The distribution user device 12A and the viewing user device 12B used by the second viewing user receive the tracking data from the server 13 and draw the second avatar object 150 using the tracking data. As a result, the second avatar object 150 output on the viewing screen 100 moves based on the selected motion data and then moves based on the motion of the first viewing user. Note that the process related to the selected motion, which includes outputting the emote selection unit to the video (step S31), accepting the emote selected by the first viewing user (step S32), acquiring selected motion data corresponding to the emote selected by the first viewing user (step S33), and using the acquired motion data for the second avatar object 150 (step S34), and the process using the tracking data, which includes acquiring tracking data (step S35) and using the acquired tracking data for the second avatar object 150 (step S36), may be performed in the reverse order. In other words, the process using the tracking data may be performed before the process related to the selected motion.

In this way, by using the selected motion data prepared in advance for the second avatar object 150, the progress of the video can be made smooth. In addition, the first viewing user can select the motion of the second avatar object 150, so that the first viewing user can feel as if the second avatar object 150 is appearing in the video. Furthermore, in a scene where the gift object 135 is handed over, the motion of the second avatar object 150 may be restricted because the second avatar object 150 is holding the gift object 135. Alternatively, if the tracking data is used while the second avatar object 150 is holding the gift object 135, depending on the type of selected motion, the appearance may appear unnatural. However, in this embodiment, the selected motion data is acquired in advance, and when the motion corresponding to the selected motion data is a motion of handing over a gift object, the tracking data is used for the second avatar object 150 after the association between the gift object 135 and the second avatar object 150 is released, so that the motion can be made natural without restrictions.

According to the third embodiment, in addition to at least the effects described in (1-4) of the first embodiment and (2-3) of the second embodiment, the following effects can be obtained.
(4-1) In the third embodiment, the user device 12 uses the selected motion data for the second avatar object 150 corresponding to the first viewing user. This makes it possible to appropriately and smoothly output scenes that are difficult to draw using only tracking data. In addition, the first viewing user can get a sense of appearing in the video. Furthermore, the user device 12 uses the tracking data of the first viewing user for the second avatar object 150 in addition to the selected motion data, so that the first viewing user can get a sense of appearing in the video. This makes it possible to increase the first viewing user's satisfaction with the video.

The above-described embodiments may be modified as follows: The above-described embodiments and the following modifications may be combined with each other to the extent that no technical contradiction occurs.
<User>
In the above embodiments, the distribution user refers to a user who transmits at least one of information related to video and information related to audio. For example, the distribution user is a user who performs independent video distribution or collaborative distribution in which multiple people can participate. Alternatively, the distribution user may be a user who hosts or organizes a video chat or voice chat in which multiple people can participate and/or watch, or an event (such as a party) in a virtual space in which multiple people can participate and/or watch. In other words, the distribution user can also be referred to as a host user, organizer user, or organizer user.

On the other hand, a viewing user refers to a user who receives at least one of the video-related information and audio-related information. However, a viewing user can be a user who is not only able to receive the above information but also able to react to it. For example, a viewing user is a user who watches a video broadcast or a collaboration broadcast. Or, a viewing user may be a user who at least participates in and watches a video chat, a voice chat, or an event. Therefore, a viewing user can also be referred to as a guest user, a participating user, a listener, a viewing user, a supporting user, etc.

The information processing system in each of the above embodiments is a digital world in which many people can participate simultaneously and freely engage in activities such as interacting, working, and playing through character objects (avatars) at a level close to that of the real world, and can be used to provide the next Internet space (metaverse) in which social activities can take place beyond the gap between reality and virtuality.

In such a space, users' avatar objects can roam freely within the world and communicate with each other.
One of the avatar objects in the virtual space may be configured to distribute video as a first avatar object of the broadcasting user. That is, one-to-many video distribution may be performed in a many-to-many metaverse virtual space. In such a space, there may be no particular distinction between the broadcasting user and the viewing user.

<Movement and Drawing of Second Avatar Object>
In the first embodiment, tracking data is used for the first part of second avatar object 150, and automatic motion data is used for the second part. Alternatively, tracking data may be used for second avatar object 150, and then automatic motion data may be used. Or, tracking data may be used after a period in which automatic motion data is used.

- In the second and third embodiments, the emote selection unit 140 may be output when an emote selection condition is met. The emote selection condition is, for example, that the sensor unit 24 of the viewing user device 12B used by the first viewing user does not detect any motion. When the first viewing user is unsure of what action to take, the sensor unit 24 does not detect any motion. In such a case, the emote selection unit 140 can be output to make it easier for the first viewing user to select an action. The emote selection condition may also be that the first viewing user has performed a predetermined motion. The predetermined motion is, for example, that the first viewing user has turned away. The emote selection condition may also be that an instruction to display the emote selection unit 140 on the screen has been given. In this way, the emote selection unit 140 can be output when the first viewing user is unsure of how to respond when appearing in the video.

In each of the above embodiments, the second avatar object 150 is operated using tracking data obtained by detecting the movement of the first viewing user or selected motion data selected by the viewing user from among a plurality of preset motion data, and automatic motion data automatically used for the second avatar object 150. In addition to this, the second avatar object 150 may be operated using tracking data obtained by detecting the movement of the first viewing user, selected motion data selected by the first viewing user from among a plurality of preset motion data, and automatic motion data automatically used for the second avatar object 150. That is, the second avatar object 150 may be operated using all of (1) the tracking data of the first viewing user who transmitted the output request for the gift object, (2) the preset automatic motion data, and (3) the selected motion data selected by the first viewing user from among the motion data prepared in advance.

In each of the above embodiments, the second avatar object 150 is operated using tracking data that detects the movement of the first viewing user or selected motion data selected by the first viewing user from among multiple preset motion data, and automatic motion data that is automatically used for the second avatar object 150. Alternatively, the second avatar object 150 may be operated using any one of (1) tracking data of the first viewing user who transmitted a request to output a gift object, (2) preset automatic motion data, and (3) selected motion data selected by the first viewing user from among previously prepared motion data.

- Gift management information that associates the gift ID of the gift object 135 with operation data for operating the second avatar object 150 may be stored in at least one of the server 13 and the user device 12.

In the second output control process, the user device 12 identifies the motion data associated with the gift ID (identification information of the gift object) based on the identification information of the motion data that moves the second avatar object 150, and uses the motion data to draw the gift object 135 and the second avatar object 150. The user device 12 may receive the gift ID from the server 13 and identify the identification information of the motion data associated with the gift ID based on the gift management information. The user device 12 may then draw the second avatar object 150 using the motion data corresponding to the identified identification information of the motion data. Alternatively, the user device 12 may receive the identification information of the motion data from the server 13 and draw the second avatar object 150 using the motion data associated with the identification information. Alternatively, the user device 12 may receive the motion data associated with the gift ID from the server 13 and draw the second avatar object 150 using the motion data associated with the identification information. In this embodiment, when drawing the gift object 135 and the second avatar object 150, the second avatar object 150 is drawn using at least one of the tracking data, the selection motion data, and the automatic motion data.

The gift management information will now be described. The gift ID included in the gift management information is associated with at least one of the multiple motion data for each gift ID. The multiple motion data are (1) tracking data of the first viewing user who transmitted an output request for the gift object, (2) preset automatic motion data, and (3) pre-prepared motion data. The pre-prepared motion data (3) is motion data that can be selected by the user as selected motion data. Alternatively, the gift ID included in the gift management information may be associated with the type of gift. The type of gift is associated with at least one of the multiple motion data. Furthermore, when the gift ID is associated with multiple of the above (1) to (3), the gift ID is associated with the order in which they are used.

The contents of the action data of (2) and (3) differ depending on the gift object 135. For example, in the gift management information, if the gift object 135 is a "bouquet", the gift ID may be associated with data of an action of passing the gift object 135. Also, in the gift management information, if the gift object 135 is a "snowball", the gift ID may be associated with data of an action of throwing the gift object 135. Alternatively, in the gift management information, the gift object 135 of "bouquet" may be associated with a type indicating "passing". The type indicating "passing" may be associated with data of an action of passing the gift object 135. The gift object 135 of "snowball" may be associated with a type indicating "throwing". The type indicating "throwing" may be associated with data of an action of throwing the gift object 135.

A case will be described in which only the server 13 stores the gift management information and the user device 12 stores the motion data (2) and (3) in advance in the storage 22. In this case, the server 13 determines the motion data corresponding to the gift ID received from the viewing user device 12B used by the first viewing user based on the gift management information. At this time, the server 13 determines that the motion data (2) or (3) is associated with the gift ID. The server 13 transmits identification information of the motion data (2) or (3) to each user device 12. The user device 12 receives the identification information of the motion data from the server 13. Then, the user device 12 acquires the motion data corresponding to the received identification information from the storage 22. The user device 12 also uses the acquired motion data to draw the second avatar object 150 by the method described in each of the above embodiments.

A case will be described in which only the server 13 stores the gift management information and the user device 12 does not store the motion data in the storage 22 in advance. In this case, the server 13 transmits the motion data corresponding to the gift ID received from the viewing user device 12B used by the first viewing user to each user device 12 based on the gift management information. The server 13 may obtain the motion data of (2) or (3) from the storage 35. Alternatively, the server 13 may receive the motion data of (2) or (3) from the viewing user device 12B used by the first viewing user. In addition, when the server 13 receives the tracking data from the viewing user device 12B used by the first viewing user, it transmits the tracking data to each user device 12. The user device 12 receives the motion data from the server 13. In addition, the user device 12 uses the received motion data to draw the second avatar object 150 by the method described in each of the above embodiments.

A case will be described where the user device 12 stores gift management information in the storage 22. In this case, the server 13 transmits the gift ID received from the viewing user device 12B used by the first viewing user to each user device 12 based on the gift management information. The user device 12 identifies the motion data corresponding to the gift ID received from the server 13 based on the gift management information. If the motion data is the tracking data (1), the user device 12 that has not acquired the tracking data receives the tracking data from the server 13. If the motion data is (2) or (3), the user device 12 acquires the identified motion data from the storage 22. The user device 12 also draws the second avatar object 150 using the motion data received from the server 13 or the acquired motion data in the manner described in each of the above embodiments.

In addition to or instead of associating the gift ID with the identification information of the motion data, the gift management information may associate the gift ID with avatar display information indicating whether or not to display the second avatar object 150. The avatar display information may be a flag indicating whether or not to display the second avatar object 150. Alternatively, the server 13 or the user device 12 may determine not to display the second avatar object 150 when the identification information of the motion data is not associated with the gift ID. When the user device 12 determines to display the second avatar object 150 based on the avatar display information, it draws the gift object 135 and the second avatar object 150. When the user device 12 determines not to display the second avatar object 150 based on the avatar display information, it draws the gift object 135 and does not display the second avatar object 150.

In addition, when the gift ID and the tracking data (1) are associated as operation data, the gift management information may further associate the gift ID with a usage time, which is a time for using the tracking data. That is, as one of the conditions for hiding the second avatar object 150, in addition to or instead of the above (A) collision determination and (B) determination of whether the position of the gift object 135 and the position of the first avatar object 110 are within a predetermined distance, (C) the usage time may be set. When drawing the second avatar object 150 using the tracking data, the user device 12 draws the second avatar object 150 using the tracking data only for the time indicated by the usage time. When the time for drawing the second avatar object 150 reaches the usage time, the user device 12 causes the second avatar object 150 to exit or hides the second avatar object 150.

In addition, when the action of the second avatar object 150 is determined in the gift management information in this manner, the condition for hiding the second avatar object 150 may be that the action of the second avatar object 150 has been completed. Whether or not the action of the second avatar object 150 has been completed may be determined based on the above-mentioned (A) collision determination, (B) determination of whether or not the position of the gift object 135 and the position of the first avatar object 110 are within a predetermined distance, or (C) a predetermined usage time.

- In each of the above embodiments, the second avatar object 150 may be output using data that imposes a smaller processing load on the drawing than the first avatar object 110. For example, the second avatar object 150 may be data with a smaller number of polygons than the first avatar object 110, two-dimensional data, data that integrates multiple parts, etc. The two-dimensional data is data for drawing a two-dimensional object. When drawing the second avatar object 150 with two-dimensional data, the position of the virtual camera is fixed to a predetermined position. The predetermined position is a position where drawing the three-dimensional first avatar object 110 and the two-dimensional second avatar object 150 does not create an awkward feeling. In this aspect, the load on the drawing process of the user device 12 can be reduced. When drawing multiple second avatar objects 150, there is a problem that the processing load on the user device 12 becomes large. By reducing the processing load on each second avatar object 150 in this way, the processing load when drawing multiple second avatar objects 150 can also be reduced. This provides more opportunities for multiple second avatar objects 150 to appear in a video.

In the above embodiments, the second avatar object 150 and the gift object 135 are output when the first viewing user who transmitted the gift output request is watching a video, and the gift object 135 is output when the first viewing user is not watching a video. Alternatively, the display mode of the gift object 135 may be changed depending on whether the first viewing user who transmitted the gift output request is watching a video. For example, when the first viewing user who transmitted the gift output request for "snowball" is watching a video, the second avatar object 150 and the snowball which is the gift object 135 may be output so as to be thrown at the first avatar object 110. Also, when the first viewing user is not watching a video, an effect of falling snow may be executed as the gift object 135 from above the head of the first avatar object 110. In this case, the effect of falling snow becomes the gift object. In other words, at least a part of the color, shape, and movement of the gift object 135 may be changed depending on whether the first viewing user who transmitted the gift output request is watching a video. According to this aspect, the variation of the display mode of the gift object 135 can be increased.

- In each of the above embodiments, the second avatar object 150 and the gift object 135 are output when the first viewing user who sent the gift output request is watching a video, and the gift object 135 is output when the first viewing user is not watching a video. Alternatively, the second avatar object 150 may be displayed regardless of whether the first viewing user is watching a video or not. In other words, step S13 may be omitted.

- The motion data used for the first avatar object 110 and the second avatar object 150 is not limited to those described in the above embodiments. For example, when the first viewing user watches a video for the first time, motion data of the first avatar object 110 and the second avatar object 150 shaking hands may be used. When the first viewing user raises their arm upward with their palm facing forward, motion data of the first avatar object 110 and the second avatar object 150 high-fiving by bringing their palms together at head height and slapping each other may be used.

- In the first and third embodiments, when the tracking data is used to depict a scene in which the second avatar object 150 hands over the gift object 135 to the first avatar object 110, there is a possibility that the first viewing user will not perform the action of handing over the gift object 135 to the first avatar object 110. Or, it is assumed that the first viewing user will fail to hand over the gift object 135 to the first avatar object 110. If the user device 12 determines that the gift object 135 and the first avatar object 110 are not associated even after a predetermined time has elapsed since the second avatar object 150 was displayed, the user device 12 may forcibly associate the gift object 135 and the first avatar object 110.

In the second embodiment, the second avatar object 150 may perform an action depending on the type of operation of the first viewing user. For example, when the first viewing user taps the screen, the user device 12 draws the second avatar object 150 performing a thumbs-up action. Also, when the first viewing user taps the screen multiple times, the user device 12 draws the second avatar object 150 performing a clapping action.

<Movement of First Avatar Object>
In the second and third embodiments, the emote selection unit 140 is output on the viewing screen in association with the second avatar object 150. Alternatively or in addition to this, as shown in FIG. 15, the emote selection unit 140 may be output on the distribution screen 103 in association with the first avatar object 110. The emote selection unit 140 shown in FIG. 1 includes selection buttons 140A and 140B. The selection button 140A is a button for selecting an action such as "open immediately" the gift object 135 passed from the second avatar object 150. The selection button 140B is a button for selecting an action such as "place" the gift object 135. Depending on which of the selection buttons 140A and 140B the broadcasting user selects, the action of the first avatar object 110 is determined. When the broadcasting user selects the selection button 140A, the selection motion data for opening the gift object 135, which is a box, is used for the first avatar object 110. On the other hand, when the broadcasting user selects selection button 140B, selection motion data for placing gift object 135 at a predetermined position on the floor or the like is used for first avatar object 110. This makes it possible to appropriately and smoothly output scenes that are difficult to render using tracking data alone. Also, it is possible to broadcast the broadcasting user's intentions in specific scenes, such as a scene where gift object 135 is received, in the video.

The distribution screen 103 shown in FIG. 15 includes a share button 116, a co-star confirmation button 137, a broadcaster ranking button 115, a setting button 138, and an audio button 160. By selecting the co-star confirmation button 137 during collaboration distribution, the profile information of the co-stars can be viewed. By selecting the setting button 138, distribution settings can be made. When the audio button 160 is selected, the broadcast user device 12A stops (mutes) the audio output or cancels the audio stop state. In addition, the distribution screen 103 includes a comment button 161, a shooting button 141, a game button 142, a video viewing button 143, and a gacha execution button 144. By operating the comment button 161, the broadcast user device 12A outputs an input form for inputting a comment. By operating the shooting button 141, the broadcast user device 12A captures the video. By operating the game button 142, the broadcast user device 12A outputs a list of games that can be played during video distribution. The game play screen is output to the video. When the video viewing button 143 is operated, the broadcast user device 12A outputs a screen for selecting videos provided by other video broadcast services. When the gacha execution button 144 is operated, the broadcast user device 12A outputs a list of available lotteries (gacha).

<Drawing of Gift Object in Response to Display Request from Broadcaster User Device>
The broadcast user may use the broadcast user device 12A to make a request to output the gift object 135. When the second avatar object 150 is being output to the video, the broadcast user performs an operation to select a gift object. The broadcast user device 12A transmits a gift ID and a gift output request. Based on the gift ID and the gift output request, the viewing user device 12B outputs the gift object 135 corresponding to the gift ID in association with the second avatar object 150. According to this aspect, the broadcast user can gift the gift object 135 to the first viewing user. This can make the interaction between the broadcast user and the viewing user more active.

<Display condition of second avatar object>
In the above embodiments, when the viewing user device 12B transmits a request to output the gift object 135, each user device 12 renders the second avatar object 150. Alternatively or in addition to this, when a condition other than the transmission of the request to output the gift object 135 is satisfied, the second avatar object 150 corresponding to the first viewing user may be rendered.

The following provides examples of conditions other than the transmission of a request to output the gift object 135, but the conditions may be other than the transmission of a request to output the gift object 135 and are not limited to the following examples. For example, the condition other than the transmission of a request to output the gift object 135 is that the value of a parameter associated with the first viewing user satisfies a predetermined condition related to the parameter. A user who satisfies this predetermined condition can transmit a request to display the second avatar object 150. The parameter may be the number of mutual followers who are mutually following users who use the video application. Alternatively, the parameter may be the ranking order of the broadcasting user or the ranking order of the viewing user. Furthermore, the ranking may be calculated regardless of whether the broadcasting user or the viewing user is a broadcasting user. The ranking of the broadcasting user is determined based on the broadcasting user's video broadcast status or the status in the video. The ranking of the viewing user is determined based on the behavior of the viewing user.

One example of a ranking of broadcasting users is the ranking of broadcasting users based on the number of gift objects they have received among all broadcasting users who broadcast videos. The ranking is also based on the number of gift objects received for each type of gift object, such as "bouquets of flowers." One example of a ranking of viewing users is the ranking of viewing users based on the number of gifts they have sent.

For example, a condition other than the transmission of a request to output the gift object 135 is that the activity related to the user satisfies a predetermined condition. A user who satisfies this predetermined condition can transmit a request to display the second avatar object 150. For example, the amount of money charged by the user in the video application may be equal to or greater than a predetermined amount. Or, the predetermined condition may be that the number of messages posted by the user in the video application is equal to or greater than a predetermined number. Or, the predetermined condition may be that the number of gift objects sent by the user in the video application is equal to or greater than a predetermined number. Or, the predetermined condition may be that the amount of money charged or the number of messages posted for the video being viewed is equal to or greater than a predetermined number.

For example, the first viewing user may post a specific message. Or the first viewing user may watch a video for the first time. Or the first viewing user may perform a specific operation. For example, when the first viewing user posts a specific message such as "Nice to meet you," motion data of the first avatar object 110 and the second avatar object 150 shaking hands may be used.

- In each of the above embodiments, the second avatar object 150 corresponding to all first viewing users who have transmitted a gift output request is drawn. In addition to this, a condition related to the first viewing user who can display the second avatar object may be set. Below, an example of a condition related to the first viewing user is given, but the condition may be related to the first viewing user and is not limited to the following examples. For example, when a first viewing user who follows a distribution user transmits a gift output request, the user device 12 may draw the second avatar object 150 of the first viewing user. In other words, it is sufficient that the follow list stored in the viewing user device 12B or the like contains the identification information of the distribution user. In addition, the condition of the first viewing user who outputs the second avatar object 150 may be at least one of the following conditions instead of or in addition to mutually following the distribution user. For example, the first viewing user belongs to a fan club that supports the distribution user, is ranked at or above a predetermined rank in the support ranking of the distribution user, has viewed a video for the first time, and the first viewing user's cumulative viewing time is at or above a predetermined time or the number of views is at or above a predetermined number. Alternatively, the conditions for the first viewing user to output the second avatar object 150 are that the number of times a gift output request has been sent is equal to or greater than a predetermined number, the number of times a message has been posted is equal to or greater than a predetermined number, a gift object of equal to or greater than a predetermined amount has been given, and the billing amount has reached a reference value. Also, once the second avatar object 150 has been output in one distribution, the second avatar object 150 may not be output in that distribution. According to this aspect, it is possible to prevent a situation in which there are too many second avatar objects to be output, resulting in insufficient distribution time and therefore the second avatar objects cannot be output. Also, by limiting the number of times the second avatar object is drawn, the load on the drawing process of the user device 12 can be reduced.

<Virtual camera position>
In each of the above embodiments, the position of the virtual camera is a common position among the broadcasting user device 12A, the first viewing user's viewing user device 12B, and the second viewing user's viewing user device 12B. Alternatively or in addition to this, the position (viewpoint) of the virtual camera may be different for each of the broadcasting user device 12A, the first viewing user's viewing user device 12B, and the second viewing user's viewing user device 12B.

The virtual camera positions that can be set by each user device 12 include the following positions.
(Position A) Position of the first avatar object 110 (Position B) Position of the second avatar object 150 (Position C) Position other than the first avatar object 110 and the second avatar object 150 An example of position A is the eye position of the first avatar object 110. That is, it is a position where a first-person perspective video of the first avatar object 110 can be displayed. More specifically, it is a center position of one of the eyes or both eyes. Alternatively, position A may be the top of the head of the first avatar object 110. The line of sight direction of the virtual camera is a direction where a first-person perspective video of the first avatar object 110 can be displayed. The line of sight direction may be constant. Or, the line of sight direction may be changed based on the user's operation of the operation unit 27.

A specific example of position B is the eye position of second avatar object 150. That is, it is a position where a first-person perspective video of second avatar object 150 can be displayed. More specifically, it is the center position of one of both eyes or both eyes. Alternatively, position B may be the top of the head of second avatar object 150. The line of sight of the virtual camera is a direction where a first-person perspective video of second avatar object 150 can be displayed. The line of sight direction may be constant. Or, the line of sight direction may be changed based on the user's operation of operation unit 27.

A specific example of position C is a position that overlooks the first avatar object 110 and the second avatar object 150. Position C may also be set by the user to any position. The line of sight of the virtual camera may be constant. Alternatively, the line of sight may be changed based on the user's operation of the operation unit 27.

The broadcast user device 12A may be configured to set a virtual camera at position A and render a video. This allows the broadcast user device 12A to render a video in which the first avatar object 110 is looking at the second avatar object 150. When the second avatar object 150 performs an action of handing over the gift object 135, the broadcast user can view a video with such realism that it feels as if the broadcast user is receiving the gift object 135 from the second avatar object 150.

The viewing user device 12B of the first viewing user may be configured to set a virtual camera at position B to render video. This allows the viewing user device 12B to render video that looks as if the first avatar object 110 is being viewed from the second avatar object 150. When the second avatar object 150 performs an action of handing over the gift object 135, the first viewing user can view a video with a sense of realism as if the first viewing user is handing over the gift object 135 to the first avatar object 110.

The viewing user device 12B of the second viewing user may be configured to set a virtual camera at position C and render a video. This allows the viewing user device 12B to render a video that appears as if the user is viewing the first avatar object 110 and the second avatar object 150.

Each user device 12 draws the second avatar object 150 using either the automatic motion data, the selected motion data, or the tracking data, regardless of whether the virtual camera is set to position A, position B, or position C. Similarly, each user device 12 draws the first avatar object 110 using either the automatic motion data, the selected motion data, or the tracking data, regardless of whether the virtual camera is set to position A, position B, or position C.

The distribution user, the first viewing user, and the second viewing user may be able to set the camera position from among positions A, B, and C. Indicators such as virtual camera icons corresponding to positions A, B, and C, respectively, may be displayed on the screen, and when each user selects one of these icons, the virtual camera may be changed to the selected position.

The user device 12 may automatically set the position of the virtual camera according to the type of the gift object 135. When drawing the gift object 135, the user device 12 sets the position of the virtual camera based on information about the position of the virtual camera associated with the gift object 135. For example, the user device 12 may set the position of the virtual camera so that the relative distance from the gift object 135 is longer or shorter according to the size of the gift object 135. When drawing a relatively large gift object 135, the user device 12 sets the position of the virtual camera so that the entire gift object 135 is included in the imaging range of the virtual camera. In addition, the user device 12 may set the position of the virtual camera so that the relative distance from a predetermined position of the movement trajectory of the gift object 135 is longer or shorter according to the movement range of the gift object 135. When drawing a "snowball" gift object 135, the user device 12 changes the position of the virtual camera so that the relative distance from the gift object 135 is longer. Then, at least a part of the movement trajectory of the gift object 135 is included in the imaging range of the virtual camera. Alternatively, the position of the virtual camera may be moved in accordance with the movement of the gift object 135.

<Gift output request by group>
In each of the above embodiments, when multiple first viewing users give gifts to a distribution user, the user device 12 may draw second avatar objects 150 corresponding to the first viewing users. For example, the viewing user device 12B is performing a group chat in which multiple viewing users chat. One viewing user device 12B that requests gift output transmits a gift ID and a group ID to the server 13. Instead of the group ID, a user ID of a user belonging to the group may be transmitted. The server 13 acquires part IDs of the body parts and part IDs of the attached parts of the avatar objects of the users belonging to the group. Then, the server 13 transmits the data together with the gift ID to each user device 12. The user device 12 that receives the part ID and the gift ID draws the second avatar object 150 of the first viewing user belonging to the group together with the gift object corresponding to the gift ID. Alternatively, the user device 12 draws the second avatar object 150 of a representative of the first viewing users belonging to the group together with the gift object corresponding to the gift ID. The representative may be the first viewing user who made the gift output request, or may be a viewing user who has been predetermined within the group. When a representative is predetermined, the user device 12 or the server 13 records the user ID of the user.

<Mode for drawing the second avatar object>
Each user device 12 may render a plurality of second avatar objects 150 based on an instruction transmitted from the server 13. The second avatar objects 150 correspond to first viewing users who have performed an operation of transmitting a gift output request within a predetermined period of time.

For example, each user device 12 may render a video in either a first mode in which one second avatar object 150 is rendered, or a second mode in which multiple second avatar objects 150 are rendered, based on instructions transmitted from the server 13. A condition for changing from the first mode to the second mode is, for example, that the number of users viewing the video in the first mode exceeds the maximum number of users in the first mode. Or, that the broadcasting user performs an operation to change from the first mode to the second mode, or an operation to change from the second mode to the first mode.

The processing load of one second avatar object 150 in the second mode is smaller than the processing load of one second avatar object 150 in the first mode. For example, the rendering data of one second avatar object 150 in the second mode may be data having a smaller number of polygons than the rendering data of one first avatar object 110 in the first mode. Alternatively, one first avatar object 110 in the first mode may be rendered as three-dimensional data, while one second avatar object 150 in the second mode may be rendered as two-dimensional data. This makes it possible to reduce the processing load when rendering multiple second avatar objects 150 in the second mode. The number of viewing users may be the cumulative number of viewing users including viewing users who stopped viewing the video midway, or may be the number of viewing users currently viewing the video.

In the second mode, a predetermined number of second avatar objects 150 may be displayed among all viewing users. For example, if there are 50 or more first viewing users, second avatar objects 150 of viewing users corresponding to 20 first viewing users may be displayed. Then, the second avatar objects 150 of all first viewing users may be displayed in multiple batches.

The second avatar object 150 may be lined up in front of the first avatar object 110 located on the stage. Alternatively, the second avatar object 150 may be lined up so as to surround the first avatar object 110.

When the first viewing user performs an operation on the viewing user device 12B to send a gift output request, the viewing user device 12B sends the gift output request to the server 13 together with the user ID and gift ID of the first viewing user, etc. The server 13 sends the gift output request, etc. to each user device 12. At this time, the server 13 sends to the user device 12 a request to make the second avatar object 150 of the first viewing user stand out more than the other second avatar objects 150. For example, the second avatar object 150 is displayed next to the first avatar object 110. When the first avatar object 110 is located on the stage, the second avatar object 150 is drawn using motion data that depicts the movement of the second avatar object 150 going up to the stage.

<Giving gift objects during collaboration broadcasts>
A broadcast user may perform a collaboration broadcast. A collaboration broadcast is a broadcast in which multiple broadcast users participate. Examples of collaboration broadcasts include a first collaboration broadcast and a second collaboration broadcast. In the first collaboration broadcast method, multiple broadcast users participate in a video broadcast from the start of the broadcast. The first avatar objects 110 of these broadcast users are displayed in the video. In the second collaboration broadcast method, there are a host user who is a broadcast user and a guest user who is a broadcast user. The host user starts broadcasting a video and allows the guest user to participate in the video. The guest user obtains permission from the host user during the broadcast of the video. As a result, the first avatar object 110 of the guest user is displayed in the video. The host user is the first broadcast user, and the guest user is the second broadcast user. There may be multiple second broadcast users.

The first viewing user can display the second avatar object 150 corresponding to the first viewing user in the video in which the collaboration distribution is being performed. In other words, the video may display multiple first avatar objects 110 corresponding to each distribution user and one or multiple second avatar objects 150. The first avatar object 110 and the second avatar object 150 have different conditions for displaying in the video and for ending the display of the video. Specifically, the first avatar object 110 is displayed when the distribution of the video starts or when the distribution user gives permission. The second avatar object 150 is displayed when the viewing user sends a gift display request. The first avatar object 110 is made invisible when the distribution of the video ends, when one of the distribution users performs an exit operation on the first avatar object 110, or when a predetermined time for displaying the first avatar object 110 has elapsed. On the other hand, the second avatar object 150 is made invisible when the non-display condition for making the second avatar object 150 invisible is satisfied. An example of a non-display condition is when the second avatar object 150 completes an action. The action of the second avatar object 150 may be associated with a gift ID.

When the second avatar object 150 is displayed in a video in which collaborative distribution is taking place, the second avatar object 150 may present gift objects to multiple first avatar objects 110.

An example of a method for giving gift objects to multiple first avatar objects 110 will be described. The first viewing user performs an operation to designate the first avatar object 110. The viewing user device 12B of the first viewing user transmits a gift ID, a user ID, a gift output request, and identification information of the designated first avatar object 110 to the server 13. The server 13 transmits at least the gift ID, the gift output request, and identification information of the designated first avatar object 110 to the user device 12. The server 13 also transmits a request to each user device 12 to draw the second avatar object 150 by designating at least one of automatic motion data, selected motion data, or tracking data. The method for drawing the second avatar object using the automatic motion data, selected motion data, or tracking data can be any of the methods described in the above embodiments.

Each user device 12 displays the second avatar object 150 and the gift object 135 in an animation. It also moves the second avatar object 150 to a position near the specified first avatar object 110. This position may be, for example, in front of the first avatar object 110. The user device 12 draws the second avatar object using at least one of the automatic motion data, the selected motion data, and the tracking data. Then, it associates the gift object 135 with the first avatar object 110.

Another example of a method for giving gift objects to multiple first avatar objects 110 will be described. When the number of gift output requests sent by the first viewing user reaches the number of first avatar objects 110 displayed in the video, the server 13 sends output requests for the second avatar object 150 and the gift object 135 to each user device 12. After selecting one gift object 135, the first viewing user repeatedly operates the gift button 122 the same number of times as the number of broadcast users displayed in the video. Alternatively, the first viewing user repeatedly operates the combination of selecting one gift object and operating the gift button 122 the same number of times as the number of first avatar objects 110. For example, when two broadcast users participate in a collaboration broadcast, the first viewing user operates the gift button 122 twice. As a result, the viewing user device 12B of the first viewing user transmits gift output requests, together with the gift ID and the user ID, the same number of times as the number of times the gift button 122 was operated.

The server 13 receives gift output requests, etc. The server 13 also compares the number of gift output requests received within a specified time period with the number of broadcast users participating in the video being viewed by the first viewing user. When the number of gift output requests reaches the number of broadcast users, the server 13 transmits a request to each user device 12 to draw the second avatar object 150 by specifying at least one of automatic motion data, selected motion data, or tracking data. The method of drawing the second avatar object using the automatic motion data, selected motion data, or tracking data can be any of the methods described in the above embodiments.

When the gift object 135 associated with the second avatar object 150 is associated with the first avatar object 110, the user device 12 moves the position of the second avatar object 150 to a position near the next first avatar object 110 to which the gift object 135 is not associated. This position is, for example, in front of the first avatar object 110. When the user device 12 moves the second avatar object 150, it draws the second avatar object 150 using at least one of automatic motion data, selected motion data, and tracking data. When using tracking data, after the second avatar object 150 completes its movement, it draws the second avatar object 150 using the tracking data of the first viewing user received by the server 13. When the user device 12 moves the second avatar object 150 to the next position, it associates a new gift object 135 with the second avatar object 150. Then, the same process as that in the case of giving the gift object 135 to the first first avatar object 110 is repeated. The user device 12 repeats the process of associating the gift object 135 with the first avatar object 110, moving it, and associating the new gift object 135 with the second avatar object 150 until all of the first avatar objects 110 have been associated with a gift object 135.

Another example of a method for giving a gift object to a plurality of first avatar objects 110 will be described. The server 13 stores the order in which the first avatar objects 110 receive the gift object 135 in the collaborative distribution. The order may be determined randomly by the server 13. Or the order may be determined based on the position of the first avatar object 110. Or the order may be determined based on the priority of the first avatar object 110. When the server 13 receives the gift ID and the gift output request, the server 13 determines the first avatar object 110 to which the gift object 135 is to be given first based on the order. Then, the server 13 transmits at least the gift ID, the gift output request, and the identification information of the first avatar object 110 to the user device 12. Each user device 12 displays the second avatar object 150 and the gift object 135 at a position near the first first avatar object 110 corresponding to the received identification information of the first avatar object 110. Furthermore, when the server 13 newly receives a gift ID and a gift output request, the server 13 determines the first avatar object 110 to which the gift object 135 is to be given second based on the order. Then, the server 13 transmits the gift ID, the gift output request, and the identification information of the first avatar object 110 to the user device 12. Each user device 12 displays the second avatar object 150 and the gift object 135 at a position near the second first avatar object 110 corresponding to the received identification information of the first avatar object 110. In this way, the server 13 transmits the gift output request and the identification information of the first avatar object 110 to the user device 12 until it receives the same number of gift IDs and gift output requests as the number of first avatar objects 110 from one viewing user device 12B. The user device 12 displays the second avatar object 150 and the gift object 135 at a position near the first avatar object 110 corresponding to the received identification information of the first avatar object 110.

In addition, when the second avatar object 150 passes on the gift object 135 in this collaborative distribution, the other first avatar objects 110 other than the first avatar object 110 with which the gift object 135 is associated may be drawn using an automatic motion. For example, the automatic motion is the action of the other first avatar object 110 staring intently at the gift object 135.

<Configuration of Information Processing System>
Although the sensor unit 24 has been described as acquiring detection data detecting a change in the user's facial expression, head movement, and the user's relative position with respect to the sensor unit 24, at least one of these may be acquired. Furthermore, the sensor unit 24 may acquire other detection data, such as detection data detecting movement other than head movement, in addition to or instead of at least one of the change in the user's facial expression, head movement, and the user's relative position.

The virtual space output in the video may be an augmented reality (AR) space. For example, an image of the real world captured by the camera of the viewing user device 12B may be output with animations of avatar objects, gift objects, etc. based on data transmitted from the broadcasting user device 12A superimposed thereon. Alternatively, the broadcasting user device 12A may generate a video by superimposing animations of avatar objects, gift objects, etc. on an image of the real world captured by its own camera and outputting the superimposed animations, encode the video, and transmit it to the server 13.

- In each of the above embodiments, the user device 12 is an information processing device such as a smartphone, a mobile phone, a tablet terminal, a personal computer, a console game machine, a wearable computer such as a head-mounted display, etc. Alternatively, the information processing system may be a system provided in a studio for video distribution. The information processing system has an information processing device, a server, a sensor unit 24 worn on the body of the distribution user, a tracking system that detects the position of the sensor unit 24, an operation unit 27, a speaker 25, and a display 28. The tracking system may be equipped with a multi-axis laser emitter that emits a pulsed laser light for synchronization. The sensor unit 24 has a sensor that detects laser light, and detects its own position and orientation while synchronizing with a synchronization pulse. For example, the sensor unit 24 may be a Vive Tracker (registered trademark) and a Vive Base Station (registered trademark) provided by HTC Corporation (registered trademark).

- The control unit 20 of the user device 12 executes a first output control process and a second output control process by executing an application program. At least one of these processes may be executed by the server 13 or another device constituting the information processing system 11. In addition, the control unit 30 of the server 13 executes a first transmission process, a gift reception process, and a second transmission process. At least one of these processes may be executed by the user device 12 or another device constituting the information processing system 11.

The technical ideas and effects obtained from the above-described embodiment and modified examples will be described below.
[1] one or more computers,
a first output control process for outputting a video including a first avatar object corresponding to the broadcast user to a display;
A reception process for receiving a selection operation of selecting a gift object by a viewing user who is viewing the video;
a gift sending process for sending gift identification information of the selected gift object and identification information of the viewing user to a server together with an output request for the gift object;
a second output control process for outputting, to the display, a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user, the video being based on data for outputting the video;
The motion data for moving the second avatar object, which is associated with the identification information of the gift object, is identified based on the identification information of the motion data, and the gift object and the second avatar object are drawn using the motion data.

[A] one or more computers,
a first output control process for outputting a video including a first avatar object corresponding to the broadcast user to a display;
A reception process for receiving a selection operation of selecting a gift object by a viewing user who is viewing the video;
a gift sending process for sending gift identification information of the selected gift object and identification information of the viewing user to a server together with an output request for the gift object;
a second output control process for outputting, to the display, a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user, the video being based on data for outputting the video;
In the second output control process,
A program that draws the second avatar object using at least one of tracking data of the viewing user and selected motion data selected by the viewing user, and preset automatic motion data.

[B] The program described in [A], in which, in the second output control process, the tracking data is used to draw at least some parts of the second avatar object, and the automatic motion data is used to draw at least some other parts of the second avatar object.

[C] The program described in [A], in which in the second output control process, the second avatar object is drawn using the automatic motion data, and then the second avatar object is drawn using the selected motion data based on an input instruction from the viewing user.

[D] The program described in [A] or [C], in which the second output control process outputs a selection unit that selects the selected motion data, and draws the second avatar object using the selected motion data according to an input instruction of the viewing user received by the selection unit.

[E] A program described in any one of [A] to [D], which, when outputting the gift object, outputs the second avatar object and the gift object corresponding to the viewing user who made the output request for the gift object and continues watching the video, and outputs the gift object without outputting the second avatar object if the viewing user does not continue watching the video.

[F] A program described in any one of [A] to [E], which, when outputting the gift object, changes the display mode of the gift object depending on whether the viewing user who made the output request for the gift object continues to watch the video or not.

[G] A program described in any one of [A] to [F], in which in the second output control process, the second avatar object is output using data that imposes a smaller rendering processing load than the first avatar object.

[H] A program described in any one of [A] to [G], which executes the second output control process when the viewing user who transmitted the gift identification information to the server satisfies a predetermined condition.

[I] The program described in [H] executes the second output control process when the value of a parameter associated with the viewing user satisfies the predetermined condition related to the parameter.

[J] The program according to claim [H], further comprising: executing the second output control process when the activity of the viewing user satisfies the predetermined condition regarding the activity.
[K] A program described in any one of [A] to [J], which, in a distribution mode in which the video is distributed, when outputting the gift object in response to a request to output the gift object transmitted from another viewing user device, draws the first avatar object corresponding to the distribution user using the tracking data of the distribution user distributing the video and at least one of the selected motion data selected by the distribution user, and predetermined automatic motion data.

[L] A program described in any one of [A] to [K], which, when a broadcast user device used by the broadcast user transmits a request to output the gift object while the second avatar object is being displayed, outputs the gift object in association with the second avatar object based on the output request.

[M] The program described in any one of [A] to [L], wherein the selected motion data is the selected motion data selected by the viewing user from a plurality of preset motion data.

[O] associating object data selected by an operation input of the viewing user with the second avatar object;
A program described in any one of [A] to [M], wherein the second output control process draws the second avatar object based on the selected object data.

[P] one or more computers,
a first output control process for outputting a video including a first avatar object corresponding to the broadcast user to a display;
A reception process for receiving a selection operation of selecting a gift object by a viewing user who is viewing the video;
a gift sending process for sending gift identification information of the selected gift object and identification information of the viewing user to a server together with an output request for the gift object;
a second output control process for outputting, on the display, a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user, based on data for outputting the video;
In the second output control process,
An information processing method, comprising: drawing the second avatar object using at least one of tracking data of the viewing user and selected motion data selected by the viewing user, and preset automatic motion data.

[Q] One or more computers:
a first transmission process for transmitting data for outputting a video including a first avatar object corresponding to a distributor user to a viewing user device used by at least a viewing user who views the video;
a gift receiving process for receiving, from a first viewing user device used by a first viewing user, gift identification information of a gift object and identification information of the first viewing user, together with an output request for the gift object;
executing a second transmission process of transmitting the gift identification information and an output request for outputting a video including the gift object corresponding to the gift identification information, the first avatar object, and a second avatar object corresponding to the first viewing user, at least to a distribution user device used by the distribution user and a second viewing user device used by a second viewing user other than the first viewing user;
An information processing method, wherein the output request transmitted in the second transmission process is transmitted together with data used to draw the second avatar object or identification information of the data, and the data used to draw the second avatar object includes tracking data of the first viewing user and at least one of selected motion data selected by the first viewing user, and predetermined automatic motion data.

[R] a first transmission unit that transmits data for outputting a video including a first avatar object corresponding to a distribution user to a viewing user device used by at least a viewing user who views the video;
a gift receiving unit that receives, from a first viewing user device used by a first viewing user, gift identification information of a gift object and identification information of the first viewing user together with an output request for the gift object;
a second transmission unit that transmits the gift identification information and an output request for outputting a video including the gift object corresponding to the gift identification information, the first avatar object, and a second avatar object corresponding to the first viewing user, at least to a distribution user device used by the distribution user and a second viewing user device used by a second viewing user other than the first viewing user,
An information processing device, wherein the output request sent by the second transmission unit is sent together with data used to draw the second avatar object or identification information of the data, and the data used to draw the second avatar object includes tracking data of the first viewing user and at least one of selected motion data selected by the first viewing user, and predetermined automatic motion data.
[1] one or more computers,
a first output control process for outputting a video including a first avatar object corresponding to the broadcast user to a display;
A reception process for receiving a selection operation of selecting a gift object by a viewing user who is viewing the video;
a gift sending process for sending gift identification information of the selected gift object and identification information of the viewing user to a server together with an output request for the gift object;
a second output control process for outputting, to the display, a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user, the video being based on data for outputting the video;
In the second output control process,
A program that draws the second avatar object using at least one of tracking data of the viewing user and selected motion data selected by the viewing user, and preset automatic motion data.
[2]
The program described in [1], in which the second output control process uses the tracking data to draw at least some parts of the second avatar object, and uses the automatic motion data to draw at least some other parts of the second avatar object.
[3]
The program described in [1], in the second output control process, the second avatar object is drawn using the automatic motion data, and then the second avatar object is drawn using the selected motion data based on an input instruction of the viewing user.
[4]
The second output control process outputs a selection unit that selects the selected motion data, and draws the second avatar object using the selected motion data in response to an input instruction from the viewing user received by the selection unit.A program described in [1] or [3].
[5]
The program described in [1], when outputting the gift object, if the viewing user who made the request to output the gift object continues watching the video, the second avatar object corresponding to the viewing user and the gift object are output, and if the viewing user does not continue watching the video, the program outputs the gift object without outputting the second avatar object.
[6]
The program described in [1] or [5], which, when outputting the gift object, changes the display mode of the gift object depending on whether the viewing user who made the output request of the gift object continues watching the video or not.
[7]
The program described in [1], in the second output control process, the second avatar object is output using data that has a smaller drawing processing load than the first avatar object.
[8]
The program described in [1], which executes the second output control process when the viewing user who transmitted the gift identification information to the server satisfies a predetermined condition.
[9]
The program according to [8], further comprising: executing the second output control process when a value of a parameter associated with the viewing user satisfies the predetermined condition regarding the parameter.
[10]
The program according to [8], further comprising: executing the second output control process when the activity of the viewing user satisfies the predetermined condition regarding the activity.
[11]
The program described in [1], in a distribution mode in which the video is distributed, when outputting the gift object in response to a request to output the gift object transmitted from another viewing user device, draws the first avatar object corresponding to the distribution user using the tracking data of the distribution user distributing the video and at least one of the selected motion data selected by the distribution user, and the automatic motion data set in advance.
[12]
The program described in [1], wherein when a broadcast user device used by the broadcast user sends a request to output the gift object while the second avatar object is displayed, the program outputs the gift object in association with the second avatar object based on the output request.
[13]
The program according to [1], wherein the selected motion data is selected by the viewing user from a plurality of preset motion data.
[14]
Corresponding object data selected by an operation input of the viewing user to the second avatar object;
The program described in [1], wherein the second output control process draws the second avatar object based on the selected object data.
[15]
One or more computers
a first output control process for outputting a video including a first avatar object corresponding to the broadcast user to a display;
A reception process for receiving a selection operation of selecting a gift object by a viewing user who is viewing the video;
a gift sending process for sending gift identification information of the selected gift object and identification information of the viewing user to a server together with an output request for the gift object;
a second output control process for outputting, on the display, a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user, based on data for outputting the video;
In the second output control process,
An information processing method, comprising: drawing the second avatar object using at least one of tracking data of the viewing user and selected motion data selected by the viewing user, and preset automatic motion data.
[16]
One or more computers
a first transmission process for transmitting data for outputting a video including a first avatar object corresponding to a distributor user to a viewing user device used by at least a viewing user who views the video;
a gift receiving process for receiving, from a first viewing user device used by a first viewing user, gift identification information of a gift object and identification information of the first viewing user, together with an output request for the gift object;
executing a second transmission process of transmitting the gift identification information and an output request for outputting a video including the gift object corresponding to the gift identification information, the first avatar object, and a second avatar object corresponding to the first viewing user, at least to a distribution user device used by the distribution user and a second viewing user device used by a second viewing user other than the first viewing user;
An information processing method, wherein the output request transmitted in the second transmission process is transmitted together with data used to draw the second avatar object or identification information of the data, and the data used to draw the second avatar object includes tracking data of the first viewing user and at least one of selected motion data selected by the first viewing user, and predetermined automatic motion data.
[17]
a first transmission unit that transmits data for outputting a video including a first avatar object corresponding to a distributor user to a viewing user device used by at least a viewing user who views the video;
a gift receiving unit that receives, from a first viewing user device used by a first viewing user, the gift identification information of the gift object and the identification information of the first viewing user together with an output request for the gift object;
a second transmission unit that transmits the gift identification information and an output request for outputting a video including the gift object corresponding to the gift identification information, the first avatar object, and a second avatar object corresponding to the first viewing user, at least to a distribution user device used by the distribution user and a second viewing user device used by a second viewing user other than the first viewing user,
An information processing device, wherein the output request sent by the second transmission unit is sent together with data used to draw the second avatar object or identification information of the data, and the data used to draw the second avatar object includes at least one of tracking data of the first viewing user and selected motion data selected by the first viewing user, and predetermined automatic motion data.

11: Information processing system 12: User device 13: Server

Claims (18)

On one or more computers,
a first output control process for outputting a video including a first avatar object corresponding to a first user to a display;
a gift sending process for sending identification information of the second user to a server together with a gift object output request based on an operation of the second user when the second user who is watching the video satisfies a predetermined condition ;
A program that executes a second output control process that outputs to the display a video based on data for outputting a video including the gift object, the first avatar object, and a second avatar object corresponding to the second user . The computer includes:
further executing a reception process for receiving a selection operation of the gift object by the second user;
The program according to claim 1 , wherein, in the gift transmission process, when the second user satisfies the predetermined condition, the gift identification information of the selected gift object and an output request for the gift object are transmitted. The program of claim 1 , further comprising: executing the gift sending process when a value of a parameter associated with the second user satisfies the predetermined condition regarding the parameter. The program according to claim 1 , further comprising: a step of executing the gift sending process when the activity of the second user satisfies the predetermined condition regarding the activity. The program described in claim 1, wherein the second output control process outputs to the display the video including the second avatar object using at least one of the tracking data of the second user, selected motion data selected by the second user, and predetermined automatic motion data. The program described in claim 1, wherein the second output control process outputs to the display the video including the second avatar object using at least one of the tracking data of the second user and selected motion data selected by the second user, and predetermined automatic motion data. The program described in claim 1, wherein the second output control process outputs to the display the video including the second avatar object in which some parts are drawn using tracking data of the second user and other parts are drawn using automatic motion data. The program described in claim 1, wherein the second output control process draws the second avatar object using preset automatic motion data, and then outputs to the display the video in which the second avatar object is drawn using selected motion data selected by the second user. The program described in claim 1, wherein the second output control process outputs a selection unit that selects selected motion data, and outputs to the display the video in which the second avatar object is drawn using the selected motion data in response to input instructions from the second user received by the selection unit. The program described in claim 1, wherein in the second output control process, when the video including the gift object is output to the display, if the second user who made the request to output the gift object continues watching the video, a video including the second avatar object corresponding to the second user and the gift object is output, and if the second user does not continue watching the video, the video including the gift object is output without outputting the second avatar object. The program described in claim 1, wherein in the second output control process, when the video including the gift object is output to the display, the video is output with the display mode of the gift object changed depending on whether the second user who made the output request of the gift object continues watching the video or not. The program according to claim 1 , wherein in the second output control process, the second avatar object is output using data that imposes a smaller processing load on drawing than the first avatar object. The program of claim 1, further causing the computer to execute a process of outputting the video in which the first avatar object corresponding to the first user is drawn using at least one of tracking data of the distribution user distributing the video and selected motion data selected by the first user, and preset automatic motion data, when outputting the gift object in response to an output request of the gift object transmitted from another second user device in a distribution mode in which the video is distributed. The program described in claim 1, wherein in the second output control process, when a first user device used by the first user sends an output request for the gift object while the second avatar object is displayed, the video including the gift object associated with the second avatar object is output based on the output request. The program according to claim 1 , wherein the second output control process outputs the video in which the second avatar object is rendered based on object data selected by an operation input of the second user. One or more computers
a first output control process for outputting a video including a first avatar object corresponding to a first user to a display;
a gift sending process for sending identification information of the second user to a server together with a gift object output request based on an operation of the second user when the second user who is watching the video satisfies a predetermined condition;
An information processing method, comprising: a second output control process for outputting a video to the display based on data for outputting a video including the gift object, the first avatar object, and a second avatar object corresponding to the second user. One or more computers
a first transmission process for transmitting data for outputting a video including a first avatar object corresponding to a first user to a second user device used by at least a second user who watches the video;
a gift receiving process for receiving identification information of the second user from the second user device used by the second user who satisfies a predetermined condition, together with a request to output a gift object;
An information processing method comprising: executing a second transmission process of transmitting an output request for outputting a video including the gift object, the first avatar object, and a second avatar object corresponding to the second user to at least a first user device used by the first user and a second user device used by the second user other than the second user who satisfies the specified condition. a first transmission unit that transmits data for outputting a video including a first avatar object corresponding to a first user to a second user device used by a second user who watches at least the video;
a gift receiving unit that receives, from the second user device used by the second user that satisfies a predetermined condition, identification information of the second user together with a request to output a gift object;
An information processing system comprising: a second transmission unit that transmits an output request for outputting a video including the gift object, the first avatar object, and a second avatar object corresponding to the second user at least to a first user device used by the first user and to a second user device used by a second user other than the second user who satisfies the specified condition.